Skip to content

Route on what a coworker can reach, not only what it is for - #144

Merged
davidmckayv merged 2 commits into
mainfrom
fix/route-to-a-bot-that-can-reach-it
Aug 22, 2026
Merged

Route on what a coworker can reach, not only what it is for#144
davidmckayv merged 2 commits into
mainfrom
fix/route-to-a-bot-that-can-reach-it

Conversation

@davidmckayv

Copy link
Copy Markdown
Contributor

The other half of #143. That change stopped a Bot browsing to a vendor it held tools for; this one stops the question reaching a Bot that holds none.

The problem

Q: In my OpenBot PRD in Google Drive, what percentage did Kai reach company-wide?

Routed to Knowledge, whose description says "company knowledge questions" and which held no Drive grants. It browsed to drive.google.com, met a sign-in page its container can never satisfy, and asked the person to sign in to an account they had already connected.

Risk Analyst, one line further down the same roster, held all four Drive tools.

The router had this to work with:

- id: knowledge
  name: Knowledge
  for: company knowledge questions

An id, a name, and a sentence somebody wrote. Nothing about what the coworker can actually do. So it matched "knowledge" to a question about a document, which is a perfectly good reading of the only evidence it had.

The change

Candidates carry the systems they hold tools for:

- id: risk-analyst
  name: Risk Analyst
  for: risk and compliance
  can reach: google-drive

and the prompt gains one instruction: prefer a coworker that can reach the system the message names.

Purpose still comes first, explicitly, because the opposite failure has the same shape. A rule that let reach outrank purpose would send every question to whichever coworker happens to hold a connector, and a specialist with no connectors is still the right answer about its own specialism. The instruction says so and there is a test on it.

Three things it does not do

  • It is a hint, not a filter. Nothing is excluded from the roster for holding no tools.
  • It never fails routing. The lookup is per request so a grant added a minute ago counts, and a store that is slow or unhappy is treated as holding nothing. A connector lookup must not turn "who is this for" into an error.
  • It is silent when there is nothing to say. A deployment with no connectors gets the prompt it got before, down to the instruction line, which is not added when nothing on the roster reaches anything. My own test caught me adding it unconditionally.

Proof

Same question, no @ mention, on the merged #143:

  • routed to risk-analyst, fallback: false
  • recorded reason: "Google Drive access and investigation of company-wide metrics" — it names the capability
  • then search_filesread_file_content, and the answer: "Kai reached 83% company-wide weekly active users."

Checked against the document rather than the answer: the file was read through the API first, and it says "Kai reached 83% company-wide; that is the bar" under weekly active users. Exact match.

Before this change, that question went to a coworker that could only browse.

result
bun run test 1153 pass, 8 skip, 0 fail, 1161 across 103 files
bun run typecheck clean, all four packages
bun run format:check clean
bunx biome lint . 1 info, identical to main

Tests

Four on the prompt: that reach appears for a coworker that has it; that it does not for one that does not; that both halves of the instruction are present, the preference and the "purpose still comes first" guard; and that a roster with no reach at all produces the prompt it used to.

Where this sits relative to #119

This is the cheap version of the same question K3 asks. K3 narrows a large catalogue before the model sees it; this tells the router what the handful of coworkers can reach. When tool retrieval lands, this is the signal it should be computed from rather than a thing to keep in step separately.

A tool array tells a model a tool exists. It does not tell it the tool is
the right way to reach that system, and it competes with COMPUTER_GUIDANCE
— a page of emphatic prose about the browser that every Bot receives
whether or not it has a single connector, and that mentions connectors
nowhere. "Never claim you cannot browse" is in there. Nothing is.

The browser prose won. A Bot holding four Google Drive tools browsed to
drive.google.com, met a sign-in page its container can never satisfy, and
asked its person to sign in to a vendor that person had already connected.
The tools were loaded the whole time.

So the grants are now said out loud, in the Bot's own instructions, and
before the browser prose rather than after it. The text is generated from
the grants rather than written anywhere: switching a connector on, or
granting one more of its tools, changes what the Bot is told on its next
run with nothing to keep in step. A Bot holding nothing is told nothing.

Both paths, because they are built by different functions and the remote
one is where it failed: a framework Bot gets COMPUTER_GUIDANCE as its
entire prompt, so it had the least reason of any to reach for a tool.

Verified against the document rather than the answer. Read a PRD's text
through the API first, took a line out of it — "Kai reached 83%
company-wide" against weekly active users — then asked in the chat. The
Bot answered 83% and named the metric, using search_files, search_files,
read_file_content, and never touching the browser. Two minutes earlier,
on the same trail, a Bot without those grants met the sign-in wall.
The router saw an id, a name and a role description. So it routed on what
somebody wrote a coworker was for, which is not the same as what it can
do.

Asked what was in a Google Drive document, it chose the coworker whose
description says "company knowledge questions" and which held no Drive
grants at all. That coworker browsed to drive.google.com, met a sign-in
page its container can never satisfy, and asked the person to sign in to
an account they had already connected. The coworker that could have
answered was one line further down the same roster.

Candidates now carry the systems they hold tools for, and the prompt says
to prefer one that can reach what the message names. Purpose still comes
first: a specialist with no connectors is still right for a question
about its specialism, and the instruction says so, because the opposite
failure is sending everything to whoever happens to hold a connector.

Asked per request rather than held, so a grant added a minute ago counts.
A store that is slow or unhappy is treated as holding nothing, because a
connector lookup must never turn "who is this for" into an error. A
deployment with no connectors gets the prompt it got before, down to the
instruction line, which is not added when nothing on the roster can reach
anything.

Driven: the same question that went to Knowledge now goes to Risk
Analyst, with no @ mention, and the recorded reason is "Google Drive
access and investigation of company-wide metrics". It then answered from
the document.
@davidmckayv
davidmckayv merged commit e905640 into main Aug 22, 2026
8 checks passed
@davidmckayv
davidmckayv deleted the fix/route-to-a-bot-that-can-reach-it branch August 22, 2026 03:33
davidmckayv added a commit that referenced this pull request Aug 22, 2026
The changelog gains what a person upgrading has to know and the six
defects fixed since the last pass. `AGENT_TOOL_TOKEN` is generated on a
laptop now, and `start.sh` stops skipping services that are already
answering, which costs about five seconds on a run that rebuilds an image
and is why a rotated secret now reaches the things that use it.

The README gains one row. Its settings table already named the two
secrets `start.sh` sets, and this is the third: without it no Bot may call
a tool, and a person setting up would otherwise not learn it exists.
Nothing else there changed — it says how to build and run this, not what
happened to it.

`docs/configuration.md` had `AGENT_TOOL_TOKEN` as "unset", which is no
longer true of the documented path. It now says start.sh generates one,
why the fail-closed default is right for a deployment and wrong for a
laptop, and that it is one of a pair with MANAGED_AGENT_TOKEN pointing the
other way — which is the thing that makes rotating either of them
confusing.

Also removes server/tests/routing.test.ts, an empty file I created by
accident while looking for the real one and committed in #144.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant