Make tabs_list usable on a large backlog, and let a clip name its vault - #14
Merged
Conversation
A 874-tab listing came back as 306 KB in one tool result — past the MCP client's ceiling, so the agent got nothing, and there was no narrower call to fall back to. Roughly a third of it was boilerplate and the rest was just unasked-for. - `query`: case-insensitive AND over whitespace-separated terms, matched against title and URL together. Not a regex — agent-authored regexes backtrack, and substring terms are what triage needs. - `groupBy: "domain"`: counts only, no tabs. The cheap first call that says what a backlog is made of and what to pass as `query` next. - `limit` (default 200) and `sort` (default `recent`), with `matched` and `truncated` so a partial answer can never read as a complete one. - BridgeTab omits every false or unknown field, and `browser` / `connectionId` are hoisted into the top-level `browsers` array unless a listing genuinely merged two browsers. `selectTabs` is shared and runs twice: in the extension so a backlog never crosses the socket whole, and again in Gullet over the merged results, where a per-browser limit would not be the limit the agent asked for. `groupBy` is the exception — Gullet groups the full filtered set, since truncating before grouping would corrupt the counts. Same reconstructed listing: 306 KB → 215 KB from shape alone (still too big, which is the point), 49 KB at the default limit, 0.4 KB grouped. Verified: bun run check.
Follow-up to the tabs_list triage work: the shape changes there cut 306 KB to 215 KB, but 23% of what remained was JSON key names repeated 874 times and another 10% was two fields carrying almost nothing. Measured per field rather than guessed. - `index` is gone. It duplicated array order under `sort: "window"`, meant nothing under the others, and nothing read it — the undo log takes position from the live `browser.tabs.Tab`. - `windowId` hoists to the top level when every tab shares one window, which on a single-window Zen is all of them. Not hoisted across a merged listing: two browsers can each call their window 1. - Titles clip at 120 with a trailing `…`. There is no gentler cap — the mean title in that backlog was ~104 chars — and what a clip loses is the site suffix the URL already gives you. - URLs are trimmed structurally, not clipped: `displayUrl` drops tracking params, `www.`, and the trailing slash, keeping the scheme (copyable) and the fragment (an SPA's whole page identity). `TAB_URL_MAX` is a backstop for data: URIs, not the mechanism. `isTrackingParam` is now exported from normalize.ts so there is one list, not two. Rendering is a separate pass from selection and runs once, in Gullet, at the very end — never in the extension. Gullet re-applies `query` over the merged results, and matching against text that clipping had removed would silently drop the tab the agent asked for. Filters see whole strings; only what reaches the model is trimmed. It also tolerates a tab missing title or url rather than throwing away the listing around it. Same 874 tabs: 363 → 185 bytes/tab, and 36 KB at the default limit (was 306 KB, which failed outright). groupBy stays 0.4 KB. Verified: bun run check.
Caught live against the 874-tab browser that motivated this work:
`tabs_list { query: "x.com", groupBy: "domain" }` answered
`matched: 874, domains: 298` — the entire backlog, byte-identical to the
unfiltered call.
The filter lived inside `selectTabs`, and the grouping branch called
`groupTabsByDomain(merged, ...)` directly, skipping it. Extracted as
`filterTabs`, now called by both paths.
What makes this worth a note: against a current extension the bug was
invisible, because the extension applies `query` before sending and the
grouping saw a pre-filtered set. It only surfaced against a real browser
running 0.2.0, which ignores `query` and hands over everything. The
version-skew tolerance that Gullet's second pass exists to provide is
precisely what kept the bug hidden — a redundant safety pass has to be
tested with the primary pass disabled or it is only ever exercised as a
no-op. Both regression tests do that: the fake browser ignores `query`.
Also: `groupBy` now defaults to `TABS_LIST_DEFAULT_GROUP_LIMIT` (50)
rather than the 200 a tab listing gets. That browser has 298 distinct
domains and everything past roughly the fiftieth is a single tab — 250
rows of noise around the ~20 that describe the backlog. `domains` still
reports the true count.
Verified: bun run check, plus live against the real browser.
Verified live: tabs_list answered `matched: 874, domains: 298`, then `matched: 160, domains: 66` with a disjoint domain set after a workspace switch. `includeHidden: false` changed neither number, and both readings hoisted the same windowId — so tabs in a non-active workspace are absent from `tabs.query` entirely, not returned flagged `hidden`. That is the condition `probeHeuristic` was written to detect; it is simply true on this Zen. Active-workspace scope is fine and is not changed here. The claim was what was wrong: tabs_list told agents `hidden: true` meant "another workspace", so an agent seeing 160 tabs would report them as the user's whole backlog, with `matched` reading as authoritative either way. The tool description and GULLET_INSTRUCTIONS now state the scoping and tell the agent not to quote a total. Also drops `index` from the tool-surface table, which the rendering change removed a commit ago. Left undone deliberately: nothing in a result says *which* workspace it is, so two listings minutes apart are not comparable. Naming one needs an API Zen does not have; surfacing probeHeuristic's verdict on the listing is a wire change, and is noted in docs/BRIDGE.md rather than made. Verified: bun run check, plus live against the real browser.
Gullet read only `tabs` from each browser's reply and let its own `selectTabs` pass derive `matched`. But an extension carrying these changes truncates to `limit` before sending, so the tabs that arrive are not the tabs that matched: 200 of 874 would come back reported as `matched: 200` with no `truncated`. That is the agent's one signal it has not seen everything, destroyed exactly when there is more to see — and it would have read as a complete listing of the whole backlog. Now each browser's reported `matched` is kept, falling back to our own count only when a browser sends none, which is how an older extension identifies itself. Resolved per connection and summed: two attached browsers can be different versions. Invisible to every live call made today, because that browser is 0.2.0 and sends everything unfiltered, so page size and match count are the same number. It would have appeared on first contact with the build that fixes the loopback cost. Both regression tests fail against the previous code. The trace that found it was after something else: whether the `Number.POSITIVE_INFINITY` limit used for `groupBy` survives the wire. It does not need to — it is spent on a `slice` inside the extension and never reaches `TabsListResult`, so `JSON.stringify` never gets to turn it into `null`. Noted at the site, since the next person to put a limit on the wire needs to send a real number. Verified: bun run check.
"Connected on 20317" in the browser and "No browser is connected" from every tool call are both true when two sidecars hold different tokens, and together they read as a broken bridge. Observed live: a session started before a token change held 4589 with the old token, a session started after it could not peer with that hub — a mismatched token must never be handed a proof — so it bound 20317, and the browser attached to whichever it found first. Every part behaved as designed and the pair of facts was still misleading. `Backend.rivalHubs()` re-probes the candidate ports on the no-connection path and the tool error names them, then points at the token as the thing that keeps two sidecars from merging. Probed live rather than read from the election's observations, because a rival can appear long after we settled — which is the case worth catching. Best-effort by construction: a throw inside the diagnosis must not replace the error it explains, and there is a test for that. Also documents what does *not* cause this: `bridgeToken` is minted only by an explicit click in options and `bridgeLastPort` lives in storage.local, so both survive an extension update. Only an uninstall clears them, which is what happened here. The port moving looks like update fragility and is not. Verified: bun run check.
The field was readonly and Generate was the only way to set it, which is wrong for a shared secret: the value may already exist at the other end — a second browser, a machine already configured, a password manager — and the extension was the one place it could not be entered. Now editable. Three things follow from it: - Persisted on `change`, not `input`. Every write of `bridgeToken` is a revocation, since the handshake pins the token it proved and a live socket drops. Saving per keystroke would tear the bridge down once per character being typed or pasted. - Trimmed. A copied secret routinely arrives with a trailing newline, and that produces a token that looks identical to the sidecar's and fails every handshake. - Empty still never persists. It is ambiguous — mid-paste, or selected-and-deleted on the way to typing — and writing it would revoke access for a keystroke rather than a decision. Turning the bridge off is the toggle; replacing the token is Generate or a paste. Also generalises the rival-sidecar note in docs/BRIDGE.md: it described one incident by port number, when the behaviour is about token realms splitting and has nothing to do with which candidates were involved. Verified: bun run check.
Two-vault users stage into an agent-managed vault by default but sometimes know a page belongs in the main one, and moving it afterwards is a step that never needed to happen. tab_clip takes an optional vault; nothing is persisted, so the blast radius of a wrong one is a single note rather than every clip after it — including the popup's, which a settings-writing tool would also have redirected. The result now reports the vault on every clip, override or not: an agent that cannot see where a note went cannot tell the user, and this is the call where that matters. A blank string is a bad-request rather than a fallback, because both readings of it are wrong — Obsidian treats an absent vault as "whichever is open", and substituting settings would report a destination nobody asked for. Path-shaped names are refused through the same vaultWarningFor the options page uses; existence is unverifiable from a WebExtension, so the tool description tells the agent not to guess instead of promising an error that never arrives.
Comments and docs only; no behaviour change.
tabsList carried one comment above two adjacent predicates, and it argued
against the one it sat on: the origin rationale ("targeted, not contributed")
was directly over `const contributors`, which after the last commit feeds only
hoistWindow. Each predicate now states its own reason — window ids collide
across browsers, so a hoist needs one contributor; ids are merged across a
multi-target listing, so every returned tab needs its origin even when one
browser came back empty. Folding the two together is the cleanup that reads as
obvious there and would un-hoist windowId on a two-browser call.
The single-use `needsOrigins` alias is inlined now that the comment is on the
predicate rather than the name.
BRIDGE.md justified protocol 2 with its two examples but never with a rule,
while tabsList deliberately tolerates `matched` and `query` skew a few lines
apart. State the test: skew the receiver can recompute is tolerated, a
confidently wrong result reported as success forces a bump.
BRIDGE.md opens by saying it is not a proposal, but the candidate-port
section was still written before its code and never revised after: automatic
mode is in storage.ts, options/, and the election in backend.ts, while the
doc kept describing what the feature would have to do.
Drop what the code and tests now record, keep what neither does.
The acceptance matrix ("not complete until all of these discriminate")
becomes a Verification note: the unit-testable rows are tests, so all that
survives is the three properties no unit test represents — shared hub across
two browsers, hub loss through peer re-election to rediscovery, exhaustion
healing in place — with event-page suspension, alarm cadence and Gecko's
reconnect delay as the reason they stay live checks.
The migration's four steps are storage.ts, pinned by storage.test.ts; the
paragraph kept is the one nothing else records, that reading a hand-picked
4589 as automatic is deliberate and unrecoverable from the old schema.
The five discovery rules are bridge-client.ts read back a step at a time,
so they collapse to a paragraph holding the constraints that are not
evident from the code: one probe per trigger, hello-ack before caching,
one dial in flight.
Left alone deliberately: the port audit, since repeating it is the cost of
appending a candidate; the election phases, which are a split-brain trap
rather than a description; and the marker-is-routing-evidence bullet, which
appears nowhere else. The blind-dial valve now says why its counter dies
with the page and points at the postmortem instead of only stating the rule.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A
tabs_listof an 874-tab Zen came back at 305,850 characters — past theclient's tool-result ceiling, so the agent received nothing at all. This makes
the listing something an agent can actually work with, and adds a per-call vault
override to
tab_clip.tabs_listShape and selection were both wrong, and only one of them changes the kind of
problem. Hoisting constants and omitting false booleans is a 29% cut (306 KB →
215 KB) — worth having, but 215 KB is still unusable. Filtering is what changes
the shape:
groupBy: "domain"query— case-insensitive AND over whitespace-split terms, matched againsttitle and URL.
limit— defaults to 200 (50 forgroupBy), max 2000. The result carriesmatchedandtruncated, so a truncated answer is always distinguishablefrom a complete one.
sort—recent/oldest/window. Tabs with unknownlastAccessedsort last under both time orders, and two unknowns tie so the browser's own
order survives the stable sort.
groupBy: "domain"— a histogram to orient on before pulling anything.www., andtrailing slashes dropped. Clipping is presentation only; filters still run
against the full strings, since a filter can depend on the tail of a URL.
Selection (
selectTabs/filterTabs,bridge-protocol.ts) is kept apart fromrendering (
renderTabs,tabs-view.ts) — the extension decides which tabs,Gullet decides how they look, and only Gullet imports the renderer.
tab_clipgains an optionalvaultTwo-vault users stage into an agent-managed vault but sometimes know a page
belongs in the main one, and moving it afterwards is a step that never needed to
happen.
Deliberately a per-call parameter rather than a tool that writes
obsidianVault:a settings-writing tool leaves the extension pointed somewhere the user never
chose, silently, for every clip after it — including the popup's. The override
expires by construction, so a wrong vault costs one note. The tool description
says to use it only when the user names a vault.
Every clip now reports the
vaultit filed into, override or not. A blank stringis a
bad-requestrather than a fallback, because both repairs mislead: anabsent
vault=means "whichever vault Obsidian has open", and substitutingsettings would report a destination the caller did not ask for. Path-shaped names
are refused via the same
vaultWarningForthe options page uses. Existence isnot checkable from a WebExtension — see the follow-up below.
Also here
groupBywas ignoringqueryentirely, and Gullet was recomputingmatchedinstead of keeping the browser's — which would have reported
matched: 200of874 with no
truncated, i.e. quietly claiming a truncated listing wascomplete.
urlno longer destroys the listing. Neither engine gives areal URL before navigation commits.
result says which one. Inactive-workspace tabs are absent from
tabs.query,not flagged
hidden— the old heuristic in the docs was simply wrong. Thetool description and
GULLET_INSTRUCTIONSnow say so rather than implying"your tabs".
second sidecar holding a different token is its own realm;
explainNoConnectionnow names the rival hub instead of leaving it as a mystery.
being forced to generate one.
Verification
bun run check— typecheck, format, lint, 386 tests, exit 0. Everything abovewas also exercised live against Zen 1.21.9b with an 874-tab profile, since none
of the browser-API surface is reachable from
bun test.No manifest or permission changes.
Follow-ups (not in this PR)
bunxdistribution.The extension cannot check this, but Gullet runs on the same host as Obsidian
and can, as a soft check.
src/tabs-view.tsships unused into bothdist-*bundles (~5 KB dead code);it moves into
gullet/src/when the sidecar becomes its own package.