The read boundary says when it fires - #188
Merged
Merged
Conversation
Two run_kaish calls through MCP, one served and one refused at the boundary, produced exactly one span and zero log lines outside the allowed set. For a product whose whole claim is read-only containment, the boundary firing is the most interesting event there is, and it left no trace at all — an operator watching a fleet could not tell a boundary that never fired from one that fired a hundred times. We shipped three fixes to exactly that boundary in #184 with no way to see it exercised in the field. Every containment refusal funnels through `containment_error`, so one `warn` there covers the whole class: `resolve_root`, `resolve_attachments`, and `read_contained_file`, reached from five MCP tools and four CLI subcommands alike. A fourth surface added later is reported the moment it uses the shared check. The split between the field and the message is kaibo's existing export policy, not a new one. `outcome` is already on SAFE_ATTRIBUTES, so *that* a refusal happened always exports; the paths live in the message body, which CONTENT_ATTRIBUTES holds back from traces unless the operator opts in. A caller-named path is content by the same rule that keeps `gen_ai.tool.arguments` off the safe list — the comment there says "which names paths" in as many words. The logs signal carries the line whole, which is what it is for. Amy chose this over adding a path to the safe set: it needs no new policy. `run_kaish` additionally opens its span *before* the containment check rather than after, so a refused call closes a `run_kaish` span tagged `outcome = "refused"` instead of producing no span at all. `tool_span.rs` settled the same question for the inner tools — a malformed-args refusal is reported as `outcome = error` rather than never appearing, "the honest reading of did this call work" — and this is the MCP surface catching up to it. The span now brackets the whole call, so its duration includes the check and the worker spawn. The other four handlers refuse before they have a phase to open a span around, so for them the logs signal is the only road. Named in telemetry.rs's module doc rather than left for the next reader to rediscover; giving them handler-level spans is a larger change than this one. Five tests, all run against the pre-fix code and failing there: three refusal surfaces with zero events captured, and the run_kaish pair with no span and no outcome field. The negative control — an allowed path stays quiet — was sabotage-checked by moving the warn to fire on every resolve, and went red. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The cross-family review of the first cut (kaibo cast `crusoe`) found that `run_kaish` recorded `outcome = "refused"` on every `resolve_root` error, not just the containment one. `resolve_root` refuses four ways — no default root, a path that does not resolve, a path that is not a directory, and the boundary check — and only the last is the boundary firing. So a typo'd path made the span claim a boundary hit that the logs, which only `containment_error` writes, knew nothing about. An operator alerting on `refused` would have been counting typos. The fix puts each writer where the knowledge is. `containment_error` records `refused` itself, from the one place that knows the boundary fired; `Span::record` is a no-op for a field the current span never declared, so it reaches exactly the spans built to receive it and costs nothing in the four handlers that resolve before they have a span at all. The handler pre-sets `error` and lets that refinement land after it, which also means a future early return closes `error` rather than closing with no outcome. A seventh test pins it, and it reproduces the reviewed bug when the first cut's recording is restored. Three prose corrections from the same review, all verified against source before acting on them: "the one place kaibo records" understated the span as a second record and is now "logs"; telemetry.rs's "four of the five MCP tools" read as exhaustive when attachment refusals are logs-only roads too; and the funnel doc now says which boundary it means, since a consult attachment outside the session root is a different class and stays quiet. Drive-by in the same file: `resolve_attachments`' doc comment was sitting on `read_contained_file`, fused ahead of that function's own doc and leaving `resolve_attachments` undocumented. Moved to the function it describes. Reviewed-by: kaibo cast `crusoe` (explorer DeepSeek-V4-Flash, synth GLM-5.3) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two
run_kaishcalls through MCP yesterday — one served, one refused at the boundary — produced exactly one span and zero log lines about the path outside the allowed set. kaibo logged 329 info + 12 warn that day and none of them was the refusal.For a product whose whole claim is read-only containment, the boundary firing is the most interesting event there is, and it left no trace at all. An operator watching a fleet could not tell a boundary that never fired from one that fired a hundred times — and #184 shipped three fixes to exactly that boundary with no way to see it exercised in the field.
Observability, not correctness. Nothing here changes what kaibo refuses.
The funnel
Every containment refusal already routes through one function,
Resolver::containment_error.resolve_root,resolve_attachments, andread_contained_fileall call it, and they are reached from five MCP tools and four CLI subcommands. Onewarnthere covers the whole class, and a fourth surface added later is reported the moment it uses the shared check.Where the path goes, and why
The split between the field and the message is kaibo's existing export policy, not a new one:
outcome = "refused"is a field, andoutcomeis already onSAFE_ATTRIBUTES— so that a refusal happened always exports, with no allowlist change.CONTENT_ATTRIBUTESholds back from traces unless the operator setscapture_content.A caller-named path is content by the same rule that keeps
gen_ai.tool.argumentsoff the safe list — the comment there says "which names paths" in as many words. The alternative was adding a path toSAFE_ATTRIBUTES; Amy chose this one, because it needs no new policy. The logs signal carries the line whole, which is what that signal is for.run_kaishopens its span before the checkIt used to build the span after
resolve_root, so a refused call was the one outcome of the tool that produced no span at all.tool_span.rshad already settled the same question for the inner tools — a malformed-args refusal is reported asoutcome = errorrather than never appearing, "the honest reading of did this call work" — and this is the MCP surface catching up. The span now brackets the whole call, so its duration includes the check and the worker spawn; nothing asserted the old bracket.The review found a real bug in the first cut
A cross-family pass (kaibo cast
crusoe— explorer DeepSeek-V4-Flash, synth GLM-5.3) caught that the first cut recordedoutcome = "refused"on everyresolve_rooterror. It refuses four ways — no default root, a path that does not resolve, a path that is not a directory, and the boundary — and only the last is the boundary firing. A typo'd path made the span claim a boundary hit the logs knew nothing about, so an operator alerting onrefusedwould have been counting typos.Fixed by putting each writer where the knowledge is:
containment_errorrecordsrefuseditself;Span::recordis a no-op for a field the current span never declared, so it reaches exactly the spans built to receive it. The handler pre-setserrorand lets that refinement land after it — which also means a future early return closeserrorrather than closing with nothing.Every citation in that review was verified against source before I acted on any of it. Three prose corrections came from the same pass, and one drive-by:
resolve_attachments' doc comment was sitting onread_contained_file, fused ahead of that function's own doc, leavingresolve_attachmentsundocumented.What still doesn't reach traces
The other four handlers refuse before they have a phase to open a span around, so for them the logs signal is the only road. That's named in
telemetry.rs's module doc rather than left for the next reader to rediscover; giving them handler-level spans is a larger change than this one. The sibling refusal for an attachment outside the session root stays quiet on purpose — that file is inside the allowed set, and the refusal is about which project the call mounts, not about the boundary.Tests
Seven, all run against the pre-fix code and failing there: three refusal surfaces (zero events captured), the
run_kaishpair (no span, no outcome field), and the vocabulary test that reproduces the reviewed bug. Two negative controls, both sabotage-checked rather than assumed:error— restored the first cut's recording, went red.New reusable capture helpers in
test_support.rs, riding the existingserialized_captureguard so they inherit the callsite-interest protection that module already documents.Suite: 1348 passed, 0 failed. Clippy clean.
🤖 Generated with Claude Code