Skip to content

fix(intent): preserve admitted subagent handoff action evidence - #4836

Merged
ScriptedAlchemy merged 4 commits into
masterfrom
fix/4540-observed-subagent-handoff
Nov 9, 2026
Merged

ScriptedAlchemy merged 4 commits into
masterfrom
fix/4540-observed-subagent-handoff

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

Focused observed-action repair — Refs #4540 (READY FOR REVIEW; NEVER MERGE)

Owner-requested review-state exception: PR #4836 is ready for review at the owner's explicit request. This supersedes the draft-only instruction for this PR only. Do not convert it back to draft solely because it includes decision-model work. The prohibition on merging or auto-merging remains unchanged, and readiness does not establish passing tests or complete issue acceptance.

The intent pass already separates user prompt inference from admitted assistant tool activity. However, tool_call_facts parsed an assistant tool's subagent_type but discarded its prompt. The existing observed_action_facts classifier recognizes a real Task/Agent handoff only when that invocation's prompt contains handoff/context evidence. That predicate was therefore unreachable from the canonical session-window intent path even for a complete, authenticated assistant Task call.

This changes the existing adapter to forward the JSON prompt from admitted assistant tool calls only. It does not scrape tool output, interpret the user's prompt as a recorded action, infer anything from invalid/truncated JSON, create an additional classifier/transport, or alter model thresholds, authorization, delivery, dedupe or receipts.

Falsifiable regression

New admitted_subagent_prompt_is_observed_but_forged_or_partial_calls_are_not uses the real SessionWindowV1 and window_intent_signals production path:

  • An assistant Task invocation with a complete implementation handoff prompt must produce SubagentStartContext as ActionFact, even if the configured semantic question receives a usable negative 0.1 answer.
  • Identical tool-call metadata in a Tool result must not qualify.
  • An incomplete JSON argument excerpt must not invent a handoff.
  • An unrelated Bash tool with the same prompt-shaped JSON must not qualify.

The original adapter fails the first assertion; the negative cases protect the source identity boundary. No existing assertions are removed or weakened. One source/test file changed (84 added lines).

Evidence and outstanding acceptance

  • Root AGENTS.md and master source read. Exact base c231fb7219f8fc2f334b911ada656bdb2c6e13a4. Independently checked the one-file GitHub commit diff and verified a one-parent, expected-old-head force=false branch fast-forward.
  • Four standalone Node checks verify the test fixture's whole/partial JSON and subagent-kind preconditions. These are not execution of the Rust production path.
  • On this commit the native macOS Bazel agent-hosts suite passed 574 cases, including the admitted-assistant handoff, forged tool-result, malformed JSON, and unrelated-tool cases. The suite skipped its one opt-in live-provider evaluation. The agent-hosts Rust formatter and both the production and test Clippy targets passed. Hosted full Linux and native macOS run.
  • Still missing from explore(intent): separate observed actions from keyword inference before expanding Kev routing #4540: independently labelled current-runtime real-session holdout cases, keyword-only/hybrid/model-first disagreement and adoption measurement, genuine model inference resource costs, and final keep/replace/defer + host hint-delivery deletion decision. Preserve no-engine fallback, genuine tool facts and typed authorization. This is Refs #4540, not complete acceptance; leave the issue open.

At 362ff7f1e6 the one-file handoff patch exactly matches 55695a9a16. The branch now uses the hosted runner setup prerequisite. Before this change, macOS lost public DNS after enrollment, and the Linux fleet and Dashboard both stopped on the launcher HTTP 504 before testing. Hosted Linux and native macOS verification on the stacked commit.

The hosted native macOS job passed. On the dependent #4836 head the runner log shows the Tailscale DNS override, restoration of Ethernet settings, a fresh download of Bazel 9.3.2 from releases.bazel.build, and 574 passing agent-hosts tests with one opt-in evaluation ignored.

Kev/decision-model merge hold: this PR must NEVER be merged or auto-merged under the current authorization. Only PR #4836's draft-only restriction has been lifted. Mixed #4776 and other decision-model PRs remain draft/never merge; beta.99 #4763 is read-only. No release, tag, production, infra, credentials, model run or external coding agents.

Refs #4540. Project an admitted assistant Task prompt into the existing
observed action classifier. Keep tool-result claims and malformed argument
excerpts excluded; verify negative semantic answers cannot erase genuine
observed handoff activity.
@changeset-bot

changeset-bot Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 9dc64ee

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ScriptedAlchemy
ScriptedAlchemy marked this pull request as ready for review November 9, 2026 05:25

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

@ScriptedAlchemy
ScriptedAlchemy changed the base branch from master to codex/preserve-macos-runner-dns November 9, 2026 06:30
@ScriptedAlchemy
ScriptedAlchemy changed the base branch from codex/preserve-macos-runner-dns to master November 9, 2026 09:08
@ScriptedAlchemy
ScriptedAlchemy merged commit ea17e50 into master Nov 9, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant