Skip to content

fix(goal): keep the /goal verifier offline instead of aborting on web tools - #467

Closed
amszuidas wants to merge 1 commit into
mainfrom
fix/goal-verifier-network-denial
Closed

amszuidas wants to merge 1 commit into
mainfrom
fix/goal-verifier-network-denial

Conversation

@amszuidas

@amszuidas amszuidas commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Change

Fixes #419.

A /goal run whose verifier child called web_fetch or web_search once was stopped and surfaced as paused(verifier_capability), even though the child still held read, grep, glob and bash and could verify from local evidence.

Root cause:

  • The Goal verifier child is an ordinary verifier delegation. The verifier role asset grants web_fetch and features.webSearch, and the canonical read-only ceiling removes neither, so the model was offered both tools (native web_fetch, and web_search from the builtin Matrix MCP server).
  • GoalVerifierChildCoordinator.beforeTool latched a capability_violation on the first refused network call, and the next beforeLlm turned the latch into an abort. The "blocked result goes back to the model" recovery path was unreachable.

Fix:

  • Withhold web_fetch and web_search from the tool catalog of any Turn whose intent is the goal-verifier-readonly profile. This applies to native tools and to MCP tools, both inline and deferred (tool_search/mcp_invoke), and to plugin capability bindings.
  • Keep the runtime guard as a fallback, counted by child turns rather than calls (parallel calls in one model response count once). A refused network call returns an instruction that verification runs offline: use local evidence, and judge PARTIAL if the evidence is missing locally. The run is stopped as capability_violation only when network tools are called in 2 consecutive child turns, or in 3 child turns overall.
  • Any other tool outside the canonical read-only ceiling (write, edit, todowrite, delegation, memory, computer use) still latches capability_violation on its first call.
  • The verifier reminder now states that the run is offline and that missing local evidence means PARTIAL.
  • route_unavailable and child_budget_exhausted behavior is unchanged.

Credit: builds on the reporter's patch (Hylouis233/minimax-code@69810ab), credited with a Co-authored-by trailer. Its unrelated goalVerifierRunId formatting change was dropped.

  • PR labels: bug, cli

Validation

  • Checks run and results: pnpm verify (full profile) on cfff8f9 with a clean tracked tree, darwin, Node 24.16.0: PASS (15 gates). capability suite 218 files, 5447 passed / 17 skipped; status-contract 9; policy 142; sandbox 48. test:windows skipped (platform), test:release-package skipped (requires an npm release archive).
  • New test file packages/local-runtime-v2/test/unit/agent/goal-verifier-readonly-guard.test.ts (registered in test/vitest-suites.json), 21 tests. It covers the turn-based fallback, non-network violations, unchanged route/budget aborts, the reminder, catalog withholding for native and builtin Matrix MCP web_search (inline and deferred), and that the verifier task-child prompt has no web-search guidance under the tui and desktop profiles. With the source changes reverted, the first 17 tests of this file ran against main: 13 failed; the 4 passing ones are intended no-regression checks.
  • eslint (repository config) on all changed .ts files: 0 errors. The remaining prettier warnings are on lines this change does not touch and are also reported for the pre-change versions of those lines.
  • Performance: not run.
  • NOT RUN, platform limitations and live-service boundaries: no live end-to-end /goal run against a real model; offline unit tests only. Windows and Linux CI not run locally. Network access through bash (for example curl) is out of scope and remains governed by the permission policy. The legacy prompt path without a prompt profile still renders the features.webSearch "Factual Freshness And Search" section for the verifier; the product always renders task children with the tui or desktop V2 profile, where that section is absent. A per-turn feature toggle would require threading the Goal verifier origin into task binding capture and is out of scope.

Publication and contribution checks

  • I have permission to contribute these changes under the existing licenses applicable to the changed files/packages; imported material and its provenance are identified and existing notices are preserved.
  • No credentials, account data, real user content, internal source history or private review material is included.
  • Added/removed source files were reviewed before regenerating release/public-source.json; new tests are declared in test/vitest-suites.json where applicable.
  • Shared English/Chinese documentation and capability/verification records are updated where applicable. Mock/offline results are not described as live-service acceptance.

Maintainer handoff

Publication scope or license changes (if any): none.

Shared-source port: pending.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…etwork call

A Goal verifier child that called `web_fetch` or `web_search` once had its
whole run stopped. `beforeTool` latched a `capability_violation` on the first
refused call and the next `beforeLlm` turned that latch into an abort, so the
Goal surfaced as `paused(verifier_capability)` even though the child still
held `read`, `grep`, `glob` and `bash` to reach the same evidence locally.

The child is an ordinary `verifier` delegation, and that role grants
`web_fetch` and `features.webSearch`; the canonical read-only ceiling does not
remove either. The model was therefore offered tools that the guard would
then treat as a violation.

- Withhold `web_fetch` and `web_search` from the tool catalog of a Turn whose
  intent is the readonly Goal verifier profile, so the child is no longer
  offered them.
- Keep the runtime guard as a fallback, but only for network tools: a refused
  call returns an offline instruction (use local evidence, judge PARTIAL when
  it is missing) and the run continues. It stops as a capability violation
  only when network tools are called in two consecutive child turns, or in
  three turns overall. Parallel calls from one model response count once.
- Treat any other tool outside the read-only ceiling (write, edit, todowrite,
  delegation, memory, computer use) as a violation on its first call.
- State in the verifier reminder that the run is offline.

`route_unavailable` and `child_budget_exhausted` are unchanged.

Fixes #419

Co-authored-by: Hylouis233 <Hylouis233@users.noreply.github.com>
@amszuidas amszuidas added bug Something isn't working cli Standalone mcode: TUI, headless, ACP and source builds/tooling labels Oct 9, 2026 — with Cursor
@amszuidas amszuidas closed this Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working cli Standalone mcode: TUI, headless, ACP and source builds/tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: /goal verifier aborts the whole run on a single denied web_fetch/web_search call (paused(verifier_capability))

1 participant