Skip to content

feat(receipt): refused (user-denied) tool calls as their own signal - #300

Open
mikehasa wants to merge 1 commit into
mainfrom
feat/refused-tool-events
Open

mikehasa wants to merge 1 commit into
mainfrom
feat/refused-tool-events

Conversation

@mikehasa

Copy link
Copy Markdown
Owner

What

Answers the community question "does your harness log refuse-before-dispatch as its own event, or treat it as if the tool never ran?" — now its own event.

When a Claude Code user declines a tool's permission prompt (refuse-before-dispatch — the tool never runs), the receipt's Actions dimension shows a distinct refused_action_count and an "N actions refused — user denied" gap line, read from the host's own transcript with transcript_scan provenance.

Pipeline

client_usage.py transcript matcher → per-session observation carrier → cli.py scoped-replace emit → work_ledger.py overlay → task_projection.py sum → receipt.py _actions_dimension. A new refused_tool_call_observed event type (deliberately not tool_activity_observed, so the executed-count reducers ignore it and it never triggers the transcript-scan supersession that would drop a Claude session's hook counts).

Honesty guarantees

  • Additive only — never folds into, nor subtracts from, executed tool counts; never touches the evidence/outcome (verified/reported) tiers. It is a capture on the "what happened" axis.
  • No fabricated zero — a client with no clean per-call refusal signal (Codex / OpenCode / dsh, or a Claude session with none) records nothing, not 0.
  • Anchored matcher — user role + a known tool_use id + is_error + the host's canonical decline prefix; an ordinary tool error (is_error alone) or a stray id never counts.
  • Forgery-guarded — a reserved contract key stamped only by the transcript-scan emit path and required by the reducer; a generic record_event / POST /events caller is stripped (strip_refused_tool_call_provenance), exactly like worksets and finding dispositions.

Adversarial review — two findings, both fixed

A 3-lens review of the first draft found two real forgery holes; both are addressed with tests that encode the exact attack:

  1. (blocker) MCP forgery — the reducer trusted the event type with no strip guard, so a single agentacct_record_event with event_type = refused_tool_call_observed and a chosen refused_action_count would forge a "user denied" claim. Fixed: reserved contract key + strip on the generic write lane; the reducer requires the stamp. Test: test_forged_refused_via_record_event_is_stripped_not_trusted.
  2. (major) matcher forgeability + a false docstring — a tool_result's content is tool-authored, so a determined agent could emit the anchor text from a failing tool; the original docstring wrongly claimed the text "cannot be injected." Fixed: the docstring now states this is a best-effort capture, not a trust boundary (it can only inflate an additive count, never touch evidence). The exact host wording/structure should be reconfirmed against a real interactive-mode transcript.

Known limits (please weigh in)

  • Claude Code only today. Other clients expose no clean per-call refusal signal (honest gap, not a fake zero).
  • The transcript matcher's exact host wording/structure is best-effort — this repo's owner runs bypass/auto-approve mode, so there are no live samples on that machine; it is validated against a fixture. Please confirm the anchor plus block structure against a real interactive-mode transcript before relying on it.
  • The additive refused_action_count field is on the /v1 wire; the human disclosure is the Actions gap line, rendered on every surface. The Swift decoder tolerates additive keys, so no golden impact is expected.

Scope / testing

  • 6 source files + tests/test_refused_tool_calls.py (12 tests: builder/reducer, transcript matcher via real discovery, honesty negatives, forgery-is-stripped, trusted-path round-trip). Full suite green: 3025 passed.
  • Part 2 of 2 responding to work-receipt design feedback (part 1 = the rollback-owner + no-observed-change PR).

…ignal

Answer "does the harness log refuse-before-dispatch as its own event?" — yes.
When a Claude Code user declines a tool's permission prompt (the tool never
runs), the receipt's Actions dimension now shows a distinct refused_action_count
and an "N actions refused — user denied" line, read from the host's own
transcript with transcript_scan provenance.

Pipeline: a new refused_tool_call_observed event type (distinct from
tool_activity_observed so the executed reducers ignore it and it never triggers
the transcript-scan supersession that would drop hook counts) -> anchored
transcript matcher (user role + known tool_use id + is_error + host canonical
decline prefix) -> per-session observation carrier -> scoped-replace emit ->
work-ledger overlay -> task projection -> Actions dimension.

Additive only: never folds into or subtracts from executed tool counts, and
never touches the evidence/outcome tiers. Forgery-guarded like worksets/finding
dispositions: only the transcript-scan emit path stamps the reserved contract
key; a generic record_event caller is stripped, so an agent cannot mint a
"user denied" claim. A client with no clean per-call refusal signal records
nothing, never a fabricated zero.

Honest limit: the transcript matcher is best-effort (tool_result content is
tool-authored, so a determined agent could emit the anchor text from a failing
tool) and the exact host wording should be reconfirmed against a real
interactive-mode transcript. Claude Code only today.

Full suite green (3025); new tests/test_refused_tool_calls.py (12) cover the
matcher via real discovery, the honesty negatives, forgery-is-stripped, and the
trusted-path round-trip.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant