Skip to content

DaveHarness approval preflight; wire file.edit - #51

Merged
DaveHomeAssist merged 2 commits into
mainfrom
claude/tool-preflight
Oct 2, 2026
Merged

DaveHomeAssist merged 2 commits into
mainfrom
claude/tool-preflight

Conversation

@DaveHomeAssist

Copy link
Copy Markdown
Owner

What

An approval-required ToolDefinition can now declare an optional preflight:

ToolDefinition(..., approval_required=True, preflight=check, preflight_version="")
def check(arguments: dict, context: ExecutionContext) -> str | None

DaveHarness runs it after schema validation and canonicalization, before the approval pause, on both approval paths (lifecycle state_engine._drain_calls and the legacy engine._process_calls). Contract: docs/DAVEHARNESS_PREFLIGHT.md.

  • Refusal: a nonempty string → an ordinary tool error whose handler never ran: status: "error", termination: "denied", empty result, the refusal as error. No pending call or nonce. It counts as one tool call and one error, the call ID enters the executed ledger, and the remaining calls in the same step continue. Events: policy_decision(pause, approval_required) + tool_result(error, preflight_refused), with no approval_required event.
  • Why denied, not error: DaveHarness already uses denied for calls refused before their handler runs (schema, policy). error would claim the handler ran and failed.
  • Fails open to approval: None, any non-string, "", an exception, a refusal larger than the tool output budget, or missing the tool deadline (min of timeout_seconds and the run deadline) → the call pauses exactly as before. A preflight can spare an approval but never authorize an effect, and the handler still checks everything after approval.
  • Isolation: the preflight gets a deep copy of the canonical arguments and runs in a worker thread with the caller's context variables (so HOST_RUN_CONTEXT is visible). It is never run for pre-approved calls, on /tools/execute, or again after approval.
  • Registration: rejects a preflight on a tool without approval_required, a coroutine, a callable that can't take (arguments, context), and preflight_version without a preflight.

Fingerprints

The preflight's provenance (module, qualname, code digest, stable state; preflight_version for opaque state) is added to the fingerprint payload only when set. Every definition without a preflight keeps its exact fingerprint; test_a_definition_without_a_preflight_keeps_its_fingerprint recomputes the old algorithm to prove it. Qualified catalog entries in tests/fixtures/davellm/tool_catalog.json are byte-identical. file.edit gains preflight/preflight_version, and the new extended_preflight_code pins its placement and source, like a handler.

file.edit

preflight_file_edit (app.py, below every handler) → davellm_edit.check_edit, which shares the handler's pre-write checks via the extracted _request/_plan: path admission, file type, hard links, UTF-8, old_text count (CRLF-aware), no-op, and size. It returns the handler's own message. The approved edit still runs every check again; the file can change after approval.

It doesn't expose new information: within the root, the extended read tools behind the same flag already give the model this without approval, and every refusal outside the root is the same fixed PathNotAllowed message.

Notion tools (PR #48)

Not wired: #48 is still an open draft. Once it merges, the hook takes the Notion checks directly: the per-run ledger is reachable from the preflight through HOST_RUN_CONTEXT.

Needs a decision: H9 edit cases

edit_outside_root_approved is now refused before the operator is asked, so the runner records 0 approvals instead of 1. Status, tool sequence, task pass and 0 unauthorized effects are unchanged. The runner unit test expectation is updated. The corpus, runner, and thresholds are not changed. "Approval never extends the root" is still covered offline by the new test_a_path_moved_outside_the_root_after_the_request_still_writes_nothing. With real models, a wrong old_text now gets an immediate retryable refusal instead of an approval card, so edit-case results from before this change (2026-09-25 run) aren't directly comparable. This is noted in docs/DAVEHARNESS_H9_LIVE_EVALUATION.md. Whether to rework that case is your call.

Other changes

  • tests/fixtures/daveharness/public_api.json: ToolDefinition gains the two optional fields (deliberate snapshot update).
  • daveharness.events: new reason code preflight_refused.
  • Capabilities manifest: each tool records preflight; regenerated with scripts/generate_capabilities_manifest.py.
  • file.edit lifecycle tests now hold one event loop (with client), the repo's pattern for runs that do threaded work; the pause path now does real I/O in a thread.
  • DaveHarness version left at 1.0.0-rc.1 (additive optional field).

Checks (local, Python 3.14.3)

All required checks from CLAUDE.md pass: py_compile, compileall, mypy daveharness (strict, clean), pytest -q 789 passed, 1 skipped, node --check, node --test (4/4), bash -n, npm ci, npm ls, npm audit (0 vulnerabilities), git diff --check, and generate_capabilities_manifest.py --check.

An approval-required ToolDefinition may now set preflight(arguments, context)
-> str | None. DaveHarness runs it after schema validation and before the
approval pause, on both the lifecycle (state_engine) and legacy executor
paths. A nonempty string refuses the call as an ordinary tool error whose
handler never ran: status "error", termination "denied", no pending call or
nonce, one tool call and one error, call ID in the executed ledger, events
policy_decision(pause) + tool_result(error, preflight_refused). Anything
else, an exception, an oversize refusal, or the tool deadline pauses as
before, so a preflight spares an approval but never authorizes an effect.

The preflight's provenance (and preflight_version for opaque state) joins the
fingerprint only when set, so definitions without one keep their exact
fingerprints. Registration rejects a preflight on a tool that needs no
approval, a coroutine, a callable that cannot take (arguments, context), and a
version without a preflight.

file.edit uses preflight_file_edit -> davellm_edit.check_edit, which shares
the handler's pre-write checks (admission, file type, links, UTF-8, old_text
count, size) and returns the handler's own message. The approved edit still
checks everything again.

Catalog fixture pins the preflight like a handler (extended_preflight_code);
qualified entries are unchanged. Public API snapshot, capabilities manifest,
tool catalog, H2/H9 docs, README and CLAUDE.md updated. The H9 runner test now
expects edit_outside_root_approved to end before approval; the corpus, runner
and thresholds are unchanged.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

…red planning

Keep existing Notion handler provenance, regenerate capabilities, and verify overlap refusal before approval and again at execution.
@DaveHomeAssist
DaveHomeAssist merged commit 6e0a6cc into main Oct 2, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant