Report every injected strategy with graded evidence

[forrestlinfeng]

Summary

Each strategy injected into a dsh turn is now reported to the Hub with graded evidence, along with correction and retention follow-ups and a durable outbox. The old turn-end capture is removed.

• Turn-end report: one per injected asset, through the Proxy's POST /asset/reuse-result. The body is exactly {asset_id, outcome, task_id, reason, evidence}, with evidence = {tier, applied, check?{kind, passed, fail_then_pass}, aborted, stage} (the Hub's strict schema).
• applied: the strategy's numbered steps are compared with the turn's bash commands, touched paths, the code its edits wrote (200,000 chars per turn, in memory only) and the final answer.
  • If there is nothing to compare, the report goes as tier: none with applied: partial, so the Hub adds no soft negative. The reason says applied: unknown.
• Checks: the last test, build, lint or type check run through bash in the turn.
  • Failure is read from isError or from failure output, because dsh's bash reports a failing run piped through tail as success.
  • fail_then_pass compares commands with output pipes (| tail, | head, | grep, …) and redirects removed.
• Follow-ups:
  • The next person prompt that corrects or approves the turn sends one L3 report under <task_id>:correction.
  • The next session in the same project sends one L2 report under <task_id>:retention when the edited files are kept or reverted.
• Outbox: ~/.evolver/state/reports-outbox.json, 8-day TTL, three states:
  • SENT: Proxy recorded it, duplicate_report, or an ineligible receipt (same_owner, daily_cap).
  • REJECTED: terminal: true, or hub 400 / hub 422 from an older Proxy. Never resent, never written to the local ledger.
  • RETRY: hub 404 and everything else.
  • Retried at every turn end, follow-up prompt and session start. Attempts are merged into the file as it stands, so a second dsh process's queued reports survive.
• Diagnostics: evolver_status gains reuse_reports.delivered, carrying the Hub's eligible / counted / weight (plus reason when ineligible), and reuse_reports.rejected with reason / status.
  • The verdict is also stored on the outbox entry as receipt.
  • evolver_asset_reuse_result returns the Proxy body unchanged, so the verdict fields reach the model.
• Model's own report: evolver_asset_reuse_result now carries L5 evidence (applied: unused for mismatched). It no longer stops the automatic report; the Hub keeps whichever report weighs more.
• Prompt redaction: prompts lose keys and tokens before recall sends them. Reasons never name the prompt, a command, code or a diff.

Engine setup (second commit)

Once per dsh process, in the background (triggered by the first agent/session-start), the plugin prepares the engine:

• evolver missing or older than 2.0.39 → npm install -g @evomap/evolver@latest, then evolver lifecycle start (fresh install) or evolver lifecycle restart (upgrade).
• evolver current but GET /proxy/status cannot connect → evolver lifecycle start. A Proxy that answers (even with 401) is left alone.
• The outcome reaches the agent as one [Evolver] notice on the next step. On failure (e.g. EACCES on the global install, or an older evolver still first on PATH) the notice gives the manual steps.
• This replaces the old daily upgrade notice. The first step no longer waits on the evolver --version probe. evolver_status probes the version fresh on each call.

There is no opt-out setting. Tests put fake evolver / npm scripts on PATH, so CI never installs anything.

Behaviour → dsh implementation / gap

Behaviour	dsh implementation	Gap
Turn-end auto report	session/event → turn/end (completed → success; aborted / blocked / error / max-tokens → failed; synthesized interrupted sends nothing and leaves the injection for the next turn). Task id dsh:<session>:<turn> uses dsh's own turn number from the session log.	—
Injection ledger	Written on injection to ~/.evolver/state/injected-<hash>.json with the strategy text. A late recall is reported by the turn that saw it.	—
Edited code / paths / commands	tools/execute wrapper: arguments of every call (command, path/file_path), and the written text of write / edit / str_replace_editor. The file hash before the first edit is taken before next().	—
Check results	tools/execute on bash, result isError + text content. tools/execute only wraps calls that were allowed to run, so a denied check is not counted as failed.	dsh's bash has no background flag in its arguments, so nothing is excluded as a background run.
Final answer	session/event → assistant/message, last non-empty text of the turn.	—
Correction follow-up	agent/pre-step: the first step's messages with source.kind === 'user'.	dsh exposes no prompt origin beyond source.kind, so any user message counts as the person's. The "label the next prompt" state is in memory, so a dsh restart between a turn and its next prompt drops that one correction.
Retention follow-up	agent/session-start (any source). Project = realpath of session.header.cwd, falling back to projectDir / DSH_PROJECT_DIR / cwd. Pending checks are kept in ~/.evolver/state/retention-pending.json.	—
Subagents	Sessions with header.origin === 'subagent' send no reports.	Their checks and edits are keyed to the child session and do not count toward the parent's turn: the plugin has no child→parent mapping.
Outbox / rejection log	Files under ~/.evolver/state (dsh gives plugins no store); rejections in evolver_status.	—
Local ledger	@evomap/evolver-core root events, once per report with evidence (never for tier none or a rejected report).	—

Removed (delete-first)

• Turn-end capture: src/capture.js, src/capture-coordinator.js, src/git.js, and the workspace-id and memory-graph half of src/workspace.js.
  • It posted to /a2a/evolution/record (the old record route; there was no /memory/record call in this repo), which the Hub does not serve.
  • It appended memory-graph rows nothing in the plugin reads.
  • Settings removed with it: gitTimeoutMs, gitMaxBufferBytes, hubTimeoutMs, captureDedupeTtlMs, captureLockStaleMs, captureLockWaitMs, untrackedHashMaxBytes. An old config that still sets them parses and ignores them.
• The session/flush listener existed only for the capture queue.

Kept, still in use:

• projectDir and the project-dir resolution (now src/project-dir.js), used as the project key for retention.
• Edit signal detection (the [Evolution Signal] nudge, minus the "attached to the turn outcome" claim).
• dissatisfaction.js, used by the follow-up label.
• local-ledger.js (root events).
• /evolver-status and /evolver-evolve still read the full engine's memory graph, which that engine writes.

Minimum Evolver

• Recall still needs 2.0.39 (MIN_EVOLVER_VERSION unchanged).
• Evidence, terminal and the verdict fields need the first Evolver release containing EvoMap/evolver-v2-private-dev#1311 (merged to main, not released yet; expected 2.0.43).
• On 2.0.42 or older, the plugin runs unchanged:
  • the Proxy silently drops evidence, so the Hub counts the report as unevidenced (0.1);
  • Hub 4xx comes back as hub 400 / hub 422, which is treated as rejected;
  • no eligible / counted / weight fields are returned.

Tests

npm test: 154 pass.

• New units: applied, checks, verification, reuse (contract field set, delivery verdicts for duplicate / ineligible / terminal / hub 400 / 422 / 404 / 5xx / 429 / 401 / network), report-outbox, retention, feedback-label, injected-assets, turn-outcomes, the report log in diagnostics, and prompt redaction.
• New end-to-end tests: test/reporting.test.js drives the plugin through fake dsh hooks against a local Proxy.
• Real-session regression: test/fixtures/fair-play-session.json comes from a recorded dsh session (home path renamed, non-check output shortened). Replaying it gives:
  • turn 1: L1 / used / test passed / fail_then_pass false (the last check is npm test … | tail -8, while the earlier failures were node --test … | tail);
  • turn 2, answer only: tier none / applied partial / reason applied: unknown;
  • a following complaint: one L3 correction for turn 2.
• Also covered:
  • fail_then_pass across different tail lengths;
  • aborted turns;
  • unused strategies;
  • kept and reverted retention, judged only by a different session in the same project and only once;
  • terminal rejection shown in evolver_status and never resent;
  • 404 retried, with the local ledger written once;
  • the model's L5 report and verdict passthrough;
  • restart durability;
  • late recall;
  • subagent and synthesized turn ends.

Not verified: a live dsh run with a model; behaviour was checked against real dsh session logs and the hook type definitions.

🤖 Generated with Claude Code

[cursor]

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.