CDD: reconcile coherence from telemetry; analyze the response after a failed call

.gitignore

@@ -238,6 +238,7 @@ examples/hivemind_governed/src/hive-r-*.txt
 # (which invokes the examples from the repo root) drops these at the top level
 # rather than beside the script.
 /hivemind-telemetry.jsonl
+examples/hivemind/src/hivemind-telemetry.jsonl
 /hivemind-governed-telemetry.jsonl
 /hivemind-governed-transcript.jsonl
 /hivemind-audit.jsonl

CLAUDE.md

@@ -224,10 +224,11 @@ include/naab/       All headers
   - `validation_outcome` (S22): folds external ground-truth pass/fail (pytest, `orchestra.enforce_convergence`) into coherence — closes the "CDD is blind to test failures" gap where syntactically valid code that fails tests scored identical coherence to passing code. Weight 0.15. Fed by `agent.record_validation(handle, passed [, detail [, evidence_count]])` (agent module — reuses `validateHandle` anti-forge; called by the orchestration script as operator ground-truth, not by the agent). **`evidence_count`** (optional int, -1/absent = not measured) is how many checks actually backed the outcome (test count, assertion count). A **pass** whose count falls below the last APPLIED count is a *shrink* and is scored AS a failure: a pytest exit code cannot separate "fixed the defect" from "deleted the failing test", because a suite with no tests left passes. The shrink also forfeits the fail→pass recovery credit and does NOT consume the outstanding failure, so a later pass that does not shed evidence still earns it — consuming it would let one shrink launder the failure away. Compared against the previous count, not a high-water mark, so erosion is charged on the transition and a one-off legitimate test consolidation does not pay forever. `DriftState.last_evidence_count` is **preserved across `agent.reset()`** (like the signal override mask): reset clears behavioural history, but scripts call it when coherence drops below a floor, so zeroing the baseline would make the signal self-cancelling exactly when it first works. Inert unless the caller reports a count — no config key, no ratchet entry, no behaviour change for existing callers. Surfaced as `validation_credit_withheld=evidence_shrank` in `CDD_TURN` `penalties_detail` and `evidence_count` in `VALIDATION_RECORDED`. Tests: `test_validation_signal.sh` Groups L and M. Note CDD sees erosion ONLY through this channel — no signal reads it out of the response stream (`test_developer_blindspot.sh` DB-04). The optional `detail` string (e.g. pytest failure lines) is keyword-extracted on a fail and stored in `DriftState.validation_failure_keywords` — it grounds the priority-0 `validation` step-up challenge type (cleared on a recorded pass). Result latches on `DriftState.has_validation_result`; the **next** `recordTurn` consumes it (one-turn lag — validation runs between sends) and clears it. **Uses flat `base_penalty`, NOT `adaptive_penalty`** — a failing test is objective, so it must never be baseline-absorbed into "failure is normal." **Recovery on fail→pass transition only**: consuming a pass after a consumed failure credits `context_drift.validation_recovery_amount` (default 0.075 = half the S22 weight, so oscillating fail/pass stays net-negative; credits ≤ failures — pass-spam cannot pump coherence; surfaced as `validation_recovery=+…` in `CDD_TURN` `penalties_detail`). Ratchet: raising the recovery amount mid-run is a loosening violation. Default-on but zero-cost until a result is fed. Telemetry: `VALIDATION_RECORDED` on the write (carries `applied`: false when an unconsumed FAILURE was preserved rather than replaced by this pass — the latch is one slot and recordTurn only runs on an AGENT_RESPONSE, so a pass landing on an unscored failure used to erase it outright, leaving no penalty and no fired count; live run 15 recorded 4 failures and 4 passes and scored none); fires in `CDD_TURN` `signals_detail` and `snapshotState` `counts["validation_failure"]`. `recordValidationOutcome()` mirrors the `recordToolOutcome()` chain.
   - `response_degenerate` (S23): fires when an AGENT_RESPONSE has `output_tokens` below `context_drift.response_min_output_tokens` (default 8; events with unreported token counts are ignored) — closes the gap where a 1-token reviewer "APPROVED" sailed through every gate. Weight 0.08. **Default OFF** (unlike S8–S22) — terse-by-design agents (single-word verdict judges) would fire constantly; scored through `adaptive_penalty` so under baselining structural terseness self-absorbs while an agent whose baseline is substantive output pays when it collapses. Enable via `context_drift.signals.response_degenerate: true`.
   - `coherence_velocity` (S6) is DETECTION-ONLY: coherence changes only via penalties/recovery, so velocity is exactly last turn's net penalty — a direct penalty would double-count evidence and cascade (each S6 penalty feeds the next velocity reading). It still fires (telemetry, dashboard) and reaches pressure escalation via the circuit breaker's `coherence_acceleration` factor (Factor 7 — it writes `coherence_acceleration`), NOT via `signal_density`; it never subtracts coherence and `weights.coherence_velocity` is inert. Note: `signal_density` is fed only by `signals_fired_this_turn`, which is incremented behind the same `if (p > 0.0)` gate as the coherence penalty — so a signal that fires but is **absorbed by the adaptive baseline** (penalty 0) contributes zero to `signal_density`. Detection-only/absorbed firings do not pump pressure through signal_density; long-session escalation comes from signal-independent composite factors (conversation depth, temporal decay) plus S6's acceleration.
-  - CDD_TURN telemetry carries `analyzed` ("true"/"false"): false = interval-skipped turn (`check_interval_turns`) where recordTurn did NOT run — the coherence/signals_detail/penalties_detail fields are STALE state re-shown from the last analyzed check. Filter on `analyzed:"true"` for per-turn forensics (the stale display misled two forensic passes before this field existed).
+  - CDD_TURN telemetry carries `analyzed` ("true"/"false"): false = interval-skipped turn (`check_interval_turns`) where recordTurn did NOT run — the coherence/signals_detail/penalties_detail fields are STALE state re-shown from the last analyzed check. Filter on `analyzed:"true"` for per-turn forensics (the stale display misled two forensic passes before this field existed). The label is read from `DriftState.analysis_seq` (incremented only past the interval check), compared before and after THIS send's `checkContextDrift()` — it used to compare `last_checked_turn == event_turn`, which read "true" for a turn an EARLIER call had analyzed (see the infrastructure-error bullet below).
+  - **Every analyzed row reconciles from telemetry**: `coherence = previous analyzed coherence - penalties + validation_recovery + sum(coherence_adjustments)`. `coherence_adjustments` (separate CDD_TURN field) carries `temporal_decay=-`, `natural_healing=+` (`coherence_natural_healing / (1 + signals_fired)`, received not granted), `floor_absorbed=+` (penalty beyond 0 the clamp discarded) and `recovery=+` (`recoverCoherence()` from a passed step-up or a pipeline-stage failure — lands BETWEEN analyzed turns, so it and decay accumulate in `DriftState.pending_*` and report on the next analyzed row). It is deliberately NOT folded into `penalties_detail`: `living-script_v3/report.py`, its gate registry and `test_signal_contract.sh` read a non-empty `penalties_detail` as "a signal paid", and healing lands on nearly every turn after damage. Before this, healing was reported nowhere, so listed penalties never matched the drop — the repo-sentinel dogfood (F-008) reported CDD's arithmetic as broken when it was exact to 4 decimals once healing was added back. `validation_recovery` now reports the credit RECEIVED (it is capped at 1.0). Test: `tests/governance_v4/test_coherence_reconcile.sh` (RC-02 proves every adjustment kind was exercised, RC-03 that the field is load-bearing).
   - CDD analyzes from turn 0: `DriftState.last_checked_turn` initializes to -1 (agent event turns start at 0; a 0 default made `0 - 0 < check_interval_turns` silently skip the first send of every handle — no fingerprint, no entity baseline, no recall check for the first response).
   - All CDD signals default enabled (S8-S22). Surfaced in: environment dict, SEMANTIC_TURN telemetry, transcript CDD section, response `semantic` section. S19 additionally produces `RECONCILIATION_TURN` telemetry and dashboard line. Environment dict includes `claim_mismatch_count`, `claim_accuracy`, `prompt_compliance_count`. Can be individually disabled via `context_drift.signals.<name>: false` in govern.json (globally — mid-run disable is a ratchet violation) or per-agent via the agents-block `context_drift_signals` map
-- **Infrastructure error classification**: API errors passed to CDD are prefixed `"infrastructure:"` (3 call sites in `agent_impl.cpp`). When `signals.exclude_infrastructure_errors` is true (default), these errors bypass the `repeated_failures` signal entirely. Prevents the "challenge death spiral" where API 500s cascade into coherence drop → governance escalation → step-up challenge → challenge also fails → all sends blocked. Only affects CDD error deque; does NOT change user-facing error messages, telemetry, or retry logic
+- **Infrastructure error classification**: API errors passed to CDD are prefixed `"infrastructure:"` (3 call sites in `agent_impl.cpp`). When `signals.exclude_infrastructure_errors` is true (default), these errors bypass the `repeated_failures` signal entirely. Prevents the "challenge death spiral" where API 500s cascade into coherence drop → governance escalation → step-up challenge → challenge also fails → all sends blocked. Only affects CDD error deque; does NOT change user-facing error messages, telemetry, or retry logic. **A retry-exhausted failure no longer takes the turn's analysis slot.** `agentSend` reports it to `checkContextDrift()` at the SAME turn number the next real response carries (a failed call does not advance the turn), so the failure's analysis set `last_checked_turn` and the next response hit the interval check and was never scored by any of the 23 signals — while its CDD_TURN read `analyzed:"true"`. After any failed call (a 429 is the common one), one response escaped CDD. Now, with `exclude_infrastructure_errors` (default) the infrastructure error returns before any analysis or event-feed watermark movement — it feeds no signal, so there is nothing to analyse; with it off the analysis still runs (it feeds `repeated_failures`) and the slot is handed back afterwards. Test: `test_coherence_reconcile.sh` IA-01/02 (IA-03 is the control that the probe response fires S21 at all)
 - **Health warning recovery awareness**: `checkGovernanceHealth()` suppresses "perfect coherence" warnings during adaptive baseline window and for 3 turns after coherence recovery (recovery resets to 1.0, which is expected behavior not a detection bypass).
 - **AGENT_RESPONSE telemetry enrichment**: now includes `content_hash` (SHA-256 of first 500 chars) and `content_length` for post-hoc audit.
 - **Truncation tracking**: AgentTracker counts truncated responses; emits advisory when majority (>50%) of responses are truncated. Exposed to scripts as `truncation_count` in `agent.usage()` and `agent.environment()` `state`.