[VerifiedOrganic]

Summary

Blind held-out-4 continuity corpus (frozen at 7a5ea85, manifest sha256 4273e4cb71647e36a18a74ac68c717c91a37ea0492678197a6517b3a81efae0e) merged with main at 2212d754, plus the one-time measurement of the before binary, the merged-tree binary with the semantic lane off, and the merged-tree binary with the semantic lane on.

• The frozen corpus, its plants, and its controls are unchanged. Both corpus hashes were recomputed from the frozen module and match the freeze record.
• Results are reported twice: as-frozen (all 56 plants) and corrected (55 valid plants) after one documented post-freeze ground-truth erratum.
• No detector or lane code changes.
• The one merge conflict was in evals/continuity_recall.py; resolution keeps the frozen held-out-4 subcommand/self-test and main's semantic dispatch.
• evals/semantic_eval.py gains held-out-4 set registration (seeding, scoring, warning precision) and reporting-only per-type plus warning TP/FP counts. Precision semantics are unchanged.

Measurement setup

Config	Binary	Semantic	Judge
a	before binary, sha256 3ee27fb8…	off	—
b	merged-tree release build, sha256 196a9990…	off	—
c	same merged-tree binary	on (then cache-only shadow)	ollama:gemma4:31b, temperature 0, seed 7

Corpus: 2 manuscripts × 16 chapters, 56 frozen plants (8 per type).

Ground-truth corrections

One frozen plant is mislabeled: vesta.dead_character_acting.nadi_ch8 is framed by Chapter 8 as an old voicemail recording made before the character's death, inside a scene about replaying old messages. A recorded voice is compatible with the death and is not a post-mortem action, so scoring it as a required dead_character_acting detection penalizes a correct detector.

The frozen corpus bytes and manifest hash are unchanged. The correction is a separate scoring overlay:

Item	Value
Frozen manifest sha256	4273e4cb71647e36a18a74ac68c717c91a37ea0492678197a6517b3a81efae0e
Frozen module sha256	3d025fb94465b39a51828dc7870780065857abe5687ab74dbee66ab7b6a1817f
Correction file	evals/continuity_held_out_4_errata.json
Correction sha256	3129737b925ce3faebf5069798e5c80cd0d2ef1162d4a81aa1cb4363a94a6411
Excluded plant	vesta.dead_character_acting.nadi_ch8 (dead_character_acting, vesta_station)
Detected by	a deterministic, b deterministic, c deterministic, c semantic-on

Because the excluded plant was detected in every configuration, its exclusion removes one detected and one planted item from each total; the semantic-on lift is unchanged.

Totals, as-frozen vs corrected

Aggregate	a	b	c deterministic	c semantic-on
As-frozen (56 plants)	30/56 (53.57%)	30/56 (53.57%)	30/56 (53.57%)	31/56 (55.36%)
Corrected (55 plants)	29/55 (52.73%)	29/55 (52.73%)	29/55 (52.73%)	30/55 (54.55%)

Semantic-on minus deterministic-after lift: +1 localized, +1 type-present in both views.

Per-type localized, as-frozen (56 plants)

Type-present is identical to localized for this corpus because score_manuscript localizes before counting; no broad-check-only credit exists.

Type	a	b	c deterministic	c semantic-on	lift vs b
character_attribute	3/8	3/8	3/8	3/8	+0
dead_character_acting	7/8	7/8	7/8	7/8	+0
timeline_chronology_rewind	4/8	4/8	4/8	4/8	+0
knowledge_leak_before_learned	7/8	7/8	7/8	7/8	+0
world_rule_cost_violation	6/8	6/8	6/8	7/8	+1
quantity_affordability	2/8	2/8	2/8	2/8	+0
secret_leak	1/8	1/8	1/8	1/8	+0

Per-type localized, corrected (55 plants)

Type	a	b	c deterministic	c semantic-on	lift vs b
character_attribute	3/8	3/8	3/8	3/8	+0
dead_character_acting	6/7	6/7	6/7	6/7	+0
timeline_chronology_rewind	4/8	4/8	4/8	4/8	+0
knowledge_leak_before_learned	7/8	7/8	7/8	7/8	+0
world_rule_cost_violation	6/8	6/8	6/8	7/8	+1
quantity_affordability	2/8	2/8	2/8	2/8	+0
secret_leak	1/8	1/8	1/8	1/8	+0

Semantic metrics

The erratum changes only the dead_character_acting denominator and one detected item; it does not change the semantic warning set.

Metric	On	Shadow
Semantic warning precision	0.50 (1 TP / 1 FP)	n/a (no emitted warnings)
Control warnings/errors by check type	none	none
Evidence-anchor validity	1.00	1.00
Repair rate per call	0.0779 (6/77)	0.50 (3/6)
Model calls / candidate scenes	77 / 64	6 / 64
Calls per scene	1.2031	0.0938
p95 prompt tokens	3282	3243
p95 output tokens	952	925
p95 latency	2430 ms	2422 ms
Tokens per scene	3552.06	287.94
Tokens per chapter	7104.12	575.88
Cache hit rate	0.00	0.9531
Shadow/on parity	ok (all 4 project checks match)	—

Shadow/on would-be warning and info counts match for every project. The planted summary_matches differences are verifier discard bookkeeping only, which the evaluator reports for diagnosis and does not gate.

Notes

• The native ollama provider name loaded in config but did not resolve a model route (route_unavailable), so the successful measurement used the Ollama Cloud OpenAI-compatible chat endpoint. Two earlier semantic attempts made zero judge calls (first rating_denied, then route_unavailable) and are excluded; only the successful run is reported.
• The successful semantic run recorded 3 unknown_candidate errors in each mode.
• Raw reports stayed outside the repository. The committed results file contains aggregates, counts, and hashes only; no plant text.

Tests

• python3 evals/continuity_recall.py self-test (includes the held-out-4 corpus self-test and the semantic self-test)
• python3 evals/continuity_recall.py held-out-4 for configs a, b, and c
• cargo build --release -p spindle-mcp from the merged tree
• CI on this PR

[VerifiedOrganic]

Changes requested: one held-out ground-truth label treats recorded speech as post-mortem action, which undermines the recall denominator and interpretation of the measurement. Reviewed the diff against origin/main and inspected seeding/scoring and result aggregates. No project programs or tests were executed, per the review constraints.

[VerifiedOrganic]

Approved / ready. The prior voicemail ground-truth finding is addressed through a documented post-freeze exclusion: the corpus bytes and module hash are unchanged, deterministic reports are recounted, semantic scoring filters the excluded plant before recall and warning matching, and recorded aggregates now consistently use 55 scored plants (29 deterministic detections and 30 semantic-on detections). Reviewed the full diff against origin/main and the changes since the prior review; no further required findings. Optional: add a small self-test for the errata recount/filter behavior to guard against future aggregate drift. Static review only; no project programs or tests were executed under the review constraints.

[VerifiedOrganic]

Approved / ready. Reviewed the PR against origin/main and the update since 8729fd8. This revision changes only the errata metadata and recorded results: as_frozen preserves the original 56-plant measurements, corrected clearly identifies the primary 55-plant aggregates, and both retain the +1 semantic lift. The frozen corpus module hash and correction-file hash match their recorded values. The previous ground-truth issue remains addressed; no further required findings. Static review only; no project programs or tests were executed and the checkout was not modified.