Repository navigation
fix(semantic): remove control false positives and repair inflation - #104
Conversation
Require balance evidence for quantity drift, strictly-prior dead state with memory framing rejection, and a shared on/shadow execution to remove repair inflation and parity drift. Control warnings are eliminated; remaining gap is recall.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Changes requested. Required fixes: make death-context slicing UTF-8 safe, invalidate cached reports produced before the new policy filters, and measure shadow parity/cost from actual shadow results rather than a copy of on-mode output. Reviewed d192552 against ed1a1d5. Static review only; no tests or project programs were run and the checkout was not modified.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Changes requested. Two correctness issues remain: the new evidence gates are bypassed by full/host reports, and deterministic dedupe can suppress distinct canon conflicts. Static review against origin/main; no programs, builds, or tests were run.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Changes requested. The full/host evidence-gate fix and secret-specific identity check address the concrete cases from the previous review. The general dedupe fallback still suppresses distinct canon conflicts in other families; one required inline finding describes the remaining issue. Static review against origin/main, including the follow-up diff; no programs, builds, or tests were run.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Approved — ready. The previously reported issues are addressed: shared evidence gates cover compact, full, and host reports, and deterministic dedupe now requires canon identity for independent fact/event families while preserving character-scoped death dedupe. The added regressions cover unrelated subjectless timeline candidates and multiple knowledge facts for the same character. No further high-confidence findings requiring changes. Static review of the diff against origin/main and the follow-up changes at ff8626f; no programs, builds, or tests were run.
What this round fixes
Precision round for the semantic continuity lane (still off by default). This PR:
d1925528.Review findings
semantic_judge.rs): the ±240-byte dead-context window now rounds both edges to character boundaries; regressions cover multibyte characters crossing the left and right edges.semantic_judge.rs):SEMANTIC_JUDGE_POLICY_VERSIONis nowsemantic-judge-policy-v3; cache-key construction is shared, and a test seeds a v2 full report (which still parses as a contradiction) and proves a v3 lookup misses it.semantic_eval.py): the evaluator runs a fresh on pass, then a real cache-only shadow pass, keeps each mode's own results, and computes parity from the two actual summaries. No derived shadow view is used as evidence.General precision fixes
Results
Judge
ollama:gemma4:31b, temperature 0, seed 7; same config for all sets.onis the fresh dispatch pass;shadowreuses the validated-report cache.Control warnings do not rise on any set. Warning precision is 1.0 where warnings exist. Evidence-anchor validity is 1.0. On-mode repair is <=4.17% of calls. Shadow/on would-be finding counts match on every set (the cache-only shadow pass makes only 0-6 retry dispatches, so its own repair ratio is reported separately in the checked-in reports and is not the gate).
Gates
--semantic offunchanged: baseline 28/28 with 3 control findings; feat(evals): held-out continuity contradiction corpus #72 54/56 with 0; held-out-2 55/56 + 56/56 type with 0; held-out-3 26/56 + 28/56 type with 0 control warnings.cargo fmt --all -- --check,cargo clippy -p spindle-core -p spindle-adapters --all-targets -- -D warnings,cargo test -p spindle-adapters --lib(900 passed),cargo test -p spindle-core --lib(358 passed).semantic_mode = off; activation gates still not met because aggregate semantic lift is +3 localized/+2 type-present (all held-out-3).Checked-in reports:
evals/results/semantic_precision_*.json.