Repository navigation
feat(semantic): add P3 judge transport and cache - #100
Conversation
Add continuity_semantic/review routing, bounded strict prompt parsing with one repair, typed fail-closed errors, V0058 validated-report cache, host report validation, shadow metrics, and budget/latency guards. Default remains semantic_mode off.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Changes requested. P3 is off by default, but its enabled path does not yet enforce the specified run budgets or dedupe, the cache can identify the wrong rating-specific judge, and report sections omit appended semantic findings. Required changes are inline. Reviewed 2d22dbb against base 1d14efb. Static inspection only; no tests, builds, or project programs executed.
Enforce run/scene/deadline and prompt/output budgets before judge and repair calls, estimate missing usage, apply deterministic-wins dedupe to emission and shadow counts, key the cache by the rating-resolved binding identity, include scene ids in semantic issues, and build report sections after the semantic stage.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Changes requested. Final report assembly and scene localization are fixed. Three earlier safeguards remain incomplete: dedupe now suppresses unrelated contradictions, output limiting is not enforced at dispatch and uses cumulative output-minus-input arithmetic, and cache identity still omits the resolved agent/endpoint. Required follow-ups are inline. Reviewed 7ad5335 against base 1d14efb, including the follow-up diff. Static inspection only; tests and project programs were not executed.
Use structured contradiction identity for dedupe, enforce a bounded 1200-token dispatch with prompt reservation and remaining-deadline propagation, and derive cache identity from the resolved agent/provider/endpoint configuration.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Changes requested. Bounded dispatch, input reservations/deadlines, and resolved agent/endpoint cache identity are now present. Two earlier issues remain: the post-response limit still compares cumulative output against prompt input, and deterministic dedupe is not canon-specific for attribute findings. Required follow-ups are inline. Reviewed daa37f7 against base 1d14efb, including the latest follow-up diff. Static inspection only; tests and project programs were not executed.
Compare per-response output tokens directly against the semantic cap and require matching canonical predicate/value for attribute dedupe. Add attribute and world-rule dedupe control tests.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Changes requested. The per-call output accounting finding is fixed. One attribute-dedupe identity gap remains: predicate/value text alone conflates different subjects with the same canonical attribute in the same scene. The required follow-up is inline. Reviewed ffbfd23 against base 1d14efb, including the latest diff. Static inspection only; tests and project programs were not executed.
Carry validator entity ids into consistency issues and caches, and require matching subject, predicate, and value before suppressing a semantic attribute finding.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Changes requested. The attribute validator now produces structured IDs, but the adapter drops them before returning or caching issues, so the prior end-to-end dedupe finding remains unresolved. The required wiring/cache follow-up is inline. Reviewed d444b16 against base 1d14efb, including the latest diff. Static inspection only; tests and project programs were not executed.
Forward validator entity ids into Phase-4 consistency issues and cached details, bump the validator output hash version, and assert attribute subject/scene identity on fresh and cached runs.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Changes requested. Attribute subject forwarding and cache invalidation are fixed. One related cache-path gap remains for world-rule/voice/reachability findings: cached IDs omit the scene ID that fresh emission adds, changing localization and dedupe on reruns. The required follow-up is inline. Reviewed 8ebe583 against base 1d14efb, including the latest diff. Static inspection only; tests and project programs were not executed.
Merge the scene id into cached Phase-4 issue entity ids so localization and semantic dedupe match fresh emission.
VerifiedOrganic
left a comment
There was a problem hiding this comment.
Approved. The remaining cache-path finding is addressed: cached_validator_issues now always merges the scene ID with stored entity IDs, preserving localization and deterministic/semantic dedupe on cache hits. The earlier bounded dispatch, input/deadline guards, resolved judge cache identity, per-response output accounting, attribute-subject forwarding, and final report assembly fixes remain intact. Reviewed 01f2ab4 against base 1d14efb, including the latest follow-up diff. No further required changes found. Static inspection only; tests and project programs were not executed.
Summary
P3 of the 0.4 semantic continuity lane: the
continuity_semanticjudge route withreviewfallback, strict prompt/parser with one repair, typed fail-closed errors, validated-report caching, host-report/report-side validation, and shadow/on accounting. Default remainssemantic_mode = off.What changed
continuity_semanticto the prose-bearing routes. It resolves beforereview; a missing/rating-denied route falls back toreview, and a built-in local stub is always rejected as a judge. TypedRouteUnavailable/RatingDeniederrors are recorded as not-evaluated coverage.SemanticJudgeReportV1output contract. The parser accepts one fenced JSON object, rejects unknown fields, validates against the core contracts, and performs at most one repair with compact validation errors. Transport/timeout errors do not consume the repair.semantic_continuity_cache, keyed by scene hash, canon version, candidate-set hash, contract version, route, and model identity. Only validated typed reports are stored; corrupt entries are deleted and treated as misses.CheckConsistencyInput.semantic_host_reportsaccepts the report side ofprepare_semantic_continuity. Reports are validated against the current request; stale/duplicate/unknown requests are typedStaleHostRequesterrors. Valid host reports are cached withroute=host.BudgetExceeded/Timeout/Truncatederrors.Live shadow sample
Run through
ai run OLLAMA_API_KEY -- ...againsthttps://ollama.com/v1withgemma4:31b; no secret was printed. One chapter-1 shadow scene per set:The router/provider did not expose a configured price, so no dollar cost was fabricated; the token and wall-latency figures are the measured per-scene cost inputs. Cache miss = 1 for each sample; a repeat scene with unchanged identity would be a cache hit.
Regression
Semantic mode is off by default. The four deterministic eval sets are unchanged:
Timeline: 4/4, 8/8, 8/8, 3/8. Temporal control warnings: 0.
Tests
cargo test -p spindle-adapters --lib semantic_judge::tests: 11 (parser/fencing/evidence, policy, review fallback, rating denial, rating-resolved cache identity, endpoint invalidation, world-rule dedupe controls, attribute subject/predicate/value dedupe).cargo test -p spindle-adapters --test semantic_judge_p3: 5 (host report on/shadow with sections/Markdown, stale report, local-stub fail-closed, cache key/deletion, deterministic-wins dedupe, typed errors).cargo test -p spindle-adapters --test semantic_continuity_p2: 3.cargo test --workspace: pass.cargo clippy --workspace --all-targets -- -D warnings: pass.cargo fmt --check: pass.Held-out-4 was not accessed. CI green, AO approval, threads resolved, checkpoint. Wait for maintainer merge.