Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 120 additions & 0 deletions .omo/evidence/20260727-v24r9-durable-source-apply-receipt/QA.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# V24R9 Durable Ordinary Source Apply Receipt QA

## Scope and boundary

V24R9 changes only the Legal Coverage observation of a successful ordinary
source merge. It does not change PilotDeck Core, O1, validator acceptance,
issue taxonomy, Progress Lease thresholds `8/2`, Router, Memory, the model, the
evaluation corpus, repair acceptance, matrix handoffs, or authority closure.

The success criterion is narrow: a successful ordinary source transaction
must leave a state-bound applied receipt which advances semantic progress once,
while apply readiness, tampering, and replay do not advance it.

## Static and automated verification

### Syntax and patch hygiene

Commands:

```sh
node --check products/legal/plugins/legal-coverage/hook.mjs
node --check products/legal/plugins/legal-coverage/scripts/legal-coverage.mjs
node --check products/legal/plugins/legal-coverage/scripts/lib/legal-coverage.mjs
node --check .omo/evidence/20260727-v24r9-durable-source-apply-receipt/replay-preserved-case09.mjs
git diff --check
```

Observed: all commands exited `0`; `git diff --check` produced no diagnostics.

Why enough: these checks cover JavaScript parse validity and whitespace
errors. They do not substitute for behavior tests.

### Focused Legal Coverage and real Gateway suite

Command:

```sh
node --test --test-force-exit --test-timeout 60000 \
dist/tests/products/legal-coverage.spec.js \
dist/tests/agent/legal-coverage-plugin-runtime.spec.js
```

Observed: `45/45` passed. Artifact: `focused-legal-gateway.log`.

The product regression reproduces the ordinary source transaction following a
successful immutable repair. It proves that apply readiness does not increment
progress; successful CLI apply writes the deterministic receipt; a tampered
receipt is ignored; an unknown source status is rejected even when the receipt
claims that new current state hash; the restored valid receipt increments
exactly once; and its replay does not increment. The four Gateway cases
continue to pass with Router and Memory disabled and O1 enabled in the
source-repair fixture.

Why enough: this covers the changed hook, CLI, receipt writer, receipt verifier,
immutable repair compatibility, real hook subscription, Gateway tool calls,
Progress Lease projection, and O1 integrity.

### Complete repository suite

Command:

```sh
pnpm test
```

Observed: `308/308` passed, with no failures, cancellations, skips, or todos.
Artifact: `full-suite.log`.

Why enough: the clean complete run covers all repository behavior on the
candidate, including Core Progress Lease, compaction, Gateway, Legal Coverage,
and O1. No unrelated component was changed.

## Preserved Case 09 failure replay

The reviewer-readable replay driver is `replay-preserved-case09.mjs`. It copies
the preserved V24R8 Case 09 failure into a disposable directory, installs the
V24R9 Legal Coverage plugin, reconstructs the canonical state immediately
before the ordinary source apply, and verifies that its hash equals the
immutable proposal's expected state before executing anything.

It then drives the actual Legal Coverage hook and CLI. Observed:

```text
proposal: source-merge-5dadf825216f.json
receipt: source-merge-applied-5dadf825216f.json
state: 74abe7... -> 4d3a3c...
applied: 4 sources, 23 facts
canonical total: 24 sources, 8 reviewed, 16 pending, 45 facts
progressOrdinal: 1 -> 2 -> 2
next work group: source-fragment-merge
```

Artifact: `case09-replay-result.json`.

Why enough: these are the exact proposal, source/fact counts, and state hashes
from the V24R8 product failure. The replay proves that the missing durable
identity, rather than error counts or opaque hash churn, now renews progress
once after the real mutation and remains deduplicated on the next hook call.

## Counterexample coverage

Automated tests cover apply-ready non-progress, deterministic receipt creation,
receipt tampering, exact source-ID binding, exact reviewed-status binding,
current state binding, immutable proposal hash binding, one-time progress, and
replay. Existing tests continue to cover stale state, changed proposals,
traversal, symlink ancestors, atomic source/fact mutation, immutable repair
receipt validation, and repair replay.

## Omitted and residual risk

No API key, provider URL, authorization header, environment dump, private
source content, report text, prompt text, or raw reasoning is included in this
evidence directory. The replay result contains only hashes, counts, stable
artifact basenames, and ordinal transitions.

The complete production-model Case 09 has not yet run on V24R9. Therefore this
change is code-, protocol-, Gateway-, and exact-failure-replay verified, but it
is not yet a product Gate pass. V25 and the 85-case campaign remain blocked
until a new immutable campaign passes Gate 0, paired smoke, Case 05, and the
complete Case 09 product Gate.
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"passed": true,
"fixture": "preserved-v24r8-case09-failure",
"proposal": "source-merge-5dadf825216f.json",
"receipt": "source-merge-applied-5dadf825216f.json",
"sourceCount": 4,
"factCountApplied": 23,
"totalSources": 24,
"reviewedSources": 8,
"pendingSources": 16,
"totalFacts": 45,
"stateHashBefore": "74abe7a06581a13c456ed97f7124a6157733ba4e0f76ad41737e62a7bf3a49ca",
"stateHashAfter": "4d3a3cc1504b6372b90b8e8472b9f0189b431c9b6e05030d7cef896448d0a059",
"progressOrdinal": [
1,
2,
2
],
"nextGroup": "source-fragment-merge"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
(node:86784) ExperimentalWarning: SQLite is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
✔ real gateway drives legal plugin milestones through bounded artifact correction (719.626292ms)
✔ real gateway blocks completion when the legal Stop hook cannot read session state (463.595ms)
✔ real gateway executes the state-bound legal authority closure with complete O1 evidence (530.795125ms)
✔ real gateway applies an immutable source repair before renewing legal progress (703.900459ms)
✔ legal coverage validator creates a current proof and removes it when the deliverable changes (174.566958ms)
✔ legal coverage initializer creates a text skeleton before source review and remains idempotent (199.284375ms)
✔ legal coverage initializer binds trusted manifests to original inputs (260.093584ms)
✔ legal coverage source bootstrap creates only deterministic pending manifest rows (293.388542ms)
✔ legal coverage injects deterministic disjoint worker batches for large pending source rooms (3605.968209ms)
✔ legal coverage CLI exposes bundled guidance through stable named references (122.752209ms)
✔ legal coverage initializer creates only explicit text formats and preserves existing content (50.801708ms)
✔ legal coverage initializer rejects traversal and symlink ancestors without external writes (132.611792ms)
✔ legal coverage validator binds runner originals and derivations into its proof (199.513167ms)
✔ legal coverage validator rejects missing lineage and mutable or derived input roots (251.643ms)
✔ legal coverage next-batch exposes one bounded deterministic repair slice (195.337583ms)
✔ legal coverage apply-batch atomically updates only the current bounded slice (288.167083ms)
✔ legal coverage apply-batch rejects record and byte limit violations before writing (183.584584ms)
✔ legal coverage validator rejects orphaned conflicts and incomplete final disclosure (94.980541ms)
✔ legal coverage validator detects un-inventoried sources and cross-fact timeline collisions (99.837208ms)
✔ legal coverage validator rejects reused generic quotes and unsupported fact coverage (94.372ms)
✔ legal coverage validator requires authority links for critical issues and legal-authority matrices (95.0945ms)
✔ legal coverage validator rejects non-object canonical JSON documents (704.504ms)
✔ legal coverage validator binds reviewed source bytes to the ledger and state hash (192.89475ms)
✔ legal coverage validator rejects ancestor symlinks and a symlinked proof (148.394417ms)
✔ legal coverage validator does not read or write through a symlinked state ancestor (119.518542ms)
✔ legal coverage validator requires unresolved disclosure for unverified material facts (95.839791ms)
✔ legal coverage validator rejects unique but irrelevant issue and authority quotes (95.227792ms)
✔ legal coverage validator enforces reciprocal and same-entry ledger relationships (95.058583ms)
✔ legal coverage validator accepts null for an optional threshold assessment (95.436084ms)
✔ legal coverage validator uses locators without quotes for binary deliverables (95.470708ms)
✔ legal coverage validator rejects empty-fact shortcuts and material facts outside matrices (205.9585ms)
✔ legal coverage hook activates only legal work and injects one observable milestone (509.73875ms)
✔ legal coverage hook groups repeated validator errors into one bounded milestone (504.349042ms)
✔ legal coverage pending-matrix selection is bounded across pages and invalid revisions cannot manufacture progress (930.852833ms)
✔ legal coverage progress ordinal advances once for a new legal phase (196.203583ms)
✔ legal coverage permits not-applicable only after exhaustive selection and rejects stale or changed matrix proposals (446.375125ms)
✔ legal coverage rejects oversized matrix evidence before making a selection receipt immutable (154.703ms)
✔ legal coverage fails closed when one matrix index item exceeds the bounded page (148.076041ms)
✔ legal coverage injects a bounded deterministic matrix relation-closure batch (200.088667ms)
✔ legal coverage authority closure uses one bounded state-bound transaction (557.200291ms)
✔ legal coverage rejects changed and symlinked authority closure proposals without ledger mutation (243.581958ms)
✔ legal coverage milestone digest ignores opaque incomplete-state hash churn (0.222709ms)
✔ legal coverage hook activates a malformed configured workspace so the agent can repair it (95.369709ms)
✔ legal coverage hook rejects a symlinked session-state ancestor without writing outside the workspace (47.996458ms)
✔ legal product plugin loads one skill and contains no benchmark-specific controls (2.958375ms)
ℹ tests 45
ℹ suites 0
ℹ pass 45
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 12283.896625
Loading