Skip to content

fix(gate): five defects found while running Feature 081 in a sync target - #19

Merged
beomeodev merged 3 commits into
masterfrom
fix/gate-self-invalidation-and-parser-bugs
Jul 27, 2026
Merged

beomeodev merged 3 commits into
masterfrom
fix/gate-self-invalidation-and-parser-bugs

Conversation

@beomeodev

Copy link
Copy Markdown
Owner

Five gate defects surfaced while running a real T3 Feature cycle (/ms.specter 081) in the suseonglm sync target. Two of them made a T3 station structurally unable to reach a valid receipt; the other three silently discarded valid reviewer output or a human's acknowledgement.

None was visible to the existing suite, because none changes a tier value — they change whether a receipt can ever be valid, whether a report is accepted, and whether prior findings survive into a re-round.

The defects

# Defect Root cause Consequence observed
1 --phase diff receipt is stale the instant it is written diff_evidence() hashes every untracked file's content, and the classifier writes its receipt into the untracked .specify/audit-tiers/ A T3 station could never reach valid: true
2 Same loop through the ledger .specify/specter-run.jsonl is tracked and is the station receipt, so emitting it changed the diff acknowledge-warn could never bind; /ms.review could not be completed
3 A fresh human acknowledgement is voided by a no-op reclassification The receipt is rewritten unconditionally, so only timestamp differs but the sha256 the ack binds to moves The user had to approve byte-identical evidence twice
4 A valid report is rejected as "wrong station" extract_field strips leading whitespace only; a Markdown hard-break (two trailing spaces) broke the exact-match compare A PASS report graded FAIL
5 Blocking findings dropped from the continuity packet The packet builder matched only CRITICAL/HIGH, but §6 calls these "blocking findings" and the prompts fix no vocabulary A round with 3 BLOCKING findings produced "(no blocking findings)" — the re-round was never told to carry them forward

The fix for 1–2, and why it is not a hole

Gate machinery is excluded from the classified bundle: receipts, the run ledger, continuity artifacts, review state, and the installed runtime copies under .specify/.

Only installed copies and outputs are excluded. The tracked sources they come from — scripts/specter/, docs/templates/ — stay in the diff, so a real policy, classifier, or gate change still forces its floor. test_gate_machinery_is_excluded_but_its_tracked_source_is_not pins exactly that, because "excluding a receipt" must never quietly become "excluding a code change".

A side effect worth noting: because the untracked bundle previously swept in the policy file and workflow prose, classification was firing audit-policy-change, secrets-crypto, and money-financial floors that no line of the Feature's code justified. That noise is gone too.

The fix for 3

The receipt write is now idempotent — identical inputs and identical verdict do not rewrite the file. timestamp is the only field exempt from the comparison, so a genuine change still rewrites and still voids the ack (test_reclassifying_changed_evidence_does_rewrite_the_receipt).

Tests

baseline: 263 passed
after:    270 passed   (7 new regression tests)

Every new test was verified to fail without its fix — the fixes were stashed and the suite re-run. Two of the new tests needed correcting before they discriminated at all:

  • the BLOCKING assertion initially checked the whole packet, where the second (finding-free) lane legitimately prints the "(no blocking findings)" line — now asserted per lane;
  • the idempotency test passed without the fix, because utc_timestamp() has second granularity and both runs landed in the same second — it now crosses a second boundary deliberately.

Not in this PR

The Antigravity plugin's scripts/commands/rescue.mjs exits 0 with no output for every input (bin/antigravity.mjs rescue works). That is a separate repository.

Full evidence, with file:line, is in the sync target's review report: docs/review/review_Claude_260727-221341.md (beomeodev/suseonglm#276).

🤖 Generated with Claude Code

beomeodev and others added 3 commits July 27, 2026 22:32
`--phase diff` was structurally unusable. `diff_evidence()` hashed every
untracked file's full content, and the classifier writes its own receipt into
the untracked `.specify/audit-tiers/`; the aggregation writes the station
receipt into the tracked `.specify/specter-run.jsonl`. Producing either receipt
therefore changed the very evidence that receipt records, so the receipt was
stale the instant it existed and a T3 station could never reach a valid one.

Exclude gate machinery — receipts, the run ledger, continuity artifacts, review
state, and the installed runtime copies under `.specify/` — from the classified
bundle, on both the tracked pathspec and the untracked scan. The tracked
sources those copies come from (`scripts/specter/`, `docs/templates/`) stay in
the diff, so a real policy, classifier, or gate change still forces its floor; a
regression test pins that, because excluding a receipt must never become a hole
that also excludes a code change.

Also make the receipt write idempotent. Re-classification that observes the same
inputs and reaches the same verdict no longer rewrites the file, because human
WARN acknowledgements bind to the receipt sha256 — the mandatory re-classify at
`/ms.implement` was voiding an acknowledgement the user had given minutes
earlier against byte-identical evidence, forcing them to approve twice. Only
`timestamp` is exempt from the comparison, so a genuine change still rewrites
and still voids the ack.

Found while running Feature 081 in the suseonglm sync target.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two parser defects made valid reviewer output look invalid or empty, and both
failed in the dangerous direction: silent under-reporting.

`extract_field` stripped leading whitespace only, so a report whose header line
ended in the Markdown hard-break (two trailing spaces) compared unequal on Mode
and Result and was rejected as coming from the wrong station. Surrounding
whitespace is never semantic in these fields; trim both sides.

The continuity packet builder matched only CRITICAL and HIGH severity cells,
while §6 of the protocol calls these "prior blocking findings" and the station
prompts fix no severity vocabulary. A round whose report carried BLOCKING
findings produced "(no blocking findings in this lane at this round)", so the
re-round was never told to carry that lineage forward. Accept BLOCKING.

Found while running Feature 081 in the suseonglm sync target.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…that runs it

`ruff format` rewrote `except (OSError, json.JSONDecodeError):` into PEP 758's
unparenthesized form, which `pyproject.toml`'s `requires-python = ">=3.14"`
permits. But this script is executed by whatever `python3` a consumer repo has
on PATH, and the test harness's own restricted PATH resolves to 3.11, where
that form is a SyntaxError — the classifier died at import and two tiered-gate
tests went red.

Split the handler into two clauses. They are valid on every interpreter and
cannot be merged back into a form the formatter will rewrite.

The underlying mismatch is left alone deliberately: the package declares 3.14
while its scripts must run under whatever the consumer has. That is a
repo-wide decision, not this PR's.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@beomeodev

Copy link
Copy Markdown
Owner Author

CI status: red on an inherited baseline, merging by operator decision

specter-release.sh classify-ci 19 returns needs_human — this is a real ruff format --check failure, not the billing/infra signature, so the automated path correctly refused to merge.

The failure is pre-existing on master. Both master and this branch fail on exactly the same three files, none of which this PR touches:

tests/specter/test_publish_release_scripts.py
tests/specter/test_specter_gate_continuity.py
tests/specter/test_specter_gate_subcommands.py

Formatting them is left out per the surgical-change rule. Operator decision: merge with CI red rather than expand this PR's scope.

One real defect found while diagnosing this, and fixed here (ca17927)

ruff format rewrote this PR's except (OSError, json.JSONDecodeError): into PEP 758's unparenthesized form, which requires-python = ">=3.14" permits. But these scripts are executed by whatever python3 a consumer repo has on PATH, and the test harness's restricted PATH resolves to Python 3.11, where that form is a SyntaxError. The classifier died at import and two tiered-gate tests went red.

Split into two except clauses — valid on every interpreter, and the formatter cannot merge them back.

The underlying mismatch is deliberately left open: the package declares >=3.14 while its scripts must run under whatever the consumer has. Any future multi-type except in scripts/specter/ will hit the same trap. That is a repo-wide decision, not this PR's.

Local verification

270 passed   (baseline 263 + 7 new regression tests)
ruff format --check scripts/specter/classify_audit_tier.py  ->  already formatted
/usr/bin/python3 (3.11) ast.parse  ->  OK

Each new regression test was verified to fail without its fix (fixes stashed, suite re-run).

@beomeodev
beomeodev merged commit 3abf864 into master Jul 27, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant