diff --git a/CLAUDE.md b/CLAUDE.md index a4755130..8a6273cf 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -284,6 +284,14 @@ diverge enough to warrant it; keep this root file general. three times). Allocate it atomically — `pwsh -NoProfile -File scripts\coord\alloc.ps1 -Kind adr -Title ""` — and add the ADR's index row in the *same* commit. A `pre-commit` hook rejects a number you did not allocate; see [`docs/LEDGER-GATE.md`](docs/LEDGER-GATE.md). +- **Never CITE a `#N` you have not allocated — allocate first, or write a reference that cannot resolve.** + The mirror of the rule above, and the more insidious half. While the number is unissued the citation + resolves to **nothing**, which is honest. The day someone legitimately allocates it, that citation + begins resolving — **to unrelated work, and with nothing anywhere reporting a problem**. A dangling + reference advertises its own brokenness; a wrongly-resolving one reads as a working cross-reference + forever. If you need to gesture at unfiled work, **name the subject, not a number** (*"the retention + runbook step, unallocated"*) — that costs nothing and cannot arm. See + [`docs/LEDGER-GATE.md`](docs/LEDGER-GATE.md) §"Citing a number you have not allocated". - **Building in two sessions at once?** Don't share the working tree — give each its own **git worktree** (`scripts/worktree/new.ps1 -Name <x>`, cleanup with `remove.ps1`). Each gets an isolated checkout + branch + `.venv`; same remote, same PR flow. See [`docs/WORKTREES.md`](docs/WORKTREES.md). diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 645aed2f..d21a2c8d 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -8341,7 +8341,9 @@ every worker session's handoff, which is the sentence the next session bases its > **WHAT LANDED 2026-08-14 IN PR #385, AND WHY IT DOES NOT CLOSE THIS ITEM.** `scripts/docs/dangling_citation_check.py` shipped with 23 unit tests. That is a real, working detector and it is the useful half. **It is not the deliverable this item asked for**, which its own Scope names as *"a rule, not a sweep"*. Three measurements at `origin/main`, each independently sufficient: **(1) the rule is unwritten** -- *"allocate the number before citing it, or write the reference so it CANNOT resolve"* appears nowhere at `origin/main` except this item's own prose (zero hits for `unallocated` across `CLAUDE.md`, `docs/LEDGER-GATE.md`, `CONTRIBUTING`, `scripts/`, `.github/`, `.claude/`). **(2) The detector is wired into nothing** -- repo-wide it is referenced by exactly **two** lines, both inside its own unit test; no workflow, no `.pre-commit-config.yaml` entry, no `.mefor-hooks/pre-commit`, no `pyproject.toml`. **(3) Even when invoked it exits 0**, because `main()` ends `return 1 if args.fail else 0` and `--fail` is opt-in and passed by nothing; a planted live-shape citation was reported correctly *and the process still exited 0*. No test runs `unresolved_citations` over the real `docs/` tree, so committing a new dangling citation turns nothing red. -> **THE TWO RESIDUAL LIMBS, for whoever picks this up.** **(i)** Write the rule where authors actually read it -- `CLAUDE.md` or `docs/LEDGER-GATE.md`. **(ii)** Wire the detector **with `--fail`**, or add a test that runs it over the real `docs/` tree; the sibling gate already does exactly this (`.github/workflows/backlog-hygiene.yml` runs its checker against the real file on every PR). **And a coverage bound that survives both:** by its own docstring the detector *"does NOT see the private companion repository"* -- which is precisely where this item's filed instances live -- so even a fully wired detector only partially covers the stated population. +> **LIMB (i) IS NOW DISCHARGED 2026-08-14.** The rule is written in both places a reader would look: `CLAUDE.md` §5, as a bullet directly beneath the never-grep-for-a-number rule it mirrors, and `docs/LEDGER-GATE.md` §*"Citing a number you have not allocated"* in full -- including which citations can actually arm (**only those above the allocation floor**; a reserved-but-never-filed number is inert **permanently**, because `alloc.ps1` issues `$observed + 1`, never fills a hole, and computes its floor from committed ledger headings that survive a fresh clone), that foreign `#N` references are not this rule's subject at all, and that **enforcement does not replace the rule** -- a checker finds only what has already been written. The `CLAUDE.md` edit is append-only inside an existing bullet list; section numbering was verified unchanged (13 headings before and after), because renumbering that file silently breaks citations across the tree. + +> **THE REMAINING LIMBS, for whoever picks this up.** **(i)** ~~Write the rule~~ **DONE, see above.** **(ii)** Wire the detector **with `--fail`**, or add a test that runs it over the real `docs/` tree; the sibling gate already does exactly this (`.github/workflows/backlog-hygiene.yml` runs its checker against the real file on every PR). **And a coverage bound that survives both:** by its own docstring the detector *"does NOT see the private companion repository"* -- which is precisely where this item's filed instances live -- so even a fully wired detector only partially covers the stated population. > **Closing on the detector's existence would have been a compensating control resting on a false premise (SDS-3.7): a rule recorded as enforced where nothing enforces it.** The check that would have justified closure -- *do the files exist?* -- passes identically whether or not the rule landed and whether or not anything invokes the detector. It measures **presence, not enforcement**. The builder itself declined to close it, recording that the ledger half was the dispatcher's to author. @@ -8621,6 +8623,8 @@ _FHIR_ID_RE.fullmatch("abc\n") -> False the fix > **ONE LIMB HAS LANDED AND THE ITEM STAYS OPEN -- read this before treating PR #382 as a closure.** `#382` ("carry unknown scalars from BOTH sources, not just the live cell (BACKLOG #1242)") merged on 2026-08-13 and fixes the **payload-only top-level key** limb: `scripts/asvs/apply.py:97` now walks `{**(live or {}), **cell}.items()` instead of the live cell alone. That limb is genuinely closed. **The limb carrying this item's severity is untouched.** At the same revision, `:98` skips `_ORDERED` and `_SUBTABLES` *before* that union is ever consulted for them, and evidence entries are re-emitted at `:101-105` by enumerating exactly `path`, `line`, `expect` -- absence entries at `:106-110` by exactly `pattern`, `positive_control`, `mutation`. **A key inside a `[[cell.evidence]]` entry is therefore still dropped**, which is precisely where the backfill put the affected keys. The top-level table-mangling limb also appears untouched. +> **LIMB 4 IS NOT A NEW REQUIREMENT -- IT IS THE UNBUILT HALF OF ONE ALREADY SPECIFIED, and that reframing is the better brief for whoever builds it.** The promotion of this writer was specified to carry through **the union of live and payload keys, so the schema could grow without hand-editing the record**. What shipped delivers that for **top-level scalars** and drops it for **sub-table entries**. The intent is visible in the shipped code's own comments -- `scripts/asvs/apply.py:30` reasons about the union being identical, and `:94-96` states the governing rule as *"enumerate what you ORDER, never what you KEEP"*, which the hardcoded `path`/`line`/`expect` emission at `:101-105` violates. **So the brief is "restore the specified behaviour", not "also handle the two keys that happen to exist today"** -- and a fix keyed to those two names would satisfy the symptom while still dropping the next field anyone adds. + > **BASE REQUIREMENT -- BUILD LIMB 4 ONLY ON A BASE THAT ALREADY CONTAINS `#382`. This is not hygiene; the obvious base silently reverts the limb that already shipped.** Measured 2026-08-14: the branch a lane was holding this on still carried `scripts/asvs/apply.py:88` as `if key in _ORDERED or key in _SUBTABLES or key in cell:` -- **`or key in cell` is exactly the clause `#382` deleted**, and it is absent at `origin/main`. A limb-4 fix authored there and merged would carry **limb 3's reversal in the same diff**: no conflict, no marker, every check green, and the item's own landed fix quietly undone by the commit claiming to extend it. Git raises nothing, because git conflicts on concurrent edits to the same lines, never on a stale base re-asserting a deleted clause. **Branch fresh from current `origin/main`, and before opening a PR, confirm `or key in cell` returns zero hits in the diff's own version of the file.** > **WHY THIS IS RECORDED RATHER THAN LEFT TO A READER TO NOTICE.** A merged PR carrying this item's number, whose title is a faithful description of what it fixed, is the strongest possible signal that the item is done -- and it is wrong. That is the **partial-move** shape: *"it was built"* is true of half, so verifying-before-closing is not enough on its own; **the verification has to ask WHICH HALF.** A closure here would retire a P1 whose stated trigger has never been addressed. **The item's proof condition is unchanged and is the one that discriminates:** put a key the writer has never heard of **inside an evidence entry**, re-render, and assert both that it survives and that the guard refuses when it is deliberately dropped. `#382` does not satisfy it, and no test asserting only top-level carry-through will. @@ -8854,6 +8858,10 @@ _FHIR_ID_RE.fullmatch("abc\n") -> False the fix **Cluster:** Security record / repository topology. **Priority:** P2. **Verdict:** build. **Severity:** no deployment axis -- nothing shipped changes; the exposure is that a ranked map of unmet security requirements **is** public today and cannot be made unpublic, only stopped from growing. +> **THIS ITEM IS NOW LOAD-BEARING RATHER THAN A PROPOSAL, and the cost of not having it accumulates per item.** Owner ruling 2026-08-14: an item whose substance is a weakness in the coordination tooling or the seat topology is a **deficit**, so it may not go in this public ledger -- and with no private ledger existing, it **parks pending this item**. That is no longer a hypothetical: a measured, reproduced defect with a named cost is parked in the coordination handoff directory today because there is nowhere correct to file it, and **the ruling applies to every future item of this class**, not only that one. Each one that arrives before this lands is a real finding held outside any ledger, discoverable only by whoever thinks to look in a handoff file. **The priority line above predates that ruling and is left unchanged deliberately** -- re-scoring is the owner's call, not a side effect of recording why it now matters. + +> **What this does NOT settle, recorded so the two questions are not merged.** The ruling answered where a **new** item of this class goes. It said nothing about items **already** in this public ledger that appear to be the same class. Those have a different cost structure -- **unfiling from a public repository does not unpublish anything** -- so the right answer may differ, and a boundary never applied retroactively is not the same thing as a boundary ignored. That question is open and is being carried separately. + ## 1253. one control-char predicate written out seven times across six files > 🔢 **Filed 2026-08-14 - not started. SUCCESSOR to #1239, which closed on its own narrower condition.** Value **4/10** · Difficulty **3/10**. `#1239` recorded two byte-identical control-char predicates in `fhir.py` and was resolved by removing one of that pair. The predicate is written out **seven times across six files**, so resolving the pair addressed two of seven sites and the drift surface is **wider today than when #1239 was filed** -- partly because the work that resolved it added sites. @@ -8935,3 +8943,62 @@ _FHIR_ID_RE.fullmatch("abc\n") -> False the fix **How to prove a fix.** Bind `(issuer, subject)` to account A, then attempt the same binding on account B, and assert it is **refused** -- on every backend, because the enforcement point may differ per backend and a test passing on SQLite alone would certify nothing. The store suites for the two server backends **skip in a local run**, so this must be demonstrated in CI. **Provenance.** The exclusivity framing is the research conclusion of **#1143**, handed over as content by the lane that concluded it, with the number allocated here rather than by that lane. The uniqueness measurement and the `NVARCHAR(MAX)` index-key consequence were measured independently during earlier #1143 work and re-verified at `origin/main` before filing. #1143's own question -- whether TOFU is the defensible ceremony -- is **separable and unresolved**, and this item does not answer it. + +## 1257. an exception after `engine.start()` unwinds nothing, so a startup refusal hangs the process instead of exiting + +> 🔢 **Filed 2026-08-14 - not started. A HUNG refusal, not a silent one -- and that distinction is the whole item.** Value **8/10** · Difficulty **5/10**. Any exception raised in the ASGI lifespan **after** `await engine.start()` leaves the engine, the upload-retention runner and the alert notifier running, and **the process never exits.** `uvicorn` itself behaves correctly: it prints the full error, logs `Application startup failed. Exiting.`, raises `SystemExit(3)` and binds no socket. The **process** then stays alive indefinitely. + +> **WHY THIS IS WORSE THAN A WRONG ANSWER, and the reason it outranks the item that found it.** An operator watching a console sees exactly the right error. **A SUPERVISOR SEES NOTHING WRONG.** The engine ships under **NSSM** ([`docs/SERVICE.md`](SERVICE.md)); systemd and container runtimes behave the same way. All of them decide by process liveness, and **running-and-dead is the one state a restart policy cannot detect and will not recover.** A service that refuses loudly and then hangs is indistinguishable, to the thing responsible for restarting it, from a service that is working. + +**Cluster:** Reliability / startup. **Priority:** P1. **Verdict:** build. +**Severity:** Conditional -- there are zero deployments (§0). On a first deployment, any startup refusal reachable after `engine.start()` would present as a live-but-serving-nothing service under its supervisor. The failure is **not** hypothetical in the sense of being unreachable: it is reachable by design the moment any post-engine startup gate refuses. + +**Measured at `HEAD`, structurally, and separately at runtime.** Each **pre-engine** step carries its own unwinding: `api/app.py:5608` (startup attestation) and `:5628` (trust-anchor preflight) each wrap in `try:` / `except BaseException:`, close the notifier and the store, and re-raise. **From `await engine.start()` at `:5731` to the `yield`, there is no `try`, no `except` and no `finally` at all** -- so nothing raised in that region unwinds anything. `auth_settings` handling sits at `:5816-5818`, squarely inside the unprotected span. Runtime behaviour was measured under `uvicorn` by timing the process directly and reading its real exit code: correct refusal output, `SystemExit(3)`, **process still alive 90 seconds after printing "Exiting." and requiring a kill.** + +**Scope, in preference order, with the rejected option recorded.** **(1) Fix the partial-startup teardown** so an exception between `engine.start()` and the `yield` unwinds what is already running -- correct, and it protects *every future startup gate* rather than one. **(2) Make a refusal terminate explicitly** rather than relying on unwinding -- smaller, but it papers over the same teardown for whatever raises next. **(3) Move the offending check before `engine.start()` -- REJECTED ON MEASUREMENT**, not on preference: for the #1020 case the bootstrap admin does not exist until `auth.initialize()`, which runs later. Recording the rejection so it is not re-proposed. + +**How to prove a fix.** Raise deliberately in the post-engine span and assert the **process exits** -- not that the error is printed, and not that `SystemExit` is raised. Both of those are already true today and neither discriminates. Time the process and read its exit code directly; a wrapper's own timeout branch must not be allowed to supply the verdict. + +**Blocks #1020**, whose gate is simply the first thing that has ever raised in this window. **This is a pre-existing defect in a lifespan every deployment runs, and it is deliberately NOT folded into an auth item** -- a lifespan-teardown change has its own blast radius and deserves its own review. + +**Provenance.** Found by the lane building #1020 when it closed that item's own open question -- *does the refusal terminate under `uvicorn` rather than under the harness?* -- and reported **against its own work**. The structural half was re-verified here at `HEAD` before filing. Its method note is recorded because it nearly cost the finding: a first probe reported "HUNG" from the lane's **own** `Wait-Job` timeout branch while the captured output beneath showed `uvicorn` exiting via `SystemExit(3)` -- a wrapper's exit path manufacturing a verdict about the thing it wraps. + +## 1258. classify existing ledger items against the publish-decisions-not-deficits ruling + +> 🔢 **Filed 2026-08-14 - not started. BLOCKED behind #1250, deliberately, and the reason is the item's own subject: its OUTPUT is the same shape as the thing it looks for.** Value **6/10** · Difficulty **3/10**. Owner-authorised. **This is a CLASSIFICATION task, not a defect report.** It names a **rule** and a **scope**; it does not name a weakness, and it must not acquire one. + +> **THE RULE.** *Publish decisions, not deficits* (owner ruling 2026-08-13, whose durable home is **ADR 0160**), extended explicitly to this ledger: a backlog item whose substance is *"this tooling can be misinstalled"* is a **deficit**, not a decision. **Engine defects file publicly as normal** -- that half is unchanged and is most of the ledger. Confirmed again 2026-08-14 when the owner declined to file a new item of this class publicly. + +> **THE SCOPE.** Existing items in `docs/BACKLOG.md`, assessed against that rule. **Not the archive, not ADRs, not the engine.** + +**Cluster:** Security record / repository topology. **Priority:** P3. **Verdict:** build. +**Severity:** no deployment axis -- nothing shipped changes. + +**WHY IT IS BLOCKED RATHER THAN MERELY UNSTARTED, and this must be settled before anyone runs it.** The sweep's product is *a list of public items that disclose tooling weaknesses*. **That list is an index to precisely what the rule protects** -- the same aggregation argument that kept the 2026-08-14 item out of this file. So the output cannot be written here, and with no private ledger existing it has **nowhere to go**. Running the sweep first would manufacture a finding that must then be parked in a handoff file, adding to the very cost #1250 exists to end. **Land #1250, then run this into it.** + +**WHAT THE SWEEP MAY AND MAY NOT PRODUCE.** It may **identify**. It may **not** presume a remedy: **unfiling from a public repository does not unpublish anything**, so what to do about anything found is a *separate question with a different cost structure*, and this item must not answer it by implication. A reclassification that moves text without reducing exposure is churn wearing the shape of a fix. + +**HOW IT MUST READ, and this is a wording constraint rather than a preference.** The ruling is dated **2026-08-13**; the items most likely to match were filed around the same time. **A boundary never applied retroactively is not a boundary ignored.** The sweep is **hygiene, not an audit of anyone's judgement**, and if its output reads as an accusation it has been written wrongly. Nothing here is a finding against any lane. + +**Do NOT enumerate candidates in this item, now or later.** Naming them here would place in the public ledger exactly the index this work exists to remove from it -- the item would perform the defect it was filed to find. The count is not recorded either, for the same reason. + +**Provenance.** Raised, and deliberately **not acted on**, by the seat that noticed it while applying the rule to a new item -- on the grounds that reclassifying existing public items is not a dispatcher's unilateral call. Escalated, authorised by the owner, and filed with the destination question settled up front rather than discovered when the result lands. + +## 1259. `parse_items` censuses a conflicted ledger without error, so a gate reports a clean count off a broken file + +> 🔢 **Filed 2026-08-14 - not started. The census AGREED WITH INTENT while the merge was broken, which is why this is worth an item rather than a note.** Value **6/10** · Difficulty **2/10** · _quick win_. `parse_items` walks a `docs/BACKLOG.md` containing `<<<<<<<`, `=======` and `>>>>>>>` and returns a **plausible, correct-looking census**, because a conflict marker is not a heading and nothing in that function is looking for one. + +> **REPRODUCED WITH A CONTROL.** The live ledger and a copy poisoned with conflict markers both parse to **287 items / 207 open**, and no exception is raised. The two counts are identical, so the census cannot be used to detect the condition -- **it is not that the number is wrong, it is that the number is right and the file is unusable.** + +**Cluster:** Ledger tooling / silent-pass gate. **Priority:** P2. **Verdict:** build. +**Severity:** no deployment axis (§0) -- nothing shipped changes. The cost is a gate that certifies a file no one can parse. + +**Why nothing else catches it.** Measured across the ledger gates: `scripts/docs/backlog_status_check.py` and `scripts/docs/backlog_citation_check.py` contain no marker detection at all. `scripts/hooks/ledger_check.py` matches a grep for *"conflict"* **only in prose** -- a comment about clean merges and an error string reading *"not as a conflict"* -- and detects no markers either. `.pre-commit-config.yaml` runs `ledger-gate`, `ruff-format`, `ruff-check`, `forbidden-content`, `gitleaks`, `actionlint` and `bandit`; **none inspects Markdown for conflict markers.** So a conflicted `docs/BACKLOG.md` can be committed and every ledger gate passes over it. + +**How it was found, and the finding underneath it.** A landing seat ran `parse_items` on a `merge-tree` output **before** checking the merge's exit code, and got a census matching the incoming batch's prediction exactly -- right item count, right open count, both new numbers present, no duplicates. The exit code was `rc=1` with six conflict markers in the blob. **The content check was a check on garbage that happened to look like data.** The general rule the seat drew is the more valuable half and belongs with the fix: **read the exit code BEFORE the content, because a content check on a conflicted tree certifies nothing.** + +**Scope.** Make the condition impossible to miss rather than relying on ordering discipline. Options to price: have `parse_items` (or its CLI) **refuse** a source containing conflict markers rather than parse it; and/or add the standard `check-merge-conflict` pre-commit hook, which is the cheap general fix and covers every file rather than this one. **The first is the one that protects programmatic callers**, which is where this bit -- a hook does not help a gate that is handed a tree in memory. + +**How to prove a fix.** Poison a copy of the real ledger with markers and assert the reader **fails**. Asserting it parses a clean file proves nothing: it already does that, and did so throughout the incident. + +**Provenance.** Reported by the seat whose own gate nearly certified the broken tree, against its own process, with the near-miss stated plainly rather than after quietly reordering its checks. diff --git a/docs/LEDGER-GATE.md b/docs/LEDGER-GATE.md index 44c4aac1..fb695054 100644 --- a/docs/LEDGER-GATE.md +++ b/docs/LEDGER-GATE.md @@ -315,3 +315,52 @@ are safe. - **Numbers leak.** An abandoned branch's number is never reclaimed. Accepted, deliberately. - **It governs ADR and BACKLOG numbers only.** Any other shared sequence (a migration version, say) would need its own `-Kind`. + +## Citing a number you have not allocated + +**The allocation rule has a mirror, and the mirror is the more insidious half.** `alloc.ps1` stops two +sessions from *issuing* the same number. Nothing stops a document from *citing* a number that was never +issued at all. + +**While the number is unissued, the citation resolves to nothing.** That is honest and harmless: a +dangling reference advertises its own brokenness, and anyone who follows it immediately sees there is +nothing there. + +**The day someone legitimately allocates that number, the citation begins resolving — to unrelated +work.** Nothing anywhere reports a problem, because nothing is broken in any mechanical sense. A +wrongly-resolving reference reads as a working cross-reference forever, which is strictly worse than a +broken one. This file's own header already states the general form of that trade: renumbering *"would +only make stale citations resolve uniquely and WRONGLY, which is worse than resolving ambiguously."* + +### The rule + +**Either allocate the number before citing it, or write the reference so it CANNOT resolve.** + +Naming the subject instead of a number is enough, and costs nothing: + + #1203 arms the day 1203 is issued + "the retention runbook step" cannot arm + "unallocated - see the runbook step" cannot arm, and says so + +### What is and is not dangerous + +**Only a citation ABOVE the current allocation floor can arm.** A number at or below the floor is +already spoken for and can never be re-issued: `alloc.ps1` issues `$observed + 1` and never fills a +hole, and the floor is computed from **committed ledger headings**, which survive a fresh clone. So a +citation to a reserved-but-never-filed number resolves to nothing **permanently** — it is inert, not +merely quiet. + +**Foreign `#N` references are not citations of this ledger at all** — an upstream driver issue, a +vendor forum thread, another project's tracker. They match a naive scan and are not this rule's +subject. + +### Enforcement + +`scripts/docs/dangling_citation_check.py` reports unresolved citations and keys its exit code on the +**live shape** — above the floor, and not foreign — rather than on a raw hit count, so the inert cases +are reported without failing anything. **It is fail-closed by default**, with `--advisory` as the +explicit escape. Note its stated coverage bound: it sees this repository only, and **not** the private +companion repository. + +**Enforcement does not replace the rule.** A checker can only find what has already been written; the +rule is what stops it being written.