workflow: stop parallel content cycles from colliding

.claude/commands/add-canon.md

@@ -13,7 +13,8 @@ the PR is **merged** (or you have reported a hard blocker).
 
 Argument (optional): `$ARGUMENTS` may name a target, e.g. `visa jp`,
 `banking br`, `emergency pl`, or a domain like `legal`. If empty, pick the gap
-yourself in step 2.
+yourself in step 3 — but run step 2 either way, so you do not collide with a
+cycle already in flight.
 
 ---
 
@@ -31,7 +32,27 @@ Prefer a descriptive name once you know the topic, e.g.
 `canon/visa-jp-work-permit`. Confirm `git config user.email` is
 `yujinhong3@gmail.com` before committing.
 
-## 2. Pick a real coverage gap (never duplicate)
+## 2. Claim a target no open PR is already working on
+
+`main` is not the whole picture. Other cycles may be running right now with
+branches already pushed, and their canons will not show up in any count of
+`data/canons/`. Three separate PRs once authored New Zealand coverage at the
+same time because each of them looked only at `main`.
+
+List the open PRs and the files they touch **before** choosing anything:
+
+```bash
+# mcp__github__list_pull_requests (state: open), then for each PR:
+# mcp__github__pull_request_read (method: get_files)
+```
+
+Treat every country appearing in an open PR as taken, and pick a different one.
+
+Step 6 excludes the canon IDs those branches touch on its own, by reading the
+pushed branches. It cannot see a PR from a fork, so note the touched IDs of any
+fork PR here and pass them along explicitly.
+
+## 3. Pick a real coverage gap (never duplicate)
 
 Count what already exists, then choose an underserved slice:
 
@@ -73,13 +94,39 @@ Selection rules:
 - The country code **must already be in** `SUPPORTED_COUNTRIES`
   (`generator/country_canon_template.py`). If you want a new country, add it
   there in the same PR.
-- Grep before writing so you do not re-author an existing slug:
-  `rg -l "your-slug" data/canons/`
+
+Then check for duplicates **by topic, not by slug**. A different slug describing
+the same dead end is still a duplicate, and the site is penalised for it. Read
+what the country already has before writing:
+
+```bash
+python - <<'PY'
+import json, pathlib
+CC = "nz"  # your target country
+for f in sorted(pathlib.Path("data/canons").rglob(f"{CC}.json")):
+    c = json.loads(f.read_text())
+    print(c["id"])
+    print("   ", c["error"]["signature"])
+    print("   ", c["verdict"]["summary"][:160])
+PY
+```
+
+For each canon you intend to write, name the existing entry it is closest to and
+say in one line why yours is a different dead end. If you cannot, drop it. Real
+examples of duplicates that got caught only at merge time: an IRD-number canon
+about bank interest when `banking/rwt-non-declaration-rate/nz` already covered
+it, and a border-declaration canon under `legal/` when
+`food-safety/undeclared-biosecurity-goods/nz` already covered it.
+
+When the overlap is partial - your canon has one genuinely new angle and the
+rest restates an existing entry - do not ship a competing page. Either narrow
+yours to the new angle alone and cross-link the two, or fold the new angle into
+the existing canon as an extra `dead_ends[]` / `workarounds[]` entry.
 
 Pick **3–5 canons** for this cycle. Fewer is fine; more than 5 makes review and
 sourcing quality slip.
 
-## 3. Research from primary sources
+## 4. Research from primary sources
 
 For each canon, find real, current, citable sources before writing a word:
 
@@ -96,7 +143,7 @@ For each canon, find real, current, citable sources before writing a word:
 A good canon captures something an AI would confidently get **wrong**: a
 non-obvious dead end, not a fact anyone can restate.
 
-## 4. Author the canons
+## 5. Author the canons
 
 Use the scaffold:
 
@@ -122,7 +169,55 @@ Follow `docs/country-canon-guide.md` and the schema in `CLAUDE.md`. Non-negotiab
 - `error.regex` must be a valid, ReDoS-safe pattern (no `(a+)+`, no `(a|b)+`)
   that matches how someone would phrase the problem.
 
-## 5. Validate — must be clean before committing
+## 6. Re-verify your assigned bucket of aging canons
+
+Roughly a thousand canons sit past the 180-day aging threshold, so each cycle
+refreshes a few. Do **not** pick "the oldest three" - that rule is deterministic,
+every parallel cycle lands on the same files, and whichever PR merges first
+leaves the rest conflicting on exactly the date fields they came to update. That
+is what stalled PRs #165, #166, #168, and #172.
+
+Ask for your slice instead. Seed it with your target country code:
+
+```bash
+git fetch origin --prune            # so the claim scan sees current branches
+python -m generator.reverify --seed <cc>
+```
+
+Each canon sits in a bucket fixed by hashing its ID, and your seed owns one
+bucket. Two cycles get the same bucket (identical picks) or share nothing -
+a partial overlap, the thing that conflicts on merge, cannot happen. The command
+also excludes canons other pushed branches already touch. Fetch first, or that
+scan reads a stale view of the remote.
+
+**Read the `Claim scan:` line it prints.** `INCOMPLETE` means some or all
+branches went unchecked and another branch may own your picks - resolve that
+before editing anything. Add `--exclude id1,id2` for what the scan cannot see:
+a PR from a fork, or canons you will touch later in this same cycle.
+
+The scan only sees branches already pushed, and you pick before you push. If
+another cycle started at the same time, re-run this command just before
+committing; if your picks are now claimed, take the new ones instead.
+
+If your bucket comes back empty or fully claimed, vary the seed (`nz` ->
+`nz-2`). Do not raise `--buckets` - that re-shuffles every canon and breaks the
+agreement with every other cycle.
+
+Then actually re-verify, in this order:
+
+1. Open every `sources[]` URL on the canon and read it.
+2. If a URL moved, update it to the new canonical location. If the claim no
+   longer holds, fix the claim - a wrong canon is worse than a stale one.
+3. Only then bump `last_confirmed`, `verdict.last_updated`, and
+   `metadata.last_verification`.
+
+Never bump a date you did not earn by re-reading the source. A refreshed date on
+an unchecked canon is a silent lie to every agent that reads it.
+
+If a source contradicts the canon, say so explicitly in the PR body - that
+finding is worth more than the new pages.
+
+## 7. Validate — must be clean before committing
 
 ```bash
 ruff check generator/ tests/
@@ -135,7 +230,7 @@ python -m generator.validate --site-only
 Fix every failure. Do not commit red. If the build emits new warnings for your
 files, resolve them too.
 
-## 6. Commit and push
+## 8. Commit and push
 
 ```bash
 git add data/ generator/
@@ -145,7 +240,7 @@ git push -u origin <branch>
 
 On network failure only, retry up to 4 times with backoff (2s, 4s, 8s, 16s).
 
-## 7. Open the PR
+## 9. Open the PR
 
 Use `mcp__github__create_pull_request` against `main`. Body should state:
 
@@ -161,7 +256,7 @@ End the body with the attribution footer:
 _Generated by [Claude Code](https://claude.ai/code)_
 ```
 
-## 8. Drive to green, then merge
+## 10. Drive to green, then merge
 
 1. Read CI status with `mcp__github__pull_request_read` (`get_status`).
 2. If checks are still running, wait with the `Monitor` tool (never a foreground
@@ -174,7 +269,7 @@ _Generated by [Claude Code](https://claude.ai/code)_
 5. If CI is red for a reason that also fails on `main` (pre-existing breakage),
    say so explicitly in the PR and stop — that one is not yours to force.
 
-## 9. Report
+## 11. Report
 
 Finish with a short summary: canons added (IDs), the PR number and merge state,
 and the new total canon count. If anything was skipped for lack of sources, say
@@ -191,3 +286,7 @@ which topic and why.
 - Duplicate or near-duplicate pages actively hurt indexing. If the only thing
   you can produce this cycle is a rewording of an existing canon, produce
   nothing and report that instead.
+- **Never** bump `last_confirmed` on a canon whose sources you did not re-read.
+- **Never** re-verify by "oldest first" — always take the bucket
+  `python -m generator.reverify --seed <cc>` assigns you, after
+  `git fetch origin --prune`.

CLAUDE.md

@@ -51,6 +51,8 @@ generator/
   lookup.py            # Programmatic error lookup SDK (lookup, lookup_all, search, batch_lookup)
   ping_search_engines.py  # Search engine ping on deploy
   pipeline.py          # Unified pipeline: validate → generate → build → test
+  reverify.py          # Assigns each content cycle a disjoint bucket of aging
+                       # canons to re-verify (see 'Re-verifying Aging Canons')
   schema.py            # ErrorCanon JSON Schema (ERRORCANON_SCHEMA)
   submit_indexnow.py   # IndexNow submission on deploy
   templates/           # Jinja2 HTML templates
@@ -101,6 +103,10 @@ ruff check generator/ tests/
 # Look up an error (CLI)
 python -m generator.lookup "error message"
 
+# Get this cycle's re-verification bucket (seed with the target country code)
+git fetch origin --prune && python -m generator.reverify --seed nz
+python -m generator.reverify --seed nz --exclude id1,id2   # additional claims
+
 # List all errors
 python -m generator.lookup --list
 
@@ -179,6 +185,55 @@ sourcing standards and confidence calibration. Quick reference:
   `generator/country_canon_template.py` gates new country codes - add there
   first before writing canons for a new country.
 
+## Re-verifying Aging Canons
+
+The validator warns at 180 days since `last_confirmed` and errors at 365, so a
+large cohort is permanently due for re-checking. Each content PR refreshes a
+few of them.
+
+**Do not pick "the oldest N".** That rule is deterministic, so every parallel
+cycle selects the same files and all but the first PR to merge is left
+conflicting on the very date fields it came to update. Ask `generator.reverify`
+for a bucket instead:
+
+```bash
+git fetch origin --prune                                  # refresh the claim scan
+python -m generator.reverify --seed nz                    # seed = target country
+python -m generator.reverify --seed nz --exclude id1,id2  # additional claims
+```
+
+Every canon belongs to a bucket determined solely by hashing its **ID**; a seed
+hashes to one bucket and the cycle takes the oldest canons in it. Two seeds
+therefore own the same bucket (identical picks - obvious immediately) or share
+nothing at all. A partial overlap, the shape that conflicts on merge, cannot
+occur.
+
+Bucketing by ID rather than by position in the aging list is the point. Two
+cycles never see the same list - each branches from a different `main` - so
+positional blocks would shift their boundaries whenever a merge aged canons in
+or refreshed them, and two cycles would land on overlapping-but-unequal slices.
+A canon's ID does not move. This is why `DEFAULT_BUCKETS` must stay constant:
+changing it re-shuffles every canon, which is only safe with no content PR open.
+
+The command also excludes every canon touched by another pushed branch
+(`claimed_canon_ids`) and always says which state it is in - excluded N, clean,
+or **incomplete**. A scan that could not read every branch reports as such
+rather than as a clean bill of health. That scan is conservative by design: it
+does not try to tell a merged branch from a live one, because squash-merging
+severs ancestry and leaves no reliable local signal. It also only sees *pushed*
+branches, so re-run it before committing rather than trusting one check at the
+start.
+
+Re-verification means re-reading the sources, in this order:
+
+1. Open every `sources[]` URL and read it.
+2. Update moved URLs; fix the claim if it no longer holds.
+3. Only then bump `last_confirmed`, `verdict.last_updated`, and
+   `metadata.last_verification`.
+
+A refreshed date on a canon nobody re-read is worse than a stale one - it tells
+every downstream agent the entry was confirmed when it was not.
+
 ## Security
 
 When editing canon JSON files or templates, follow these rules:
@@ -261,5 +316,6 @@ Tests are in `tests/` using pytest. Key test files:
 - `test_build.py` - Site builder unit tests
 - `test_build_integration.py` - Integration tests for full site build
 - `test_pipeline.py` - Pipeline tests
+- `test_reverify.py` - Re-verification bucket selection (disjointness, determinism)
 
 Shared fixtures in `conftest.py`: `valid_canon` (deep copy of a valid canon) and `make_canon` (factory with overrides).