Skip to content

Sort the AB2025 registry's name searches - #251

Merged
MaxGhenis merged 4 commits into
mainfrom
uk/ab2025-name-search-sort
Nov 5, 2026
Merged

MaxGhenis merged 4 commits into
mainfrom
uk/ab2025-name-search-sort

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

pipeline/build_ab2025_registry.py now sorts the variables each name search matches before it prints their count and the first six. policyengine-core adds variables in glob order, which differs by filesystem, so --check failed wherever that order differed from the one that wrote data/uk/ab2025_measures.json. In our Linux build container, main's builder fails --check and the new builder passes it. We held the sort out of #247, and #230 tracks it.

The builder now takes its name-search text from pipeline/name_search.py, a pure function of the pattern, the parameter paths, the variable names and the engine pin, so CI tests it without policyengine-uk installed. A Hypothesis test in tests/test_name_search.py checks that every permutation of the variable list gives the same text. A second test pins the sorted first six, and both tests fail when we remove the sorted.

We regenerated the registry with policyengine-uk 2.89.2 and policyengine-core 3.34.5, and the new file matches 640bdd3f's copy byte for byte. The builder rewrites the name searches of 37 of its 161 measures: it names a different six variables for 19, reorders the six for 18, and prints every count unchanged. --check and validate_budget_2026_registry.py --resolve both pass, with 83 parameter paths resolved.

We rebuilt data/scorecard.db with scorecard_db.build_db and reran scorecard_db.export_populations with main's built date. The export rewrote the annotation text of 212 AB2025 pe_gap rows in data/populations.json and its copy in app/public/data. In 62 of those rows the search now names a different six variables, and in the other 150 it reorders them. No value, verdict, count, key or row order moves, and data/populations-pmd/ and data/lanes.json stay as main has them.

tests/fixtures/uk_replay_ab2025_hashes.json now pins the registry's new sha256, a37e8a73c5f0664bb7ade2a90df841ed058f57357a61e179aecd4c62485f228b, for tests/test_uk_replay_legacy.py. We searched the repository for the old sha256, its 8- and 12-character prefixes and the file's git blob id, and found no other pin, in a replay seal, a receipt or anywhere else.

uvx ruff format --check be nz scorecard_db tests pipeline passes, and we ran the full pytest tests/ suite with CI's dependencies on four workers: 3,114 tests pass, 17 skip, and one fails. That test fails because the Modal factory refuses any interpreter but Python 3.12, and we ran the suite on 3.13; it fails the same way on main under 3.13 and passes here under 3.12. .github/scripts/uk_replay_regenerate.sh verifies every replay seal and regenerates every namespace unchanged, and the suite and the export leave the tree clean.

🤖 Generated with Claude Code

policyengine-core adds variables in glob order, which differs by
filesystem, so build_ab2025_registry.py --check failed on a filesystem
whose order differed from the one that wrote the registry. ns() now sorts
the matching variables. Regenerated at policyengine-uk 2.89.2 with
policyengine-core 3.34.5, the registry matches 640bdd3's copy byte for
byte, and --check and --resolve pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6JAsQ5xLsK6aNQd6E8
scorecard_db.export_populations, run against a fresh build_db at main's
built date, reorders the variable lists in 212 AB2025 pe_gap rows'
annotations. 62 of those rows now name a different six variables; the
printed totals, values, verdicts, keys and row order stay the same.
data/populations-pmd/ does not move. app/public/data carries the same copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6JAsQ5xLsK6aNQd6E8
tests/test_uk_replay_legacy.py compares the registry with this fixture,
so the regenerated registry's sha256 replaces main's. No replay seal or
receipt pins these bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6JAsQ5xLsK6aNQd6E8
The builder calls policyengine-uk at import, so ns() now delegates to
pipeline/name_search.py, which takes the parameter paths and variable
names as arguments. A Hypothesis test checks that every permutation of
the variable list gives the same text, and a second test pins the sorted
first six. Both fail with the sort removed. The builder still reproduces
the registry byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6JAsQ5xLsK6aNQd6E8
@vercel

vercel Bot commented Nov 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-scorecard Ready Ready Preview Nov 5, 2026 1:02pm UTC

Request Review

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review

Head: 47e842920934e6fbd19cd31f313a24d53763e091, four commits on uk/ab2025-name-search-sort, draft.

We built the registry in a venv with policyengine-uk 2.89.2 and policyengine-core 3.34.5. The builder pins only policyengine-uk, at 2.89.2, through engine_dumps and the default replay variant. That variant records policyengine-core 3.27.1, and 640bdd3f used 3.34.5. The variable names come from policyengine-uk, so the two cores give the same registry, and ours matches 640bdd3f's copy byte for byte. Main's unsorted builder fails --check in this container, and the sorted one passes --check and --resolve.

Our row-diff script compares each regenerated file with main's copy and asserts that only the variable list inside a name search moves. It reports:

  • data/populations.json: 5,896 rows, 212 changed, all pe_gap rows for autumn_budget_2025, by producer: HM Treasury 97, Resolution Foundation 48, CenTax 43, OBR 13, SMF 4, IPPR 2, JRF 2, Demos 1, IEA 1, Tax Policy Associates 1.
  • In each changed row, one annotation in latest and its twin in results move. That gives 424 name-search occurrences: 124 name a different six variables and 300 only reorder them.
  • 62 rows name a different six variables, and 150 only reorder them. Every printed total, value, verdict, key and the row order stay the same.
  • In the registry, 37 of 161 measures change their name search: 19 name a different six and 18 reorder them.
  • app/public/data/populations.json matches data/populations.json. data/populations-pmd/ didn't move, and we restored data/lanes.json, whose export changed only its updated date.

The grep found main's sha256 (08b8dbb1…) only in tests/fixtures/uk_replay_ab2025_hashes.json, which we re-pinned to a37e8a73…. Neither its 8- and 12-character prefixes nor the old git blob id (15fc971e…) appear anywhere else. No seal or receipt in results/uk, docs/uk_replay, tests/fixtures or the workflows pins the registry's bytes. results/uk/events/LEGACY_VERIFICATION.json names the fixture but records no registry hash.

Test results:

  • uvx ruff format --check be nz scorecard_db tests pipeline passes.
  • pytest tests/ with CI's dependencies on four xdist workers under Python 3.13 gives 3,114 passed, 17 skipped and 1 failed. test_modal_uk_event.py::test_modal_factory_has_one_blocked_network_worker_and_explicit_uv_venv fails because modal_function refuses any interpreter but 3.12. It fails the same way on main under 3.13 and passes on this branch under 3.12.
  • tests/test_name_search.py passes. With the sorted removed from pipeline/name_search.py, both of its tests fail, and with the sort restored, the builder still reproduces the registry byte for byte.
  • .github/scripts/uk_replay_regenerate.sh verifies every seal and regenerates every namespace unchanged, and the tree stays clean after the suite.

We didn't run CI's build_db determinism pair (two builds and a hash comparison), the replay-variants matrix or baseline-probe.yml. CI on this PR runs them.

We found three things outside this change and left them alone:

  • results/uk/events/DETERMINISM_VERIFICATION.json, DETERMINISM_CURRENT_REGISTRY.json and REVIEW_FIXES.json hash pipeline/uk_engine_registry.py, and none of them holds main's current sha256 for that file. We put the pure function in a new module, pipeline/name_search.py, so uk_engine_registry.py, which modal_uk_event.py ships into the replay image, keeps main's bytes.
  • scorecard_db.export_populations writes today's date into built and into data/lanes.json's updated, so a plain rerun moves both. We passed main's built date to export() and restored lanes.json.
  • pipeline/build_uk_event_registry.py already sorts its variable matches, so the event registries didn't need this change.

Generated by Claude Code

@MaxGhenis
MaxGhenis marked this pull request as ready for review November 5, 2026 13:21
@MaxGhenis
MaxGhenis merged commit 011d4b3 into main Nov 5, 2026
16 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged at 47e84292 with a merge commit (011d4b3f).

  • GPT-6.1 Sol approved this head (job 20261105-080757-rev-251-47e84292). It re-ran the row diff and confirmed the merge order against Ingest the Autumn Budget 2026 harvest (tranche 2 of #230) #247's differential. It listed three low gaps in my local row-diff script, which didn't touch this PR, and a footer point that doesn't apply, because our attribution line requires that footer.
  • CI at this head: gh pr checks exits 0 with every check passing, and GitHub reported MERGEABLE and CLEAN.
  • I checked the feed and registry locally: 5,896 rows, 212 changed only in annotation text, and the registry's sha256 matches 640bdd3's copy byte for byte.

We land #231, #247, #246 and #249 after this, each with main merged in, so #247's differential compares against the sorted registry.

@MaxGhenis
MaxGhenis deleted the uk/ab2025-name-search-sort branch November 5, 2026 13:21
MaxGhenis added a commit that referenced this pull request Nov 5, 2026
#251 sorted AB2025's name search on main. We merge it here so #247 can take it
and still show only tranche 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Nov 5, 2026
This takes #231's final head and main, with #251, which sorted AB2025's name
search.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 47e84292 Deployed Nov 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants