Skip to content

Refuse malformed printed amounts and invented conditions in the AB2026 ingest - #253

Merged
MaxGhenis merged 2 commits into
mainfrom
uk/ab2026-ingest-guards
Nov 5, 2026
Merged

MaxGhenis merged 2 commits into
mainfrom
uk/ab2026-ingest-guards

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

We close two gaps that a cloud reviewer found in #247's guards at a28223f2, and a third that we found with mutants, as part of #230. We promised this PR in #247's description. We staged no amount in the forms we now refuse, so the ingest gives every claim the same id, value and conditions as on main.

  • Printed amounts (L2). _PRINTED_AMOUNT accepted "£0,123 million" and "£01 million", and a mutant that allowed "£1234,567 million" passed every test. The pattern now accepts a lone zero, digits with no leading zero, or groups of three digits after a lead of one to three digits. We add those three forms to the test that requires the ingest to refuse a reviewed twin that a producer prints in another form.
  • Invented conditions (L3). _keeps_conditions checked each staged key and ignored the keys the ingest adds, so a mutant that wrote housing_costs: AHC on every claim passed. The helper now allows a condition only when we staged it, or when the ingest canonicalises that condition or program and stores the text in a _verbatim field. It also allows the conditions in INGEST_CONDITIONS, the test's list of what _score, _verdict_conditions and with_baseline_condition write.
  • The static scoring method. We staged no row under exchequer_impact_static, so the ingest writes its static scoring_method on no claim. A new test sets that metric on an HM Treasury costings row and requires the ingest to add scoring_method: static, which the helper accepts.

Invariants

  1. _PRINTED_AMOUNT reads every whole number from 0 to 10^12 as that number, with or without thousands commas. Hypothesis draws the numbers.
  2. It refuses a single comma anywhere except the thousands place, and any amount with a leading zero. Hypothesis draws the numbers and the comma's position.
  3. For each of the 4,687 claims, the ingest writes only the conditions we staged on its row, the _verbatim texts of those it canonicalises, and the conditions in INGEST_CONDITIONS.

Checks

  • tests/test_uk_ab2026_ingest.py: 105 passed with two workers at 9123e32. We deselected test_ab2025_ingest_matches_main, because our sparse tree lacks its inputs, and CI runs it.
  • A local script that hashes every claim prints 4687 5952 4687 1265 9d62ee7ab0d90ced017a3f8159aeb2a91ada314262b87e7bb64278e5938a5311, the same line it prints at a28223f2.
  • We ran 25 mutants at acd5340a: seven of the pattern, three that make the ingest write a condition we never staged, two of the helper, and 13 that each drop one name from INGEST_CONDITIONS. A test killed 24. At 9123e32, the new test kills the 25th, which dropped scoring_method, and two mutants that change what the ingest writes for a static row.

🤖 Generated with Claude Code

…6 ingest

We answer L2 and L3 of the cloud review of #247 at a28223f. The ingest's
printed-amount pattern now refuses a comma after a lone zero, a leading zero
and a lead group of four digits. The preservation test now refuses a claim
condition that neither the staged row nor the ingest supplies. No staged row
prints those forms, and no claim moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-scorecard Ready Ready Preview Nov 5, 2026 9:21pm UTC

Request Review

We staged no row under exchequer_impact_static, so the ingest wrote its
static scoring_method on no claim, and a mutant that dropped it from the
test's list of added conditions survived. We now set that metric on an
HM Treasury costings row and require the ingest to add a static scoring
method that the preservation helper accepts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis MaxGhenis closed this Nov 5, 2026
@MaxGhenis MaxGhenis reopened this Nov 5, 2026
@MaxGhenis
MaxGhenis marked this pull request as ready for review November 5, 2026 23:08
@MaxGhenis
MaxGhenis merged commit 81a358d into main Nov 5, 2026
28 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

We merged 9123e324 with a merge commit, under the merge-your-own-PR gates.

  • Independent review. GPT-6.1 Sol approved 9123e324 with three low findings, all on test coverage, and we'll answer them in the follow-up PR where we stage the OBR's paragraph 4.30 and Table 6.3 rows. If we change the ingest to set scoring_method to "static" on every claim without one, _keeps_conditions and the new test still pass (L1). The static test tries only a row with no scoring method (L2), and the decimal tests generate at most three places (L3). Sol replayed both patterns over all 5,952 staged rows and found the same matches, and when it restored the old pattern, the ingest wrote the same 4,687 claims and 1,265 drops. In its read-only sandbox, Sol ran 102 of the file's 106 tests in one process, and they passed.
  • CI. After Pair Autumn Budget 2026 claims with the dashboard's Budget-day runs (tranche 3 of #230) #249 merged, we closed and reopened this PR so that CI would run against the new main. gh pr checks then returned 0 at this head, with all 15 checks listed, and GitHub reported the merge state as clean.
  • Scope. In this PR we tightened the ingest's pattern for printed amounts, and we now test that the ingest writes no condition beyond the staged ones and its own list. We changed no staged row, no file under app/, and no site or lane data.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

This branch was successfully deployed

1 active deployment
Preview — 9123e324 Deployed Nov 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant