Skip to content

model-v2: certified bundle, engine port and guarded rehearsal - #24

Merged
vahid-ahmadi merged 336 commits into
mainfrom
model-v2
Oct 15, 2026
Merged

vahid-ahmadi merged 336 commits into
mainfrom
model-v2

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Ports the long-horizon engine to the released certified policyengine[uk]==6.2.6 bundle: UK 2.125.1, core 3.33.0 and Enhanced FRS 1.58.0. The released wheel, immutable HF manifest and dataset SHA-256 are verified. Publication, scenario and ageing entry points enforce certification and the behavioural Housing Benefit receipt-passport preflight. The historical 2.120.0 / 1.56.16 pilot remains uncertified, with retained fiscal values and original provenance authenticated by frozen source archives and privacy proofs.

The pre-Budget bridge and rehearsal are now complete and verified; not for quoting. Original calculation head is b31137ea055ab7e2b2ef7dd8b63df1302f5affdf. The committed importer reconstructs exact-cache fiscal/coverage tables through Part J's binders, verifies the dry C2 handoff, saved passports, core/coverage invariants and 20 Enhanced FRS + 2 Microcosm sample, and commits only permitted aggregate receipts. Both import and direct verification disable optional Git index writes in the original calculation checkout. Rehearsal fiscal values stay private. All linked gross/net/offset, geographic and HB subset/complement families pass the ten-record rule.

The central bridge holds data at 1.56.16 for the combined model/core/port step, then changes only data to 1.58.0. UK central net changes by −£0.064m/−£0.192m in 2034–35/2039–40 in the model step and +£4.942m/+£42.653m in the data step. Intermediate paired jobs were not planned, so paired attribution combines both steps. The certified UK 2039–40 paired net is £7.653775bn with total Monte Carlo SE £0.059125bn; these describe precision of the frozen forty-path model-conditional design. The bridge does not isolate #1927's fiscal or HB-offset effect.

Certified 2025–26 GB/DWP ratios: Pension Credit spending 1.0252, PC caseload 1.0459, pension-age HB 0.9985, total HB 0.6011. The data step improves PC caseload and pension-age HB fit; PC spending moves slightly farther from DWP. Later PC, total HB and State Pension coverage gaps remain. Released code supports the documented uk-data #490/#510/#513 mechanisms and #1927 receipt correction; certification does not settle calibration accuracy or the substantive coverage gate.

Completed wrapper walls are 106/318/312/431/171/1,862 seconds for C2/model bridge/certified bridge/core/coverage/sample. Safe per-job timing metadata contains 43 completed cold jobs and four warm rows. The committed planning method estimates post-Budget build plus validation at 12.32 hours after a C2 pass or 3.94 hours after a failure, using 3 Enhanced FRS + 1 Microcosm worker. Two Microcosm workers project 8.91 hours after a pass; concurrent memory headroom is unmeasured. Input preparation, C2 scoring, review, scheduling, cache reuse and contention remain outside these model-runtime estimates.

Part L head bf0e6ee8d6157a5b88c08f7982057450c1c34c3e adds the heavy-wrapper caller requirement and synchronizes the pilot-table generator with the completed bridge caveat. The shared historical footnote distinguishes the certified forty-path SE from an isolated receipt-correction SE. No table values or JSON receipts changed. The reboot salvage snapshot has the same tree as 4df5857; no unpushed edits need recovery.

Cache-only re-verification passes: all five regenerated committed artifacts match exactly, including both bridges, certified coverage, completion and runtime metadata; no model jobs were started. All 10 renderer tests pass.

Exact-head CI is green. Full Pipeline pytest completed 1653 passed, 1 skipped and only the 3 documented stale-results failures in 663.02 seconds. The completion gate verified all 1657 selected tests finished and accepted exactly those three failures. Dashboard passed default lint, all 95 tests, and build. The Vercel preview also passed. These manual Actions runs are bound to bf0e6ee8d6157a5b88c08f7982057450c1c34c3e; their direct commit checks are successful.

Independent subfleet run --task review --tier hard job 20261012-104314-review returned APPROVE, with no actionable findings. The reviewer inspected all 21 changed files across the complete b31137e..bf0e6ee range, including Part K's 0e02f26, both runner integrations, the completed Part J imports/write-up, and both Part L commits. All 364 targeted tests passed, and the reviewer independently reconstructed all five committed artifacts byte for byte, checked the receipt/privacy proofs and planning arithmetic, and confirmed the Fleet reservation and loaded C2 step. The normal clone provided shell and test access. No new full model job was launched by the review. Full corrected-head CI is green as recorded above.

Completed report, bridge and coverage tables, completion receipt, runtime receipt, certification and REBUILD retain the evidence and commands.

OPEN draft. After committing September CPI, published May–July earnings, the 28 October Budget/OBR inputs and newly complete forecast origins, run binding C2, then the certified fiscal build and ageing validation. A primary pass retains a model-conditional expected value beside the scenario envelope; a failure automatically retains scenarios only. Verify full Python/dashboard checks and obtain independent review before quoting any rebuilt figure. From 29 October, uk-triple-lock-c2 is a step inside the loaded launchd job com.maxghenis.triple-lock-2027.

The Fleet ops reservation triple-lock-rebuild-2026-10 runs from 29 October to 2 November. Mark reservation.py ready if inputs are ready early and release after rebuild and review. Every command through heavy_run.sh, lockrun.sh or heavy_queue.zsh requires HEAVY_CALLER=uk-triple-lock. Start with 3 Enhanced FRS + 1 Microcosm worker; the measured method gives the labelled 12.32/3.94-hour pass/failure planning estimates above. Rehearsal and historical pilot outputs remain pre-Budget and not for quoting. Nothing is merged and no comments are posted.

@vercel

vercel Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
uk-triple-lock Ready Ready Preview Oct 15, 2026 3:55pm UTC

Request Review

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Post-merge review of #23's ageing method, read against 0d042a3; the points that still apply carry over to model-v2.

  1. Re-typed records keep their old-system amount. pension_components splits each record's survey-year reported pension at the caps, so a basic record moved to the new pension carries its basic-era level. This probably understates the saving.
    • Evidence: "types" moves the 2039-40 bill only from £187.2bn to £189.7bn, and the GB gap to DWP widens from −£15bn (2024-25) to −£37bn (2030-31).
    • Could you add a level rule for re-typed cohorts, such as the transitional-rate starting amount (Pensions Act 2014 s.4), or at least a sensitivity run?
  2. Reweight vs frozen mixes two population paths. Frozen uses the engine's own uprated weights; reweight uses ONS 2024-based cells anchored at 2025. A run matching only the total population would isolate the age-structure effect.
  3. Base-year weights. 2025 is preserved exactly, but the 2024 data year is raked backwards (GB State Pension £116.1bn → £114.9bn). Could we rake only forwards from the anchor?
  4. Diagnostics: the largest-household and concentration diagnostics are suppressed in every ageing mode. Please check that the new ten-record measure (44a6dec) covers them.
  5. Geography: the country table is withheld, so the gate 2 comparison by geography is still outstanding.
  6. Minor: key the represented-age ordering on the dataset hash, not its name (demography.py:223).

Confirmed:

  • the cohort cutoffs (6 April 1951 for men, 6 April 1953 for women; s.1(2));
  • CPI uprating of the additional pension under both rules;
  • fixed household composition;
  • a rake that fails safely (bounds 0.2–5×, no silent relaxation).

The interaction arithmetic reproduces £0.158bn gross and £0.115bn net, and the pilot figures match docs/AGEING_PILOT_RESULTS.md and data/ageing_validation.json. Tests at 0d042a3 on the old engine: 585 passed, with only test_not_stale failing, as expected. No record-level data is committed. The pilot figures come from policyengine-uk 2.90.2, so they shouldn't be quoted until the re-pilot on the new engine.

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Thanks Max. I've reviewed b994a55 independently:

  • Tests: a fresh venv on 2.120.0 gives 656 passed, 3 failed. The failures are the expected staleness and percentile tests.
  • Dashboard: 77/77 pass, and the build succeeds.

#16, #17 and #18 are all in, with my commits and authorship intact. What moved (run_child into jobs.py, the bundle provenance into model, the type rule into part B's treatment) is a reasonable adaptation, and nothing is lost.

I couldn't find an engine defect that moves a number. These all check out:

  • State Pension age by date of birth;
  • the statutory triple lock, read back exactly;
  • gross to net from PE-UK's lists;
  • the GB masks.

Two things stop me approving, though:

  1. The description isn't the branch any more. The title and body say part A on 2.118.0 ("the pin stays at 2.118.0"). The head pins 2.120.0, has Model v2 C1: pre-register uncertainty screen, backtest five forms and gate full runs #22 and Model v2: age the pension population with anchored ONS inputs #23 merged in, and defaults to DEMOGRAPHY = "both". So the £0.66bn/£0.43bn bridge, the 8.12 → 8.13 draws and the APPROVE all refer to 432bb79/60d0aeb, not what this branch builds. Could you either rewrite the PR as A+B+C1 on 2.120.0 and rerun the bridge, or split the later work off?
  2. The pilot evidence is only on your machine. Please commit the aggregate bridge and the GB-vs-DWP tables (redacted, tagged with commit and model version), plus the review notes. Then María's gate 2 has something to attribute the rebuild against.

Smaller points:

  • Pension Credit take-up: the §2 sensitivity is missing. Your own table has GB reporters at 1.68m against DWP's 1.31m, so gate 1 needs a stated route (uk-data #510/#513) before part C.
  • Certification: running on an uncertified pair (data 1.56.16 built on 2.89.2) should get María's explicit agreement in Model v2: population ageing, PolicyEngine UK upgrade, uncertainty and the DWP comparison #14.
  • Core pin: the lock pins core 3.32.16, but a fresh install resolves 3.32.17. Please pin it in pyproject.toml.
  • Body details: the test count (263; it's now 656) and the coverage ranges (Pension Credit +28–48%, pension-age Housing Benefit +69–71%) need updating.
  • Identity check: gov_balance is the sum of its components, so the check at engine.py:475 can't fail. The float32 total check is the real one, and it's sound.

Happy to re-review once the body and evidence match the head. My separate comment above covers the ageing method from #23.

@MaxGhenis MaxGhenis changed the title model-v2 part A: engine on policyengine-uk 2.118.0, statutory triple lock, State Pension age by date of birth, gross to net from PE-UK's lists, GB coverage model-v2: parts A, B and C integrated: engine on policyengine-uk 2.120.0, static ageing, the C1 uncertainty screen, and the rebuild runbook Oct 5, 2026
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Thanks Max. I've re-reviewed 30a1be9. Tests in a fresh venv: 657 passed and only the three stale-results tests fail. The dashboard passes 77/77 and the build succeeds.

Fixed, thanks:

  • the title and body now describe A+B+C on 2.120.0;
  • weights through 2025 stay exactly the dataset's own, 2024 included, and it's property-tested;
  • the double-counted band between the flat rates, plus the rule for reports below State Pension age;
  • #2138, #2139 and uk-data#538 are filed and listed as limitations.

The pilot arithmetic checks out (2.739 gross and 1.867 net in 2039-40), and the −£24.5bn gap to DWP in 2030-31 fits B's reweight run.

Still open:

  1. Evidence. The 2.120.0 pilot tables are still only in the PR body and on your machine. Please commit the aggregates with commit and version tags, and mark AGEING_PILOT_RESULTS.md/ageing_validation.json as the 2.90.2 record.
  2. Re-typed level. The per-year split fixes the double count, but a basic-era record below the new flat rate still keeps its basic-era amount. Could we have a level rule (s.4 starting amount) or a sensitivity run?
  3. C1 adapter. build(run=True) still raises, so the runbook can't run end to end yet.
  4. Determinism. Please rerun at the head; f2ea89a and 30a1be9 change the rake fallback.
  5. Smaller points:
    • pin core 3.32.16 in pyproject.toml (a fresh install gets 3.32.17);
    • the gov_balance check at engine.py:469 still can't fail;
    • the first-phase SE still uses N=50,000;
    • the ±0.5pt mean paths are still gated;
    • key the age ordering on the dataset hash;
    • a total-only control for reweight vs frozen;
    • the country table;
    • María's sign-off on the uncertified pair in Model v2: population ageing, PolicyEngine UK upgrade, uncertainty and the DWP comparison #14;
    • drop the stale Part A text from the body.

vahid-ahmadi and others added 25 commits October 15, 2026 15:58
…ecile ticks 1-10

Example pensioners: pensioner buttons, three cards (State Pension under
each rule, income change with what made up the lower pension, share made
up from unrounded figures), and one panel switching between an item-by-
item waterfall and each year, with The numbers and How to read this as
joined footers. The by-group table moves into its chart's panel as The
numbers; income deciles show 1-10 with one axis label.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tral note, DWP first, neutral wording

The headline card shows the expected saving without the Monte Carlo error in
the value and notes that it is provisional until the Budget; the note on why
the OBR central forecast saves little sits under the yearly chart; the
comparison panel opens on DWP; scenario names and the earnings-shift note
come from the file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… household and triple-lock tabs

- The four example paths have the same names on every tab; the
  Distributional tab opens on the OBR central forecast and labels its
  figures 'on this path'.
- How the triple lock works comes second.
- 'Years the rules differ' becomes 'Pension below the triple lock'.
- Plain, neutral wording: no 'ratchet', 'losing' or code words; one
  phrasing for the Housing Benefit and council tax reduction caveat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…parison titles, 'points' not 'pp', plain past-forecast test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ll on phones

BoxTitle gives each chart or table box its title and one line saying what
it shows, and a test fails if a box heading lacks one. The site header
hides two nav links below the small breakpoint so the page fits 390px.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ets from the path's rows; no year-by-year table on Forecast paths

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the household figures are calculated; one header layout for every chart box

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rd; its notes as five bullets with the replayed-vs-paid list

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Root cause: the chart used ChartFrame's grow mode (left from its old
card layout), which set only a min-height, so the 100%-height chart
measured 0 tall inside the panel and drew nothing. ChartFrame now always
sets an explicit height, and a test clicks every view switch on every tab
and checks each chart frame has one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the line where it matters

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…wn; headline card with one detail line, the error on hover and a short Budget note

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ette test also blocks raw primary colours on data marks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-earlier box; the selected path's line moves into 'What each path is'

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ged)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pre-Budget certified release: rebuild on policyengine 6.2.6, model-conditional expected value, dashboard
model-v2 already strips single-record fields in the pipeline (redact_records)
and tests it. Take model-v2's StepPopulation, dataHelpers, trajectoryHelpers
and test_results; keep #26's redaction script, CI changes and its dashboard
guard test. pytest 1,729 passed; dashboard 152 passed; lint and build pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vahid-ahmadi and others added 2 commits October 15, 2026 17:18
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merge main into model-v2 (clears #24's conflict)

This branch was successfully deployed

1 active deployment
Preview — abcf5ea7 Deployed Oct 15, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants