Repository navigation
Binding rebuild on the October 2026 Budget inputs (scenarios only) - #49
Merged
Merged
Conversation
The central path starts in 2027, so every run kept the installed model's 2026 calendar CPI and earnings growth: policyengine-uk 2.151.0's earlier forecast, not the October 2026 EFO the rebuild uses to 2030 (d2124). Pavel's review of #45 flagged the mix. engine.base_year_growth() reads the central forecast file's single calendar-year 2026 row for each series. econ_changes puts it in the model on every path, moving RPI and CPIH with CPI as for later years. run_path refuses a job whose economic assumptions differ from the path's in any year from 2026 to 2039 by more than 1e-12, and records the 2026 growth, its source and the installed model's figure in path_following.calendar_base_year. The forecast file joins ENGINE_DATA, because jobs now read it, so a change to it changes every job's cache key. central.py, config.py and the CSV are unchanged, so the C2 handoff's fingerprints still hold. The HB allowance diagnostic reproduces the 2026 CPI each file's runs recorded. The pre-Budget file has no record, and its runs kept the model's own. Tests: the changes cover 2026, from the file's row; a scenario simulation on an invented situation reads every year back exactly; a missing or duplicated row refuses; every run in a post-Budget file carries the record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The coverage string called DWP's costing "one deterministic uprating path, not stated". DWP's note says it "applies a path for uprating" and publishes no rates. HL3774 (answered 15 October 2026) says it "did not use single year assumptions for earnings or prices growth". METHOD already says so. The string and the module docstring now match those sources, the results carry HL3774's link, and the docstring repeats DWP's caveat that its savings are long-term estimates, compared only in the years DWP publishes. A test pins the wording to the saved DWP text, and the compared years to 2039-40 and 2049-50. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#47 records gaining_pct and mean_gain_gbp at the same £1 threshold as the losers. The households card on the path tab now gives both. "No household is better off" covers an empty group. A file that records losers only, like the pre-Budget release, shows no line about gains. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Max ruled on 28 October (d2123 = A) that the plan is named as the government names it: the adjusted Triple Lock (Budget 2026, Box 5.A; DWP's factsheet of 6 October). The results file now writes the label "Adjusted Triple Lock (Budget 2026)" through pipeline.published_policies(). The id burnham_2030 stays, because it keys the results, handoffs and receipts. config.POLICIES keeps its old label, because the C2 handoff fingerprints config.py byte for byte. The modelling assumption and the trajectory notes that went into the file by the old name now use the new one. Job arguments strip labels and sources, so no job key changes. Tests: published_policies changes only the label. A post-Budget file names the plan the government's way. The provisional pre-Budget file keeps its old label. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d2123 adds the Budget's long-term saving and the OBR's costing as benchmarks; d955's fallback (a) means the binding file has no expected value. So: - benchmarks.csv: the DWP and Budget rows compare the central path's GB gross saving; the EFO medium-term row compares our April 2030 rate (both versions pay 2.5%, EFO fn 17); the EFO historical replay compares the replay's 2026 level ratio; the FRS 2026 row compares the OBR-wedge scenario. The Budget row quotes paragraph 1.29 (fn 61), not Box 5.A. - pipeline: a file without an expected value lists the benchmarks that need one under benchmarks_omitted, with the reason, instead of dropping them silently; the dashboard shows that list. - trajectories: the OBR-wedge label names the March 2026 long-term determinants (d2124). - data/raw: the Budget page JSON and EFO PDF, hashed in sources.md and checked by a test that the quoted words are in the saved page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the October file, calendar 2026 CPI is 3.1128%, within the test's 0.1pp tolerance of its 3.1% case, so the last assertion could no longer tell September CPI from lagged calendar CPI. Use 4.2% and assert the precondition so a future forecast file fails clearly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scenarios first (the explainer's ask, which it reports Max approved: "Yes, scenarios first"): - `triple-lock-build --without-mean-paths --out PATH` runs every job of the full build except the paired mean paths and writes an interim file labelled `scenarios_first`. It never writes the results file, its dashboard copy or a scenario file. - The full build reuses those jobs from the cache (same kind, args, engine semantics and packages). - Allowed only where the file has no expected value: ruling (a), or a binding (c) handoff whose primary failed C2. It refuses before any job otherwise, and refuses with --pre-budget. d2124: `provenance.forecast_vintages` reads from the central path's own sources which years come from the October 2026 EFO (calendar 2026-2030, April rises to 2031) and which from the March 2026 long-term determinants. The limitations say so, and the dashboard row names both. The Pension Credit guarantee's earnings proxy (April-June average earnings, as the OBR publishes no May-July forecast) is now a stated limitation and in METHOD. Its figures come from the central path, with a property test on when CPI sets the April 2028 rise. Fixes the AST protection broken in b16793a/aba2f17: - pipeline.py's module-level imports and assignments are fingerprinted for C2 correspondence; - the label map and the benchmarks import now live inside functions; - the frozen METHOD_LIMITATIONS text is restored, renamed only in method_limitations(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
households_affected() rounds each share to two decimals of a percent and gives an empty group a share and a mean of zero. A nonempty group whose share rounds to zero kept a positive mean gain, and the card said "No household is better off." The card now treats a group as empty only when both the share and the mean are zero, and writes "Fewer than 0.1%" when a nonempty group's share shows as 0.0%. A fast-check property pins the empty-group rule (Sol review r1, P2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…); vintage line (d2124) d2123: the dashboard, README, METHOD, REBUILD and RESULTS_SCHEMA call burnham_2030 the government's adjusted Triple Lock (Budget 2026, paragraph 5.23 and Box 5.A). The intro names Burnham once, for his 29 September speech and the earlier releases' name. getPolicyLabel maps an old file's "Burnham plan" label, and trajectoryHelpers reuses it. Tests: every tab names Burnham only in the intro; the label map; a Python check that the dashboard constant equals published_policies(). d2124 (in progress, untested): getForecastVintages/forecastInputsText and a ForecastInputs context. TabLayout provides it on the budget, paths and households tabs, and each Section prints one line naming the OBR forecast and long-term vintages. Tests still to write. pyproject.toml, engine.py, __init__.py, config.py and rules.py are untouched (cache key and C2 fingerprint). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…I ties Sol review delta2 (REQUEST_CHANGES) on f2df512..224666f: - P2, cli.py: --without-mean-paths refused only exact paths and direct children of data/scenarios. It now refuses anything under data/scenarios, and any destination that already has a second name (a hardlink to data/results.json would have been truncated in place). write_interim writes a new inode and renames it over the destination, so a link made during the build keeps the old contents. Its text is pipeline.write's (a differential test). - P3, MethodTab.jsx: the guarantee summary keeps its April 2028-2031 window and says both legs follow calendar-year earnings growth from April 2032 (central.statutory_source is "calendar" from 2031). The row shows only when the limitation carries those bounds. - The review's tie note: the CPI clause now uses rules.triple_lock_source, which labels a tie above the floor "cpi". Also tests for the d2124 vintage line: getForecastVintages examples and a fast-check property, every section of the budget, paths and households tabs shows it, and no section of the past-years or method tabs does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sol review of 224666f..ba22f3c (REQUEST_CHANGES): - P2: case aliases (data/RESULTS.JSON, DATA/results.json, data/SCENARIOS/nested/...) passed the interim guard on case-insensitive APFS, because resolve() keeps the spelling. interim_refusal now compares by file identity (os.path.samefile on each existing ancestor and on the destination, and casefolded names in the results file's directory). - P2: a symlinked entry under data/scenarios resolved outside it, and the rename replaced the entry. Any symlinked destination is now refused, and a symlinked parent counts as the directory it names. write_interim repeats the identity and symlink checks before the rename; it skips the hardlink count there, because the rename leaves other names alone. - P2: the vintage line now says it describes the central path, and that a scenario setting its own rates names their source. obr_premium pays the March 2026 uprating line from April 2028, and its label says so. - P3: the provisional pre-Budget file's benchmark prose says "Burnham plan". The binding build's benchmarks.csv does not. Two new tests: any results or interim file built after d2123 has no capitalised "Burnham" in any string, and no benchmarks.csv cell says it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
triple-lock-build --workers 1 --sensitivity-workers 1 --uncertainty-ruling c --uncertainty-handoff data/pilot/c2-binding --without-mean-paths --out data/interim/scenarios-first.json, built at 224666f from 11:44Z to 12:17Z on 29 October 2026. The bundle is policyengine 6.11.1 (UK 2.151.0, core 3.33.3), with Microcosm a6299504 as primary and Enhanced FRS 1.58.2 as the sensitivity. The binding C2 primary failed, so d955 falls back to (a): scenarios only, no expected value. The file holds the central path (October 2026 EFO to 2030, March 2026 long-term determinants after, d2124), both coverage runs, 4 forward paths, 4 past-year cases, the example households and the obr_premium scenario. It omits the paired mean paths. The full build reuses these jobs from the job cache. Not for quoting until the review of the figures approves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d2124) The OBR wedge paid the March 2026 determinants' 'Triple lock' row from April 2028. For the Aprils 2028 to 2031 that row is the March forecast's 2.5%, which the October 2026 outlook supersedes under d2124: the October path's September 2027 CPI sets April 2028 at 3.4%. So the scenario mixed a March forecast-period rate into the October window, and the adjusted Triple Lock anchored on a different 2029-30 level than on the central path. obr_premium_spec now specifies the triple lock only from the first April whose inputs the central path takes from the long-term determinants (April 2032, read from the path's statutory sources). Before it both rules pay the central path's rates; the adjusted Triple Lock pays them in every year. The label and source name the April and both vintages. The central spec's source still said "OBR March 2026 forecast to 2030"; it now names the October outlook. Job arguments exclude id, label and source, so only the obr_premium job reruns; trajectories.py is outside ENGINE_FILES and the C2 handoff's hashed inputs. tests/test_obr_wedge.py checks the definition on the rules alone, with a Hypothesis property over any statutory inputs and OBR row. The old definition fails all three of its checks. test_inputs_are_the_scenario_as _the_code_defines_it fails against the provisional pre-Budget scenario file until the build regenerates it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
27 tasks done
… a held directory, label the forecast from the file The review of 3fb05f9 (20261029-080330-pr49-review-rename-delta) asked for six changes. 1. P2. interim_refusal used os.path.abspath, which resolves '..' by spelling, but the OS follows a symbolic link before the '..' after it. `--out L/../results.json`, with L linking to data/scenarios, passed the guard and replaced data/results.json. destination() now resolves the directory with realpath and keeps the final name. A name that is '', '.' or '..' is refused. 2. P2. The paths and method tabs hard-coded "the OBR's March 2026 forecast", "OBR March 2026 Economic and fiscal outlook" and "August 2026 CPI". They now read the month from the central path's own source titles (obrVintageMonths), and the April 2027 inputs from central.path.april_2027_inputs (april2027InputsText). The binding file reads October 2026 and September 2026 CPI; the pre-Budget file still reads March 2026 and August 2026 CPI. 3. P3. write_interim opens the destination directory once (O_DIRECTORY, O_NOFOLLOW) and checks it again by identity. The check walks '..' from the handle to the scenario directory and compares the entry's inode and folded name with the results file and its copy. It then writes the temporary file and renames it through that handle (openat, renameat), so a path changed after the check cannot redirect the write. 4. P3. A case or Unicode alias of a protected file that doesn't exist, or whose directory doesn't exist, is refused by its folded resolved spelling (NFD and casefold). A test now covers it. 5. P3. A dashboard test holds the OBR wedge note to the scenario's own source, with the generic line only when there is none. 6. P3. --scenario and --mean-path-scenarios refuse the results file and its copy through the same identity-and-alias check (overwrite_refusal), not resolve() spellings. tests/test_scenarios_first.py adds a Hypothesis property. For any --out spelled through links, '..', '.' and case aliases, the write is refused or fails, or it writes a file outside the publication outputs, and every output and link is unchanged. About 55% of examples write and 28% are refused. Mutants check the new tests: - abspath in place of realpath fails 7 tests; - no handle check fails 1; - no folding fails 1; - the old resolve() guard for --scenario fails 2; - the hard-coded March labels and the generic wedge line fail 3 dashboard tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le build fixtures The wedge review (20261029-083250-pr49-review-wedge, APPROVE) had one P3. The central and OBR wedge sources hard-coded "October 2026", so a path on the March 2026 outlook (a historical replay) was misnamed. The wedge source also said April 2027 follows the October outlook, but published inputs set it. trajectories.vintage_months now reads both months from central["sources"] titles. The wedge source says April 2027 uses published inputs, the October outlook follows, and the OBR row applies from April 2032. A path whose sources name no month gets no invented one. Five tests in tests/test_uncertainty_adapter.py failed on main too (Pipeline CI at da346e3 and b4c56c2): - Two stubbed central_path with a dict that has no sources. Since the d2124 limitations (d186671), method_limitations reads them. The tests now stub it with the real central path, which runs no model. - Three built a synthetic history group without level_ratio. #45's OBR historical-replay benchmark (aba2f17) reads that field, and trajectories.history writes it. The fixture now has it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…one; name Microcosm as the binding data The interim figures review of 94d3bb6 (20261029-083208-interim-figures-review-r2) found that method_limitations still said "The expected value rests on one statistical model", although the scenarios-only file has none. - pipeline.method_limitations(omitted=) words the model's note for a file without an expected value: the simulated example paths rest on the model and carry no probability weight, because the model failed its frozen C2 screen ("c2") or the d955 ruling publishes scenarios only ("ruling"). build passes the reason from the ruling's provenance. A file with an expected value keeps the historical note unchanged. The note never names an expected value, so the dashboard lists it with the limitations, not in the box for the omitted estimate. - data/benchmarks.csv: the OBR comparison's note says the wedge pays the OBR row from April 2032 (d40796c); a test ties the year to forecast_vintages(). The binding file's data is Microcosm (d2011), but the dashboard still called it the Enhanced FRS and dropped every sentence naming Microcosm: - getPrimaryDataset reads the file's primary dataset; the inputs list, coverage table, coverage headline and population line name it ("Microcosm UK 2024-25 (microcosm-data 0.1.0)", "Microcosm UK, GB", "the data"). - withoutCrossCheck drops Microcosm sentences only when Microcosm is the cross-check, not the data. - The population line mentions the top code of 80 only when the primary's ageing record says represented top-coding applied (the Enhanced FRS); Microcosm keeps uncapped ages. - A LIMIT_ROWS line for the example-paths note, so the binding file never shows "The average and range rest on one model". Tests: method_limitations for both reasons and for a file with an expected value; the C2-failure and direct-(a) builds' notes; helper tests with a fast-check property for the sentence filter; rendered tests on a Microcosm file and the pre-Budget file, and every tab of the Microcosm file. Mutants (call site back to method_limitations(), no example-paths row, hard-coded Enhanced FRS header, always stripping Microcosm, unconditional top code) each fail at least one test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… failed stream, word vintages from the sources The review of d40796c..8b6b31b (20261029-090305-pr49-review-delta3) approved, with three P3s. 1. A refused interim write could leave new directories in data/scenarios. With work/ swapped for a link to the scenario directory after the path check, mkdir(parents=True) created work/new/nested there before the handle check refused. write_interim now opens the destination directory component by component through handles (_open_directory). It never follows a link (O_NOFOLLOW), and it creates a missing component only where the handle is outside the scenario directory. After the rename, a last check removes the file if the held directory moved into the scenario directory during the write. A directory moved there and back between that check and the rename stays beyond what a check can see, as the docstring says. 2. If os.fdopen raised, the raw descriptor stayed open. It is now closed before the temporary file is removed. 3. method_limitations and forecast_vintages named the October 2026 outlook and an older March set for any path. Both now take the months from the path's own sources (trajectories.vintage_months). They give the older-set reason and the rebuild sentence only when the long-term set is older than the outlook. On the binding sources both texts are word for word what they were (a test pins them). Mutants (no scenario check before mkdir, the old mkdir by path, no close on a failed fdopen, no check after the rename, the older-set clause always, a note for a single-vintage path) each fail at least one new test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-uk 2.151.0 takes it engine.poverty's docstring said policyengine-uk's relative measure used a household-weighted median. Installed 2.151.0's in_relative_poverty_ahc weights the median by household_weight times household_count_people: the median person's income, as in HBAI. The docstring now says so, and names the absolute line (poverty_threshold_ahc, a reference-year median held constant in real terms). The explainer session found the mismatch; the interim figures review had repeated it. Docstring only. engine_semantics() drops docstrings, so the job cache key is unchanged: the same hash before and after, and equal to the running build's (checked with engine_semantics()). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hard review of 8b6b31b..125bb99 approved with six P3 findings (~/reviews/uk-triple-lock-2026-09-29/release-night/pr49/review-sol-delta4.md). 1. A directory moved into data/scenarios before a mkdir could leave the directory it made there. _open_directory now records each directory it makes as (handle on its parent, name, identity), and _release removes them, deepest first, whenever the write is refused or fails: only while empty, and only while the name still holds the directory it made, so it removes no directory another process put in its place. 2. An exception in the post-rename ancestry check left the new file in place. The writer now removes the file and re-raises: it fails closed. The docstring names the remaining limit, a move after the last check. 3. The vintage wording treated any two different months as an older long-term set. forecast_vintages and method_limitations now compare the months in time (_month_order), so a March outlook with an October long-term set, or October 2026 with March 2027, gets no older-set reason and no rebuild promise. 4. A pre-Budget (a) build, which awaits the binding score, received the d955 "scenarios only" reason. It now keeps the note that release published, which the dashboard files under the expected saving to come. 5. The dashboard matched dataset ids by prefix, so unknown vintages took 2024-25 labels. getPrimaryDataset now matches the exact id before any "@Version"; an unknown id names no dataset and strips no sentence. An ageing record without the top-code flag claims no top code, even on the Enhanced FRS; only a file with no ageing record falls back to it. 6. The C2 panel hard-coded the dry run's six and twelve forecasts and its one-in-six published-earnings result. getC2Origins now reads the forecasts from c2_scores.origins and the hits from the published rows, so the binding file shows its own counts; a file that lists no forecasts names none. The review's coverage gaps: the Landing assumption and Method coverage call sites now render a Microcosm file without an expected value, so reverting either argument fails a test. A walk that fails to record a directory it made now closes the new handle too. Tests: the full suite (2422 passed) fails only the four stale-file assertions the build clears; the dashboard passes 233 tests, lint and build. A mutation run reverted each fix in turn (15 mutants), and a test caught each one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rename branch carries d2123's name, d2124's vintages, the OBR wedge fix and the answers to the delta reviews through delta5, which approved 1f0a18e (~/reviews/uk-triple-lock-2026-09-29/release-night/pr49/review-sol-delta5.md). Engine semantics are unchanged, so the running build's cache stays valid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fstat, locale-free months The hard review of 125bb99..1f0a18e approved, with one P3, three P4s and four notes (~/reviews/uk-triple-lock-2026-09-29/release-night/pr49/review-sol-delta5.md). 1. (P3) _release finds each directory the write made under its parent and name, so one that was itself moved or renamed stays behind, empty. 1f0a18e's docstring and commit message claimed otherwise. The docstrings of _release and write_interim now say what the cleanup reaches and what it misses. 2. (P4) _open_directory now reads a new directory's identity before it dups the parent handle, so a failed fstat leaves no descriptor open. 3. (P4) The _release docstring now names all three ways the identity check can remove another process's empty directory: a swap between the mkdir and the open after it, a swap between the check and the rmdir, and a file system that reuses a removed directory's identity. 4. (P4) The C2 panel says "only" only when the models missed some final years. A new test covers the guard that drops the published-earnings sentence when the rows scored different numbers of years; mutant D6 survived without it. 5. (Note) write_interim releases the directories it made even when closing its directory handle raises. 6. (Note) _month_order reads the English month names itself instead of through strptime, so a process-wide LC_TIME can no longer stop the results file's assembly after every job has run. 7. (Note) BindingOutcomeNotice keeps its prefix match. It shows only on the pre-Budget file, whose id it knows, and the binding release deletes it with the rest of #48's withdrawal code. 8. (Note) The build call site's comment now says the pre-Budget file keeps the historical note (METHOD_LIMITATIONS). 1f0a18e's message called that "the note that release published", but the published release is a provisional (c) file. Tests: the full suite (2431 passed) fails only the four stale-file assertions the build clears; the dashboard passes 235 tests, lint and build. A test catches each of the five new mutants (dup before fstat, release skipped on a failed close, strptime months, "only" always, no scored guard). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The delta6 review approved 393e6d1 with one P4 (~/reviews/uk-triple-lock-2026-09-29/release-night/pr49/review-sol-delta6.md): the docstrings still promised more than the cleanup does. - A directory made just before a failed open, fstat or dup goes unrecorded, so the cleanup never sees it and it stays, empty. The write_interim docstring now says so, and that the write tries to remove only the directories _open_directory recorded. - If unlinking the file after a failed post-rename check fails too, the file stays and that error propagates. The docstring now says so instead of promising the removal. Docstrings only: cli.py's syntax tree without docstrings is identical to 393e6d1's, and the writer's tests pass unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The binding file records the C2 outcome itself, so the page no longer needs withBindingOutcome, BindingOutcomeNotice or BINDING_C2_OUTCOME. Rendering the interim file showed what those had hidden: - The Budget tab and the Method tab printed the file's own reason, "failed the frozen C2 statutory screen; automatic d955(a) scenario-only fallback: A/suspended: |floor_bias| 0.267 exceeds 0.25", after "The original primary failed C2, so the build automatically takes (a)". - The Method tab's headline said coverage fell "short of the pass mark". Coverage passed; the floor-frequency bias failed (docs/C2_BINDING.md). getC2FailureText now words the failure from the file's own scores, and the headline states the counts without judging them. A failure the page cannot word gives only "The forecast model failed its test on past forecasts, so only scenarios are shown"; the page never prints the file's reason. Ten tests and one suite pinned the pre-Budget file and would have failed on the binding commit. They now run on a file of the right shape, or only on the pre-Budget file they describe. ScenariosOnly.test.jsx keeps the withdrawn page's checks on a failed-C2 file, and adds the wording's. The suite passes on the committed file, on data/interim/scenarios-first.json in its place, and in both fixture modes. This must land with the binding results: on the pre-Budget file the page would show the expected value again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… pre-Budget file
Running the suite with the interim file as data/results.json, its
mean_path_scenarios filled from the pre-Budget file, found what the
binding commit would have hit under strict CI:
- hb_allowance_diagnostic read its mean-path specs from
data/pilot/c2-dry-run, the pre-Budget draws. It now reads
data/pilot/c2-binding, whose specs match out/uncertainty-c2 byte for
byte. The test pins follow: the binding central path's crossings and
gaps, 160 paths in each group, and paired_baseline/draw_23404 in place
of draw_40124, which the binding draws do not hold.
- test_results had no population wording for Microcosm's
("ons_projection", "survey_year", "cohort"), the treatment demography
records when a dataset's ages need no adjusting.
- test_backtest_gap_rounds_like_the_fiscal_runs read the random path from
the results file; it now pins those four years' inputs itself.
- test_a_scenarios_only_file_says_why_it_has_no_paired_sample read a
release block, which the binding file doesn't carry.
docs/REBUILD.md now states Microcosm's population record, and
docs/HB_ALLOWANCE_REVIEW.md says which vintage its tables hold.
The binding build must regenerate data/diagnostics/pension_age_allowances
(scripts/diagnose_pension_age_allowances.py). Until then, on the
pre-Budget file, test_committed_artifacts_reproduce_exactly fails, so
this lands with the binding results. hb_allowance_diagnostic.py is not an
engine file, so no fiscal job reruns.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Method tab showed five limitations under the title "Other", with the decision ids d2011 and d1753, a named colleague's post, and the claim that Enhanced FRS 1.58.2 "is the paired sensitivity", which this build never ran. The limitation-row table now gives each a name, a short line and its upstream links, with a new "Data and model" group. The file's own text still shows whenever a figure a short line quotes has moved. The README and REBUILD described the pre-Budget release. They now state the binding outcome, the four scenarios and the build that reproduces them, and the README no longer quotes the expected-value sensitivity range or a top-code age treatment that Microcosm does not use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The full build ran under ruling (c) on the certified policyengine.py 6.11.1 bundle with Microcosm a6299504 and the binding C2 handoff. The primary failed the frozen screen on one check, A/suspended floor bias 0.267 against the limit of 0.25, so the build took the automatic fallback to (a) and wrote scenarios with no expected value. The 480 path jobs and interim jobs ran at 94d3bb6. This file's build ran at c441f42 with an unchanged job cache key, reused them all from the cache and ran only the OBR premium scenario cold, as its specification changed. The Housing Benefit diagnostic reads these results and the binding specs, so it follows them here. Checks on this file: check_assumptions, validate_ageing --central-only (11 jobs), the dashboard suite, lint and build, and the full pytest (one timing-sensitive interrupt test failed once and passed 15 reruns). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delta 7: - A form with no verdict no longer crashes the Budget tab: getC2Screen returns null, as it does for a missing score row. - getC2FailureText words only the suspended treatment, the one C2 scores. - The coverage note drops its code names on the page. - HB_ALLOWANCE_REVIEW gives the binding report's calendar-2026 CPI and the pre-Budget tables' separately. Delta 8: - The uprating row's needs guard its April date. - The donors row says a nonzero figure needs ten contributing records: disclosure.py publishes a zero with no contributors. - The C2 details call the binding score final and say the Budget's inputs were in. They no longer say it scored the Budget's figures: it scores past OBR forecasts against outturns. - REBUILD and the README take the binding handoff's 160 unique runs, so 491 Microcosm jobs, in place of the dry run's 161 and 494. - The scenario-heading and asymmetry tests, and the three damaged assumption-item cases, now run on the binding file. The explainer's read of the file (limitations 2, 5 and 13, C2_BINDING, the example-path labels): - The example-path labels name the size of the gap, not a place in the distribution. - The scenarios-only limitations say the build runs no paired sensitivity and give the file's reason for making no probability claim. METHOD's opening described the pre-Budget file as committed. It now describes the binding release, and the expected-saving section says only a passing build publishes that estimator. The frozen C2 section is untouched. pipeline.py and trajectories.py hold wording only and neither is an engine file, so the results need a re-assembly from cache and no model run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every job came from the cache: 491 Microcosm path and coverage jobs and the one Enhanced FRS coverage job, none run cold, at e43f598 with the job key unchanged. Against 26fbc89, 12 leaves in results.json changed: - wording: method_limitations 2, 5 and 13, the population assumption's text, and the two model paths' label and source; - provenance: generated_at, git_revision and the pipeline.py and trajectories.py source hashes. No figure moved. The scenario file, the diagnostic and the dashboard copy follow from those. check_assumptions passes and the dashboard suite, lint and build pass on this file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delta 9's full run failed one Hypothesis health check on a loaded host. No value assertion failed: generating inputs took over a second, and the check reads the wall clock. tests/conftest.py now loads a profile with no deadline and no too_slow check for the whole suite, and the one test that sets its own health checks keeps too_slow off too. A generator that sleeps 0.4 s fails without the profile and passes with it. A new test pins the profile. Two damaged-item cases protected nothing. The missing-text case damaged "paths", which a scenarios-only page leaves out anyway, and the non-object case checked only the known keys, so a blank "assumption-undefined" card passed. The damage now falls on a card every layout shows, and the check reads every card on the page. Delta 9's two uncaught mutations now each fail their own case. METHOD's run count described the dry run's handoff as the committed design. It now gives the binding handoff's 159 distinct indices, one check and 160 runs, and the dry run's 160 and 161 beside them. The frozen C2 section is unchanged. The Method tab's "Verdict" column repeats the model's overall result on every row, so a test set that passed could sit beside "Fail". It now reads "Model's overall verdict", with a test. Also retitle a test that still said "typical path". No file under src/ changes, so no figure, hash or job key moves and the committed results stand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The earnings scenario cards printed unique_full_runs as the number of paths averaged. That field counts engine runs: each distinct draw once, plus the identical-rates check. expected_value.estimates averages only design["sample"], so it leaves the check out and counts a repeated draw twice. The binding file's 159 distinct draws and one check gave the right number, 160, only by chance; the pre-Budget file's gave 161 against the same 160 draws. getScenarioHeadline now reads sample_slots, and the page says "draws", because only 159 distinct paths carry weight. The README's table and the schema's notes follow. No file under src/ or data/ changes, and the binding page prints the same number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Method tab called the longer forecast set "a wider check". Delta 10 noted that both sets count: ts_uncertainty.adequacy loops over tests A and B, and a failure in either fails the model. A failure in the longer set alone would have read as a side check. The line now names both sets and says a model must pass on both. No file under src/ or data/ changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
631d10f said "all 13 since June 2010", which is false: test B holds the complete spring forecasts (ts_backtest.complete_statutory_origins), one a year, and leaves out the other OBR forecasts of those years. The pass-mark sentence now says "one a year" and gives each set's first and last forecast. The failure sentence's "the OBR's forecasts from March 2016 to March 2022" implied the same, so it now counts them too. A new test checks that each set in the file holds one forecast a year; a second forecast in any year fails it. C2_BINDING says "spring". The schema note held only for this release: sampling can draw the check itself, and then the average uses it and it adds no run. The note now says so. No file under src/ or data/ changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README's release section and three lines of REBUILD still implied that the score used every OBR forecast in its period. The README now gives the same words as the page, and REBUILD and UNCERTAINTY_PILOT say spring origins, one a year. METHOD's frozen C2 section keeps its own text; line 498 there already says spring. Documents only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
marked this pull request as ready for review
October 31, 2026 01:55
Contributor
Author
Audit: state of #49 at 9d99213This comment records the gates at the head. The merge into main is Max's call, because Vahid asked on 28 October that agents merge only what someone has checked by hand. No agent will merge this PR without that go. Head: Independent reviews (GPT-6.1 Sol, hard). The PR description lists every round and what fixed each finding. The last five:
Delta 13 also searched the whole repository for statements that imply the binding score used every OBR forecast in a period, and found none left to change. Tests.
Results. After the merge. The Vercel deployment follows main, so the dashboard shows the binding release straight away. PolicyEngine/policyengine-app-v2#1493 updates the post, and it is pinned to this head. Max's decision on the post's wording is d2223. One point for the person who merges. The page does not print the unshifted average, but anyone can recover it from the file by subtracting |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the binding post-Budget rebuild under Max's rulings:
It runs under the
triple-lock-rebuild-2026-10booking, marked ready at 29 October 11:44Z: one Microcosm worker plus one Enhanced FRS sensitivity worker,HEAVY_CAP_MB=32768,HEAVY_CALLER=uk-triple-lock. The job cache key is the job's kind and arguments, the engine's semantics and the package versions. Changes outsideengine.ENGINE_FILES,ENGINE_DATAand pyproject need only a re-assembly from cache.Status (31 October, 02:30Z)
src/ordata/, sodata/results.jsonkeeps its blob. The primary failed C2 on one check, the A/suspended floor bias of 0.267 against the limit of 0.25, so the build took the automatic fallback to (a) and wrote scenarios with no expected value. Coverage passed, at 5 of 7 final years and 83% of annual checks.obr_premiumspecification, so that one job ran cold. e43f598 changed only wording, so every job came from the cache: against 26fbc89, 12 leaves changed, all wording or provenance, and no figure moved. The C2 fallback ran no Enhanced FRS sensitivity phase, so Enhanced FRS 1.58.2 ran only for the coverage check.check_assumptions, the dashboard suite (212 passed, 23 skipped), lint and build.validate_ageing --central-only(11 jobs) ran at c441f42; wording changes no draw or job. At 0b47c87 the full pytest run had 2,437 passed and 5 skipped, and CI passed. At d4e94d1 the full pytest run had 2,438 passed and 5 skipped, the pre-build checker passed, and CI passed. At e6afafb, which changes only the dashboard, the README and two docs, the dashboard suite passes in all four data modes (216 passed and 23 skipped on the committed file), with lint and build; 9d99213 then changes only documents. All checks pass at 9d99213, and the PR is MERGEABLE.data/interim/scenarios-first.json) stays in the repository as a record. It predates the OBR wedge change, so itsobr_premiumfigures are superseded.Reviews (GPT-6.1 Sol, hard)
origin/main...f2df512(r1)f2df512..224666f(delta2)--outbypassed the interim guard; P3 the guarantee summary lost its 2028–2031 bounds. r1's finding resolved224666f..ba22f3c(rename)obr_premium; P3 legacy "Burnham plan" prose in the provisional file's benchmarksba22f3c..3fb05f9(rename delta)..passed the guard (regression fromabspath); P2 the paths and method tabs hard-coded "March 2026" and "August 2026 CPI"; P3 a post-check race, an alias of a missing protected file untested, the wedge note untested,--scenario/--mean-path-scenarioscomparedresolve()spellings. All four earlier findings resolved3fb05f9..d40796c(OBR wedge)d40796c..8b6b31b(delta 3)data/scenarios; P3 a failedfdopenleaked its descriptor; P3 the vintage wording hard-coded October8b6b31b..125bb99(delta 4)data/scenariosbefore a mkdir left one there; an exception in the post-rename check kept the file; unequal vintages read as an older long-term set; pre-Budget (a) builds gave the d955 reason; the dashboard matched dataset ids by prefix; the C2 panel hard-coded the dry run's forecast counts. Interim r2 finding 1 resolved125bb99..1f0a18e(delta 5, with f322e58)fstatafterdupleaked a descriptor; P4 the docstring named one of three identity gaps; P4 the C2 panel said "only" when every year hit, and one guard had no test. Notes: a failing close skipped the cleanup;strptimedepended on the locale1c0c8d7..393e6d1(delta 6)8cc7bd8..c441f42(delta 7)tests/test_microcosm_primary.pypins the ids in the raw text); P3 the failure wording accepted thepublishedtreatment, which C2 does not score; P3 the HB review mixed the two vintages' 2026 CPIc441f42..26fbc89(delta 8, the results)needsdid not guard its April date; P3 the donors row left out the zero exception. Notes: "Scored on the Budget's figures" misdescribed the score; REBUILD counted the dry run's 161 unique runs; five tests that apply to the binding file skipped on it. All fixed26fbc89..0b47c87(delta 9)0b47c87..d4e94d1(delta 10)unique_full_runs, which counts engine runs, as the paths averaged; it matched the 160 draws only by chanced4e94d1..631d10f(delta 11)631d10f..e6afafb(delta 12)e6afafb..9d99213(delta 13, the head)Plan
engine.base_year_growth()reads the forecast file's calendar-year 2026 row, andrun_pathrefuses any year 2026–2039 the model holds differently from the path.triple-lock-build --without-mean-paths --out PATHwrites only an interim file. It refuses any destination that is, by file identity, the results file, its dashboard copy, anything underdata/scenarios, a symlink or a multiply linked file. It writes atomically.provenance.forecast_vintagesand the limitations name both vintages. Every section of the budget, paths and households tabs shows one line describing the central path's mix, and says that a scenario setting its own rates names their source. The Pension Credit guarantee proxy (April–June earnings for the April 2028–2031 rises) is disclosed in METHOD and the limitations.burnham_2030. The intro names Andy Burnham once, for his 29 September speech and the earlier releases' name. It quotes the Budget's rule (¶5.23 and Box 5.A), checked against the saved Budget page.test_uncertainty_adapterfailures that also fail main's Pipeline CI (8b6b31b). Two fixtures were stale:level_ratiothat Binding C2 re-score on October 2026 Budget inputs #45's OBR replay benchmark reads.rename-adjusted-triple-lock(1c0c8d7, then 393e6d1 and 8cc7bd8 after delta6 approved).test_inputs_are_the_scenario_as_the_code_defines_itfails against the provisionaldata/scenarios/obr_premium.jsonuntil the build regenerates it.--uncertainty-ruling c --uncertainty-handoff out/uncertainty-c2, 492 jobs: 491 on Microcosm and one on Enhanced FRS). Then ageing validation (--central-only),check_assumptions, the full test suite and the dashboard checks, and the regenerated HB allowance diagnostic.results.json:BINDING_C2_OUTCOME,withBindingOutcome/getBindingOutcome,BindingOutcomeNotice, thepage.jsxwrapper, thebindingbranches inMethodTab,BindingOutcome.test.jsx), keepingSCENARIO_PATH_NAMES/pathNames;Invariants
path_calendar(spec)in every year 2026–2039, to 1e-12, checked on every job. 2026 is the same on every path and equals the forecast file's calendar-year 2026 row.(kind, arg)jobs, so the interim figures are the final ones. The delta2 review checked this with mocked builds and the real spec builders.tests/test_published_label.py).rules.triple_lock_sourcelabels the April 2028 rise "cpi" (a Hypothesis property).tests/test_obr_wedge.py, a Hypothesis property).Not for quoting until this merges: this PR adds no figures to its description.
🤖 Generated with Claude Code