Repository navigation
Dashboard: withdraw the provisional expected saving after the binding C2 failure - #48
Merged
Merged
Conversation
… C2 failure The binding C2 score on the Budget's inputs (#45) failed, so ruling (c) falls back to (a): no expected value, scenarios only. Until the rebuild replaces it, the committed file is still the provisional pre-Budget (c) release, which carries the expected value. withBindingOutcome() presents that file as the fallback requires: it drops expected_value and records the (c) to (a) fallback the way the binding build does, so the page uses its existing scenarios-only rendering. Every other figure is the file's own. A dated notice says what the page showed, why it was withdrawn and what it shows now, and the method tab labels the scores it shows as the dry run's. Without an expected value the model's paths are scenarios, so they are named for their gap ("Model path: a large gap") with no typical or percentile claim. That also applies to the rebuild's scenarios-only file. Any other file is unchanged; delete the transform and notice with the rebuild. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…post's note The reason string said the model overpredicts "the 2.5% floor setting the rise". C2's floor frequency is the share of years with earnings growth strictly below 2.5% (ts_backtest.py, METHOD), distinct from the older both-inputs-below-floor diagnostic. It now says so, as the explainer post does. The notice no longer says the model failed "on the Budget's figures", which implied the Budget caused it. The rescore added a seventh past forecast once September's CPI completed the March 2022 window. The notice and method tab now say that and use the post's words. The second paragraph names what the pre-Budget release used (read from the file) and what the rebuild uses (engine.py's binding bundle and d2124's forecast mix). The "What each path is" list now uses the file's path names for the random path too. The misplaced isPreBudget docstring is back on its function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3 of 4 tasks
… expected value The Sol review at 588b696 found four gaps in the withdrawal (its fifth finding, the floor wording, was fixed in 101f3ba): - Distributional impact still described "the typical path" in the poverty card, and its footer pointed readers to an average over all paths. Without an expected value, neither appears now. - The "What each path is" list described the scenario paths by median, 90th percentile and percentile band. It now says they were picked for the size of their gap. - The transform changed a provisional sample. Samples now pass through untouched. - Path names required ruling (a) as well as no expected value. They now follow the expected value alone (`pathsAreScenarios`), whatever the ruling. The envelope rendering still uses `isScenariosOnly`. Each tab's whole text, collapsed <details> footers included, is now scanned for typical, percentile, median and average-over-paths wording. Only four phrases are allowed: the envelope's earnings scenarios, the past-years check score, and the two lines saying no path is typical and no average is shown. Each fix, reverted alone, fails a test. Two Steps tests now follow whether the file has an expected value, so c2-fail mode fails only where main already does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
11 tasks done
MaxGhenis
marked this pull request as ready for review
October 28, 2026 22:30
Contributor
Author
|
Merge audit for head
Merged with |
MaxGhenis
added a commit
that referenced
this pull request
Oct 30, 2026
…fstat, locale-free months The hard review of 125bb99..1f0a18e approved, with one P3, three P4s and four notes (~/reviews/uk-triple-lock-2026-09-29/release-night/pr49/review-sol-delta5.md). 1. (P3) _release finds each directory the write made under its parent and name, so one that was itself moved or renamed stays behind, empty. 1f0a18e's docstring and commit message claimed otherwise. The docstrings of _release and write_interim now say what the cleanup reaches and what it misses. 2. (P4) _open_directory now reads a new directory's identity before it dups the parent handle, so a failed fstat leaves no descriptor open. 3. (P4) The _release docstring now names all three ways the identity check can remove another process's empty directory: a swap between the mkdir and the open after it, a swap between the check and the rmdir, and a file system that reuses a removed directory's identity. 4. (P4) The C2 panel says "only" only when the models missed some final years. A new test covers the guard that drops the published-earnings sentence when the rows scored different numbers of years; mutant D6 survived without it. 5. (Note) write_interim releases the directories it made even when closing its directory handle raises. 6. (Note) _month_order reads the English month names itself instead of through strptime, so a process-wide LC_TIME can no longer stop the results file's assembly after every job has run. 7. (Note) BindingOutcomeNotice keeps its prefix match. It shows only on the pre-Budget file, whose id it knows, and the binding release deletes it with the rest of #48's withdrawal code. 8. (Note) The build call site's comment now says the pre-Budget file keeps the historical note (METHOD_LIMITATIONS). 1f0a18e's message called that "the note that release published", but the published release is a provisional (c) file. Tests: the full suite (2431 passed) fails only the four stale-file assertions the build clears; the dashboard passes 235 tests, lint and build. A test catches each of the five new mutants (dup before fstat, release skipped on a failed close, strptime months, "only" always, no scored guard). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Oct 30, 2026
The binding file records the C2 outcome itself, so the page no longer needs withBindingOutcome, BindingOutcomeNotice or BINDING_C2_OUTCOME. Rendering the interim file showed what those had hidden: - The Budget tab and the Method tab printed the file's own reason, "failed the frozen C2 statutory screen; automatic d955(a) scenario-only fallback: A/suspended: |floor_bias| 0.267 exceeds 0.25", after "The original primary failed C2, so the build automatically takes (a)". - The Method tab's headline said coverage fell "short of the pass mark". Coverage passed; the floor-frequency bias failed (docs/C2_BINDING.md). getC2FailureText now words the failure from the file's own scores, and the headline states the counts without judging them. A failure the page cannot word gives only "The forecast model failed its test on past forecasts, so only scenarios are shown"; the page never prints the file's reason. Ten tests and one suite pinned the pre-Budget file and would have failed on the binding commit. They now run on a file of the right shape, or only on the pre-Budget file they describe. ScenariosOnly.test.jsx keeps the withdrawn page's checks on a failed-C2 file, and adds the wording's. The suite passes on the committed file, on data/interim/scenarios-first.json in its place, and in both fixture modes. This must land with the binding results: on the pre-Budget file the page would show the expected value again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Max said go (d2146, 28 Oct): merge on the gates below. This changes a published page. The explainer post went live with the matching note at 20:52Z (PolicyEngine/policyengine-app-v2#1473, d2147), so until this merges the dashboard still shows the average the post has withdrawn.
The binding C2 score on post-Budget data (#45,
docs/C2_BINDING.md) failed: test A's floor-frequency bias is 0.267 against a limit of 0.25. The floor frequency is the share of years with earnings growth strictly below 2.5% (ts_backtest.py, METHOD); the model predicted 44.6% against 17.9% realised. Test A gained a seventh forecast (March 2022) once September's CPI completed its window; the dry run had six. Under Max's d955(c), the expected value then falls back automatically to (a): scenarios only, no expected value. The rebuild on the Budget's inputs, booked from 29 October, writes that file itself, but takes about a day of Microcosm compute (REBUILD.md's runtime paragraph). Until it lands, the live dashboard is the provisional pre-Budget (c) release. That release leads with the expected saving and its middle-80% range.The explainer post is getting the matching note in PolicyEngine/policyengine-app-v2#1473. This PR's notice uses the same facts, order and wording, except two deliberate differences: the notice quotes no withdrawn figure, and the model's paths stay as labelled illustrations, which d955(a) lists (the post drops them).
What changes on the page
A dated notice at the top: "Updated 28 October 2026". It says what the pre-Budget release showed, that the model failed a test we had registered before the Budget (linking Binding C2 re-score on October 2026 Budget inputs #45), and what was withdrawn. It adds what the pre-Budget release uses (forecast, PolicyEngine UK version and survey, read from the file) and what the rebuild uses: PolicyEngine UK 2.151.0, the Microcosm UK dataset (engine.py's binding bundle, d2011), and the OBR's October 2026 forecast to 2030 with its March 2026 long-term assumptions after (d2124).
No expected value anywhere.
withBindingOutcome()(dataHelpers.js) takes the provisional (c) file and:expected_value;uncertainty_ruling,expected_value_omission.reason);The page then uses its existing, tested scenarios-only rendering:
Path names. Without an expected value, the model's paths are scenarios (d955(a): "model paths labelled as scenarios"). They become:
"Typical", "high-saving" and the per-path percentile are gone, and the paths tab says these are illustrations, not a typical or likely path. The names follow the expected value alone (
pathsAreScenarios), whatever the ruling, so they also apply to the rebuild's scenarios-only file.No typical path or average anywhere. The Distributional impact tab's poverty card no longer gives "the typical path" as context, and its footer no longer points to an average over all paths. The "What each path is" list no longer places the scenario paths at the median, the 90th percentile or a percentile band.
Methodology tab:
Any other file, a sample included, passes through unchanged. Remove
BINDING_C2_OUTCOME,withBindingOutcomeandBindingOutcomeNoticewith the rebuild, whose file records the outcome itself.Checks
BindingOutcome.test.jsx(13 tests):<details>footers included, scanned for typical, percentile, median and average-over-paths wording. Only four phrases are allowed: the envelope's earnings scenarios, the past-years check score, and the two lines saying no path is typical and no average is shown;Steps.test.jsxnow follow the file's path names, and whether it has an expected value, instead of hard-coding "90th percentile" or the typical-path context. One new test checks that a file without an expected value names no typical path.bun run test: 188 passed, 4 skipped.TRIPLE_LOCK_RESULT_FIXTURE=c2-pass: 174 passed.TRIPLE_LOCK_RESULT_FIXTURE=c2-fail: 1 failure plus the ruling-(b) suite. Both already fail on main in that mode and are untouched here; they belong to the rebuild.20261028-172859-pr48-review-r2).bun run lintshows the same 2 warnings as main.bun run buildpasses.next startof this branch: the notice, the Budget-impact scenario envelope, the paths tab names and the methodology text. There were no console errors.Merge gates
🤖 Generated with Claude Code