Repository navigation
Dashboard: state what the headline assumes (#9 A1-A3) - #12
Conversation
…A1-A3) Under the Budget impact cards, three conditions the figures rest on, each read from results.json: the pensioner population and pension types held fixed to 2039-40 (A3); one model of prices and earnings, with the gross range across every reweighting of the same runs (A2); and survey Pension Credit and pension-age Housing Benefit baselines above DWP's, with the net saving on the Microcosm dataset (A1). A test checks each number against the file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Program reviewBase repository: PolicyEngine/uk-triple-lock Source documentsThe DWP State Pension uprating analysis was checked live for GB coverage and its dynamic population projection. Two OBR workbooks were verified against publisher bytes. The OBR long-term and DWP benchmark workbooks were inspected as committed copies without publisher-byte authentication. CriticalNone. Should AddressA1 — SHOULD ADDRESS — Do not present uncertain reweightings as a precise gross range (OPEN)Location: Coordinator assessment: Independent code tracing confirms the helper correctly computes extrema over all 12 rows; the problem is the explanation of those rows and omission of the high endpoint's £5.184bn sampling SE. The new UI test checks the numeric extrema but not their interpretation. A2 — SHOULD ADDRESS — Date, geography and paired sample qualify the benefit comparison (OPEN)Location: Coordinator assessment: Code review confirmed the displayed coverage values are the 2026–27 primary Enhanced FRS rows; Microcosm baseline rows sit below the cited DWP benchmarks. The new test does not assert the period, geography or paired-path context. SuggestionsS1 — Coverage row validation couples the visible primary comparison to an unused field (MINOR) (OPEN)Location: Trigger and observed output: Expected / impact: Validate fields needed by the consumer and guard the row container with Reproduction: Bounded Node helper invocation in Evidence Gaps
Notes
Validation SummaryAt head 76885b7, GitHub Dashboard, Pipeline and Vercel checks: SUCCESS. Read-only Node helper diagnostics reproduced the displayed values and current-data rendering; no model or results JSON changed. Bounded local React test attempt failed before collection with Timingsetup seconds: 183.00s; scope seconds: 19.00s; parallel review seconds: 546.00s; policy role seconds: 426.00s; code role seconds: 524.00s; adjudication seconds: 0.00s; consolidation cleanup seconds: 210.00s; elapsed seconds: 917.00s Review SeverityCOMMENT. Open findings: 0 critical, 2 should address, 1 suggestions. |
The tab holds the population and distributional results as well as the example pensioners (María, Slack). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Budget impact: the households-losing card says plainly what its numbers are (expected share in 2039-40; the 10th-90th percentile span across paths). - Forecast paths: the drawn paths' first line now explains, from the file's selection fields, how each was picked. 50,000 simulated paths; the middle and 90th-percentile views show one real path, chosen among the draws whose 2039-40 gap lies within 2.5 points of that percentile as the one whose statutory CPI and earnings are closest to those draws' average (trajectories.py select_draws); the expected saving rests on the full runs sampled across all paths. Falls back to the file's sentence if fields are missing. - Distributional impact: income breakdown by decile only (default), label "Income decile"; quintile option removed. - Assumptions strip: the reweighting item says these are point estimates of reweighting the same full runs (gap variance, lead switching, in some versions the floor share) and gives the highest one's effective runs and standard error; the benefits item names the coverage year and UK vs GB, and gives the paired Microcosm minus Enhanced FRS net difference; the population item says headcount moves only through weights and SPA. - getCoverageRow guards the rows array and needs only dwp and primary; getSensitivityRange returns the highest row's effective runs and SE. - Tab "Method" renamed "Methodology" (id unchanged), with references. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lier page Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Incremental re-review at
|
… other model is tested Answers María's A1 in full: the strip no longer reads as a bounded range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks! All addressed in f65e57f, 1883242 and 10e25c2:
Also from your Slack notes:
The modelling parts of #9 (benefit calibration, another macro model or mean path, ageing) are still open. |
Re-review at
|
… fiscal range; the path pick's weight filter From María's re-review of PR #12: the range excludes other CPI/earnings models rather than none being tested, and the percentile path is picked among candidates with at least their median weight (all of them, as every path has the same weight). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, both fixed in the latest commit:
|
Program reviewBase repository: PolicyEngine/uk-triple-lock Source Documents
CriticalNone. Should AddressA1 — SHOULD ADDRESS — Do not present uncertain reweightings as a precise gross range (RESOLVED)Location: Coordinator assessment: Independent code tracing confirms the helper correctly computes extrema over all 12 rows; the problem is the explanation of those rows and omission of the high endpoint's £5.184bn sampling SE. The new UI test checks the numeric extrema but not their interpretation. Coordinator assessment: At 10e25c2, the strip identifies gap variance, lead switches and floor-frequency variants; reports separately estimated endpoints with each SE; and states the upper endpoint rests on about 44 effective full runs. The original missing-target/precision problem is resolved. A distinct new false statement about other models is recorded below. A2 — SHOULD ADDRESS — Date, geography and paired sample qualify the benefit comparison (RESOLVED)Location: Coordinator assessment: Code review confirmed the displayed coverage values are the 2026–27 primary Enhanced FRS rows; Microcosm baseline rows sit below the cited DWP benchmarks. The new test does not assert the period, geography or paired-path context. Coordinator assessment: At 10e25c2, the strip labels the 2026–27 UK-model versus DWP-GB baseline and reports the £0.613bn paired Microcosm net difference (SE £0.089bn) across the same 40 paths, rather than presenting an unmatched total. A3 — Limit the “no other model” claim to the sensitivity range (RESOLVED)P2; PR12 head Coordinator assessment: Code adjudication confirmed that Coordinator assessment: At 963cc60, LandingTab confines the statement to the fiscal sensitivity range and explicitly describes the Student-t and Gaussian backtests. Statutory backtest code and committed results include both variants; only the boot variant receives full fiscal runs. SuggestionsS1 — Coverage row validation couples the visible primary comparison to an unused field (MINOR) (RESOLVED)Location: Trigger and observed output: Expected / impact: Validate fields needed by the consumer and guard the row container with Reproduction: Bounded Node helper invocation in Coordinator assessment: At 10e25c2, S2 — Path-selection explanation omits its weight eligibility filter (MINOR) (RESOLVED)Location: Coordinator assessment: At 963cc60, trajectoryHelpers explains the median candidate-weight threshold. The selector implements it, and the published primary paths have equal weights, so all 2,500 band candidates remain eligible. S3 — Update the PR description's superseded model claim (OPEN)P3 documentation suggestion, PR body A2 table. The coordinator's live Evidence Gaps
Notes
Validation SummaryAt head 963cc60, GitHub Dashboard lint/test/build, Pipeline tests/committed-output checks, and Vercel succeeded. Scoped read-only review traced the three-file delta through statutory backtests, fiscal-path gate and committed results. Five cached source items passed SHA-256 integrity. Local React tests/build and browser rendering were NOT RUN in the isolated snapshot (node_modules/Vitest unavailable); prior full review validation and its gaps are inherited. No model/results data changed. Timingsetup seconds: 98.00s; scope seconds: 16.00s; parallel review seconds: 137.00s; policy role seconds: 121.00s; code role seconds: 60.00s; adjudication seconds: 8.00s; consolidation cleanup seconds: 13.00s; elapsed seconds: 245.00s Review SeverityCOMMENT. Open findings: 0 critical, 0 should address, 1 suggestions. |
Answers the presentation side of María's review in #9. The review's point is that the headline figures are conditional, so the dashboard now says on what, next to the headline and with the numbers.
What changes
Under the four Budget impact cards, a "What these figures assume" strip gives three conditions. Every number comes from
results.json.A2's point about the path-sampling standard error was already covered by #11. The chart's explanation says the average line's ±£0.1bn is "how precisely the full runs pin down the average, not how much the saving could vary", and the headline shows the spread across paths.
A new test checks each number in the strip against the file.
Also, from María's Slack notes:
getCoverageRownow checks the rows are a list and needs onlydwpandprimary(review S1).Still open in #9 (need model work, not in this PR)
sandboxlacksallow-scripts. That file exists only on thepaperbranch, so the fix belongs in Working paper: Public Pension Indexation Under Uncertainty: Fiscal and Distributional Effects (/paper route) #8.Refs #9.
Checks
bunx vitest run: 63 passed.bun run build: succeeds.🤖 Generated with Claude Code