Skip to content

Assumptions strip reads an assumptions block from the results (#14 §5) - #18

Closed
vahid-ahmadi wants to merge 4 commits into
mainfrom
assumptions-block
Closed

vahid-ahmadi wants to merge 4 commits into
mainfrom
assumptions-block

Conversation

@vahid-ahmadi

@vahid-ahmadi vahid-ahmadi commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Refs #14 (section 5, "Dashboard strip"), #12.

What and why

The Budget impact tab's "What these figures assume" strip (#12) worded its three items in the dashboard. Max's point in #14: "Otherwise merging model-v2 would leave 'today's pensioners, held fixed' live and wrong." This PR moves the wording into the model, so the strip changes when the model does.

  • Engine. Each path run records how it treats the population in fixed_inputs.population, as {weights, ages, pension_types}. This comes from engine.population_treatment.
    • Weights and ages are declared by the code that loads the data. _managed attaches engine.LOADED_POPULATION ({"weights": "survey", "ages": "survey_year"}), and a simulation without a declaration fails.
    • Why a declaration, not a comparison with stored survey values: the dataset is multi-year (2024-2030, weights uprated by year), and §3.1's rake is anchored to the base year. A rake or redraw done inside the dataset changes the very values a comparison would use, so nothing is left to compare against. The code that makes such a change has to declare it, for example "ons_projection" or "adjusted".
    • Checks on the declaration: pinned weights fail a "survey" declaration. Ages the run pins, or that the model moves between years, fail a "survey_year" declaration; ages that move between years are recorded as "aged_forward".
    • Pension types: recorded as the rule pinned_inputs follows (PENSION_TYPE_RULE = "survey_year"), or "model" if no types are pinned. pinned_inputs raises on a rule it does not implement, so a cohort rule (§3.3) has to add its own branch and name. The earlier pinned-versus-survey comparison could never fail, so I removed it. held_pension_types still checks that the model used the pinned types.
    • I checked this on the unreformed simulation with the local Enhanced FRS 2024-25 file, using the declaration _managed sets: it gives today's survey / survey_year / survey_year.
  • Pipeline (pipeline.assumptions): builds [{key, title, text, facts}] and build() writes it as results["assumptions"].
    • Today's treatment keeps today's wording exactly. Other treatments get wording per axis, using §3's vocabulary: raked to the ONS population projection, types by cohort, ages adjusted or aged forward.
    • The central run and every trajectory must record the same treatment, because the strip describes all of them.
    • The build fails (MissingFigure) if a record is missing, a value has no wording, or a figure the text quotes is missing.
  • Dashboard (LandingTab.jsx, dataHelpers.getAssumptions): renders the block's valid items. It computes the strip itself only for a file with no assumptions key, such as today's.

The block

"assumptions": [ { "key": "population" | "paths" | "benefits", "title", "text", "facts": {...} } ]
  • population: weights, ages, pension_types, final_year, data_year, the final year's state_pension_age (a list) and the year it settles, and max_age. When ages and types are both from the survey year, the build also fails if the held type counts move once the State Pension age stops changing.
  • paths: the lowest and highest final-year gross saving across the reweightings to past dynamics (shift_dynamics*), with SEs and the highest's effective runs. Other sensitivity families, such as §4's mean-path sensitivity, are listed in other_sensitivities and counted in the text as left out of the range; they don't fail the build.
    • These parts of the prose come from the results: the calibration sentence comes from expected_value.primary, and the floor figure comes from TRIPLE_LOCK_FLOOR. The "in some versions" clause depends on how many reweightings include the floor.
    • The models tested but not run come from trajectories.models. A model with no short name is named by its own label.
  • benefits: the coverage year; Pension Credit claims and pension-age Housing Benefit for the survey and DWP; UK against Great Britain; and the paired sensitivity-dataset net difference (mean, SE, paths, dataset name).

This is documented in docs/RESULTS_SCHEMA.md, including fixed_inputs.population.

Wording parity: with today's population record, the block's text is byte-identical to what the dashboard computes now. I re-checked this with a one-off render, not committed.

Pilot checklist (#14 §5)

assumptions() must be on the §5 pilot checklist. Run PYTHONPATH=src python scripts/check_assumptions.py <pilot results.json> on the pilot's results: in seconds it prints each item's title, or the reason the block can't be written. That way a gap shows before the 8-16 hour rebuild, not at its end. A test covers the script.

Fallback

  • No assumptions key (today's file): the dashboard computes the strip as it does now.
  • A block is present: it renders only the valid items, meaning a non-empty key, title and text, and the first item of each key. If no item is valid, it shows no strip. A damaged block never brings back the computed "held fixed" wording. Test ids are unchanged.

Why draft

tests/test_results.py::test_not_stale fails, and only on the engine.py and pipeline.py source hashes. engine.py is in the job cache key, so this lands with the model-v2 rebuild, like #16 and #17. The committed data/results.json has neither the block nor fixed_inputs.population; I have not rebuilt it or edited it by hand. Until the rebuild, the dashboard shows the computed strip, which reads the same as today.

The same claim elsewhere (not fixed here)

#18 alone does not remove the stale-text risk. The same "held fixed" claim is hard-coded in three other places:

  • METHOD_LIMITATIONS in src/triple_lock/pipeline.py (lines 73-75: "The survey is not aged forward … State Pension type … held at its survey-year value");
  • dashboard/src/components/SummaryTab.jsx (lines 163-164: "survey here is not aged: ages stay at their survey values … each pensioner keeps the …");
  • dashboard/src/components/MethodTab.jsx (line 197: "each person keeps their survey-year State Pension type").

These need the same treatment, or an update in the model-v2 PR. That is a follow-up issue.

Tests

  • Python: 191 passed, 1 skipped, and test_not_stale fails as described above.
    • test_engine_pure.py, on stubs:
      • a simulation without a declaration fails;
      • a declared ONS rake or redrawn ages is recorded as declared;
      • ages that move are recorded as aged_forward;
      • pinned weights, or pinned ages that differ from the survey's, contradict a survey declaration and fail;
      • unpinned types are recorded as model;
      • pinned_inputs raises on an unimplemented type rule.
    • test_results.py:
      • the facts equal the figures elsewhere in the results;
      • weights alone, types alone, ages alone, and all of §3 together each move the item off "held fixed";
      • a trajectory that records a different treatment, or none, fails the build;
      • a §4-style extra sensitivity leaves the range unchanged, and the text says one sensitivity is left out;
      • a model with no short name is named by its label;
      • the build fails on a missing record, an unknown value, or a missing paired_difference, trajectories.models, strata or max_age;
      • the pilot check script reports the block or the reason it can't be written;
      • the integer-year-key round trip works, and a "block is current" test is skipped until the rebuild.
  • Dashboard: vitest 72 passed. eslint: 0 errors, with 2 warnings that were already there. bun run build succeeds.

Follow-up

The strip lays its items out in three columns (md:grid-cols-3). If model-v2 writes a fourth item, the layout will need a look.

🤖 Generated with Claude Code

The pipeline writes `assumptions` ([{key, title, text, facts}]) from the
model's configuration and the assembled results, so the Budget impact tab's
"What these figures assume" strip changes when the model does. The population
item's wording follows pipeline.POPULATION, which the build checks against the
central run's held State Pension type counts. The dashboard renders the block
when it is valid and otherwise computes the strip as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
uk-triple-lock Ready Ready Preview Oct 1, 2026 3:30pm UTC

Request Review

…build's runs carry integer ones

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…peline constant

The engine records how a run treats the population in fixed_inputs.population
(weights, ages, pension types), read from what it pins and the model's ages,
and the pipeline words the item from it, with #14 §3's vocabulary for ONS
reweighting and cohort types; a missing record, a value without wording or a
missing figure fails the build. The dashboard computes the strip only for a
file with no block; a damaged block shows its valid items or nothing. The
paths item's calibration, floor and shock-model wording come from the results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… by the dataset load

The dataset load (_managed) declares what it did to the survey's weights and
ages (engine.LOADED_POPULATION), since a change made inside the dataset leaves
nothing to compare against; population_treatment fails without a declaration
and checks it against pinned weights and the model's ages. Pension types are
recorded from the rule pinned_inputs follows (PENSION_TYPE_RULE), replacing a
comparison that could not fail. The block fails unless the central run and
every trajectory record the same treatment. Sensitivities outside the
reweightings to past dynamics, and models without a short name, are worded
generically; scripts/check_assumptions.py checks a pilot's results in seconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor Author

Folded into model-v2 (#24) with all commits and authorship kept, and checked independently: nothing missing or reverted. Closing in favour of #24. The branch stays, so this can be reopened if model-v2 doesn't land.

This branch was successfully deployed

1 active deployment
Preview — 891265fc Deployed Oct 1, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant