Skip to content

Specified rates per policy (#14) and the OBR-wedge scenario run, published reproducibly (#13) - #17

Closed
vahid-ahmadi wants to merge 10 commits into
mainfrom
scenario/specified-rates
Closed

vahid-ahmadi wants to merge 10 commits into
mainfrom
scenario/specified-rates

Conversation

@vahid-ahmadi

@vahid-ahmadi vahid-ahmadi commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Refs #13, #14

Draft only because tests/test_results.py::test_not_stale has to go red (see "Why draft" below). Everything else passes.

The mechanism (#14 section 2)

A path's spec may now carry specified_rates, {policy: {April: rate}}. In those years that policy pays the given rate instead of its rule (rules.rates_matrix(..., specified=), rules.uprating_path(..., specified=), engine.spec_specified). This is the engine's one scenario mechanism. It merges the approach from scenario/obr-premium (commits 3235f65 and 48da03d, cherry-picked onto main after #10) and replaces that branch's triple-lock-only triple_lock_rates field.

  • Before the switch. Both rules are the triple lock until April 2029. A specified triple lock is therefore also what the Burnham plan pays in those years, and the plan anchors its earnings path on the level that produces.
  • Burnham-plan rates. A specified rate for burnham_2030 is allowed from the switch only (an earlier one raises ValueError). It replaces the plan's own rule that year, and later years run the rule from the level it leaves.
  • Rounding. Specified rates are rounded with rules.round_rate like every other rate, and not floored: they are inputs, not a rule, so negative rates are allowed. An unknown policy, a year outside the path, or a rate that is NaN or infinite raises an error (in the rules and in engine.spec_specified).
  • Labels. rate_sources labels a specified year specified for that policy. Before the switch the plan stays triple_lock. The dashboard's describeSource reads specified as "a specified rate", so a specified run would never show null, and docs/RESULTS_SCHEMA.md documents the label.
  • No specified rates. Every rate is then exactly what main's rules gave. Two differential tests check this: one against a frozen copy of the rules before specified rates (hypothesis), one against every path run's recorded rates in main's results.json.
  • Running scenarios. trajectories.SCENARIOS names the scenarios. Every full build runs every scenario after the paths, inside the build's start/end check, and writes each to data/scenarios/NAME.json (pipeline.build returns them with the results), so one rebuild refreshes results.json and the scenarios together. triple-lock-build --scenario NAME [--out PATH] reruns one alone (pipeline.scenario). Each file carries the main results' provenance, and pipeline.redact_records removes record-level fields. The single rerun refuses to write the results file or its dashboard copy, and refuses a dirty tree unless given --allow-dirty; --out without --scenario is an error. Scenario files are build outputs like results.json: an uncommitted data/scenarios/*.json does not make the tree dirty. tests/test_scenarios.py::test_not_stale compares a scenario's source, input and engine hashes with today's, and test_fixed_inputs_are_the_central_runs checks it holds the central run's fixed inputs (a key only later engines record, such as Assumptions strip reads an assumptions block from the results (#14 §5) #18's population, is compared once both files have it).

The OBR wedge (#13)

obr_premium is the central path with the triple lock paying the 'Triple lock' row of the OBR's long-term economic determinants (March 2026 EFO). Each April from 2028 gets the row's value for the fiscal year of the inputs. April 2027 is set by published inputs (May-July 2026 AWE), so it follows them as on the central path (3.9%), not the OBR row's 3.7%. For 2033-34 onwards that row is "average earnings growth plus 0.6 percentage points" (OBR's note). This is a stylised average-rate assumption for how often CPI or the floor beats earnings, not a CPI and earnings path the triple lock could produce. Only the triple lock is specified. The Burnham plan runs its own rule on the central path's CPI and earnings.

£bn, nominal, UK 2034-35 gross 2034-35 net 2039-40 gross 2039-40 net
Central path (main) 0.54 0.36 0.66 0.43
OBR wedge (this PR) 2.59 1.73 7.84 5.10
Expected value (main, primary) 2.59 1.68 8.41 5.39
DWP (Pensim3, GB) 15

The first run in this PR, with April 2027 at the OBR row's 3.7%, reproduced the earlier scenario/obr-premium run exactly (2039-40 £7.83bn / £5.06bn; 2034-35 £2.585bn / £1.722bn). So #10's rounding changes and the generalisation kept the semantics. Paying April 2027 on the published inputs instead (3.9%) raises gross by 0.19% in both years (2039-40 £7.829bn to £7.844bn; 2034-35 £2.585bn to £2.590bn). Net rises 0.25% in 2034-35 (£1.722bn to £1.726bn) and 0.8% in 2039-40 (£5.063bn to £5.104bn). Most of the extra 2039-40 net change is a smaller Housing Benefit offset (£0.062bn to £0.034bn). Housing Benefit here moves with a few heavily weighted records near Pension Credit eligibility (method limitations), so a small level shift can move it this much; I have not traced which records.

Other figures in the OBR-wedge run:

  • Net saving, 2039-40. The £5.10bn net figure is the £7.84bn gross saving less:
    • £0.94bn more Pension Credit;
    • £1.71bn less income tax;
    • £0.03bn more Housing Benefit;
    • small amounts from the other components.
  • Households losing. 20.2% in both years, losing £263 a year on average in 2034-35 and £766 in 2039-40.
  • Weekly new State Pension, 2039-40. £375.09 under the triple lock against £358.45 under the plan (2034-35: £306.82 against £301.22).
  • Checks. Every path-following check passes (same tolerances as main). The largest single record contributes −£0.11bn (2.2%) of the 2039-40 net change.

Exact rate inputs

These are the specified_rates the file records, from April 2028: the OBR row, unrounded, by April paid. The rate paid is that value rounded to 0.1 point, and the plan's rates come from its own rule.

April OBR row (fiscal year of inputs) Triple lock pays Burnham plan pays (what set it)
2027 not specified (OBR row 0.037 unused) 3.9% (published inputs: earnings) 3.9% (triple lock)
2028 0.025 2.5% 2.5% (triple lock)
2029 0.025 2.5% 2.5% (triple lock)
2030 0.025 2.5% 2.5% (floor)
2031 0.025 2.5% 2.5% (floor)
2032 0.029927 3.0% 2.5% (floor)
2033 0.034855 3.5% 2.8% (earnings path)
2034 0.039782 4.0% 3.3% (earnings path)
2035 0.040195 4.0% 3.5% (earnings path)
2036 0.040609 4.1% 3.5% (earnings path)
2037 0.041022 4.1% 3.5% (earnings path)
2038 0.041435 4.1% 3.6% (earnings path)
2039 0.041848 4.2% 3.6% (earnings path)

Reproduce

The command needs a clean tree (or --allow-dirty), with the Enhanced FRS 2024-25 (sha256 e433e532b17bd8ce76030156285816e33d44e93edabd2204adbef71d19a68712, certified build policyengine-uk-data-1.56.16) in the worker directory. Without Hugging Face credentials, copy it to .cache/workers/efrs0/data/enhanced_frs_2024_25.h5. The run takes about 40 seconds.

PYTHONPATH=src .venv/bin/python -m triple_lock.cli --scenario obr_premium   # = triple-lock-build --scenario obr_premium

Versions

From the file's provenance:

  • policyengine 5.3.0 (bundle uk-5.3.0), policyengine-uk 2.90.2, policyengine-core 3.30.1;
  • dataset enhanced_frs_2024_25 @ 1.56.16;
  • numpy 2.5.3, pandas 3.0.6, scipy 1.18.1, microdf-python 1.3.10.

The file also records the git revision (f3a5a88), the source, input and engine hashes, and the full redacted run (rates, sources, weekly amounts, savings with components, totals, poverty, households affected, distribution, path-following checks).

tests/test_scenarios.py checks the committed file against the code:

  • its specified rates are what SCENARIOS["obr_premium"] gives today, and everything else is the central path;
  • both rules' rates and labels recompute from its inputs;
  • it passes every check a path run in results.json passes (reusing test_results' run checks);
  • it publishes no survey record id, weight or amount (test_no_survey_record_is_published applied to the file).

Caveats

  • Premium on the triple lock only. The plan gets no credit for the volatility that sets its own floor: it runs on the smooth central path, where the OBR's premium stands in for that volatility under the triple lock alone. This is a comparison with the OBR's costing convention, not a forecast.
  • Static survey bill. No ageing beyond the survey weights, as on every path here.
  • UK, not GB. DWP's £15bn is for Great Britain, from a dynamic population model.
  • The wedge opens only from April 2032. For the Aprils 2028 to 2031 the OBR row is 2.5%, as the central path is.
  • The premium is 0.56 points, not 0.6. Against the fiscal-year earnings growth the central path uses, the row is 0.56 points above (OBR's note says 0.6).
  • Provenance after a squash merge. The file's git_revision (f3a5a88) is a commit on this branch. Please merge with a merge commit, or rerun the scenario after merging (40 s, from the cache). docs/RESULTS_SCHEMA.md now says a revision may be a branch commit, and the hashes are what tie an output to the code.

Why draft: test_not_stale is red

test_not_stale compares the raw byte hashes of every src/triple_lock/*.py with those data/results.json recorded, so any edit under src turns it red. Only a rebuild clears it. rules.py, engine.py and config.py are engine files whose code changed, so the cached jobs in .cache/jobs no longer match: the next full build reruns every job (about 250 model runs), not a replay from the cache. With no specified rates every rate is unchanged (tested), so a rebuild should reproduce main's figures exactly. I have not run it; data/results.json and the dashboard copy are untouched.

Tests

  • Python, PYTHONPATH=src .venv/bin/python -m pytest -q: 201 passed, 2 failed.
  • Dashboard, bun run test: 64 passed, including the new describeSource test.
  • tests/test_scenarios.py::test_not_stale passes: the committed file's provenance was refreshed at the head of the code (f3a5a88) from the cached model job of 75ee7c9, whose engine files are unchanged since (no model rerun; figures identical).

🤖 Generated with Claude Code

vahid-ahmadi and others added 6 commits October 1, 2026 15:26
…run as a single path

rules.rates_matrix/uprating_path take optional given triple lock rates (the plan
follows them before the switch); engine.spec_rates reads spec['triple_lock_rates'];
trajectories.obr_premium_spec builds the scenario; triple-lock-build --obr-premium OUT
runs it alone and writes the redacted run to OUT. Defaults unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A path's spec may carry specified_rates, {policy: {April: rate}}, paid instead
of that policy's rule in those years. This replaces the triple-lock-only field
the OBR-premium branch added. A specified triple lock before the switch is also
what the Burnham plan pays, and the plan anchors on the level it gives; a
Burnham-plan rate (from the switch only) replaces the plan's own rule that
year. Specified rates are rounded with round_rate; rate_sources labels them
"specified". With none, every rate is the rules' own (tested).

triple-lock-build --scenario NAME runs one scenario (trajectories.SCENARIOS)
and writes it to data/scenarios/NAME.json with the main results' provenance,
records redacted by pipeline.redact_records. The dashboard names "specified";
METHOD.md describes scenario runs and the OBR wedge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… its inputs (#13)

data/scenarios/obr_premium.json: one full run of the central path with the
triple lock paying the OBR long-term determinants' 'Triple lock' row, from
triple-lock-build --scenario obr_premium at f9a56d9 (clean tree, Enhanced FRS
2024-25 1.56.16, policyengine 5.3.0, policyengine-uk 2.90.2). Saving in
2039-40 £7.83bn gross, £5.06bn net; 2034-35 £2.59bn and £1.72bn.

tests/test_scenarios.py checks the committed run against the code: its
specified rates are the scenario's today, both rules' rates and labels
recompute from them, it passes every path run's checks from test_results, and
it publishes no survey record id, weight or amount.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… tree dirty

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…or this revision

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
uk-triple-lock Ready Ready Preview Oct 1, 2026 3:27pm UTC

Request Review

vahid-ahmadi and others added 2 commits October 1, 2026 16:02
…scenarios fail

- obr_premium specifies the OBR 'Triple lock' row from April 2028 only: April
  2027 is set by published inputs (May-July 2026 AWE), as on the central path.
- Specified rates must be finite (NaN or infinity refused by the rules and by
  engine.spec_specified); negative rates stay allowed.
- tests/test_scenarios.py::test_not_stale: a scenario run goes stale on any
  source, input or engine change, like the results file; README and METHOD say
  to rerun every scenario with every full rebuild.
- The no-specified-rates test is now differential: against a frozen copy of
  the rules before specified rates, and against every run in results.json.
- RESULTS_SCHEMA: git_revision may be a branch commit; the hashes tie an
  output to the code.
- Build outputs are data/scenarios/*.json (narrower exclusion, named in the
  dirty-tree message); --out without --scenario is an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2039-40 £7.84bn gross, £5.10bn net (was £7.83bn, £5.06bn); 2034-35 £2.59bn,
£1.73bn (was £2.59bn, £1.72bn).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vahid-ahmadi and others added 2 commits October 1, 2026 16:25
…n and data/scenarios

pipeline.build runs every registered scenario after the paths (the same cached
engine jobs as --scenario) inside the build's start/end check and returns them
with the results; triple-lock-build writes each to data/scenarios/NAME.json.
--scenario NAME still reruns one alone. A test checks that a scenario run holds
the central run's fixed inputs (a key only later engines record, such as
population, is compared once both files have it); stubbed tests check the
build writes every scenario and what the scenario jobs and records hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…el run; figures unchanged)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor Author

Folded into model-v2 (#24) with all commits and authorship kept, and checked independently: nothing missing or reverted. Closing in favour of #24. The branch stays, so this can be reopened if model-v2 doesn't land.

This branch was successfully deployed

1 active deployment
Preview — 8a788b5e Deployed Oct 1, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant