Repository navigation
Specified rates per policy (#14) and the OBR-wedge scenario run, published reproducibly (#13) - #17
Closed
vahid-ahmadi wants to merge 10 commits into
Closed
vahid-ahmadi wants to merge 10 commits into
vahid-ahmadi wants to merge 10 commits into
Conversation
…run as a single path rules.rates_matrix/uprating_path take optional given triple lock rates (the plan follows them before the switch); engine.spec_rates reads spec['triple_lock_rates']; trajectories.obr_premium_spec builds the scenario; triple-lock-build --obr-premium OUT runs it alone and writes the redacted run to OUT. Defaults unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A path's spec may carry specified_rates, {policy: {April: rate}}, paid instead
of that policy's rule in those years. This replaces the triple-lock-only field
the OBR-premium branch added. A specified triple lock before the switch is also
what the Burnham plan pays, and the plan anchors on the level it gives; a
Burnham-plan rate (from the switch only) replaces the plan's own rule that
year. Specified rates are rounded with round_rate; rate_sources labels them
"specified". With none, every rate is the rules' own (tested).
triple-lock-build --scenario NAME runs one scenario (trajectories.SCENARIOS)
and writes it to data/scenarios/NAME.json with the main results' provenance,
records redacted by pipeline.redact_records. The dashboard names "specified";
METHOD.md describes scenario runs and the OBR wedge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… its inputs (#13) data/scenarios/obr_premium.json: one full run of the central path with the triple lock paying the OBR long-term determinants' 'Triple lock' row, from triple-lock-build --scenario obr_premium at f9a56d9 (clean tree, Enhanced FRS 2024-25 1.56.16, policyengine 5.3.0, policyengine-uk 2.90.2). Saving in 2039-40 £7.83bn gross, £5.06bn net; 2034-35 £2.59bn and £1.72bn. tests/test_scenarios.py checks the committed run against the code: its specified rates are the scenario's today, both rules' rates and labels recompute from them, it passes every path run's checks from test_results, and it publishes no survey record id, weight or amount. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… tree dirty Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…or this revision Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…scenarios fail - obr_premium specifies the OBR 'Triple lock' row from April 2028 only: April 2027 is set by published inputs (May-July 2026 AWE), as on the central path. - Specified rates must be finite (NaN or infinity refused by the rules and by engine.spec_specified); negative rates stay allowed. - tests/test_scenarios.py::test_not_stale: a scenario run goes stale on any source, input or engine change, like the results file; README and METHOD say to rerun every scenario with every full rebuild. - The no-specified-rates test is now differential: against a frozen copy of the rules before specified rates, and against every run in results.json. - RESULTS_SCHEMA: git_revision may be a branch commit; the hashes tie an output to the code. - Build outputs are data/scenarios/*.json (narrower exclusion, named in the dirty-tree message); --out without --scenario is an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2039-40 £7.84bn gross, £5.10bn net (was £7.83bn, £5.06bn); 2034-35 £2.59bn, £1.73bn (was £2.59bn, £1.72bn). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n and data/scenarios pipeline.build runs every registered scenario after the paths (the same cached engine jobs as --scenario) inside the build's start/end check and returns them with the results; triple-lock-build writes each to data/scenarios/NAME.json. --scenario NAME still reruns one alone. A test checks that a scenario run holds the central run's fixed inputs (a key only later engines record, such as population, is compared once both files have it); stubbed tests check the build writes every scenario and what the scenario jobs and records hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…el run; figures unchanged) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #13, #14
Draft only because
tests/test_results.py::test_not_stalehas to go red (see "Why draft" below). Everything else passes.The mechanism (#14 section 2)
A path's spec may now carry
specified_rates,{policy: {April: rate}}. In those years that policy pays the given rate instead of its rule (rules.rates_matrix(..., specified=),rules.uprating_path(..., specified=),engine.spec_specified). This is the engine's one scenario mechanism. It merges the approach fromscenario/obr-premium(commits 3235f65 and 48da03d, cherry-picked onto main after #10) and replaces that branch's triple-lock-onlytriple_lock_ratesfield.burnham_2030is allowed from the switch only (an earlier one raisesValueError). It replaces the plan's own rule that year, and later years run the rule from the level it leaves.rules.round_ratelike every other rate, and not floored: they are inputs, not a rule, so negative rates are allowed. An unknown policy, a year outside the path, or a rate that is NaN or infinite raises an error (in the rules and inengine.spec_specified).rate_sourceslabels a specified yearspecifiedfor that policy. Before the switch the plan staystriple_lock. The dashboard'sdescribeSourcereadsspecifiedas "a specified rate", so a specified run would never show null, anddocs/RESULTS_SCHEMA.mddocuments the label.results.json.trajectories.SCENARIOSnames the scenarios. Every full build runs every scenario after the paths, inside the build's start/end check, and writes each todata/scenarios/NAME.json(pipeline.buildreturns them with the results), so one rebuild refreshesresults.jsonand the scenarios together.triple-lock-build --scenario NAME [--out PATH]reruns one alone (pipeline.scenario). Each file carries the main results' provenance, andpipeline.redact_recordsremoves record-level fields. The single rerun refuses to write the results file or its dashboard copy, and refuses a dirty tree unless given--allow-dirty;--outwithout--scenariois an error. Scenario files are build outputs likeresults.json: an uncommitteddata/scenarios/*.jsondoes not make the tree dirty.tests/test_scenarios.py::test_not_stalecompares a scenario's source, input and engine hashes with today's, andtest_fixed_inputs_are_the_central_runschecks it holds the central run's fixed inputs (a key only later engines record, such as Assumptions strip reads an assumptions block from the results (#14 §5) #18'spopulation, is compared once both files have it).The OBR wedge (#13)
obr_premiumis the central path with the triple lock paying the 'Triple lock' row of the OBR's long-term economic determinants (March 2026 EFO). Each April from 2028 gets the row's value for the fiscal year of the inputs. April 2027 is set by published inputs (May-July 2026 AWE), so it follows them as on the central path (3.9%), not the OBR row's 3.7%. For 2033-34 onwards that row is "average earnings growth plus 0.6 percentage points" (OBR's note). This is a stylised average-rate assumption for how often CPI or the floor beats earnings, not a CPI and earnings path the triple lock could produce. Only the triple lock is specified. The Burnham plan runs its own rule on the central path's CPI and earnings.The first run in this PR, with April 2027 at the OBR row's 3.7%, reproduced the earlier
scenario/obr-premiumrun exactly (2039-40 £7.83bn / £5.06bn; 2034-35 £2.585bn / £1.722bn). So #10's rounding changes and the generalisation kept the semantics. Paying April 2027 on the published inputs instead (3.9%) raises gross by 0.19% in both years (2039-40 £7.829bn to £7.844bn; 2034-35 £2.585bn to £2.590bn). Net rises 0.25% in 2034-35 (£1.722bn to £1.726bn) and 0.8% in 2039-40 (£5.063bn to £5.104bn). Most of the extra 2039-40 net change is a smaller Housing Benefit offset (£0.062bn to £0.034bn). Housing Benefit here moves with a few heavily weighted records near Pension Credit eligibility (method limitations), so a small level shift can move it this much; I have not traced which records.Other figures in the OBR-wedge run:
Exact rate inputs
These are the
specified_ratesthe file records, from April 2028: the OBR row, unrounded, by April paid. The rate paid is that value rounded to 0.1 point, and the plan's rates come from its own rule.Reproduce
The command needs a clean tree (or
--allow-dirty), with the Enhanced FRS 2024-25 (sha256e433e532b17bd8ce76030156285816e33d44e93edabd2204adbef71d19a68712, certified buildpolicyengine-uk-data-1.56.16) in the worker directory. Without Hugging Face credentials, copy it to.cache/workers/efrs0/data/enhanced_frs_2024_25.h5. The run takes about 40 seconds.PYTHONPATH=src .venv/bin/python -m triple_lock.cli --scenario obr_premium # = triple-lock-build --scenario obr_premiumVersions
From the file's
provenance:uk-5.3.0), policyengine-uk 2.90.2, policyengine-core 3.30.1;enhanced_frs_2024_25@ 1.56.16;The file also records the git revision (
f3a5a88), the source, input and engine hashes, and the full redacted run (rates, sources, weekly amounts, savings with components, totals, poverty, households affected, distribution, path-following checks).tests/test_scenarios.pychecks the committed file against the code:SCENARIOS["obr_premium"]gives today, and everything else is the central path;results.jsonpasses (reusingtest_results' run checks);test_no_survey_record_is_publishedapplied to the file).Caveats
git_revision(f3a5a88) is a commit on this branch. Please merge with a merge commit, or rerun the scenario after merging (40 s, from the cache).docs/RESULTS_SCHEMA.mdnow says a revision may be a branch commit, and the hashes are what tie an output to the code.Why draft:
test_not_staleis redtest_not_stalecompares the raw byte hashes of everysrc/triple_lock/*.pywith thosedata/results.jsonrecorded, so any edit undersrcturns it red. Only a rebuild clears it.rules.py,engine.pyandconfig.pyare engine files whose code changed, so the cached jobs in.cache/jobsno longer match: the next full build reruns every job (about 250 model runs), not a replay from the cache. With no specified rates every rate is unchanged (tested), so a rebuild should reproduce main's figures exactly. I have not run it;data/results.jsonand the dashboard copy are untouched.Tests
PYTHONPATH=src .venv/bin/python -m pytest -q: 201 passed, 2 failed.test_not_stale: expected, explained above.test_moment_tilt_hits_every_target_whatever_the_scale: the known flaky test in Flaky test: test_moment_tilt_hits_every_target_whatever_the_scale (tilt error just above 1e-6) #15. It also fails onorigin/main(ee02c8d) here.bun run test: 64 passed, including the newdescribeSourcetest.tests/test_scenarios.py::test_not_stalepasses: the committed file's provenance was refreshed at the head of the code (f3a5a88) from the cached model job of75ee7c9, whose engine files are unchanged since (no model rerun; figures identical).🤖 Generated with Claude Code