Repository navigation
Pre-Budget release mode: scenarios-only certified build, clearly labelled - #28
Closed
vahid-ahmadi wants to merge 6 commits into
Closed
vahid-ahmadi wants to merge 6 commits into
vahid-ahmadi wants to merge 6 commits into
Conversation
A certified-bundle build records certified: true (datasets.provenance), so the results and scenario provenance tests asserted the wrong thing: they required certified is False. They now check the flag against what the build recorded: true only with the registry's certification_bundle for the dataset and matching model/core versions; false, with the uncertified reason and no bundle, for the historical pilot. A unit test exercises both branches, since the committed files predate the model key. The dashboard's replication line no longer assumes an uncertified pair: it names the certifying policyengine.py bundle when the file says certified. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
triple-lock-build --pre-budget --uncertainty-ruling a writes an interim certified file on today's committed inputs and says so: - a top-level release block (also in provenance): stage pre-budget, the OBR March 2026 forecast, April 2027's provisional August 2026 CPI leg, the expected value withheld until the binding C2 score on the 28 October 2026 Budget inputs, and Max's recorded d955 ruling (c); - uncertainty_ruling and the mean-path provenance keep ruling a (the scenario-only execution, no requested_ruling) and add release_stage, interim and recorded_ruling c: an interim release under (c), not a new d955 ruling; - expected_value_omission.reason reads "Withheld pending the post-Budget binding score", not "d955 scenario-envelope ruling", and the assumptions strip quotes it. --pre-budget refuses any ruling but a (CLI and pipeline.build, before any job), --scenario, --mean-path-scenarios and --uncertainty-handoff, and refuses once CPI_PERIOD or the central forecast no longer describe the pre-Budget inputs, so the label cannot go stale. The dashboard shows a pre-Budget banner while release.stage is pre-budget, and the scenario envelope no longer says the recorded ruling omits an expected value in that case. RESULTS_SCHEMA.md documents release; REBUILD.md adds the interim step before the binding rebuild. A results-file check verifies the labels whenever a file carries a release block. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drives pipeline.build with --pre-budget on the adapter's fake-engine handoff fixture. Only PolicyEngine- and git-bound steps are stubbed; EV.build, the mean paths, coverage, redaction, assumptions and benchmarks run for real, and the post-build verifiers from test_results accept the file. An assembly error on this path can no longer first appear after the build's model jobs. Also covers the unlabelled direct-(a) shape and every --pre-budget refusal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After the post-Budget inputs are committed (CPI_PERIOD 2026 SEP) and before the binding rebuild replaces it, a committed pre-Budget file must fail only the documented stale-results test. The results check now reads the file's own release block; the assembly test still proves the build derives it from config. RESULTS_SCHEMA.md lists provenance.release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Dashboard: a scenarios-only file (ruling a, no expected value) now leads the Budget impact tab with one scenario-envelope headline read from the file: the central forecast, the OBR wedge, the paired +/-0.5pp earnings mean paths (2034-35 and 2039-40, gross and net) and the April 2017 replay, with a drop-down line on the earnings asymmetry when the paired differences show it. The envelope is no longer rendered twice, Methodology no longer prints "unavailable", and the intro, backtest note, limitations and path text keep expected-value wording out of a scenarios-only release. Tests: pytest branches on the file's own state. The historical-publication proof and the historical estimator run on the pilot's exact pinned bytes (archived under tests/fixtures/historical-publication once a certified build replaces them); the paired-ageing tests read the paired draws from the same historical file when the committed one omits an expected value; assumptions tests follow the population treatment the runs record; a weight key may hold a label or a check, never a number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
17 of 31 tasks
Contributor
Author
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #27 (base
fix-24-footnote-ci), so this diff is only the pre-Budget release work. @MaxGhenis, this is for your review. Vahid authorised it, and Nikhil said "fine to proceed".The change lets us publish a certified, scenarios-only build before the Budget, labelled as an interim release under your recorded ruling (c). It is not a new d955 ruling.
certifiedis now checked against the certification the build actually recorded. A certified-bundle build passes withtrue; the historical pilot passes withfalse. The replication line on the dashboard now reads the value from the file.triple-lock-build --pre-budget --uncertainty-ruling a. It writes a top-levelreleaseblock, also copied into provenance.uncertainty_rulingand the mean-path provenance keepruling: "a", with norequested_ruling, and addrecorded_ruling: "c"andinterim: true. The omission reason reads "Withheld pending the post-Budget binding score". The dashboard shows a banner and corrects the scenario-envelope wording. RESULTS_SCHEMA.md and REBUILD.md are updated.--pre-budgetrefuses any ruling other than (a).--scenario,--mean-path-scenariosand--uncertainty-handoff.CPI_PERIODor the central EFO source stop describing the pre-Budget inputs.pipeline.buildon the adapter's fake-engine handoff fixture. EV.build, the mean paths, coverage, redaction, assumptions and benchmarks all run for real, and the existing post-build verifiers accept the resulting file.cold_source_correspondence.current_files()reads the archived pilot tarball, not the working tree, so editingpipeline.pyandcli.pyleaves your byte bindings untouched. The determinism and bridge correspondence tests pass unchanged.Local checks on the head (policyengine 6.2.6 venv):
ALLOW_PREBUILD_STALE=true scripts/check_prebuild_tests.pyreports "Completed 1647 tests; 3 documented pre-build stale-results failures." and exits 0.Expect the Pipeline CI on this PR to be red on those same 3 stale tests.
python.ymlonly grantsALLOW_PREBUILD_STALEto a draft PR whose head ismodel-v2.Nothing has been merged and no model jobs have run.
🤖 Generated with Claude Code