Repository navigation
Review fixes from #10: backtests use the fiscal rounding; stop what a finished job left behind - #16
Closed
vahid-ahmadi wants to merge 2 commits into
Closed
vahid-ahmadi wants to merge 2 commits into
vahid-ahmadi wants to merge 2 commits into
Conversation
…at a finished job left in its group - ts_backtest.gap_pct rounds inputs and rates to 0.1 point (rules.round_rate) as the fiscal runs do, so the expected-value backtest and the past-years check score the same policy arithmetic as the expected saving (decimals=None keeps the unrounded gap). Test: María's example, the random path's April 2030-33 inputs, 0.3846% rounded against 0.5263% unrounded. - engine.run_child kills the job's process group once the job exits, before the worker directory is released, so a descendant that closed its output cannot outlive it. Test: a job that exits 0 leaving a grandchild. engine.py is in the cache key, so every job reruns; results rebuilt in the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…est reads the figure, not a literal With the backtests on the fiscal 0.1-point rounding, the realised 2012-start gap sits at the 55.8th percentile of the untilted model (was 55.2), so the method text and docs/METHOD.md say 56th; the tilted model stays 93rd. The test now checks the text quotes whatever percentile the file records, so it fails until the rebuild, like test_not_stale, and can't drift again. Also notes that a descendant starting its own session (setsid) is not reached by run_child. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the two should-address points from María's review of #10. Neither changes a published number, so the results are rebuilt with the model-v2 rebuild (#14) rather than now.
Changes
ts_backtest.gap_pctnow rounds inputs and rates to 0.1 point (rules.round_rate), as the fiscal runs do. The expected-value backtest and the past-years check now score the same policy arithmetic as the expected saving.decimals=Nonekeeps the unrounded gap.engine.run_childnow kills the job's process group once the job exits, before the worker directory is released. A descendant that closed its output can no longer outlive the job.Why the results check is red
engine.pyis in the model-run cache key, so this change makes every job rerun.test_not_staletherefore fails until the next full rebuild, and so doestest_method_text_percentiles_match_the_past_years_check: it now checks that the text quotes whatever percentile the file records, and the committed file still says 55. Fix 2 changes no output. Fix 1 changes only backtest diagnostics, not the expected saving, its range or the household results:expected_value.backtest;trajectories.backtest(*_crps,*_inside,realised_gap_pct).The calibration choice still holds after rounding: means_shift bias is −0.60 (SE 0.57) as published and 0.20 (SE 0.45) as in law, and
test_the_calibration_choice_still_holdspasses. Merge this into the model-v2 branch, or merge it just before that rebuild.Checks
test_not_stale, as above.test_jobsandtest_cache_keypass.🤖 Generated with Claude Code