Skip to content

Review fixes from #10: backtests use the fiscal rounding; stop what a finished job left behind - #16

Closed
vahid-ahmadi wants to merge 2 commits into
mainfrom
review-fixes-10
Closed

vahid-ahmadi wants to merge 2 commits into
mainfrom
review-fixes-10

Conversation

@vahid-ahmadi

@vahid-ahmadi vahid-ahmadi commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Fixes the two should-address points from María's review of #10. Neither changes a published number, so the results are rebuilt with the model-v2 rebuild (#14) rather than now.

Changes

  1. The backtests use the fiscal rounding (her A1).
    • ts_backtest.gap_pct now rounds inputs and rates to 0.1 point (rules.round_rate), as the fiscal runs do. The expected-value backtest and the past-years check now score the same policy arithmetic as the expected saving. decimals=None keeps the unrounded gap.
    • Test: María's example, the random path's April 2030-33 inputs, gives 0.3846% rounded against 0.5263% unrounded.
  2. A finished job can't leave anything running (her A2).
    • engine.run_child now kills the job's process group once the job exits, before the worker directory is released. A descendant that closed its output can no longer outlive the job.
    • Test: a job that exits 0 leaving a grandchild running.

Why the results check is red

engine.py is in the model-run cache key, so this change makes every job rerun. test_not_stale therefore fails until the next full rebuild, and so does test_method_text_percentiles_match_the_past_years_check: it now checks that the text quotes whatever percentile the file records, and the committed file still says 55. Fix 2 changes no output. Fix 1 changes only backtest diagnostics, not the expected saving, its range or the household results:

  • expected_value.backtest;
  • the past-years check: the untilted model's realised 2012-start percentile goes from 55.2 to 55.8, so the method text and METHOD.md now say 56th, and the tilted model stays 93rd;
  • trajectories.backtest (*_crps, *_inside, realised_gap_pct).

The calibration choice still holds after rounding: means_shift bias is −0.60 (SE 0.57) as published and 0.20 (SE 0.45) as in law, and test_the_calibration_choice_still_holds passes. Merge this into the model-v2 branch, or merge it just before that rebuild.

Checks

  • 177 passed locally. The only failure is test_not_stale, as above.
  • test_jobs and test_cache_key pass.

🤖 Generated with Claude Code

…at a finished job left in its group

- ts_backtest.gap_pct rounds inputs and rates to 0.1 point (rules.round_rate) as the fiscal runs do, so the
  expected-value backtest and the past-years check score the same policy arithmetic as the expected saving
  (decimals=None keeps the unrounded gap). Test: María's example, the random path's April 2030-33 inputs,
  0.3846% rounded against 0.5263% unrounded.
- engine.run_child kills the job's process group once the job exits, before the worker directory is released,
  so a descendant that closed its output cannot outlive it. Test: a job that exits 0 leaving a grandchild.

engine.py is in the cache key, so every job reruns; results rebuilt in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
uk-triple-lock Ready Ready Preview Oct 1, 2026 3:23pm UTC

Request Review

…est reads the figure, not a literal

With the backtests on the fiscal 0.1-point rounding, the realised 2012-start gap sits at the 55.8th percentile of
the untilted model (was 55.2), so the method text and docs/METHOD.md say 56th; the tilted model stays 93rd. The test
now checks the text quotes whatever percentile the file records, so it fails until the rebuild, like test_not_stale,
and can't drift again. Also notes that a descendant starting its own session (setsid) is not reached by run_child.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor Author

Folded into model-v2 (#24) with all commits and authorship kept, and checked independently: nothing missing or reverted. Closing in favour of #24. The branch stays, so this can be reopened if model-v2 doesn't land.

This branch was successfully deployed

1 active deployment
Preview — b892f253 Deployed Oct 1, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant