Repository navigation
test_built_total_gains fails on reduced (TESTING=1) builds of main #535
Description
policyengine_uk_data/tests/test_cgt_band_donors.py::test_built_total_gains fails on reduced (TESTING=1, 32-epoch) builds of main, so PR CI can go red on PRs that have nothing to do with capital gains.
Measured locally on main b45c373 (TESTING=1 build): £242.6bn of above-AEA gains against HMRC's £65.9bn, a relative error of 268%. The reduced-build tolerance is 0.5 × 5.0 = 250%.
Seen in PR CI:
- severe-disability-flag-tax-credit-definition, run 36784812236: 255%. The same branch passed two hours earlier.
- Keep FRS-reported dividends and key them on person_id #498: 274% and 260%.
Full builds pass. Production-setting builds of main and #498 (512 epochs) pass all eight test_cgt_band_donors.py tests at the strict bounds.
So the reduced-build check sits at its threshold. It catches calibration noise rather than the "order-of-magnitude pathologies" its comment describes. Options:
- loosen the reduced-build tolerance with evidence of its spread across seeds;
- run more epochs for this check;
- skip it under TESTING, as
test_built_cgt_taxpayer_countalready does.
Activity
MaxGhenis commentedon Oct 12, 2026
Correction, and closing as resolved.
The "measured locally on main" figure above (268%) doesn't reproduce CI:
PE_UK_DATA_OA_CLONES=1(.github/workflows/pull_request.yaml).So the check sat near its threshold, but "fails on main" was wrong for CI.
Resolved on main. d39371d ("Bound the reduced-build gains check on both sides", released in the 10/8 batch, #544) now does two things:
#498's CI values (3.6-3.9 times HMRC) are inside that guard.
🤖 Generated with Claude Code