From 6d8e86610f27db04ea91e2c2040f2e18982497c1 Mon Sep 17 00:00:00 2001 From: Vahid Ahmadi Date: Wed, 27 May 2026 12:14:50 +0200 Subject: [PATCH 1/5] Document policyengine.py household-impact test diagnosis (#1628) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #1628 reported that two policyengine.py household-impact integration tests started asserting 0.0 for expected-positive tax / child-benefit values after bumping to policyengine-uk==2.88.0. Reproducing the same scenarios directly against policyengine-uk shows the model is healthy: a £50k single adult correctly returns £7,486 income tax, and a couple with one child correctly returns £1,354.60 child benefit at 2025 rates. The figures are stable across 2022-2025. So the regression is upstream in policyengine.py's test harness, not a model bug. Most likely cause: the harness depending on the model's default_input_period (2025) vs the year encoded in the test fixture, or being broken by the default-dataset env var requirement introduced since the previous 2.74 pin. Persist the diagnosis as a docs page so the next reviewer of #1628 gets a one-stop explanation. Guidance for the upstream fix: - pass explicit period= to every calculate() call, - use Microsimulation(situation=...) for single-household queries (no dataset needed), - assert against the fixture-encoded year rather than the model's default. Cross-links to mp30k-uk-switch-plan.md which documents the env-var contract this issue collides with. --- changelog.d/1628.md | 1 + .../policyengine-py-household-impact.md | 111 ++++++++++++++++++ 2 files changed, 112 insertions(+) create mode 100644 changelog.d/1628.md create mode 100644 docs/book/validation/policyengine-py-household-impact.md diff --git a/changelog.d/1628.md b/changelog.d/1628.md new file mode 100644 index 0000000000..8ef6c4eddc --- /dev/null +++ b/changelog.d/1628.md @@ -0,0 +1 @@ +- Document the resolution of the `policyengine.py` household-impact zero-return regression at `docs/book/validation/policyengine-py-household-impact.md`; the UK model itself is healthy (£7,486 income tax for a £50k single adult and £1,354.60 child benefit for a couple with one child at 2025 rates) and the upstream fix is in the `policyengine.py` harness around default-period / explicit dataset arguments. diff --git a/docs/book/validation/policyengine-py-household-impact.md b/docs/book/validation/policyengine-py-household-impact.md new file mode 100644 index 0000000000..fc08dbd525 --- /dev/null +++ b/docs/book/validation/policyengine-py-household-impact.md @@ -0,0 +1,111 @@ +# Household-impact tests returning 0 (resolved) + +```{note} +**Resolved.** [#1628](https://github.com/PolicyEngine/policyengine-uk/issues/1628) +reported that two `policyengine.py` household-impact integration tests +started asserting `0.0` for expected-positive tax and child-benefit +values after bumping the UK extra to `policyengine-uk==2.88.0`. This +page documents the diagnosis from the PolicyEngine UK side and the +guidance for the upstream `policyengine.py` repo. It is not a model +bug — the UK model produces the expected non-zero values when called +correctly. +``` + +## Reproduction from the UK side + +Running the two scenarios directly against `policyengine-uk` main: + +```python +from policyengine_uk import Microsimulation +YEAR = 2025 + +# Scenario 1 — single adult with employment income. +s1 = { + "people": {"adult": {"age": {YEAR: 30}, + "employment_income": {YEAR: 50_000}}}, + "benunits": {"benunit": {"members": ["adult"]}}, + "households": {"household": {"members": ["adult"]}}, +} +sim = Microsimulation(situation=s1) +print("income_tax:", sim.calculate("income_tax", YEAR).sum()) +# -> 7486.00 + +# Scenario 2 — couple with one child. +s2 = { + "people": { + "a1": {"age": {YEAR: 35}, "employment_income": {YEAR: 30_000}}, + "a2": {"age": {YEAR: 33}}, + "c1": {"age": {YEAR: 5}}, + }, + "benunits": {"benunit": {"members": ["a1", "a2", "c1"]}}, + "households": {"household": {"members": ["a1", "a2", "c1"]}}, +} +sim = Microsimulation(situation=s2) +print("child_benefit:", sim.calculate("child_benefit", YEAR).sum()) +# -> 1354.60 +``` + +Both return the expected non-zero values across the range 2022-2025 +(the income-tax figure is invariant because the personal allowance is +£12,570 throughout this window). So **the underlying model is healthy**. + +## What changed between 2.74 and 2.88 + +The most likely cause of the upstream test failure is the +**default-period** contract: + +```python +# policyengine_uk/simulation.py +class Simulation(CoreSimulation): + default_input_period: int = 2025 + default_calculation_period: int = 2025 +``` + +If `policyengine.py`'s household-impact harness was building a 2024 +situation but the UK model now defaults to 2025 for calculation, any +benefit / tax variable that hadn't been populated for 2025 in the +situation would compute against missing inputs and could return 0. + +The other plausible cause is the **dataset-required** invariant in +the `Simulation()` constructor: from +`policyengine_uk==2.79`-ish onwards, `Simulation()` without an +explicit `dataset=` argument and without the +`POLICYENGINE_UK_DEFAULT_DATASET` env var raises a `ValueError`. If +the `policyengine.py` harness was catching that error and falling +through to a degenerate code path (zero-filled `Microsimulation`), +the tests would see zeros. + +## Guidance for the upstream fix + +The `policyengine.py` household-impact harness should: + +1. Pass an **explicit `period`** to every `calculate(...)` call + matching the year encoded in the situation YAML, rather than relying + on the model default. +2. Build the situation using the **`Microsimulation(situation=...)`** + form rather than the dataset-backed `Simulation()` — single-household + queries don't need a dataset and shouldn't bring in the default- + dataset env-var dependency. +3. **Assert against the year encoded in the test fixture**, not the + model's `default_input_period`. The fixture-encoded year is what the + test author intended. + +The two failing tests should produce ~£7,486 income tax for a +£50k-earning single adult and ~£1,354.60 of child benefit for a +couple-with-one-child benefit unit at 2025 rates. + +## Why this is documented in `policyengine-uk` rather than fixed there + +The actual fix lives in `policyengine.py`'s test harness and is +explicitly out of scope for this repo (sister-repo edits aren't +made from `policyengine-uk`). This page exists so the next reviewer +who sees `#1628` and is unfamiliar with the policyengine.py setup has +a one-stop diagnosis instead of having to re-derive it. + +## References + +- Issue: [#1628](https://github.com/PolicyEngine/policyengine-uk/issues/1628). +- Reproducer: [`PolicyEngine/policyengine.py#280`](https://github.com/PolicyEngine/policyengine.py/pull/280). +- Default-period contract: [`policyengine_uk/simulation.py`](../../../policyengine_uk/simulation.py) (`default_input_period`, `default_calculation_period`). +- Default-dataset env var contract: documented in + [mp30k-uk-switch-plan.md](../assumptions/mp30k-uk-switch-plan.md#current-state). From 25383e88d82e92f6c410e60c412a3551729c24aa Mon Sep 17 00:00:00 2001 From: vahid-ahmadi Date: Thu, 8 Oct 2026 15:49:31 +0100 Subject: [PATCH 2/5] Refresh the page against current main and Microcosm UK; type the changelog fragment Co-Authored-By: Claude Opus 5.5 --- changelog.d/1628.added.md | 1 + changelog.d/1628.md | 1 - .../policyengine-py-household-impact.md | 32 +++++++++++-------- 3 files changed, 19 insertions(+), 15 deletions(-) create mode 100644 changelog.d/1628.added.md delete mode 100644 changelog.d/1628.md diff --git a/changelog.d/1628.added.md b/changelog.d/1628.added.md new file mode 100644 index 0000000000..9bf39b9e40 --- /dev/null +++ b/changelog.d/1628.added.md @@ -0,0 +1 @@ +Documented the policyengine.py household-impact zero-return diagnosis (docs/book/validation/policyengine-py-household-impact.md); the UK model returns the expected values, re-checked on 2.123.7. diff --git a/changelog.d/1628.md b/changelog.d/1628.md deleted file mode 100644 index 8ef6c4eddc..0000000000 --- a/changelog.d/1628.md +++ /dev/null @@ -1 +0,0 @@ -- Document the resolution of the `policyengine.py` household-impact zero-return regression at `docs/book/validation/policyengine-py-household-impact.md`; the UK model itself is healthy (£7,486 income tax for a £50k single adult and £1,354.60 child benefit for a couple with one child at 2025 rates) and the upstream fix is in the `policyengine.py` harness around default-period / explicit dataset arguments. diff --git a/docs/book/validation/policyengine-py-household-impact.md b/docs/book/validation/policyengine-py-household-impact.md index fc08dbd525..d3c33d0f5c 100644 --- a/docs/book/validation/policyengine-py-household-impact.md +++ b/docs/book/validation/policyengine-py-household-impact.md @@ -9,6 +9,11 @@ page documents the diagnosis from the PolicyEngine UK side and the guidance for the upstream `policyengine.py` repo. It is not a model bug — the UK model produces the expected non-zero values when called correctly. + +Re-checked in October 2026 on policyengine-uk 2.123.7: both scenarios +below still return £7,486 and £1,354.60. The upstream reproducer +(policyengine.py#280) was closed without merging; #1628 can be closed +once the policyengine.py household-impact tests are confirmed green. ``` ## Reproduction from the UK side @@ -16,7 +21,7 @@ correctly. Running the two scenarios directly against `policyengine-uk` main: ```python -from policyengine_uk import Microsimulation +from policyengine_uk import Simulation YEAR = 2025 # Scenario 1 — single adult with employment income. @@ -26,7 +31,7 @@ s1 = { "benunits": {"benunit": {"members": ["adult"]}}, "households": {"household": {"members": ["adult"]}}, } -sim = Microsimulation(situation=s1) +sim = Simulation(situation=s1) print("income_tax:", sim.calculate("income_tax", YEAR).sum()) # -> 7486.00 @@ -40,7 +45,7 @@ s2 = { "benunits": {"benunit": {"members": ["a1", "a2", "c1"]}}, "households": {"household": {"members": ["a1", "a2", "c1"]}}, } -sim = Microsimulation(situation=s2) +sim = Simulation(situation=s2) print("child_benefit:", sim.calculate("child_benefit", YEAR).sum()) # -> 1354.60 ``` @@ -67,10 +72,10 @@ benefit / tax variable that hadn't been populated for 2025 in the situation would compute against missing inputs and could return 0. The other plausible cause is the **dataset-required** invariant in -the `Simulation()` constructor: from -`policyengine_uk==2.79`-ish onwards, `Simulation()` without an -explicit `dataset=` argument and without the -`POLICYENGINE_UK_DEFAULT_DATASET` env var raises a `ValueError`. If +the `Simulation()` constructor: `Simulation()` with no `situation=`, +no explicit `dataset=` and no `POLICYENGINE_UK_DEFAULT_DATASET` env +var raises `ValueError: Simulation() requires an explicit dataset when +no situation is provided`. If the `policyengine.py` harness was catching that error and falling through to a degenerate code path (zero-filled `Microsimulation`), the tests would see zeros. @@ -82,10 +87,9 @@ The `policyengine.py` household-impact harness should: 1. Pass an **explicit `period`** to every `calculate(...)` call matching the year encoded in the situation YAML, rather than relying on the model default. -2. Build the situation using the **`Microsimulation(situation=...)`** - form rather than the dataset-backed `Simulation()` — single-household - queries don't need a dataset and shouldn't bring in the default- - dataset env-var dependency. +2. Build single-household cases with **`Simulation(situation=...)`** + rather than a dataset-backed simulation — they don't need a dataset + and shouldn't bring in the default-dataset env-var dependency. 3. **Assert against the year encoded in the test fixture**, not the model's `default_input_period`. The fixture-encoded year is what the test author intended. @@ -106,6 +110,6 @@ a one-stop diagnosis instead of having to re-derive it. - Issue: [#1628](https://github.com/PolicyEngine/policyengine-uk/issues/1628). - Reproducer: [`PolicyEngine/policyengine.py#280`](https://github.com/PolicyEngine/policyengine.py/pull/280). -- Default-period contract: [`policyengine_uk/simulation.py`](../../../policyengine_uk/simulation.py) (`default_input_period`, `default_calculation_period`). -- Default-dataset env var contract: documented in - [mp30k-uk-switch-plan.md](../assumptions/mp30k-uk-switch-plan.md#current-state). +- Default-period contract: [`policyengine_uk/simulation.py`](https://github.com/PolicyEngine/policyengine-uk/blob/main/policyengine_uk/simulation.py) (`default_input_period`, `default_calculation_period`). +- Default-dataset env var contract: `POLICYENGINE_UK_DEFAULT_DATASET`, + defined in [`policyengine_uk/tax_benefit_system.py`](https://github.com/PolicyEngine/policyengine-uk/blob/main/policyengine_uk/tax_benefit_system.py). From ad466f5f261a8ea3d0d51553ae73c7c3386290fe Mon Sep 17 00:00:00 2001 From: vahid-ahmadi Date: Mon, 12 Oct 2026 11:05:47 +0200 Subject: [PATCH 3/5] Correct the 2.74 to 2.88 diagnosis and note the upstream test rewrite The default period was already 2025 at 2.74.0, so it did not change in that window; the dataset-required error arrived in 2.76.0. Note that policyengine.py now tests with explicit years in TestUKCalculateHousehold, and remove em dashes. Co-Authored-By: Claude Opus 5.5 --- .../policyengine-py-household-impact.md | 46 ++++++++++--------- 1 file changed, 25 insertions(+), 21 deletions(-) diff --git a/docs/book/validation/policyengine-py-household-impact.md b/docs/book/validation/policyengine-py-household-impact.md index d3c33d0f5c..7ebe47bb26 100644 --- a/docs/book/validation/policyengine-py-household-impact.md +++ b/docs/book/validation/policyengine-py-household-impact.md @@ -7,13 +7,16 @@ started asserting `0.0` for expected-positive tax and child-benefit values after bumping the UK extra to `policyengine-uk==2.88.0`. This page documents the diagnosis from the PolicyEngine UK side and the guidance for the upstream `policyengine.py` repo. It is not a model -bug — the UK model produces the expected non-zero values when called +bug: the UK model produces the expected non-zero values when called correctly. Re-checked in October 2026 on policyengine-uk 2.123.7: both scenarios below still return £7,486 and £1,354.60. The upstream reproducer -(policyengine.py#280) was closed without merging; #1628 can be closed -once the policyengine.py household-impact tests are confirmed green. +(policyengine.py#280) was closed without merging. policyengine.py has +since replaced the `TestUKHouseholdImpact` tests with +`TestUKCalculateHousehold` in `tests/test_household_impact.py`, which +passes an explicit `year` to every case and pins policyengine-uk +2.125.1. #1628 can be closed once those tests are confirmed green. ``` ## Reproduction from the UK side @@ -24,7 +27,7 @@ Running the two scenarios directly against `policyengine-uk` main: from policyengine_uk import Simulation YEAR = 2025 -# Scenario 1 — single adult with employment income. +# Scenario 1: single adult with employment income. s1 = { "people": {"adult": {"age": {YEAR: 30}, "employment_income": {YEAR: 50_000}}}, @@ -35,7 +38,7 @@ sim = Simulation(situation=s1) print("income_tax:", sim.calculate("income_tax", YEAR).sum()) # -> 7486.00 -# Scenario 2 — couple with one child. +# Scenario 2: couple with one child. s2 = { "people": { "a1": {"age": {YEAR: 35}, "employment_income": {YEAR: 30_000}}, @@ -56,8 +59,19 @@ Both return the expected non-zero values across the range 2022-2025 ## What changed between 2.74 and 2.88 -The most likely cause of the upstream test failure is the -**default-period** contract: +The change in that window that most plausibly explains the upstream +failure is the **dataset-required** invariant, added in 2.76.0 +("Stop defaulting to private dataset"): `Simulation()` with no +`situation=`, no explicit `dataset=` and no +`POLICYENGINE_UK_DEFAULT_DATASET` environment variable raises +`ValueError: Simulation() requires an explicit dataset when no +situation is provided`. If the `policyengine.py` harness was catching +that error and falling through to a degenerate code path (a +zero-filled `Microsimulation`), the tests would see zeros. + +The **default-period** contract is a second thing to rule out, although +it did not change in that window (both 2.74.0 and 2.88.0 default to +2025): ```python # policyengine_uk/simulation.py @@ -66,19 +80,9 @@ class Simulation(CoreSimulation): default_calculation_period: int = 2025 ``` -If `policyengine.py`'s household-impact harness was building a 2024 -situation but the UK model now defaults to 2025 for calculation, any -benefit / tax variable that hadn't been populated for 2025 in the -situation would compute against missing inputs and could return 0. - -The other plausible cause is the **dataset-required** invariant in -the `Simulation()` constructor: `Simulation()` with no `situation=`, -no explicit `dataset=` and no `POLICYENGINE_UK_DEFAULT_DATASET` env -var raises `ValueError: Simulation() requires an explicit dataset when -no situation is provided`. If -the `policyengine.py` harness was catching that error and falling -through to a degenerate code path (zero-filled `Microsimulation`), -the tests would see zeros. +The failing tests built 2024 households. Any calculation that relied on +the default period would compute 2025, and any tax or benefit input +populated only for 2024 would be missing for that year. ## Guidance for the upstream fix @@ -88,7 +92,7 @@ The `policyengine.py` household-impact harness should: matching the year encoded in the situation YAML, rather than relying on the model default. 2. Build single-household cases with **`Simulation(situation=...)`** - rather than a dataset-backed simulation — they don't need a dataset + rather than a dataset-backed simulation. They don't need a dataset and shouldn't bring in the default-dataset env-var dependency. 3. **Assert against the year encoded in the test fixture**, not the model's `default_input_period`. The fixture-encoded year is what the From dd3061261a8259987aacd2cdb0439e9bd4809812 Mon Sep 17 00:00:00 2001 From: vahid-ahmadi Date: Mon, 12 Oct 2026 11:23:07 +0200 Subject: [PATCH 4/5] Format the code block for ruff Co-Authored-By: Claude Opus 5.5 --- docs/book/validation/policyengine-py-household-impact.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/book/validation/policyengine-py-household-impact.md b/docs/book/validation/policyengine-py-household-impact.md index 7ebe47bb26..1926c110cc 100644 --- a/docs/book/validation/policyengine-py-household-impact.md +++ b/docs/book/validation/policyengine-py-household-impact.md @@ -25,12 +25,12 @@ Running the two scenarios directly against `policyengine-uk` main: ```python from policyengine_uk import Simulation + YEAR = 2025 # Scenario 1: single adult with employment income. s1 = { - "people": {"adult": {"age": {YEAR: 30}, - "employment_income": {YEAR: 50_000}}}, + "people": {"adult": {"age": {YEAR: 30}, "employment_income": {YEAR: 50_000}}}, "benunits": {"benunit": {"members": ["adult"]}}, "households": {"household": {"members": ["adult"]}}, } From 9340cb37ad63af7b65d83cca8dbc0cf0494f93da Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Fri, 23 Oct 2026 02:10:40 -0400 Subject: [PATCH 5/5] Correct the household-impact zeros diagnosis to the core 3.24 bug The page reached main through #2386 and its changelog line shipped in 2.143.0, so this drops the released .added fragment and corrects main's copy. The page blamed the 2.76.0 dataset-required change and the default period; the harness at policyengine.py#280's head passed an explicit dataset and period, and its tests built 2026 households. The actual cause, per policyengine-core#475 and core's 3.24.4 changelog, was core 3.24.0 to 3.24.3 wiping set_input values on apply_reform; #280 raised the core floor to 3.24.1 alongside the UK bump, and policyengine.py#283 moved it to 3.25.0. - Cut the guidance and "why documented here" sections, which rested on the wrong diagnosis and cited a repo rule that does not exist. - policyengine.py now pins policyengine-uk 2.132.6 and its eight TestUKCalculateHousehold cases pass on main CI (run 39822499276). - Re-ran the page's code on 2.148.2: 7486.0 and 1354.60, non-zero for 2022 to 2025. Co-Authored-By: Claude Opus 5.5 --- ...1628-household-impact-diagnosis.changed.md | 1 + changelog.d/1628.added.md | 1 - .../policyengine-py-household-impact.md | 132 ++++++++---------- 3 files changed, 59 insertions(+), 75 deletions(-) create mode 100644 changelog.d/1628-household-impact-diagnosis.changed.md delete mode 100644 changelog.d/1628.added.md diff --git a/changelog.d/1628-household-impact-diagnosis.changed.md b/changelog.d/1628-household-impact-diagnosis.changed.md new file mode 100644 index 0000000000..cc962b3f32 --- /dev/null +++ b/changelog.d/1628-household-impact-diagnosis.changed.md @@ -0,0 +1 @@ +Corrected the policyengine.py household-impact page (docs/book/validation/policyengine-py-household-impact.md): the zeros in #1628 came from a policyengine-core 3.24 bug fixed in 3.24.4, not from policyengine-uk; re-checked on 2.148.2. diff --git a/changelog.d/1628.added.md b/changelog.d/1628.added.md deleted file mode 100644 index 9bf39b9e40..0000000000 --- a/changelog.d/1628.added.md +++ /dev/null @@ -1 +0,0 @@ -Documented the policyengine.py household-impact zero-return diagnosis (docs/book/validation/policyengine-py-household-impact.md); the UK model returns the expected values, re-checked on 2.123.7. diff --git a/docs/book/validation/policyengine-py-household-impact.md b/docs/book/validation/policyengine-py-household-impact.md index 1926c110cc..3cede5ab7e 100644 --- a/docs/book/validation/policyengine-py-household-impact.md +++ b/docs/book/validation/policyengine-py-household-impact.md @@ -2,26 +2,57 @@ ```{note} **Resolved.** [#1628](https://github.com/PolicyEngine/policyengine-uk/issues/1628) -reported that two `policyengine.py` household-impact integration tests -started asserting `0.0` for expected-positive tax and child-benefit -values after bumping the UK extra to `policyengine-uk==2.88.0`. This -page documents the diagnosis from the PolicyEngine UK side and the -guidance for the upstream `policyengine.py` repo. It is not a model -bug: the UK model produces the expected non-zero values when called -correctly. - -Re-checked in October 2026 on policyengine-uk 2.123.7: both scenarios -below still return £7,486 and £1,354.60. The upstream reproducer -(policyengine.py#280) was closed without merging. policyengine.py has -since replaced the `TestUKHouseholdImpact` tests with -`TestUKCalculateHousehold` in `tests/test_household_impact.py`, which -passes an explicit `year` to every case and pins policyengine-uk -2.125.1. #1628 can be closed once those tests are confirmed green. +reported that two `policyengine.py` household-impact integration tests, +which assert that income tax and child benefit are positive, got `0.0` +after +[policyengine.py#280](https://github.com/PolicyEngine/policyengine.py/pull/280) +moved the UK extra from `policyengine-uk==2.74.0` to `==2.88.0` and +raised the `policyengine_core` floor from 3.23.6 to 3.24.1. + +It was not a PolicyEngine UK bug. policyengine-core 3.24.0 to 3.24.3 +discarded inputs that a country package set before applying a reform +during simulation construction, the pattern `policyengine_uk.Simulation` +uses. +[policyengine-core#475](https://github.com/PolicyEngine/policyengine-core/pull/475) +fixed this in core 3.24.4, and +[policyengine.py#283](https://github.com/PolicyEngine/policyengine.py/pull/283) +raised policyengine.py's core floor to 3.25.0, both on 18 April 2026. + +On 22 October 2026 all eight UK cases in policyengine.py's +`TestUKCalculateHousehold` (`tests/test_household_impact.py`, which +replaced `TestUKHouseholdImpact` in the v4 launch) passed on its main CI +with policyengine-uk 2.132.6, so #1628 can be closed. ``` +## What went wrong + +policyengine-core's changelog for 3.24.4 describes the bug: its cache +invalidation after `apply_reform` "wiped every variable's +`_memory_storage._arrays`, which also wiped user-provided dataset inputs +loaded via `set_input`. Country-package subclasses calling `set_input` +during construction and then applying a structural reform (the +`policyengine-uk` pattern) silently lost their datasets." In the failing +tests, inputs such as age, `employment_income` and `would_claim_*` were +gone before any calculation ran, so tax and benefits came out as zero. + +Two other explanations were considered and do not fit the harness code +at policyengine.py#280's head (`6772ecc`): + +- **The dataset-required change** (policyengine-uk 2.76.0, where + `Simulation()` with no situation, no dataset and no + `POLICYENGINE_UK_DEFAULT_DATASET` raises a `ValueError`). The harness + passed an explicit dataset (`Microsimulation(dataset=...)` in + `src/policyengine/tax_benefit_models/uk/model.py`), so it never reached + that path. +- **The default period** (`default_input_period` and + `default_calculation_period`, 2025 in both 2.74.0 and 2.88.0). The + harness passed `period=` to every calculation, and the failing tests + built 2026 households. + ## Reproduction from the UK side -Running the two scenarios directly against `policyengine-uk` main: +Two comparable scenarios, run directly against `policyengine-uk` main for +2025 (the failing tests used 2026 and a one-adult, two-child family): ```python from policyengine_uk import Simulation @@ -53,67 +84,20 @@ print("child_benefit:", sim.calculate("child_benefit", YEAR).sum()) # -> 1354.60 ``` -Both return the expected non-zero values across the range 2022-2025 -(the income-tax figure is invariant because the personal allowance is -£12,570 throughout this window). So **the underlying model is healthy**. - -## What changed between 2.74 and 2.88 - -The change in that window that most plausibly explains the upstream -failure is the **dataset-required** invariant, added in 2.76.0 -("Stop defaulting to private dataset"): `Simulation()` with no -`situation=`, no explicit `dataset=` and no -`POLICYENGINE_UK_DEFAULT_DATASET` environment variable raises -`ValueError: Simulation() requires an explicit dataset when no -situation is provided`. If the `policyengine.py` harness was catching -that error and falling through to a degenerate code path (a -zero-filled `Microsimulation`), the tests would see zeros. - -The **default-period** contract is a second thing to rule out, although -it did not change in that window (both 2.74.0 and 2.88.0 default to -2025): - -```python -# policyengine_uk/simulation.py -class Simulation(CoreSimulation): - default_input_period: int = 2025 - default_calculation_period: int = 2025 -``` - -The failing tests built 2024 households. Any calculation that relied on -the default period would compute 2025, and any tax or benefit input -populated only for 2024 would be missing for that year. - -## Guidance for the upstream fix - -The `policyengine.py` household-impact harness should: - -1. Pass an **explicit `period`** to every `calculate(...)` call - matching the year encoded in the situation YAML, rather than relying - on the model default. -2. Build single-household cases with **`Simulation(situation=...)`** - rather than a dataset-backed simulation. They don't need a dataset - and shouldn't bring in the default-dataset env-var dependency. -3. **Assert against the year encoded in the test fixture**, not the - model's `default_input_period`. The fixture-encoded year is what the - test author intended. - -The two failing tests should produce ~£7,486 income tax for a -£50k-earning single adult and ~£1,354.60 of child benefit for a -couple-with-one-child benefit unit at 2025 rates. - -## Why this is documented in `policyengine-uk` rather than fixed there - -The actual fix lives in `policyengine.py`'s test harness and is -explicitly out of scope for this repo (sister-repo edits aren't -made from `policyengine-uk`). This page exists so the next reviewer -who sees `#1628` and is unfamiliar with the policyengine.py setup has -a one-stop diagnosis instead of having to re-derive it. +Re-checked on 23 October 2026 on policyengine-uk 2.148.2 (policyengine-core +3.33.2): both scenarios return £7,486 and £1,354.60. Both also return +non-zero values for every year from 2022 to 2025. The income-tax figure is +the same each year because the personal allowance (£12,570), the basic +rate limit (£37,700) and the 20% basic rate are unchanged from 2021-22; +child benefit follows the eldest-child rate (£1,133.60 in 2022, £1,248.00 +in 2023, £1,331.20 in 2024). ## References - Issue: [#1628](https://github.com/PolicyEngine/policyengine-uk/issues/1628). -- Reproducer: [`PolicyEngine/policyengine.py#280`](https://github.com/PolicyEngine/policyengine.py/pull/280). +- Where the failure showed: [`PolicyEngine/policyengine.py#280`](https://github.com/PolicyEngine/policyengine.py/pull/280) (closed without merging, superseded by #282). +- The fix: [`PolicyEngine/policyengine-core#475`](https://github.com/PolicyEngine/policyengine-core/pull/475) (core 3.24.4) and [`PolicyEngine/policyengine.py#283`](https://github.com/PolicyEngine/policyengine.py/pull/283) (core floor 3.25.0). - Default-period contract: [`policyengine_uk/simulation.py`](https://github.com/PolicyEngine/policyengine-uk/blob/main/policyengine_uk/simulation.py) (`default_input_period`, `default_calculation_period`). - Default-dataset env var contract: `POLICYENGINE_UK_DEFAULT_DATASET`, - defined in [`policyengine_uk/tax_benefit_system.py`](https://github.com/PolicyEngine/policyengine-uk/blob/main/policyengine_uk/tax_benefit_system.py). + defined in [`policyengine_uk/tax_benefit_system.py`](https://github.com/PolicyEngine/policyengine-uk/blob/main/policyengine_uk/tax_benefit_system.py) + and read in `policyengine_uk/simulation.py`.