Skip to content

Guard weighted incidence calculations against zero totals - #2448

Merged
juaristi22 merged 1 commit into
mainfrom
fix-2296-zero-weight-incidence
Oct 19, 2026
Merged

juaristi22 merged 1 commit into
mainfrom
fix-2296-zero-weight-incidence

Conversation

@juaristi22

Copy link
Copy Markdown
Collaborator

Fixes #2296

A dataset can contain wealth or consumption held only by households with zero weight. The raw total is then positive while the weighted total is zero, causing infinite or NaN shares that propagate into corporate incidence and household tax.

Check the actual weighted denominator in shareholding, consumption_shareholding and consumer_incident_tax_revenue_change. If it is zero, retain each formula's existing fallback. Nonzero weighted normalisation, incidence assumptions and policy parameters are unchanged.

Reproducer: policyengine_uk/tests/test_weighted_incidence.py::test_zero_weight_holder_uses_existing_fallback

Validation: five zero-weight-holder assertions failed before the fix. The 19 focused tests now pass, covering all three formulas, corporate incidence and household tax, positive weighted totals, zero-valued inputs and actual dataset-free Core controls. Another 16 existing corporate-land, employer-NI and excise regressions pass. make format, Ruff and git diff --check pass.

Tests use synthetic data and literal arithmetic expectations. Local validation used Python3.13 / Core3.32.27 in an existing offline environment; the full suite, locked Core3.33.2 and private population calculations were not run locally.

axiom: n/a: numerical dataset-normalisation guard; statutory provisions and existing incidence-allocation assumptions are unchanged.

@juaristi22 juaristi22 self-assigned this Oct 16, 2026

@vahid-ahmadi vahid-ahmadi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks right, fine to approve.

@juaristi22, the guard now tests the weighted denominator, which is the one the formula divides by. That is the fix #2296 asked for, and each formula keeps its old fallback. One follow-up point.

  1. nit, policyengine_uk/variables/contrib/policyengine/employer_ni/employer_ni_response_capital_incidence.py:31 and employer_ni_response_consumer_incidence.py:30: these two divide by a weighted total with no zero check at all, so the same zero-weight holder gives inf or NaN there when the employer NI response reform is on. They are outside #2296, so a follow-up issue is enough. corporate_land_value.py:16 already guards its total.

What I checked

  • Read the three changed formulas against the issue's reproduction. Computing household_weight before the check, rather than only when the raw total is non-zero, does not change any result.
  • pytest policyengine_uk/tests/test_weighted_incidence.py: 19 passed. With the three formula changes reversed: 5 failed, 14 passed.
  • ruff check and ruff format --check on the four changed Python files: clean.
  • #2363 also edits shareholding.py and consumption_share.py (unit only). git merge-tree shows no conflict.
  • Reproducer and axiom lines are present; fixed fragment type is right. CI is green on the current head. I did not run a population calculation.

🤖 Generated with Claude Code

@vahid-ahmadi vahid-ahmadi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, per my review above. The shoulds and nits can follow.

@juaristi22
juaristi22 added this pull request to the merge queue Oct 19, 2026
Merged via the queue into main with commit 0aaa946 Oct 19, 2026
14 checks passed
@juaristi22
juaristi22 deleted the fix-2296-zero-weight-incidence branch October 19, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wealth and consumption shares divide by a weighted total that can be zero

2 participants