diff --git a/changelog.d/1354-takeup-plan-corrections.changed.md b/changelog.d/1354-takeup-plan-corrections.changed.md new file mode 100644 index 0000000000..c292ddac2a --- /dev/null +++ b/changelog.d/1354-takeup-plan-corrections.changed.md @@ -0,0 +1 @@ +Corrected the effective take-up analysis plan (docs/book/assumptions/takeup-analysis-plan.md): Microcosm UK as the certified default dataset, the SPI-before-reweighting order, the size of seed-to-effective drift, the Council Tax Reduction, Carer's Allowance, State Pension and disability rows, new legacy-closure and JSA/ESA rows, and two broken gov.uk links. diff --git a/docs/book/assumptions/takeup-analysis-plan.md b/docs/book/assumptions/takeup-analysis-plan.md index 64cd8e5679..519da82353 100644 --- a/docs/book/assumptions/takeup-analysis-plan.md +++ b/docs/book/assumptions/takeup-analysis-plan.md @@ -10,41 +10,62 @@ and per-programme target list. ``` ```{admonition} Status, October 2026 -The analysis below has not been run yet. Take-up flags are drawn in the -data build, not in this repository. The certified UK default dataset is -still the Enhanced FRS built by +The analysis below has not been run yet. Stochastic take-up flags are +drawn in the data build, not in this repository; the Housing Benefit, +Income Support, tax credit and Council Tax Reduction flags are formulas +here (see below). Since policyengine.py 6.10.0 (21 October 2026) the +certified UK default dataset is Microcosm UK (`microcosm_uk_2024_25`), +built by [Microcosm](https://github.com/PolicyEngine/microcosm). The +Enhanced FRS built by [`policyengine-uk-data`](https://github.com/PolicyEngine/policyengine-uk-data) -(release 1.58.0). [Microcosm UK](https://github.com/PolicyEngine/microcosm) -builds the replacement national dataset with the same take-up design. -Both draw Universal Credit, Pension Credit and Child Benefit take-up -anchored on reported receipt, so the analysis should run on whichever -dataset is the certified default at the time. +stays loadable by name at release 1.58.1, and it is the dataset this +repository's credentialed tests pin +(`policyengine_uk/tests/certified_dataset.yaml`). Both draw Universal +Credit, Pension Credit and Child Benefit take-up anchored on reported +receipt, and Microcosm adds some draws the Enhanced FRS lacks (see the +table), so the analysis should run on whichever dataset is the certified +default at the time. ``` ## Why this matters PolicyEngine UK does not compute benefit take-up from scratch. The data -build **seeds** stochastic take-up flags (`would_claim_*`) per programme -from published target rates, then runs them through: +build **seeds** stochastic take-up flags (`would_claim_*`) for most +programmes from target rates (some published take-up estimates, some +fitted to caseload or spending, and Universal Credit's 0.55, which has no +cited source in policyengine-uk-data and is held in Microcosm for parity +with the earlier pipeline), then runs them through: 1. **Reported anchoring.** Universal Credit, Pension Credit and Child Benefit flags are inputs, drawn so that units reporting receipt claim. Housing Benefit, Income Support, the tax credits and Council - Tax Reduction instead derive their flags at runtime from reported - receipt (or `claims_all_entitled_benefits`); see + Tax Reduction instead derive their flags at runtime: on data-built + simulations they equal reported receipt, and hand-entered households + also claim when `claims_all_entitled_benefits` is true; see [#1621](https://github.com/PolicyEngine/policyengine-uk/issues/1621). -2. **Reweighting** to match aggregate caseload and expenditure - targets. -3. **SPI integration** for income-tax-relevant variables (which can +2. **SPI integration** for income-tax-relevant variables (which can shift the income distribution and therefore who falls into each benefit's eligible population). +3. **Reweighting** to match aggregate caseload and expenditure + targets. + +Both builds integrate the SPI before reweighting. Pension Credit take-up +(both builds), and Child Benefit and Carer's Allowance take-up +(Microcosm), are redrawn after the SPI step. By the end of the pipeline the *effective* take-up rate, the share of the eligible population that ends up with `would_claim_X == True`, can -diverge from the seed rate by a few percentage points. The seeded rates -themselves were calibrated against pre-reweighted aggregates, so the -real interest is how the post-pipeline shares stack up against -published outturn share figures. +diverge a long way from the seed rate: the initial analysis linked below +found 74.9% of the benefit units entitled to Universal Credit in 2026 +receiving it, against a 55% seed (Enhanced FRS 2023-24, policyengine-uk +2.53.1). Part of the gap is by construction, because the data builds +apply most seed rates to every benefit unit, not only to eligible ones. +Some seed rates are published take-up estimates (Pension Credit, Child +Benefit, Scottish Child Payment, Marriage Allowance); others were fitted +to caseload or spending targets, some at calibrated weights (Tax-Free +Childcare in policyengine-uk-data, the Universal Credit childcare element +in Microcosm). The real interest is how the post-pipeline shares stack up +against published outturn share figures. ## Where the seed rates live @@ -52,10 +73,13 @@ The take-up rate parameters in this repository (for example `gov.dwp.universal_credit.takeup_rate` and `gov.dwp.housing_benefit.takeup`) are reference values: no formula reads them. The rates the data build draws from are in `policyengine_uk_data/parameters/take_up/` in -policyengine-uk-data and in `build/uk/take_up_contract.json` in -Microcosm. The analysis should read the seed rate from the data build +policyengine-uk-data and in +`packages/microcosm-build/src/microcosm/build/uk/take_up_contract.json` +in Microcosm. The analysis should read the seed rate from the data build that produced the dataset, and check the matching parameter here is in -sync. +sync. Two already differ: Pension Credit is 0.7 here against 0.62 in +policyengine-uk-data from April 2023, and Scottish Child Payment for +children aged 6 and over is 0.86 here against 0.85 in both builds. ## Programmes to cover @@ -64,26 +88,28 @@ Per #1354, with current PolicyEngine UK take-up handling: | Programme | Variable | Take-up flag | Reference parameter here | Handling | |-----------|----------|--------------|--------------------------|----------| | Universal Credit | `universal_credit` | `would_claim_uc` (input) | `gov.dwp.universal_credit.takeup_rate` | Drawn in the data build, anchored on reported receipt | -| UC childcare element | `uc_childcare_element_claimed` | `would_claim_uc_childcare` (input) | – | Drawn in Microcosm at single and couple rates; the Enhanced FRS does not set it, so it defaults to true | +| UC on legacy closure | `claims_uc_at_legacy_closure` | `would_claim_uc_at_legacy_closure` (input) | – | Drawn in Microcosm at DWP's Move to Universal Credit claim rates by legacy benefit combination; the Enhanced FRS does not set it, so it defaults to true | +| UC childcare element | `uc_childcare_element` (claimed via `uc_childcare_element_claimed`) | `would_claim_uc_childcare` (input) | – | Drawn in Microcosm at single and couple rates; the Enhanced FRS does not set it, so it defaults to true | | Pension Credit | `pension_credit` | `would_claim_pc` (input) | `gov.dwp.pension_credit.takeup` | Drawn in the data build, anchored on reported receipt | | Child Benefit | `child_benefit` | `would_claim_child_benefit` (input) | `gov.hmrc.child_benefit.takeup.overall` and `.by_age` | Drawn in the data build, anchored on reported receipt | | Housing Benefit | `housing_benefit` | `would_claim_housing_benefit` (formula) | `gov.dwp.housing_benefit.takeup` | Follows reported receipt (#1621) | -| Income Support | `income_support` | `would_claim_IS` (formula) | `gov.dwp.income_support.takeup` | Follows reported receipt (#1621) | -| Child Tax Credit | `child_tax_credit` | `would_claim_CTC` (formula) | `gov.dwp.tax_credits.child_tax_credit.takeup` | Follows reported receipt; scheme ended 2025-04-06 | -| Working Tax Credit | `working_tax_credit` | `would_claim_WTC` (formula) | `gov.dwp.tax_credits.working_tax_credit.takeup` | Follows reported receipt; scheme ended 2025-04-06 | -| Council Tax Reduction | `council_tax_reduction` | `would_claim_council_tax_reduction` (formula) | – | Local schemes simulated; take-up follows reported receipt (#1669 open) | +| Income Support | `income_support` | `would_claim_IS` (formula) | `gov.dwp.income_support.takeup` | Follows reported receipt (#1621); scheme closed 2026-04-01 | +| Income-based JSA / income-related ESA | `jsa_income`, `esa_income` | none | `gov.dwp.JSA.income.takeup`, `gov.dwp.ESA.income.takeup_estimate_fye_2019` | Paid on reported awards that pass capital and work tests; income-based JSA closed 2026-04-01 | +| Child Tax Credit | `child_tax_credit` | `would_claim_CTC` (formula) | `gov.dwp.tax_credits.child_tax_credit.takeup` | Follows reported receipt; scheme ended on 5 April 2025 | +| Working Tax Credit | `working_tax_credit` | `would_claim_WTC` (formula) | `gov.dwp.tax_credits.working_tax_credit.takeup` | Follows reported receipt; scheme ended on 5 April 2025 | +| Council Tax Reduction | `council_tax_reduction` | `would_claim_council_tax_reduction` (formula) | – | Simulated for England's pension-age scheme, the Scottish and Welsh schemes and five English councils' working-age schemes, reported reductions elsewhere; on data-built simulations take-up follows reported receipt | | Tax-Free Childcare | `tax_free_childcare` | `would_claim_tfc` (input) | – | Drawn in the data build at a flat rate | | Universal childcare entitlement | `universal_childcare_entitlement` | `would_claim_universal_childcare` (input) | – | Drawn in the data build at a flat rate | | Extended childcare entitlement | `extended_childcare_entitlement` | `would_claim_extended_childcare` (input) | – | Drawn in the data build at a flat rate | | Targeted childcare entitlement | `targeted_childcare_entitlement` | `would_claim_targeted_childcare` (input) | – | Drawn in the data build at a flat rate | | Marriage Allowance | `marriage_allowance` | `would_claim_marriage_allowance` (input) | `gov.hmrc.income_tax.allowances.marriage_allowance.takeup_rate` (0.5, the post-2019 steady state) | Drawn in the data build at a flat rate | | Scottish Child Payment | `scottish_child_payment` | `would_claim_scp` (input) | `gov.social_security_scotland.scottish_child_payment.takeup_rate` (under 6, 6 and over) | Drawn in the data build by age | -| Carer's Allowance | `carers_allowance` | `would_claim_carers_allowance` (input) | – | Set from reported receipt in Microcosm; the Enhanced FRS does not set it, so it defaults to true | +| Carer's Allowance | `carers_allowance` | `would_claim_carers_allowance` (input) | – | Set from reported receipt in Microcosm, whose main branch since 20 October 2026 also draws a claim at 0.6 for carers outside Scotland whom only the earnings limit keeps from the allowance; the Enhanced FRS does not set it, so it defaults to true | | 16 to 19 Bursary Fund | `bursary_fund_16_to_19` | `would_claim_bursary_fund_16_to_19` (input) | – | Defaults to true | | Student support grants | `adult_dependants_grant`, `childcare_grant`, `parents_learning_allowance`, `travel_grant`, `disabled_students_allowance` | `would_claim_adult_dependants_grant`, `would_claim_childcare_grant`, `would_claim_parents_learning_allowance`, `would_claim_travel_grant`, `would_claim_disabled_students_allowance` (inputs) | – | Default to true | | Care to Learn | `care_to_learn` | `would_claim_care_to_learn` (input) | – | Defaults to true | -| State Pension | `state_pension` | n/a (effectively universal) | – | See [state-pension.md](../programs/gov/dwp/state-pension.md) | -| PIP / DLA / AA / SDA | various | none | – | Category inputs set from reported receipt; see [disability-legacy-benefits.md](../programs/gov/dwp/disability-legacy-benefits.md) | +| State Pension | `state_pension` | none | – | Paid from reported State Pension (`state_pension_reported`); see [state-pension.md](../programs/gov/dwp/state-pension.md) | +| PIP / DLA / AA / SDA | various | none | – | PIP, DLA and AA category inputs set from reported receipt (PIP categories can also come from points); SDA paid at the maximum rate where `sda_reported` is positive; see [disability-legacy-benefits.md](../programs/gov/dwp/disability-legacy-benefits.md) | ## Methodology @@ -107,21 +133,26 @@ For each programme above, the analysis should produce: 4. **Pipeline drift:** `effective − seed` per programme, with a sign convention so positive means more claimants than seeded. -The published reference rates change roughly annually. The analysis -should be repeatable, ideally as a notebook under +Most published reference rates change roughly annually (HMRC's tax +credit take-up rates stop at 2017 to 2018). The analysis should be +repeatable, ideally as a notebook under [`docs/book/validation/`](../validation/) so future builds can re-run -it and update the gaps table. +it and update the gaps table. A notebook needs its own `file:` entry +under Validation in `myst.yml` to appear in the docs, since only +Markdown pages are picked up by folder. ## Output A new validation notebook `docs/book/validation/take-up-rates.ipynb` should: -- pull each programme's seed take-up parameter, +- pull each programme's seed take-up rate from the data build that + produced the dataset, - compute the effective rate against the built dataset, -- compare to the most recent published share (DWP take-up tables for - income-related benefits, HMRC Tax-Free Childcare and tax credits - statistics for the rest), +- compare to the most recent published share (DWP's income-related + benefits take-up tables; HMRC's Child Benefit, Tax-Free Childcare and + tax relief statistics; Scottish Government take-up rates; DfE + childcare statistics), - flag drifts of more than ±2 percentage points for follow-up. The first run is the deliverable for #1354. After that the notebook @@ -132,20 +163,21 @@ data refresh. - **Reported-anchored take-up** is in both data builds for UC, Pension Credit and Child Benefit. The programmes whose flags still follow - reported receipt at runtime are the most likely source of drift - between seed and effective rates. + reported receipt at runtime have no seed draw: their effective rate is + the share of eligible units that report receipt, which the analysis + should compare with the published rate directly. - The **`would_claim_*` input-only conversion** (also in #1621) would give the analysis a cleaner ladder: with all of `would_claim_*` as - inputs, the effective rate is exactly the share assigned by the - data build's take-up stage, and pipeline drift becomes a pure - reweighting question. + inputs, every flag comes from the data build's take-up stage, and + pipeline drift comes only from the steps after it (SPI income + imputation and reweighting). ## References - Issue: [#1354](https://github.com/PolicyEngine/policyengine-uk/issues/1354). -- Initial analysis (UC and CTC): -- DWP, [Income-related benefits: estimates of take-up](https://www.gov.uk/government/collections/income-related-benefits-estimates-of-take-up): the main reference rates for Housing Benefit, Income Support, Pension Credit and Universal Credit. +- Initial analysis (UC and CTC): (policyengine-uk 2.53.1; it measured take-up by setting `gov.dwp.universal_credit.takeup_rate` to 1, which no longer changes take-up on main). +- DWP, [Income-related benefits: estimates of take-up](https://www.gov.uk/government/collections/income-related-benefits-estimates-of-take-up--2): reference rates for Pension Credit and pensioners' Housing Benefit, and (from the financial year ending 2025 edition) a composite Universal Credit (or equivalent benefits) rate. - HMRC, [Personal tax credits statistics](https://www.gov.uk/government/collections/personal-tax-credits-statistics). -- HMRC, [Child Benefit statistics](https://www.gov.uk/government/collections/child-benefit-statistics) and Marriage Allowance statistics. +- HMRC, [Child Benefit: annual release](https://www.gov.uk/government/collections/child-benefit-geographical-statistics), and Marriage Allowance claimants and cost in HMRC's [tax relief statistics](https://www.gov.uk/government/statistics/tax-reliefs/tax-relief-statistics-january-2026). - Pipeline alignment: [#1621](https://github.com/PolicyEngine/policyengine-uk/issues/1621). - Cross-links: [`state-pension.md`](../programs/gov/dwp/state-pension.md), [`disability-legacy-benefits.md`](../programs/gov/dwp/disability-legacy-benefits.md), [`tax-credits.md`](../programs/gov/dwp/tax-credits.md).