diff --git a/changelog.d/disable-simulated-benefits-scoped-awards.fixed.md b/changelog.d/disable-simulated-benefits-scoped-awards.fixed.md new file mode 100644 index 0000000000..3b3224c49a --- /dev/null +++ b/changelog.d/disable-simulated-benefits-scoped-awards.fixed.md @@ -0,0 +1,7 @@ +- A clone or branch that applies a reform now links the tax-benefit system to itself while it does, so a reform that sets inputs, such as `disable_simulated_benefits`, sets the clone's or branch's inputs and never its parent's. A reform applied after the simulation is built (one passed as `reform=`, or applied to a built simulation, clone or branch) now keeps through `reset_calculations` (and, before policyengine-core 3.33.0, `derivative`) every input it stores and leaves in place, however it stores it, including one that replaces an input set since the build; an input set since the build that the reform did not store is still reset, as before, unless the reform stores an input of the same variable (a reset keeps or clears whole variables). +- Under `disable_simulated_benefits`, set the claimant's and partner's income-related ESA and income-based JSA (`claimant_or_partner_esa_income`, `claimant_or_partner_jsa_income`) for each year from the dataset-year reports of whoever is the claimant or partner that year, and set Income Support from their reports alone. The reform holds each benefit at its dataset-year amount while the reported amounts are uprated, so in a later year the stored award matched neither reading of the reports and another member's award was taken to be the claimant's. Another member's own Income Support report is left out of household income, as it is with the reform off, where Income Support is calculated only for the claimant's family. +- Under `disable_simulated_benefits`, no family is moved off its legacy benefits: the reform holds `legacy_benefits_closed` false for its ten years, since it pays every reported legacy award and Universal Credit in every year. So, after the closure of income-based JSA and Income Support from April 2026, an adult outside the couple is still on the income-related ESA the reform pays them, and the closure ends no family's Housing Benefit or tax credit eligibility and prompts no Universal Credit claim. Only the closure is held: the model's other rules still apply, though the reform pays the amounts reported. So a family the model has claim Universal Credit anyway is still not eligible for Housing Benefit or tax credits, and an adult outside the couple is on the income-based JSA they report, as their own claim, only in the years the model has the scheme open (to 2025); the claimant and partner are on the award the reform holds for them. +- Under `disable_simulated_benefits`, count a Scottish carer's reported Carer's Allowance once. The reform set `carers_allowance` from the report in every country and left `carer_support_payment` simulated, so from 2025, when the model pays a carer in Scotland Carer Support Payment in place of Carer's Allowance, a Scottish carer was paid both. Now, as with the winter heating payments, in each year the report is the `carer_support_payment` of a carer living in Scotland that year, from 2025, and their `carers_allowance` before then, and everyone else's `carers_allowance`; neither benefit is simulated beside it. The entitlements before the overlapping-benefit reduction stay the model's for both. This split and the winter heating one read where the household lives in each year, as the model does, when the reform is applied, so a household that moves into or out of Scotland is paid under that year's scheme: before, both read the dataset year's country, so a carer who moved to Scotland was paid Carer's Allowance and lost the Scottish Carer Supplement. +- Neutralizing `esa_income` or `jsa_income` also neutralizes the claimant's and partner's award, so an award entered for them, or set by the reform, no longer outlives the benefit it is part of. +- Read the claimant's and partner's income-related ESA and income-based JSA in the benefit cap (its total of welfare benefits, and the ESA support component exemptions from both the Universal Credit and Housing Benefit caps) from `claimant_or_partner_esa_income` and `claimant_or_partner_jsa_income` wherever the cap's couple is the claimant and partner. Under `disable_simulated_benefits` the cap read an award held for another member of the benefit unit (a non-dependent adult claiming in their own right) as the couple's in every year after the dataset year, so it raised the cap's total and could exempt a claimant with a support component who receives no ESA. An award entered for those variables directly now reaches the cap too, and an award within half a penny of zero is no award there, as elsewhere. A Universal Credit couple that is not the claimant and partner (only with `is_uc_claimant` or `is_uc_assessed_claimant` entered, or more than two members entered as claimant or partner) is still read from `esa_income` and `jsa_income` by value. +- Read the claimant's and partner's income-related ESA and income-based JSA in `income_support_eligible` from `claimant_or_partner_esa_income` and `claimant_or_partner_jsa_income`, as the means tests and passports do, so an award entered for them directly, or set by the reform, is read the same way everywhere. The two variables share one formula, which now treats an award within half a penny of zero as no award, as `income_support_eligible` did, so the float residual left by reports that exactly match the tariff income is no longer the claimant's or partner's award, and no passport or status that reads their award counts it. A member outside the couple is still on their own award as before: it is read from their own report, without that rule. When `esa_income` or `jsa_income` holds the plain total of the reported awards, Income Support now reads the plain total of the claimant's and partner's reports, as the two variables already did; it used to read their award after the capital test. diff --git a/policyengine_uk/reforms/policyengine/disable_simulated_benefits.py b/policyengine_uk/reforms/policyengine/disable_simulated_benefits.py index 315604c9c5..720eb6432f 100644 --- a/policyengine_uk/reforms/policyengine/disable_simulated_benefits.py +++ b/policyengine_uk/reforms/policyengine/disable_simulated_benefits.py @@ -1,15 +1,18 @@ +import numpy as np from policyengine_core.model_api import * -# Benefits set to the amount the survey reports, _reported, in place -# of the simulated amount. Attendance Allowance, DLA and PIP are not listed: -# they are paid at the rate of the award category the survey reports -# (aa_category, dla_sc_category, dla_m_category, pip_dl_category and -# pip_m_category), so they already follow the reported award. The winter -# heating payments are set separately (see below). +from policyengine_uk.utils.benefit_unit import CLAIMANT_OR_PARTNER_AWARDS + +# Benefits whose simulated amount the reform replaces with the amount reported +# in the dataset year. Each needs a _reported variable. Attendance +# Allowance, DLA and PIP are not listed: the data records their receipt as a +# rate category (aa_category, dla_m_category, dla_sc_category, pip_dl_category, +# pip_m_category), and the model pays that category's rate. The model has no +# reported amount for them to copy. Carer's Allowance and the winter heating +# payments are set separately (see below). BENEFITS = [ "afcs", "bsp", - "carers_allowance", "child_benefit", "child_tax_credit", "council_tax_benefit", @@ -29,6 +32,52 @@ "working_tax_credit", ] +# Tax credits are not paid below a minimum award, and each has a variable +# before that minimum, which takes the same reported amount. +PRE_MINIMUM = { + "child_tax_credit": "child_tax_credit_pre_minimum", + "working_tax_credit": "working_tax_credit_pre_minimum", +} + +# CLAIMANT_OR_PARTNER_AWARDS (utils/benefit_unit.py) names the benefit-unit +# awards whose claimant's and partner's part has its own variable. The reform +# sets that part from their own reports. The reported amounts of anyone else +# in the benefit unit (such as a non-dependent adult, who claims in their own +# right) stay in esa_income and jsa_income, which count in household income. +# They never enter the claimant's or partner's award. + +# Income Support is the claimant's family's award, so it is set from the +# claimant's and partner's reports only. That is all the model ever pays as +# income_support: with the reform off, would_claim_IS and +# income_support_eligible read only their reports too. Another member's own +# Income Support report (a non-dependent adult claiming in their own right) is +# in household income neither with the reform nor without it. ESA and JSA +# differ: esa_income and jsa_income count every member's award either way. +CLAIMANT_OR_PARTNER_ONLY = ["income_support"] + +# The model's move off legacy benefits (legacy_benefits_closed): once a legacy +# benefit a family reports has closed, DWP's migration notice ends all its +# legacy awards, whether or not the family then claims Universal Credit, +# unless DWP sent it to Pension Credit instead. The reform pays every listed +# benefit, the legacy ones and Universal Credit alike, at its dataset-year +# report in every year, so it moves no family: it holds these false for its +# years. The closure then ends none of the family's legacy benefits (their +# eligibility and the "on" statuses read from it, and Housing Benefit and tax +# credit eligibility), prompts no Universal Credit claim +# (claims_uc_at_legacy_closure) and takes no family off the Pension Credit +# route. Only the closure is held: the model's other rules still apply, and +# the reform still pays the amounts reported. A family the model has claim +# Universal Credit anyway (would_claim_uc) is still not eligible for Housing +# Benefit or tax credits. An adult outside the couple who reports +# income-based JSA is on it, as their own claim, only in the years the model +# has the scheme open (gov.dwp.JSA.income.active: to 2025). +NOT_MOVED = ["legacy_benefits_closed"] + +# The first year in which the model pays a carer in Scotland Carer Support +# Payment in place of Carer's Allowance: carers_allowance_pre_overlap and +# carer_support_payment_pre_overlap both switch on period.start.year >= 2025. +CARER_SUPPORT_PAYMENT_FROM = 2025 + YEARS_IN_FUTURE = 10 @@ -36,37 +85,154 @@ def disable_simulated_benefits(parameters, period): if parameters(period).gov.contrib.policyengine.disable_simulated_benefits: class DisableSimulatedBenefits(Reform): + """Use the dataset year's reported benefit amounts in every year. + + Each listed benefit is set, in the dataset year and the nine years + after it, to its amount reported in the dataset year. The amount + is not uprated, while most of the *_reported variables are. + Carer's Allowance and the winter heating payment are each reported + once and set the same way, under the scheme that pays them in each + year: for a household in Scotland that year, Carer Support Payment + and the Pension Age Winter Heating Payment in the years the model + pays those instead. + + The claimant-or-partner awards are set the same way, from the + dataset-year reports of whoever is the claimant or partner in + each year (is_claimant_or_partner); Income Support is set from + those reports alone. So every amount is the same in every year, + except that these three change when who is the claimant or + partner changes. Readers of those awards, the Income Support + gate and the benefit cap included, cannot tell an award set here + from one entered directly. In a later year, esa_income and + jsa_income match neither the award on that year's uprated reports + nor their total. Without the claimant-or-partner awards set here, + the readers would take the whole of esa_income or jsa_income, + including another member's award, to be the claimant's or + partner's. The benefit cap reads them through award_of_couple; + for a Universal Credit couple that is not the claimant and + partner (see award_of_couple), it still reads esa_income and + jsa_income by value. + + No family is moved off its legacy benefits (NOT_MOVED): the + reform pays them as reported in every year, after the dates + from which the model closes them included. Only the closure is + held; the model's other eligibility rules still apply. + + The reform reads what it uses when it is applied: the dataset + year's reports, and who is the claimant or partner and where each + household lives in each year. When the parameter switches it on, + that is in the constructor, after the data is built and before a + scenario's modifier applied after data load runs. The amounts it + sets do not follow a change made after that (by such a modifier, + or set_input): to have it read one, make the change in the + situation or dataset, or apply the reform after the change + (Simulation.apply_reform). + + As the amounts are inputs, a reform that changes a benefit's + parameters does not change them. A reform that neutralizes a + benefit removes it, and neutralizing esa_income or jsa_income + also neutralizes its claimant-or-partner award + (CountryTaxBenefitSystem.variable_parts). A neutralized variable + ignores what the reform sets for it. + """ + def apply(self): simulation = self.simulation - time_period = int(simulation.dataset.time_period) + system = simulation.tax_benefit_system + time_period = int(str(simulation.dataset.time_period)[:4]) years = range(time_period, time_period + YEARS_IN_FUTURE) - for variable in BENEFITS: - entity = simulation.tax_benefit_system.variables[ - variable - ].entity.key - reported_value = simulation.calculate( - variable + "_reported", time_period, map_to=entity + def reported(variable, members=None): + """A benefit's reported amount in the dataset year, for + the given members, summed over the benefit's entity.""" + amounts = simulation.calculate(variable + "_reported", time_period) + if members is not None: + amounts = amounts * members + entity = system.variables[variable].entity.key + population = simulation.populations[entity] + if population.entity.is_person: + return amounts + return population.sum(amounts) + + for year in years: + claimant_or_partner = simulation.calculate( + "is_claimant_or_partner", year ) - for year in years: - simulation.set_input(variable, year, reported_value) + values = {} + for variable in BENEFITS: + members = ( + claimant_or_partner + if variable in CLAIMANT_OR_PARTNER_ONLY + else None + ) + values[variable] = reported(variable, members) + if variable in PRE_MINIMUM: + values[PRE_MINIMUM[variable]] = values[variable] + if variable in CLAIMANT_OR_PARTNER_AWARDS: + values[CLAIMANT_OR_PARTNER_AWARDS[variable]] = reported( + variable, claimant_or_partner + ) + for variable in NOT_MOVED: + entity = system.variables[variable].entity.key + count = simulation.populations[entity].count + values[variable] = np.zeros(count, dtype=bool) + for target, value in values.items(): + simulation.set_input(target, year, value) - if variable in ["child_tax_credit", "working_tax_credit"]: - # CTC and WTC have their own pre_minimum variables because tax credits aren't paid if - # below a threshold. - variable = variable + "_pre_minimum" - for year in years: - simulation.set_input(variable, year, reported_value) + # Where a household lives is read in each year, as the model + # reads it (country is a YEAR variable). The reported amounts + # are held at the dataset year, but a household that moves + # into or out of Scotland is paid under that year's scheme. + # It is read now, when the reform is applied (see the class + # docstring). + in_scotland = { + year: np.asarray(simulation.calculate("country", year)) + == "SCOTLAND" + for year in years + } + + # The survey reports Carer's Allowance for every respondent, + # Scotland included (carers_allowance_reported). From + # CARER_SUPPORT_PAYMENT_FROM the model pays a carer in + # Scotland Carer Support Payment in its place, so in each year + # the report of a carer living in Scotland that year becomes + # their carer_support_payment from then, and their + # carers_allowance before. Both variables are household + # income, so each report is counted once, under the scheme + # that pays it, and neither benefit is simulated beside it. + # The entitlements before the overlapping-benefit reduction + # (carers_allowance_pre_overlap, + # carer_support_payment_pre_overlap) stay the model's, for + # both schemes alike. + reported_care = np.asarray( + simulation.calculate("carers_allowance_reported", time_period) + ) + households = simulation.populations["household"] + for year in years: + paid_as_csp = households.project(in_scotland[year]) & ( + year >= CARER_SUPPORT_PAYMENT_FROM + ) + simulation.set_input( + "carers_allowance", + year, + np.where(paid_as_csp, 0, reported_care), + ) + simulation.set_input( + "carer_support_payment", + year, + np.where(paid_as_csp, reported_care, 0), + ) # The survey reports one winter heating payment: FRS benefit # code 62, which policyengine-uk-data loads into # winter_fuel_allowance_reported for every respondent, # Scotland included. Scotland's payment has been the Pension # Age Winter Heating Payment since the 2024 qualifying week, - # so in each year a Scottish household's report becomes its - # pawhp while PAWHP is paid, and its winter_fuel_allowance - # before then. Both variables are household income, so each - # report is counted once, under the scheme that pays it. + # so in each year the report of a household living in Scotland + # that year becomes its pawhp while PAWHP is paid, and its + # winter_fuel_allowance before then. Both variables are + # household income, so each report is counted once, under the + # scheme that pays it. reported_winter_heating = np.asarray( simulation.calculate( "winter_fuel_allowance_reported", @@ -74,15 +240,13 @@ def apply(self): map_to="household", ) ) - in_scotland = ( - np.asarray(simulation.calculate("country", time_period)) - == "SCOTLAND" - ) + # Read from the system of the simulation applying the reform: + # when the parameter switches the reform on, that is the tree + # it was built from; a reform class applied later may have been + # built from another. for year in years: - paid_as_pawhp = ( - in_scotland - & parameters(year).gov.social_security_scotland.pawhp.active - ) + scheme = system.parameters(year).gov.social_security_scotland + paid_as_pawhp = in_scotland[year] & scheme.pawhp.active simulation.set_input( "winter_fuel_allowance", year, diff --git a/policyengine_uk/simulation.py b/policyengine_uk/simulation.py index 3f769e8950..29e2c17a02 100644 --- a/policyengine_uk/simulation.py +++ b/policyengine_uk/simulation.py @@ -105,6 +105,46 @@ def _canonical_parameter_paths(changes: dict) -> dict: from microdf import MicroDataFrame + +class _InputKeyLog(set): + """A simulation's record of its inputs while Simulation.apply_reform runs. + + policyengine-core records each input in the simulation's + ``_user_input_keys`` as (variable, branch, period). ``Holder._set`` adds + the key once it has stored the value, again when it replaces one, and on + every route: ``Simulation.set_input``, ``Holder.set_input``, a + ``set_input`` helper, another thread. A value core does not store adds + no key: one ``Simulation.set_input`` gets for a period after the + variable's end, one for a neutralized variable, or one for a sub-period + that a ``set_input`` helper skips because it already holds an input. + This set also adds each key to every set in ``logs``: one for each + reform being applied to the simulation, the innermost last. + + A copy or pickle of it is a plain set, so a simulation deep-copied or + pickled while a reform runs logs nothing. A shallow copy of the + simulation shares its record, as it shares its holders. + """ + + def __init__(self, keys=(), logs: list = ()): + super().__init__(keys) + self.logs = logs + + def add(self, key) -> None: + super().add(key) + for log in self.logs: + log.add(key) + + def __reduce__(self): + return set, (list(self),) + + +# The logs (see _InputKeyLog) of the reforms Simulation.apply_reform is +# applying, by the id of the simulation it applies them to, while it does. +# Kept outside the simulation, because core's clone copies its __dict__: +# a clone or branch made while a reform runs must not log for it. +_reform_logs: Dict[int, list] = {} + + # Cache for fully-loaded, uprated, enum-pre-encoded multi-year datasets keyed # by URL and, when selected, the CGT projection fingerprint and runtime. # Avoids repeating HDF5 reading (0.84s), uprating (0.69s) and enum @@ -331,14 +371,12 @@ def __init__( self.tax_benefit_system.reset_parameter_caches() - # Core applies a reform by calling its apply() with the tax-benefit - # system as self, so reforms that change the data (such as - # disable_simulated_benefits) reach the simulation through - # tax_benefit_system.simulation. Core's Simulation.__init__ sets it; - # this __init__ replaces core's, so it sets it too. + # Apply structural reforms based on parameters. Core applies a reform + # to the tax-benefit system, so reforms that set inputs (such as + # disable_simulated_benefits and adjust_budgets) reach the simulation + # through it. Core's own constructor sets this link; this one does not + # call it, so set it here (apply_reform also sets it for each reform). self.tax_benefit_system.simulation = self - - # Apply structural reforms based on parameters structural_reform = create_structural_reforms_from_parameters( self.tax_benefit_system.parameters, period_(self.default_input_period), @@ -393,8 +431,85 @@ def _refresh_capital_gains_projection(self) -> None: def apply_reform(self, reform: dict | tuple | Type[Reform]) -> None: """Apply a runtime reform while keeping generated CGT assumptions coherent.""" - super().apply_reform(reform) - self._refresh_capital_gains_projection() + # Reforms that set inputs (disable_simulated_benefits, adjust_budgets) + # reach the simulation through its tax-benefit system's link. A clone + # or branch may share its parent's system, so link the system to this + # simulation while the reform is applied, then restore the link. + system = self.tax_benefit_system + linked = getattr(system, "simulation", None) + system.simulation = self + # Log the inputs the reform stores (see _InputKeyLog and the + # _user_input_keys property). Within another reform applied to this + # simulation (a tuple's parts, or a reform that applies one), log + # them for that one too. + logs = _reform_logs.setdefault(id(self), []) + stored = set() + logs.append(stored) + self._user_input_keys = getattr(self, "_user_input_keys", None) or set() + try: + super().apply_reform(reform) + self._refresh_capital_gains_projection() + finally: + logs.pop() + if not logs: + del _reform_logs[id(self)] + # A plain set again, unless an enclosing reform is still logging. + self._user_input_keys = set(self._user_input_keys) + if linked is not None: + system.simulation = linked + # input_variables is taken when the simulation is built. + # reset_calculations deletes the values of every variable it does not + # name (as core's derivative did before policyengine-core 3.33.0, from + # which it keeps every input), and the marginal rate formulas those of + # every such variable that has a formula. A reform applied afterwards + # (a Reform class passed as reform=, or one applied to a clone or + # branch) can store inputs: new ones, or ones that replace an input + # set since the build. Name every variable with an input the reform + # stored that this simulation still holds, under a branch name it + # reads and a reset deletes, or a reset would delete what the reform + # stored and recalculate the formula. input_variables names whole + # variables, so a reset then keeps every input of the variable, any + # set since the build included. A reform that raises names nothing it + # stored itself (a reform it applied first has named its own). Assign + # a new list: a clone or branch shares its parent's. + input_variables = getattr(self, "input_variables", None) + if input_variables is not None: + branches = set(self._get_visible_branch_names()) + supplied = { + name + for name, branch, _ in stored & self._user_input_keys + if branch in branches + } + added = sorted(supplied - set(input_variables)) + if added: + self.input_variables = list(input_variables) + added + + @property + def _user_input_keys(self) -> set: + # policyengine-core's record of the simulation's inputs, kept in + # __dict__ as core keeps it. While apply_reform applies a reform to + # this simulation, a set assigned here becomes a log (see + # _InputKeyLog), so a record core replaces mid-reform still logs: a + # branch's, when a reform applied to its parent rebuilds it, or one + # rebuilt when the system is rebound. + try: + return self.__dict__["_user_input_keys"] + except KeyError: + raise AttributeError("_user_input_keys") from None + + @_user_input_keys.setter + def _user_input_keys(self, keys) -> None: + logs = _reform_logs.get(id(self)) + if logs and keys is not None: + keys = _InputKeyLog(keys, logs) + self.__dict__["_user_input_keys"] = keys + + @_user_input_keys.deleter + def _user_input_keys(self) -> None: + try: + del self.__dict__["_user_input_keys"] + except KeyError: + raise AttributeError("_user_input_keys") from None def delete_arrays(self, variable: str, period: Period = None) -> None: super().delete_arrays(variable, period) diff --git a/policyengine_uk/tax_benefit_system.py b/policyengine_uk/tax_benefit_system.py index 8a67912988..cfd1f67801 100644 --- a/policyengine_uk/tax_benefit_system.py +++ b/policyengine_uk/tax_benefit_system.py @@ -58,6 +58,7 @@ add_lsr_deprecation_aliases, canonicalize_lsr_parameter_path, ) +from policyengine_uk.utils.benefit_unit import CLAIMANT_OR_PARTNER_AWARDS from policyengine_uk.utils.dated_reforms import ( BACKDATED_FROM, CALENDAR_YEAR_SERIES, @@ -223,6 +224,20 @@ class CountryTaxBenefitSystem(TaxBenefitSystem): variables: Dict[str, Variable] + # Variables that hold part of another variable's value: each award's + # claimant-or-partner part (CLAIMANT_OR_PARTNER_AWARDS). Neutralizing a + # variable neutralizes its parts, so a value entered or set for a part + # (as disable_simulated_benefits sets the claimant's and partner's awards + # for each year) cannot outlive the award it is part of. + variable_parts: Dict[str, Tuple[str, ...]] = { + award: (part,) for award, part in CLAIMANT_OR_PARTNER_AWARDS.items() + } + + def neutralize_variable(self, variable_name: str) -> None: + super().neutralize_variable(variable_name) + for part in self.variable_parts.get(variable_name, ()): + super().neutralize_variable(part) + def reset_parameter_caches(self): """Reset all caches in the tax-benefit system.""" self._parameters_at_instant_cache = {} diff --git a/policyengine_uk/tests/test_disable_simulated_benefits.py b/policyengine_uk/tests/test_disable_simulated_benefits.py new file mode 100644 index 0000000000..079089376a --- /dev/null +++ b/policyengine_uk/tests/test_disable_simulated_benefits.py @@ -0,0 +1,794 @@ +"""disable_simulated_benefits pays each family the benefits it reported. + +The reform (gov.contrib.policyengine.disable_simulated_benefits) sets each +listed benefit, in the dataset year and the nine years after it, to the +amount reported in the dataset year. esa_income_reported and +jsa_income_reported are uprated, so in a later year a stored esa_income or +jsa_income equals neither the award on that year's reports nor their plain +total. Read by value alone, it would then +be taken to be wholly the claimant's or partner's, another member's award +included. So the reform also sets the claimant's and partner's awards +(claimant_or_partner_esa_income, claimant_or_partner_jsa_income) for each +year, from the dataset-year reports of whoever is the claimant or partner +that year, and Income Support from theirs alone. Both of Income Support's income-related limbs (SSCBA 1992 +s.124(1)(f) and (h)) and the council tax reduction passport read those +awards. + +Each family below is its own benefit unit and household. Reported amounts are +entered for the dataset year only, so later years uprate them. Everything +else is entered for every year. +""" + +from types import SimpleNamespace + +import numpy as np +import pandas as pd +import pytest +from hypothesis import HealthCheck, given, settings +from hypothesis import strategies as st + +from policyengine_core.periods import period + +from policyengine_uk import CountryTaxBenefitSystem, Simulation +from policyengine_uk.data import UKMultiYearDataset, UKSingleYearDataset +from policyengine_uk.model_api import Reform +from policyengine_uk.reforms.policyengine.disable_simulated_benefits import ( + BENEFITS, + CLAIMANT_OR_PARTNER_AWARDS, + CLAIMANT_OR_PARTNER_ONLY, + NOT_MOVED, + PRE_MINIMUM, + YEARS_IN_FUTURE, + disable_simulated_benefits, +) +from policyengine_uk.utils.benefit_unit import claimant_or_partner_award +from policyengine_uk.utils.scenario import Scenario +from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award + +DATASET_YEAR = 2025 +# The dataset year and two later years. +YEARS = [DATASET_YEAR, DATASET_YEAR + 1, DATASET_YEAR + 2] + +REFORM = Scenario( + applied_before_data_load=True, + parameter_changes={"gov.contrib.policyengine.disable_simulated_benefits": True}, +) + + +def every_year(value): + return {year: value for year in YEARS} + + +def adult(age, claimant_or_partner, carer=False, **reported): + person = { + "age": every_year(age), + "is_claimant_or_partner": every_year(claimant_or_partner), + "receives_carer_benefit": every_year(carer), + "current_education": every_year("NOT_IN_EDUCATION"), + } + for name, amount in reported.items(): + person[name] = {DATASET_YEAR: amount} + return person + + +# A carer who reports Income Support satisfies every other condition of +# income_support_eligible, so in these families only the income-related +# limbs can bar the claim. +CARER = dict(age=40, claimant_or_partner=True, carer=True) +NOT_IN_COUPLE = dict(age=30, claimant_or_partner=False) + +# family: (members, capital) +FAMILIES = { + # An adult outside the couple, such as a non-dependent, claims ESA and + # JSA in their own right. + "excluded_adult": ( + [ + adult(**CARER, income_support_reported=1_000), + adult( + **NOT_IN_COUPLE, esa_income_reported=3_000, jsa_income_reported=2_000 + ), + ], + 0, + ), + # The same, with no Income Support award to passport the claimant. + "excluded_adult_no_award": ( + [ + adult(age=40, claimant_or_partner=True), + adult( + **NOT_IN_COUPLE, esa_income_reported=3_000, jsa_income_reported=2_000 + ), + ], + 0, + ), + # The claimant's own income-related ESA. + "own_esa": ( + [adult(**CARER, income_support_reported=1_000, esa_income_reported=3_000)], + 0, + ), + # The claimant's own income-based JSA. + "own_jsa": ( + [adult(**CARER, income_support_reported=1_000, jsa_income_reported=2_000)], + 0, + ), + # The claimant's own £200 of ESA with £10,000 of capital: tariff income of + # £832 a year leaves no award on the formula's reading, but the reform + # pays the reported amount, before any screen. + "own_esa_over_tariff": ( + [adult(**CARER, income_support_reported=1_000, esa_income_reported=200)], + 10_000, + ), + # Only an adult outside the couple reports Income Support. + "excluded_income_support": ( + [ + adult(age=40, claimant_or_partner=True), + adult(**NOT_IN_COUPLE, income_support_reported=1_000), + ], + 0, + ), +} +NAMES = list(FAMILIES) + + +def situation(): + people, benunits, households = {}, {}, {} + for family, (members, capital) in FAMILIES.items(): + names = [] + for i, person in enumerate(members): + name = f"{family}_{i}" + people[name] = person + names.append(name) + benunits[family] = { + "members": names, + "income_support_assessable_capital": every_year(capital), + "esa_income_assessable_capital": every_year(capital), + "jsa_income_assessable_capital": every_year(capital), + } + households[family] = {"members": names} + return {"people": people, "benunits": benunits, "households": households} + + +def by_family(simulation, variable, year): + return dict(zip(NAMES, simulation.calculate(variable, year).tolist())) + + +def by_person(simulation, variable, year): + names = [ + f"{family}_{i}" + for family, (members, _) in FAMILIES.items() + for i in range(len(members)) + ] + return dict(zip(names, simulation.calculate(variable, year).tolist())) + + +@pytest.fixture(scope="module") +def reformed(): + return Simulation(situation=situation(), scenario=REFORM) + + +@pytest.fixture(scope="module") +def unreformed(): + return Simulation(situation=situation()) + + +def test_every_listed_benefit_has_what_the_reform_reads_and_sets(): + # The reform used to list Attendance Allowance, DLA and PIP, which have + # no reported amount, and so raised before setting anything. + variables = CountryTaxBenefitSystem().variables + for benefit in BENEFITS: + assert benefit in variables, benefit + assert f"{benefit}_reported" in variables, benefit + for name in [ + *PRE_MINIMUM.values(), + *CLAIMANT_OR_PARTNER_AWARDS.values(), + *CLAIMANT_OR_PARTNER_ONLY, + *NOT_MOVED, + ]: + assert name in variables, name + assert set(PRE_MINIMUM) <= set(BENEFITS) + assert set(CLAIMANT_OR_PARTNER_AWARDS) <= set(BENEFITS) + assert set(CLAIMANT_OR_PARTNER_ONLY) <= set(BENEFITS) + + +def test_the_reform_is_off_by_default(): + simulation = Simulation(situation=situation()) + for variable in ["esa_income", "claimant_or_partner_esa_income"]: + assert simulation.get_holder(variable).get_known_periods() == [] + + +def test_the_reform_sets_the_awards_for_ten_years(reformed): + expected = list(range(DATASET_YEAR, DATASET_YEAR + YEARS_IN_FUTURE)) + for variable in [ + "esa_income", + "jsa_income", + "income_support", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ]: + periods = reformed.get_holder(variable).get_known_periods() + assert sorted(int(str(p)[:4]) for p in periods) == expected, variable + + +@pytest.mark.parametrize("year", YEARS) +def test_the_stored_awards_are_the_dataset_years_reports(reformed, year): + # Every member's report, at its dataset-year amount in every year. + assert by_family(reformed, "esa_income", year) == { + "excluded_adult": 3_000, + "excluded_adult_no_award": 3_000, + "own_esa": 3_000, + "own_jsa": 0, + "own_esa_over_tariff": 200, + "excluded_income_support": 0, + } + assert by_family(reformed, "jsa_income", year) == { + "excluded_adult": 2_000, + "excluded_adult_no_award": 2_000, + "own_esa": 0, + "own_jsa": 2_000, + "own_esa_over_tariff": 0, + "excluded_income_support": 0, + } + + +@pytest.mark.parametrize("year", YEARS[1:]) +def test_later_years_need_the_awards_set_explicitly(reformed, year): + # In a later year the reports have been uprated, so the stored £3,000 + # equals neither their plain total nor the award on them. Read by value + # alone, it would be the claimant's or partner's: the excluded adult's + # ESA would become the couple's. + reported_total = reformed.calculate("esa_income_reported", year, map_to="benunit") + excluded = NAMES.index("excluded_adult") + assert reported_total[excluded] > 3_000 + by_value_alone = claimant_or_partner_award( + reformed.populations["benunit"], + period(year), + "esa_income", + "esa_income_reported", + income_related_esa_award, + ) + assert by_value_alone[excluded] == 3_000 + + +@pytest.mark.parametrize("year", YEARS) +def test_the_claimant_or_partner_awards_are_their_own_reports(reformed, year): + assert by_family(reformed, "claimant_or_partner_esa_income", year) == { + "excluded_adult": 0, + "excluded_adult_no_award": 0, + "own_esa": 3_000, + "own_jsa": 0, + "own_esa_over_tariff": 200, + "excluded_income_support": 0, + } + assert by_family(reformed, "claimant_or_partner_jsa_income", year) == { + "excluded_adult": 0, + "excluded_adult_no_award": 0, + "own_esa": 0, + "own_jsa": 2_000, + "own_esa_over_tariff": 0, + "excluded_income_support": 0, + } + # Income Support is the claimant's family's award, from their own reports. + assert by_family(reformed, "income_support", year) == { + "excluded_adult": 1_000, + "excluded_adult_no_award": 0, + "own_esa": 1_000, + "own_jsa": 1_000, + "own_esa_over_tariff": 1_000, + "excluded_income_support": 0, + } + + +@pytest.mark.parametrize("year", YEARS) +def test_both_income_support_limbs_read_the_claimant_or_partner_awards(reformed, year): + # The carer with an Income Support award is eligible beside an adult + # outside the couple who reports ESA and JSA: neither limb bars the claim. + # The same carer's own ESA bars it under s.124(1)(h), and their own JSA + # under s.124(1)(f), every year. The reform pays the £200 of ESA the + # tariff income would extinguish, so that bars it too. + assert by_family(reformed, "income_support_eligible", year) == { + "excluded_adult": True, + "excluded_adult_no_award": False, + "own_esa": False, + "own_jsa": False, + "own_esa_over_tariff": False, + "excluded_income_support": False, + } + + +@pytest.mark.parametrize("year", YEARS) +def test_the_council_tax_reduction_passport(reformed, year): + # The working-age schemes passport an applicant or partner on Income + # Support, income-based JSA or income-related ESA (SI 2012/2886 Sch, + # para 33(2)). Another member's award passports nobody else. + assert by_family( + reformed, "council_tax_reduction_relevant_income_based_benefit", year + ) == { + "excluded_adult": True, + "excluded_adult_no_award": False, + "own_esa": True, + "own_jsa": True, + "own_esa_over_tariff": True, + "excluded_income_support": False, + } + + +@pytest.mark.parametrize("year", YEARS) +def test_no_family_is_moved_off_its_legacy_benefits(reformed, unreformed, year): + # Income-based JSA and Income Support close from April 2026, and the + # model then moves a working-age family reporting either, unless it is on + # Pension Credit, off all its legacy benefits (legacy_benefits_closed), so + # their awards end. Every family here reports one of them. The reform pays every reported award in every + # year, so it holds the move off; otherwise an adult outside the couple + # would stop being on the ESA the reform still pays them, and on its JSA + # while income-based JSA is open (test_each_person_is_on_their_own_award). + moved = year > DATASET_YEAR + assert by_family(unreformed, "legacy_benefits_closed", year) == dict.fromkeys( + NAMES, moved + ) + assert by_family(reformed, "legacy_benefits_closed", year) == dict.fromkeys( + NAMES, False + ) + ten_years = list(range(DATASET_YEAR, DATASET_YEAR + YEARS_IN_FUTURE)) + for variable in NOT_MOVED: + assert known_years(reformed, variable) == ten_years, variable + + +@pytest.mark.parametrize("year", YEARS) +def test_each_person_is_on_their_own_award(reformed, year): + on_esa = by_person(reformed, "is_on_income_related_esa", year) + on_jsa = by_person(reformed, "is_on_income_based_jsa", year) + # An adult outside the couple claims in their own right. The reform pays + # the income-based JSA they report in every year, but their own claim is + # on it only in the years the model has income-based JSA open + # (gov.dwp.JSA.income.active, false from 2026): the reform holds the move + # off legacy benefits (NOT_MOVED), not the closure of the scheme. The + # claimant and partner are on the award the reform holds for them in + # every year. + jsa_open = reformed.tax_benefit_system.parameters( + str(year) + ).gov.dwp.JSA.income.active + assert jsa_open == (year == DATASET_YEAR) + assert not on_esa["excluded_adult_no_award_0"] + assert on_esa["excluded_adult_no_award_1"] + assert not on_jsa["excluded_adult_no_award_0"] + assert on_jsa["excluded_adult_no_award_1"] == jsa_open + assert on_esa["own_esa_0"] and not on_jsa["own_esa_0"] + assert on_jsa["own_jsa_0"] and not on_esa["own_jsa_0"] + + +def test_a_multi_year_dataset_uses_its_first_year(): + # A situation's dataset stores its year as text and a multi-year dataset + # as a number; both start the reform's ten years. Here the second year's + # reports differ, but the reform reads the first year's. The second + # member is 18 and the first 40, so after the data's two years the + # model's presumption reads them as the claimant's child, as the data + # does, and the roles are the same in every year. + def year(fiscal_year, esa): + person = pd.DataFrame( + { + "person_id": [1, 2, 3], + "person_benunit_id": [1, 1, 2], + "person_household_id": [1, 1, 2], + "age": [40, 18, 40], + "is_claimant_or_partner": [True, False, True], + "esa_income_reported": esa, + } + ) + return UKSingleYearDataset( + person=person, + benunit=pd.DataFrame({"benunit_id": [1, 2]}), + household=pd.DataFrame({"household_id": [1, 2]}), + fiscal_year=fiscal_year, + ) + + dataset = UKMultiYearDataset( + datasets=[ + year(DATASET_YEAR, [0.0, 3_000.0, 3_000.0]), + year(DATASET_YEAR + 1, [0.0, 3_100.0, 3_100.0]), + ] + ) + simulation = Simulation(dataset=dataset, scenario=REFORM) + for year_ in YEARS: + roles = simulation.calculate("is_claimant_or_partner", year_) + assert roles.tolist() == [True, False, True] + assert simulation.calculate("esa_income", year_).tolist() == [3_000, 3_000] + assert simulation.calculate( + "claimant_or_partner_esa_income", year_ + ).tolist() == [0, 3_000] + + +REPORTED = ["esa_income_reported", "jsa_income_reported", "income_support_reported"] + + +@st.composite +def reporting_families(draw): + """One to three adults, the first one or two the claimant and partner, + each reporting ESA, JSA and Income Support in the dataset year.""" + amounts = st.sampled_from([0, 0, 200, 3_000, 65_536.01]) + n_couple = draw(st.integers(1, 2)) + n_others = draw(st.integers(0, 2)) + members = [] + for i in range(n_couple + n_others): + members.append( + { + "age": every_year(40 - i), + "is_claimant_or_partner": every_year(i < n_couple), + "receives_carer_benefit": every_year(draw(st.booleans())), + "current_education": every_year("NOT_IN_EDUCATION"), + **{name: {DATASET_YEAR: draw(amounts)} for name in REPORTED}, + } + ) + return members, draw(st.sampled_from([0, 10_000, 20_000])) + + +@settings( + max_examples=10, + deadline=None, + derandomize=True, + suppress_health_check=[HealthCheck.too_slow, HealthCheck.data_too_large], +) +@given(st.lists(reporting_families(), min_size=1, max_size=8)) +def test_the_reform_splits_every_familys_reports_between_the_awards(drawn): + people, benunits, households = {}, {}, {} + for i, (members, capital) in enumerate(drawn): + names = [f"p{i}_{j}" for j in range(len(members))] + people.update(zip(names, members)) + benunits[f"b{i}"] = { + "members": names, + "income_support_assessable_capital": every_year(capital), + "esa_income_assessable_capital": every_year(capital), + "jsa_income_assessable_capital": every_year(capital), + } + households[f"h{i}"] = {"members": names} + simulation = Simulation( + situation={"people": people, "benunits": benunits, "households": households}, + scenario=REFORM, + ) + + def total(name, couple_only): + return np.array( + [ + sum( + member[name][DATASET_YEAR] + for member in members + if member["is_claimant_or_partner"][DATASET_YEAR] or not couple_only + ) + for members, _ in drawn + ], + dtype=np.float32, + ) + + for year in YEARS: + for award, scoped in CLAIMANT_OR_PARTNER_AWARDS.items(): + stored = simulation.calculate(award, year) + couple = simulation.calculate(scoped, year) + # Conservation: the stored award is everyone's dataset-year + # reports, and the couple's part is their own. + np.testing.assert_allclose( + stored, total(f"{award}_reported", False), rtol=1e-6 + ) + np.testing.assert_allclose( + couple, total(f"{award}_reported", True), rtol=1e-6 + ) + assert ((couple >= 0) & (couple <= stored)).all() + np.testing.assert_allclose( + simulation.calculate("income_support", year), + total("income_support_reported", True), + rtol=1e-6, + ) + # Either income-related award of the couple bars Income Support, and + # the passport follows the couple's awards alone. + esa = simulation.calculate("claimant_or_partner_esa_income", year) > 0 + jsa = simulation.calculate("claimant_or_partner_jsa_income", year) > 0 + income_support = simulation.calculate("income_support", year) > 0 + eligible = simulation.calculate("income_support_eligible", year) + assert not (eligible & (esa | jsa)).any() + np.testing.assert_array_equal( + simulation.calculate( + "council_tax_reduction_relevant_income_based_benefit", year + ), + esa | jsa | income_support, + ) + + +# The reform class itself, as create_structural_reforms_from_parameters builds +# it when the parameter is on. +ON = SimpleNamespace( + gov=SimpleNamespace( + contrib=SimpleNamespace( + policyengine=SimpleNamespace(disable_simulated_benefits=True) + ) + ) +) +DisableSimulatedBenefits = disable_simulated_benefits(lambda _: ON, None) +SET_BY_THE_REFORM = [ + "esa_income", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + "income_support", +] + + +def known_years(simulation, variable): + holder = simulation.get_holder(variable) + return sorted(int(str(p)[:4]) for p in holder.get_known_periods()) + + +def test_a_clone_or_branch_applies_the_reform_to_itself_alone(unreformed): + # A clone's copy of the tax-benefit system links to the clone, and a + # branch shares its parent's system. The reform reaches the simulation + # through that link, so it must reach the clone or branch applying it, + # never the parent. + parent = Simulation(situation=situation()) + clone = parent.clone() + assert clone.tax_benefit_system.simulation is clone + clone.apply_reform(DisableSimulatedBenefits) + branch = parent.get_branch("reformed") + assert branch.tax_benefit_system is parent.tax_benefit_system + branch.apply_reform(DisableSimulatedBenefits) + assert parent.tax_benefit_system.simulation is parent + ten_years = list(range(DATASET_YEAR, DATASET_YEAR + YEARS_IN_FUTURE)) + for variable in SET_BY_THE_REFORM: + assert known_years(parent, variable) == [], variable + assert known_years(clone, variable) == ten_years, variable + excluded = NAMES.index("excluded_adult") + for year in YEARS: + for simulation in (clone, branch): + assert simulation.calculate("esa_income", year)[excluded] == 3_000 + assert ( + simulation.calculate("claimant_or_partner_esa_income", year)[excluded] + == 0 + ) + # The parent still simulates ESA, as a simulation without the reform + # does: the excluded adult's report in the dataset year, and nothing + # once the closure of income-based JSA and Income Support has moved + # the family off its legacy benefits. + np.testing.assert_array_equal( + parent.calculate("esa_income", year), + unreformed.calculate("esa_income", year), + ) + assert parent.calculate("esa_income", DATASET_YEAR)[excluded] == 3_000 + + +# Applied after the simulation is built: passed as reform=, or applied to a +# clone or a branch. +LATER = ["reform=", "clone", "branch"] + + +@pytest.mark.parametrize("route", LATER) +def test_a_reform_applied_later_keeps_what_it_sets_through_a_reset(route): + # input_variables is taken when a simulation is built, and + # reset_calculations deletes every other variable's values (as + # derivative did before policyengine-core 3.33.0). + # A reform applied later names what it sets, or a reset deletes the + # claimant-or-partner awards and recalculates them. esa_income is entered + # here too, so a reset kept it while deleting its claimant-or-partner + # part, and the formula then read the excluded adult's ESA as the couple's. + entered = situation() + for benunit in entered["benunits"].values(): + benunit["esa_income"] = {DATASET_YEAR: 3_000} + if route == "reform=": + parent = None + simulation = Simulation(situation=entered, reform=DisableSimulatedBenefits) + else: + parent = Simulation(situation=entered) + names = parent.input_variables + recorded = list(names) + simulation = ( + parent.clone() if route == "clone" else parent.get_branch("reformed") + ) + simulation.apply_reform(DisableSimulatedBenefits) + # A clone or branch shares its parent's list until it names more. + assert parent.input_variables is names + assert parent.input_variables == recorded + assert "claimant_or_partner_esa_income" not in recorded + set_by_the_reform = [ + *BENEFITS, + *PRE_MINIMUM.values(), + *CLAIMANT_OR_PARTNER_AWARDS.values(), + *NOT_MOVED, + "carers_allowance", + "carer_support_payment", + "winter_fuel_allowance", + "pawhp", + ] + for variable in set_by_the_reform: + assert variable in simulation.input_variables, variable + excluded = NAMES.index("excluded_adult") + year = YEARS[1] + before = { + variable: simulation.calculate(variable, year) + for variable in [*set_by_the_reform, "income_support_eligible"] + } + assert before["claimant_or_partner_esa_income"][excluded] == 0 + assert before["income_support_eligible"][excluded] + # derivative recalculates on a clone: before policyengine-core 3.33.0 + # after the same deletions, and from 3.33.0 keeping every input. + assert not simulation.derivative( + "claimant_or_partner_esa_income", "employment_income", year + ).any() + simulation.reset_calculations() + for variable, value in before.items(): + np.testing.assert_array_equal( + simulation.calculate(variable, year), value, err_msg=variable + ) + + +@pytest.mark.parametrize("route", ["itself", "clone", "branch"]) +def test_a_reform_that_replaces_inputs_set_after_the_build_keeps_them(route): + # Inputs set after the build are recorded as inputs, but input_variables + # does not name them. A reform applied to the same simulation, or to a + # clone, that replaces one adds no new record, so the reform's inputs are + # named by what it sets. A branch stores its inputs under its own name, + # so there every record is new. + entered = situation() + for benunit in entered["benunits"].values(): + benunit["esa_income"] = {DATASET_YEAR: 3_000} + parent = Simulation(situation=entered) + for year in range(DATASET_YEAR, DATASET_YEAR + YEARS_IN_FUTURE): + parent.set_input( + "claimant_or_partner_esa_income", year, np.full(len(NAMES), 100) + ) + # Set after the build and not by the reform, so the reform does not name it. + parent.set_input( + "housing_benefit_applicable_income", YEARS[1], np.zeros(len(NAMES)) + ) + names = parent.input_variables + recorded = list(names) + assert "claimant_or_partner_esa_income" not in recorded + if route == "itself": + simulation = parent + elif route == "clone": + simulation = parent.clone() + else: + simulation = parent.get_branch("reformed") + simulation.apply_reform(DisableSimulatedBenefits) + if route != "itself": + assert parent.input_variables is names + assert parent.input_variables == recorded + for variable in SET_BY_THE_REFORM: + assert variable in simulation.input_variables, variable + assert "housing_benefit_applicable_income" not in simulation.input_variables + excluded = NAMES.index("excluded_adult") + year = YEARS[1] + before = { + variable: simulation.calculate(variable, year) + for variable in [*SET_BY_THE_REFORM, "income_support_eligible"] + } + assert before["claimant_or_partner_esa_income"][excluded] == 0 + assert before["income_support_eligible"][excluded] + assert not simulation.derivative( + "claimant_or_partner_esa_income", "employment_income", year + ).any() + simulation.reset_calculations() + for variable, value in before.items(): + np.testing.assert_array_equal( + simulation.calculate(variable, year), value, err_msg=variable + ) + + +class RemoveIncomeRelatedAwards(Reform): + def apply(self): + self.neutralize_variable("esa_income") + self.neutralize_variable("jsa_income") + + +def test_neutralizing_an_award_removes_the_claimant_or_partners_award(): + # The claimant's and partner's award is part of esa_income or + # jsa_income: a reform that removes the award removes their part too, + # including the part the reform set as an input. + simulation = Simulation(situation=situation(), scenario=REFORM) + simulation.apply_reform(RemoveIncomeRelatedAwards) + for year in YEARS: + for variable in [ + "esa_income", + "jsa_income", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ]: + assert simulation.calculate(variable, year).tolist() == [0] * len(NAMES), ( + variable, + year, + ) + # Without the income-related awards nothing bars Income Support, and + # only Income Support passports council tax reduction. + assert by_family(simulation, "income_support_eligible", year) == { + "excluded_adult": True, + "excluded_adult_no_award": False, + "own_esa": True, + "own_jsa": True, + "own_esa_over_tariff": True, + "excluded_income_support": False, + } + assert by_family( + simulation, "council_tax_reduction_relevant_income_based_benefit", year + ) == dict( + zip(NAMES, (simulation.calculate("income_support", year) > 0).tolist()) + ) + assert not simulation.calculate("is_on_income_related_esa", year).any() + assert not simulation.calculate("is_on_income_based_jsa", year).any() + + +def test_an_award_neutralized_first_is_not_set(): + # Neutralized before the reform runs: neither the award nor the + # claimant's and partner's part of it takes what the reform sets. + def remove(simulation): + simulation.tax_benefit_system.neutralize_variable("esa_income") + + scenario = Scenario( + applied_before_data_load=True, + parameter_changes=REFORM.parameter_changes, + simulation_modifier=remove, + ) + simulation = Simulation(situation=situation(), scenario=scenario) + assert known_years(simulation, "esa_income") == [] + assert known_years(simulation, "claimant_or_partner_esa_income") == [] + assert len(known_years(simulation, "claimant_or_partner_jsa_income")) == ( + YEARS_IN_FUTURE + ) + for year in YEARS: + assert not simulation.calculate("claimant_or_partner_esa_income", year).any() + assert ( + by_family(simulation, "claimant_or_partner_jsa_income", year)["own_jsa"] + == 2_000 + ) + + +def test_neutralizing_an_award_removes_an_entered_claimant_or_partner_award(): + # The same without the reform, for an award entered directly. + simulation = Simulation(situation=situation()) + simulation.set_input( + "claimant_or_partner_esa_income", DATASET_YEAR, np.full(len(NAMES), 3_000) + ) + assert simulation.calculate("claimant_or_partner_esa_income", DATASET_YEAR).all() + simulation.apply_reform(RemoveIncomeRelatedAwards) + assert not simulation.calculate( + "claimant_or_partner_esa_income", DATASET_YEAR + ).any() + + +@pytest.mark.parametrize("year", YEARS) +def test_another_members_income_support_is_never_the_familys(reformed, year): + # The model pays Income Support only to the claimant's family, so + # another member's own report is left out of income_support (and so of + # household income) with the reform off, as the reform leaves it out. + for simulation in (Simulation(situation=situation()), reformed): + income_support = by_family(simulation, "income_support", year) + assert income_support["excluded_income_support"] == 0 + + +def test_the_claimant_and_partner_are_taken_in_each_year(): + # The amounts are held at the dataset year, but who the claimant and + # partner are is read in each year, as every reader of the awards reads + # it. Here the claimant in the dataset year is someone else's + # non-dependant afterwards, and the other adult claims instead. + first, later = DATASET_YEAR, YEARS[1:] + + def member(age, claimant_or_partner_in_first_year, **reported): + person = adult(age, claimant_or_partner_in_first_year, **reported) + person["is_claimant_or_partner"] = { + first: claimant_or_partner_in_first_year, + **{year: not claimant_or_partner_in_first_year for year in later}, + } + return person + + people = { + "a": member(40, True, esa_income_reported=3_000, income_support_reported=1_000), + "b": member(35, False, esa_income_reported=1_000), + } + simulation = Simulation( + situation={ + "people": people, + "benunits": {"family": {"members": ["a", "b"]}}, + "households": {"home": {"members": ["a", "b"]}}, + }, + scenario=REFORM, + ) + expected = {first: (3_000, 1_000)} + expected.update({year: (1_000, 0) for year in later}) + for year, (claimant_or_partner_esa, income_support) in expected.items(): + assert simulation.calculate("esa_income", year)[0] == 4_000 + assert ( + simulation.calculate("claimant_or_partner_esa_income", year)[0] + == claimant_or_partner_esa + ) + assert simulation.calculate("income_support", year)[0] == income_support diff --git a/policyengine_uk/tests/test_disable_simulated_benefits_benefit_cap.py b/policyengine_uk/tests/test_disable_simulated_benefits_benefit_cap.py new file mode 100644 index 0000000000..cf8dbee648 --- /dev/null +++ b/policyengine_uk/tests/test_disable_simulated_benefits_benefit_cap.py @@ -0,0 +1,288 @@ +"""The benefit cap reads the couple's held income-related ESA and income-based +JSA under gov.contrib.policyengine.disable_simulated_benefits. + +The reform holds esa_income and jsa_income at the dataset year's reported +total for ten years, while the reports are uprated, and sets the claimant's +and partner's part (claimant_or_partner_esa_income, +claimant_or_partner_jsa_income) from their own reports. In a later year the +held total matches neither reading award_of_members makes of a stored award +(the award on everyone's reports, or their plain total), so it took all of +it to be the couple's. The benefit cap's readers called award_of_members on +esa_income and jsa_income, so another member's held award (a non-dependent +adult's, claimed in their own right) raised the cap's total of welfare +benefits, and lifted the cap for a claimant with an ESA support component +who receives no ESA (review r11 on #2075). They now read award_of_couple. + +Invariants: + +1. Metamorphic, reform on: where the cap's couple is the claimant and + partner, in each of the reform's years, the cap's total, both schemes' + specified-benefit exemptions and both reductions are the same whether or + not a member outside the couple reports ESA or JSA in the dataset year. +2. Differential, reform off: the cap's total equals the code it replaces + (policyengine-uk main at b4dbaf57a) to within a penny, and the couple's + award the exemptions read equals award_of_members for the couple, except + that an award within half a penny of zero is none. +3. For a couple that is not the claimant and partner, award_of_couple is + award_of_members for that couple. +4. A claimant-or-partner award entered directly reaches the cap. +""" + +import numpy as np +import pytest +from hypothesis import given +from hypothesis import strategies as st +from policyengine_core.periods import period as as_period + +from policyengine_uk import Simulation +from policyengine_uk.tests.test_income_support_eligibility_properties import ( + DIFFERENTIAL_SETTINGS, + FAMILIES, + input_settings, + situation as property_situation, +) +from policyengine_uk.utils.benefit_cap import benefit_cap_couple +from policyengine_uk.utils.benefit_unit import ( + HALF_PENNY, + add_for_members, + award_of_couple, + award_of_members, +) +from policyengine_uk.utils.scenario import Scenario +from policyengine_uk.variables.gov.dwp.benefit_cap_welfare_benefits import ( + CLAIMANT_OR_PARTNER_BENEFITS, + FAMILY_BENEFITS, +) +from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award +from policyengine_uk.variables.gov.dwp.jsa_income import income_related_jsa_award + +REFORM_ON = Scenario( + applied_before_data_load=True, + parameter_changes={"gov.contrib.policyengine.disable_simulated_benefits": True}, +) +YEARS = [2025, 2026, 2027] +CAP = [ + "benefit_cap_welfare_benefits", + "is_uc_benefit_cap_exempt_specified_benefit", + "is_housing_benefit_benefit_cap_exempt_specified_benefit", + "uc_benefit_cap_reduction", + "housing_benefit_benefit_cap_reduction", + "benefit_cap_reduction", +] +AWARDS = { + "esa_income": ("esa_income_reported", income_related_esa_award), + "jsa_income": ("jsa_income_reported", income_related_jsa_award), +} + + +def in_each_year(value): + return {year: value for year in YEARS} + + +def cap_families(support_component): + """Four benefit units in London, each a claimant of 40 and another + adult of 30 outside the couple who claims in their own right. Two have + £25,000 of Universal Credit before the cap, two £25,000 of Housing + Benefit before the cap, so each is capped (the London cap for a single + claimant without children is below £25,000). In one of each pair the + other adult reports £3,000 of ESA and £2,000 of JSA in 2025; in the + control they report none. The claimant reports no ESA, and has an ESA + support component when support_component is true, so only a misread + ESA award would exempt them.""" + people, benunits, households = {}, {}, {} + for scheme in ("uc", "hb"): + for other_reports in (False, True): + unit = f"{scheme}_{'other_reports' if other_reports else 'control'}" + members = [f"{unit}_claimant", f"{unit}_other"] + for member, is_claimant in zip(members, (True, False)): + reports = other_reports and not is_claimant + people[member] = { + "age": in_each_year(40 if is_claimant else 30), + "is_claimant_or_partner": in_each_year(is_claimant), + "is_uc_claimant": in_each_year(is_claimant), + "current_education": in_each_year("NOT_IN_EDUCATION"), + "esa_includes_support_component": in_each_year( + support_component and is_claimant + ), + "esa_income_reported": {2025: 3_000 if reports else 0}, + "jsa_income_reported": {2025: 2_000 if reports else 0}, + } + benunits[unit] = { + "members": members, + "esa_income_assessable_capital": in_each_year(0), + "jsa_income_assessable_capital": in_each_year(0), + "universal_credit_pre_benefit_cap": in_each_year( + 25_000 if scheme == "uc" else 0 + ), + "housing_benefit_pre_benefit_cap": in_each_year( + 25_000 if scheme == "hb" else 0 + ), + "uc_LCWRA_element": in_each_year(0), + "uc_carer_element": in_each_year(0), + "uc_childcare_element": in_each_year(0), + } + households[unit] = {"members": members, "region": in_each_year("LONDON")} + return {"people": people, "benunits": benunits, "households": households} + + +UNITS = ["uc_control", "uc_other_reports", "hb_control", "hb_other_reports"] + + +def by_unit(simulation, variable, year): + return dict(zip(UNITS, np.asarray(simulation.calculate(variable, year)).tolist())) + + +@pytest.mark.parametrize("support_component", [False, True]) +def test_another_members_held_award_never_enters_the_benefit_cap(support_component): + simulation = Simulation( + situation=cap_families(support_component), scenario=REFORM_ON + ) + for year in YEARS: + # The reform holds the other adult's awards in every year, and the + # couple's own awards are nil. + esa = by_unit(simulation, "esa_income", year) + assert esa["uc_other_reports"] == esa["hb_other_reports"] == 3_000 + for name in [ + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ]: + assert set(by_unit(simulation, name, year).values()) == {0}, (name, year) + for name in CAP: + values = by_unit(simulation, name, year) + for scheme in ("uc", "hb"): + assert ( + values[f"{scheme}_other_reports"] == values[f"{scheme}_control"] + ), (name, year, scheme, values) + # The cap bites in every family: no misread award lifts it. + reductions = by_unit(simulation, "benefit_cap_reduction", year) + assert all(reduction > 0 for reduction in reductions.values()), ( + year, + reductions, + ) + + +def replaced_benefit_cap_welfare_benefits(benunit, period): + """benefit_cap_welfare_benefits at policyengine-uk main b4dbaf57a + (comments removed).""" + couple = benefit_cap_couple(benunit, period) + income_based = award_of_members( + benunit, + period, + "jsa_income", + "jsa_income_reported", + income_related_jsa_award, + couple, + ) + award_of_members( + benunit, + period, + "esa_income", + "esa_income_reported", + income_related_esa_award, + couple, + ) + return income_based + add_for_members( + benunit, period, CLAIMANT_OR_PARTNER_BENEFITS + FAMILY_BENEFITS, couple + ) + + +@DIFFERENTIAL_SETTINGS +@given(FAMILIES, st.data()) +def test_the_benefit_cap_matches_the_replaced_readers_with_the_reform_off(drawn, data): + capital_as_savings, esa_income, jsa_income = data.draw(input_settings(len(drawn))) + units = [(*family, extra) for family, extra in drawn] + simulation = Simulation( + situation=property_situation(units, capital_as_savings, esa_income, jsa_income) + ) + period = as_period(2025) + benunit = simulation.populations["benunit"] + np.testing.assert_allclose( + simulation.calculate("benefit_cap_welfare_benefits", 2025), + replaced_benefit_cap_welfare_benefits(benunit, period), + rtol=0, + atol=2 * HALF_PENNY, + ) + # The couple's award as the specified-benefit exemptions read it, for + # the claimant and partner (Housing Benefit) and the benefit cap's + # couple (Universal Credit). + couples = [ + benunit.members("is_claimant_or_partner", period), + benefit_cap_couple(benunit, period), + ] + for award, (reported, award_on_reports) in AWARDS.items(): + stored = benunit(award, period) + for couple in couples: + replaced = award_of_members( + benunit, period, award, reported, award_on_reports, couple + ) + switched = award_of_couple( + benunit, period, award, reported, award_on_reports, couple + ) + np.testing.assert_array_equal( + switched, + np.where((stored > HALF_PENNY) & (replaced > HALF_PENNY), replaced, 0), + ) + + +def test_a_couple_that_is_not_the_claimant_and_partner_reads_the_stored_award(): + """A couple of whom only the claimant is in the rule's couple (as for a + Universal Credit claim where is_uc_claimant is entered for one member): + the claimant-or-partner award is not that couple's, so award_of_couple + reads the stored award by value, as award_of_members does.""" + simulation = Simulation( + situation={ + "people": { + "claimant": {"age": 40, "esa_income_reported": 3_000}, + "partner": {"age": 38, "esa_income_reported": 2_000}, + }, + "benunits": {"couple": {"members": ["claimant", "partner"]}}, + "households": {"home": {"members": ["claimant", "partner"]}}, + } + ) + period = as_period(2025) + benunit = simulation.populations["benunit"] + claimant_only = np.array([True, False]) + for award, (reported, award_on_reports) in AWARDS.items(): + np.testing.assert_array_equal( + award_of_couple( + benunit, period, award, reported, award_on_reports, claimant_only + ), + award_of_members( + benunit, period, award, reported, award_on_reports, claimant_only + ), + ) + # The two differ here: the claimant-or-partner award covers both. + assert ( + simulation.calculate("claimant_or_partner_esa_income", 2025)[0] + > ( + award_of_members( + benunit, + period, + "esa_income", + "esa_income_reported", + income_related_esa_award, + claimant_only, + )[0] + ) + ) + + +@pytest.mark.parametrize("award", ["esa_income", "jsa_income"]) +def test_an_entered_claimant_or_partner_award_reaches_the_benefit_cap(award): + """An award entered for claimant_or_partner_* is the couple's award for + the benefit cap too, as for the Income Support gate and the passports.""" + + def total(entered): + person = {"age": 40} + unit = {"members": ["claimant"]} + if entered is not None: + unit[f"claimant_or_partner_{award}"] = entered + simulation = Simulation( + situation={ + "people": {"claimant": person}, + "benunits": {"family": unit}, + "households": {"home": {"members": ["claimant"]}}, + } + ) + return simulation.calculate("benefit_cap_welfare_benefits", 2025)[0] + + assert total(5_000) - total(None) == pytest.approx(5_000, abs=0.01) diff --git a/policyengine_uk/tests/test_disable_simulated_benefits_carers.py b/policyengine_uk/tests/test_disable_simulated_benefits_carers.py new file mode 100644 index 0000000000..388f2a4b73 --- /dev/null +++ b/policyengine_uk/tests/test_disable_simulated_benefits_carers.py @@ -0,0 +1,391 @@ +"""gov.contrib.policyengine.disable_simulated_benefits counts a carer's +reported Carer's Allowance once. + +The survey reports Carer's Allowance (carers_allowance_reported) for every +respondent, Scotland included. From 2025 the model pays a carer in Scotland +Carer Support Payment in its place. The reform used to set carers_allowance +from the report in every country and leave carer_support_payment simulated, +so a Scottish carer was paid both from 2025. Invariants, for every person +and each of the reform's years: + +1. Conservation: carers_allowance + carer_support_payment equals the + person's reported Carer's Allowance, so the report is counted once in + household income (both variables are in household_benefits once). +2. Placement: the report is carer_support_payment for a carer living in + Scotland that year, from CARER_SUPPORT_PAYMENT_FROM, and + carers_allowance otherwise. Where the carer lives is read in each year, + so a carer who moves into or out of Scotland changes scheme with the move. +3. Differential: the reform places the report under the scheme the model + itself pays with the reform off (the pre-overlap entitlements), and + household_benefits exceeds that of the same dataset with no report by the + report plus the Scottish Carer Supplement that Carer Support Payment + brings. +""" + +from types import SimpleNamespace + +import numpy as np +import pandas as pd +import pytest +from hypothesis import HealthCheck, given, settings +from hypothesis import strategies as st + +from policyengine_uk import Microsimulation, Simulation +from policyengine_uk.data import UKMultiYearDataset +from policyengine_uk.data.dataset_schema import UKSingleYearDataset +from policyengine_uk.reforms.policyengine.disable_simulated_benefits import ( + BENEFITS, + CARER_SUPPORT_PAYMENT_FROM, + YEARS_IN_FUTURE, + disable_simulated_benefits, +) +from policyengine_uk.utils.scenario import Scenario + +REFORM_ON = Scenario( + applied_before_data_load=True, + parameter_changes={"gov.contrib.policyengine.disable_simulated_benefits": True}, +) +REGIONS = [ + "NORTH_EAST", + "LONDON", + "WALES", + "SCOTLAND", + "NORTHERN_IRELAND", +] +PROPERTY_SETTINGS = settings( + max_examples=4, + deadline=None, + derandomize=True, + suppress_health_check=[HealthCheck.too_slow, HealthCheck.data_too_large], +) + + +def dataset(year, regions, reported_care, care_hours=None): + """One adult of 40 per household, with a reported Carer's Allowance and + weekly care hours (40 unless given).""" + n = len(regions) + ids = np.arange(1, n + 1) + person = pd.DataFrame( + { + "person_id": ids, + "person_benunit_id": ids, + "person_household_id": ids, + "age": [40] * n, + "care_hours": [float(x) for x in (care_hours or [40] * n)], + "carers_allowance_reported": [float(x) for x in reported_care], + } + ) + household = pd.DataFrame( + { + "household_id": ids, + "region": regions, + "household_weight": [1.0] * n, + "tenure_type": ["OWNED_OUTRIGHT"] * n, + "council_tax": [0.0] * n, + "rent": [0.0] * n, + } + ) + return UKSingleYearDataset( + person=person, + benunit=pd.DataFrame({"benunit_id": ids}), + household=household, + fiscal_year=year, + ) + + +def values(simulation, variable, year): + return np.asarray(simulation.calculate(variable, year), dtype=float) + + +def moving_dataset(first_year, regions_by_year, reported_care, care_hours=None): + """The same carers in each of the reform's years, living in each year + where regions_by_year says. The reform reads the first year's reports.""" + return UKMultiYearDataset( + datasets=[ + dataset(year, regions_by_year[year], reported_care, care_hours) + for year in range(first_year, first_year + YEARS_IN_FUTURE) + ] + ) + + +def check_care(simulation, regions, reported_care, first_year): + """regions: each carer's region, the same in every year, or a dict of + them by year.""" + reported = np.asarray(reported_care, dtype=float) + for year in range(first_year, first_year + YEARS_IN_FUTURE): + regions_in_year = regions[year] if isinstance(regions, dict) else regions + in_scotland = np.asarray(regions_in_year) == "SCOTLAND" + ca = values(simulation, "carers_allowance", year) + csp = values(simulation, "carer_support_payment", year) + np.testing.assert_allclose(ca + csp, reported, err_msg=str(year)) + paid_as_csp = in_scotland & (year >= CARER_SUPPORT_PAYMENT_FROM) + np.testing.assert_allclose( + csp, np.where(paid_as_csp, reported, 0), err_msg=str(year) + ) + np.testing.assert_allclose( + ca, np.where(paid_as_csp, 0, reported), err_msg=str(year) + ) + + +def test_carers_allowance_is_set_by_the_carer_split_alone(): + """carers_allowance is set with carer_support_payment from the one + report. A generic BENEFITS entry as well would set it from the report in + Scotland too, if it ran after the split.""" + assert "carers_allowance" not in BENEFITS + assert "carer_support_payment" not in BENEFITS + + +@pytest.mark.parametrize("simulation_class", [Simulation, Microsimulation]) +def test_a_scottish_carer_is_paid_the_report_once(simulation_class): + """The case reported on #2075: two carers of 40 caring 40 hours a week, + each reporting £4,000 of Carer's Allowance in 2025, one in Scotland and one + in London. The Scottish carer was paid the report as Carer's Allowance + and a simulated Carer Support Payment as well.""" + regions = ["SCOTLAND", "LONDON"] + simulation = simulation_class( + dataset=dataset(2025, regions, [4_000, 4_000]), scenario=REFORM_ON + ) + for year in [2025, 2026]: + np.testing.assert_allclose( + values(simulation, "carers_allowance", year), [0, 4_000] + ) + np.testing.assert_allclose( + values(simulation, "carer_support_payment", year), [4_000, 0] + ) + + +@pytest.mark.parametrize("simulation_class", [Simulation, Microsimulation]) +@pytest.mark.parametrize("first_year", [2023, 2025]) +def test_reported_care_is_counted_once(simulation_class, first_year): + regions = ["SCOTLAND", "LONDON", "WALES", "NORTHERN_IRELAND", "SCOTLAND"] + reported_care = [4_000, 3_500, 4_200.5, 3_000, 0] + simulation = simulation_class( + dataset=dataset(first_year, regions, reported_care), scenario=REFORM_ON + ) + check_care(simulation, regions, reported_care, first_year) + + +@pytest.mark.parametrize("simulation_class", [Simulation, Microsimulation]) +def test_reform_set_after_the_data_load_counts_the_report_once(simulation_class): + regions = ["SCOTLAND", "LONDON", "WALES", "NORTHERN_IRELAND", "SCOTLAND"] + reported_care = [4_000, 3_500, 4_200.5, 3_000, 0] + simulation = simulation_class( + dataset=dataset(2023, regions, reported_care), + reform={"gov.contrib.policyengine.disable_simulated_benefits": True}, + ) + check_care(simulation, regions, reported_care, 2023) + + +def test_the_report_goes_to_the_scheme_the_model_pays(): + """With the reform off the model's pre-overlap entitlements say which + scheme pays each carer: CARER_SUPPORT_PAYMENT_FROM must agree with them in + every year, before and after the switch.""" + regions = ["SCOTLAND", "LONDON"] + first_year = CARER_SUPPORT_PAYMENT_FROM - 3 + reported_care = [4_000, 4_000] + off = Simulation(dataset=dataset(first_year, regions, reported_care)) + on = Simulation( + dataset=dataset(first_year, regions, reported_care), scenario=REFORM_ON + ) + for year in range(first_year, first_year + YEARS_IN_FUTURE): + model_pays_csp = values(off, "carer_support_payment_pre_overlap", year) > 0 + model_pays_ca = values(off, "carers_allowance_pre_overlap", year) > 0 + # Each carer is entitled under exactly one scheme with the reform off. + np.testing.assert_array_equal(model_pays_csp ^ model_pays_ca, True) + np.testing.assert_array_equal( + values(on, "carer_support_payment", year) > 0, + model_pays_csp, + err_msg=str(year), + ) + np.testing.assert_array_equal( + values(on, "carers_allowance", year) > 0, model_pays_ca, err_msg=str(year) + ) + + +def test_household_benefits_include_the_report_once(): + regions = ["SCOTLAND", "LONDON", "WALES", "NORTHERN_IRELAND"] + reported_care = [4_000, 3_500, 4_200.5, 3_000] + with_report = Simulation( + dataset=dataset(2023, regions, reported_care), scenario=REFORM_ON + ) + without_report = Simulation( + dataset=dataset(2023, regions, [0] * len(regions)), scenario=REFORM_ON + ) + in_scotland = np.asarray(regions) == "SCOTLAND" + for year in range(2023, 2023 + YEARS_IN_FUTURE): + change = { + name: values(with_report, name, year) - values(without_report, name, year) + for name in [ + "household_benefits", + "hbai_household_net_income", + "scottish_carer_supplement", + ] + } + # The Scottish Carer Supplement is paid with Carer Support Payment + # (from March 2026), so only to the Scottish carer. + supplement = change["scottish_carer_supplement"] + np.testing.assert_allclose(supplement[~in_scotland], 0, err_msg=str(year)) + assert (supplement >= 0).all(), year + for name in ["household_benefits", "hbai_household_net_income"]: + np.testing.assert_allclose( + change[name], + np.asarray(reported_care) + supplement, + err_msg=f"{name} {year}", + ) + + +@PROPERTY_SETTINGS +@given( + first_year=st.sampled_from([2022, 2023, 2024, 2025, 2026]), + carers=st.lists( + st.tuples( + st.sampled_from(REGIONS), + st.sampled_from([0, 100, 150.5, 3_000, 4_000, 4_331.6]), + st.sampled_from([0, 20, 35, 40]), + ), + min_size=1, + max_size=6, + ), +) +def test_reported_care_is_conserved(first_year, carers): + regions = [region for region, _, _ in carers] + reported_care = [amount for _, amount, _ in carers] + care_hours = [hours for _, _, hours in carers] + simulation = Simulation( + dataset=dataset(first_year, regions, reported_care, care_hours), + scenario=REFORM_ON, + ) + check_care(simulation, regions, reported_care, first_year) + + +def carer_situation(regions_by_year): + """The case reported on #2075 at ab800e8bb: one carer, 40 in 2025, caring + 40 hours a week in every year, who reports £4,000 of Carer's Allowance in + 2025 and lives in each year where regions_by_year says. No earnings and + no overlapping benefit.""" + years = list(regions_by_year) + return { + "people": { + "carer": { + "age": {year: 40 + year - years[0] for year in years}, + "care_hours": {year: 40 for year in years}, + "carers_allowance_reported": {years[0]: 4_000}, + } + }, + "benunits": {"family": {"members": ["carer"]}}, + "households": {"home": {"members": ["carer"], "region": regions_by_year}}, + } + + +INCOME = [ + "carers_allowance", + "carer_support_payment", + "scottish_carer_supplement", + "household_benefits", + "hbai_household_net_income", + "household_net_income", +] + + +@pytest.mark.parametrize( + "before, after", [("LONDON", "SCOTLAND"), ("SCOTLAND", "LONDON")] +) +@pytest.mark.parametrize("route", ["scenario", "reform"]) +def test_a_carer_who_moves_is_paid_under_each_years_scheme(before, after, route): + """Reported on #2075 at ab800e8bb: the split read the country in the + dataset year only, so a carer in London in 2025 and Scotland from 2026 + was paid Carer's Allowance in 2026 and lost the Scottish Carer + Supplement. Each year's outputs now equal those of a carer who has lived + in that year's country throughout.""" + first = 2025 + years = range(first, first + YEARS_IN_FUTURE) + moved = {year: before if year == first else after for year in years} + stayed = {year: after for year in years} + switch = {"gov.contrib.policyengine.disable_simulated_benefits": True} + + def simulate(regions_by_year): + if route == "scenario": + return Simulation( + situation=carer_situation(regions_by_year), scenario=REFORM_ON + ) + return Simulation(situation=carer_situation(regions_by_year), reform=switch) + + simulation, control = simulate(moved), simulate(stayed) + check_care(simulation, {year: [moved[year]] for year in years}, [4_000], first) + for year in years[1:]: + for name in INCOME: + np.testing.assert_allclose( + values(simulation, name, year), + values(control, name, year), + err_msg=f"{name} {year}", + ) + supplement = values(simulation, "scottish_carer_supplement", first + 1) + if after == "SCOTLAND": + # The supplement comes with Carer Support Payment, and reaches + # household income. + assert supplement[0] > 0 + else: + np.testing.assert_allclose(supplement, 0) + for name in ["household_benefits", "hbai_household_net_income"]: + np.testing.assert_allclose( + values(simulation, name, first + 1), 4_000 + supplement, err_msg=name + ) + + +def test_a_reform_applied_after_a_move_is_set_reads_the_move(): + """The reform reads where each household lives when it is applied + (review r11 on #2075). Switched on by the parameter, it is applied in the + constructor, before a scenario's modifier applied after data load, so + the amounts it sets do not follow a region that modifier or set_input + changes later. Applied after the change, it reads the new region: a + carer in London in the situation, moved to Scotland from 2026 by + set_input, is paid Carer Support Payment from 2026.""" + first = 2025 + years = range(first, first + YEARS_IN_FUTURE) + simulation = Simulation( + situation=carer_situation({year: "LONDON" for year in years}) + ) + for year in years[1:]: + simulation.set_input("region", year, ["SCOTLAND"]) + switched_on = SimpleNamespace( + gov=SimpleNamespace( + contrib=SimpleNamespace( + policyengine=SimpleNamespace(disable_simulated_benefits=True) + ) + ) + ) + simulation.apply_reform(disable_simulated_benefits(lambda _: switched_on, None)) + regions = {year: ["LONDON" if year == first else "SCOTLAND"] for year in years} + check_care(simulation, regions, [4_000], first) + assert values(simulation, "scottish_carer_supplement", first + 1)[0] > 0 + + +@PROPERTY_SETTINGS +@given( + first_year=st.sampled_from([2023, 2024, 2025, 2026]), + carers=st.lists( + st.tuples( + st.lists( + st.sampled_from(["SCOTLAND", "LONDON", "WALES"]), + min_size=YEARS_IN_FUTURE, + max_size=YEARS_IN_FUTURE, + ), + st.sampled_from([0, 150.5, 4_000]), + ), + min_size=1, + max_size=4, + ), +) +def test_reported_care_follows_each_years_country(first_year, carers): + """Conservation and placement in every year, with each carer's region + drawn afresh for each year (a multi-year dataset).""" + years = range(first_year, first_year + YEARS_IN_FUTURE) + regions_by_year = { + year: [path[i] for path, _ in carers] for i, year in enumerate(years) + } + reported_care = [amount for _, amount in carers] + simulation = Simulation( + dataset=moving_dataset(first_year, regions_by_year, reported_care), + scenario=REFORM_ON, + ) + check_care(simulation, regions_by_year, reported_care, first_year) diff --git a/policyengine_uk/tests/test_disable_simulated_benefits_winter_heating.py b/policyengine_uk/tests/test_disable_simulated_benefits_winter_heating.py index 0925b751f8..0eb5b8d7ef 100644 --- a/policyengine_uk/tests/test_disable_simulated_benefits_winter_heating.py +++ b/policyengine_uk/tests/test_disable_simulated_benefits_winter_heating.py @@ -16,8 +16,11 @@ 1. Conservation: winter_fuel_allowance + pawhp equals the household's reported winter heating payment, so the report is counted once in household income (both variables are in household_benefits once). -2. Placement: the report is pawhp in Scotland in the years PAWHP is paid - (from the 2024 qualifying week) and winter_fuel_allowance otherwise. +2. Placement: the report is pawhp for a household living in Scotland that + year, in the years PAWHP is paid (from the 2024 qualifying week), and + winter_fuel_allowance otherwise. Where the household lives is read in + each year, so a household that moves into or out of Scotland changes + scheme with the move. 3. Differential: household_benefits exceeds that of the same dataset with no reported winter heating payment by the report, plus the pensioner Cost-of-Living Payment it brings in 2022-23 and 2023-24. @@ -30,6 +33,7 @@ from hypothesis import strategies as st from policyengine_uk import CountryTaxBenefitSystem, Microsimulation, Simulation +from policyengine_uk.data import UKMultiYearDataset from policyengine_uk.data.dataset_schema import UKSingleYearDataset from policyengine_uk.reforms.policyengine.disable_simulated_benefits import ( BENEFITS, @@ -61,6 +65,7 @@ # set from BENEFITS, and why. NOT_IN_BENEFITS = { "winter_fuel_allowance": "set with pawhp from the one reported winter heating payment", + "carers_allowance": "set with carer_support_payment from the one reported Carer's Allowance", "maternity_allowance": "already the reported amount (adds maternity_allowance_reported)", "employee_pension_contributions": "not a benefit", "tax": "not a benefit", @@ -113,9 +118,12 @@ def values(simulation, variable, year): def check_winter_heating(simulation, regions, winter_heating, first_year): + """regions: each household's region, the same in every year, or a dict of + them by year.""" reported = np.asarray(winter_heating, dtype=float) - in_scotland = np.asarray(regions) == "SCOTLAND" for year in range(first_year, first_year + YEARS_IN_FUTURE): + regions_in_year = regions[year] if isinstance(regions, dict) else regions + in_scotland = np.asarray(regions_in_year) == "SCOTLAND" wfa = values(simulation, "winter_fuel_allowance", year) pawhp = values(simulation, "pawhp", year) np.testing.assert_allclose(wfa + pawhp, reported, err_msg=str(year)) @@ -238,3 +246,84 @@ def test_reported_winter_heating_is_conserved(first_year, households): dataset=dataset(first_year, regions, winter_heating), scenario=REFORM_ON ) check_winter_heating(simulation, regions, winter_heating, first_year) + + +def pensioner_situation(regions_by_year): + """One pensioner of 72 who reports a £300 winter heating payment in the + first year and lives in each year where regions_by_year says.""" + years = list(regions_by_year) + return { + "people": { + "pensioner": { + "age": {year: 72 + year - years[0] for year in years}, + "winter_fuel_allowance_reported": {years[0]: 300}, + } + }, + "benunits": {"family": {"members": ["pensioner"]}}, + "households": {"home": {"members": ["pensioner"], "region": regions_by_year}}, + } + + +@pytest.mark.parametrize( + "before, after", [("LONDON", "SCOTLAND"), ("SCOTLAND", "LONDON")] +) +def test_a_household_that_moves_is_paid_under_each_years_scheme(before, after): + """The split reads the country in each year: a household's outputs in a + year equal those of a household that has lived in that year's country + throughout.""" + first = 2025 + years = range(first, first + YEARS_IN_FUTURE) + moved = {year: before if year == first else after for year in years} + stayed = {year: after for year in years} + simulation = Simulation(situation=pensioner_situation(moved), scenario=REFORM_ON) + control = Simulation(situation=pensioner_situation(stayed), scenario=REFORM_ON) + check_winter_heating( + simulation, {year: [moved[year]] for year in years}, [300], first + ) + for year in years[1:]: + for name in [ + "winter_fuel_allowance", + "pawhp", + "household_benefits", + "hbai_household_net_income", + ]: + np.testing.assert_allclose( + values(simulation, name, year), + values(control, name, year), + err_msg=f"{name} {year}", + ) + + +@PROPERTY_SETTINGS +@given( + first_year=st.sampled_from([2022, 2023, 2024, 2025]), + households=st.lists( + st.tuples( + st.lists( + st.sampled_from(["SCOTLAND", "LONDON", "NORTHERN_IRELAND"]), + min_size=YEARS_IN_FUTURE, + max_size=YEARS_IN_FUTURE, + ), + st.sampled_from([0, 150.5, 300]), + ), + min_size=1, + max_size=4, + ), +) +def test_reported_winter_heating_follows_each_years_country(first_year, households): + """Conservation and placement in every year, with each household's region + drawn afresh for each year (a multi-year dataset).""" + years = range(first_year, first_year + YEARS_IN_FUTURE) + regions_by_year = { + year: [path[i] for path, _ in households] for i, year in enumerate(years) + } + winter_heating = [amount for _, amount in households] + simulation = Simulation( + dataset=UKMultiYearDataset( + datasets=[ + dataset(year, regions_by_year[year], winter_heating) for year in years + ] + ), + scenario=REFORM_ON, + ) + check_winter_heating(simulation, regions_by_year, winter_heating, first_year) diff --git a/policyengine_uk/tests/test_entered_directly_lifecycle.py b/policyengine_uk/tests/test_entered_directly_lifecycle.py index 8da15defc3..ebf572ed41 100644 --- a/policyengine_uk/tests/test_entered_directly_lifecycle.py +++ b/policyengine_uk/tests/test_entered_directly_lifecycle.py @@ -2,6 +2,7 @@ income_support_eligible reads income-related ESA and income-based JSA from the claimant's and partner's reported awards (SSCBA 1992 s.124(1)(f), (h)), +through claimant_or_partner_esa_income and claimant_or_partner_jsa_income, unless esa_income or jsa_income holds a different award from the one all the reported amounts give, which is then taken to be the couple's. The rule depends only on the value the simulation reads, so it holds through @@ -54,7 +55,15 @@ def family(award): def eligible(simulation): - simulation.delete_arrays("income_support_eligible") + # The gate reads the claimant-or-partner awards, which are calculated and + # cached like any other variable. Recalculate them too after changing an + # input they depend on. + for variable in ( + "income_support_eligible", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ): + simulation.delete_arrays(variable) return bool(simulation.calculate("income_support_eligible", YEAR)[0]) diff --git a/policyengine_uk/tests/test_income_support_direct_inputs.py b/policyengine_uk/tests/test_income_support_direct_inputs.py index 6f519f99c0..597c6de878 100644 --- a/policyengine_uk/tests/test_income_support_direct_inputs.py +++ b/policyengine_uk/tests/test_income_support_direct_inputs.py @@ -1,9 +1,10 @@ """A jsa_income that differs from the reported award is the couple's award. income_support_eligible reads income-based JSA (SSCBA 1992 s.124(1)(f)) from -the claimant's and partner's reported awards after the jsa_income capital -screen. Where jsa_income holds a different award from the one all the -reported amounts give, that award is used instead and taken to be the +claimant_or_partner_jsa_income: the claimant's and partner's reported awards +after the jsa_income capital screen, or the raw total of their reports where +jsa_income holds the raw total of everyone's. Where jsa_income holds a +different award from either, that award is used instead and taken to be the claimant's or partner's. That covers an award entered directly (when the simulation is built or later, or on a branch) and a reform that changes or removes jsa_income. An award entered for one year says nothing about another. @@ -53,7 +54,15 @@ def carer_family(benunit_inputs=None, other_adult=None): def eligible(simulation, year=YEAR): - simulation.delete_arrays("income_support_eligible") + # The gate reads the claimant-or-partner awards, which are calculated and + # cached like any other variable. Recalculate them too after changing an + # input they depend on. + for variable in ( + "income_support_eligible", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ): + simulation.delete_arrays(variable) return bool(simulation.calculate("income_support_eligible", year)[0]) @@ -115,13 +124,16 @@ def test_a_reform_that_removes_income_based_jsa_lifts_the_bar(): def test_the_raw_reported_total_is_read_through_the_reports(): - """disable_simulated_benefits sets jsa_income to the raw total of every - member's report. Here the partner reports £200 and an adult outside the - couple £3,000, with capital of £10,000. Tariff income of - ceil(4,000 / 250) x £1 x 52 = £832 a year extinguishes the partner's - award, so the claimant and partner have no income-based JSA. The raw - total, £3,200, differs from the screened award, £3,200 - £832 = £2,368, - but is still what the reports give, so the reports decide whose it is.""" + """A jsa_income equal to the raw total of every member's report, before + the capital test, is read as reported amounts paid in full (as + disable_simulated_benefits pays them): the claimant's and partner's + award is the raw total of their own reports. Here the partner reports + £200 and an adult outside the couple £3,000, with capital of £10,000. + Tariff income of ceil(4,000 / 250) x £1 x 52 = £832 a year extinguishes + the partner's award on the formula's reading, so the formula's £2,368 + (£3,200 - £832) does not bar the claim. The raw total, £3,200, pays the + partner's £200 unscreened, so it does. The excluded adult's £3,000 is + theirs on either reading.""" members = ["carer", "partner", "other_adult"] simulation = Simulation( situation={ @@ -156,7 +168,8 @@ def test_the_raw_reported_total_is_read_through_the_reports(): assert simulation.calculate("jsa_income", YEAR)[0] == 2_368 assert eligible(simulation) simulation.set_input("jsa_income", YEAR, [3_200]) - assert eligible(simulation) + assert not eligible(simulation) + assert simulation.calculate("claimant_or_partner_jsa_income", YEAR)[0] == 200 # A different award entered directly is the couple's. simulation.set_input("jsa_income", YEAR, [4_000]) assert not eligible(simulation) @@ -220,10 +233,12 @@ def test_an_excluded_adults_report_can_explain_a_direct_award(): def test_a_claimants_report_can_explain_a_direct_award(): - # Intended: a direct £200 with £10,000 of capital and no reports is the - # claimant's, and bars the claim. If the claimant reports exactly £200, - # the raw total explains it, and the claimant's £200 is extinguished by - # tariff income of £832 a year, so it no longer bars the claim. + # Intended: with £10,000 of capital, the formula's reading of the + # claimant's own £200 report leaves nothing after tariff income of £832 a + # year, so it does not bar the claim. A direct £200 with no report is the + # claimant's, and bars it. A direct £200 equal to the claimant's report is + # the raw total of the reports, read as the claimant's £200 paid in full + # (as disable_simulated_benefits pays it), so it bars the claim too. claimant_reports = { "age": {YEAR: 40}, "is_claimant_or_partner": {YEAR: True}, @@ -231,25 +246,26 @@ def test_a_claimants_report_can_explain_a_direct_award(): "income_support_reported": {YEAR: 1_000}, } - def single(report): + def single(report, entered=None): person = {**claimant_reports, "jsa_income_reported": {YEAR: report}} + benunit = { + "members": ["carer"], + "income_support_assessable_capital": {YEAR: 10_000}, + "jsa_income_assessable_capital": {YEAR: 10_000}, + } + if entered is not None: + benunit["jsa_income"] = {YEAR: entered} return Simulation( situation={ "people": {"carer": person}, - "benunits": { - "family": { - "members": ["carer"], - "income_support_assessable_capital": {YEAR: 10_000}, - "jsa_income_assessable_capital": {YEAR: 10_000}, - "jsa_income": {YEAR: 200}, - } - }, + "benunits": {"family": benunit}, "households": {"home": {"members": ["carer"]}}, } ) - assert not eligible(single(0)) assert eligible(single(200)) + assert not eligible(single(0, entered=200)) + assert not eligible(single(200, entered=200)) def test_the_half_penny_tolerance_endpoints(): diff --git a/policyengine_uk/tests/test_income_support_esa_entered_directly.py b/policyengine_uk/tests/test_income_support_esa_entered_directly.py index bf19537350..ce9b42862f 100644 --- a/policyengine_uk/tests/test_income_support_esa_entered_directly.py +++ b/policyengine_uk/tests/test_income_support_esa_entered_directly.py @@ -1,10 +1,12 @@ """Income-related ESA that is not the award on the reported amounts. -income_support_eligible reads the claimant's and partner's reported -income-related ESA, after the esa_income capital test. When esa_income holds -anything else (an award entered directly, or a reform that replaces or -removes it), the reported amounts do not say whose award it is, so the gate -takes esa_income itself to be the claimant's or partner's. The test is on the +income_support_eligible reads the claimant's and partner's income-related ESA +from claimant_or_partner_esa_income: their reported awards after the +esa_income capital test, or the plain total of their reports where esa_income +holds the plain total of everyone's. When esa_income holds anything else (an +award entered directly, or a reform that replaces or removes it), the +reported amounts do not say whose award it is, so the gate takes esa_income +itself to be the claimant's or partner's. The test is on the value the simulation reads, so it holds however that value got there: set before or after the simulation is built, on a clone or a branch, for this year or another, deleted and recalculated. @@ -60,7 +62,15 @@ def family(benunit=None): def eligible(sim): - sim.delete_arrays("income_support_eligible") + # The gate reads the claimant-or-partner awards, which are calculated and + # cached like any other variable. Recalculate them too after changing an + # input they depend on. + for variable in ( + "income_support_eligible", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ): + sim.delete_arrays(variable) return bool(sim.calculate("income_support_eligible", YEAR)[0]) @@ -143,11 +153,14 @@ def test_abolishing_income_related_esa_removes_the_bar(): def test_the_plain_reported_total_is_read_through_the_reports(): - # disable_simulated_benefits sets esa_income to the plain total of the - # reported awards, before the capital test. With £10,000 of capital the - # tariff income (£832 a year) extinguishes the partner's own £200, so the - # claimant and partner have no income-related ESA; the £3,000 is the - # excluded adult's. + # An esa_income equal to the plain total of the reported awards, before + # the capital test, is read as reported amounts paid in full (as + # disable_simulated_benefits pays them): the claimant's and partner's + # award is the plain total of their own reports. With £10,000 of capital + # the tariff income (£832 a year) would extinguish the partner's own + # £200 on the formula's reading, so the formula's award does not bar the + # claim. The plain total pays the partner's £200 unscreened, so it does. + # Either way the excluded adult's £3,000 is theirs, not the couple's. partner = { "age": {YEAR: 42}, "is_claimant_or_partner": {YEAR: True}, @@ -157,8 +170,18 @@ def test_the_plain_reported_total_is_read_through_the_reports(): {"carer": CARER, "partner": partner, "other_adult": EXCLUDED_ADULT}, {"esa_income_assessable_capital": {YEAR: 10_000}}, ) - sim.set_input("esa_income", YEAR, np.array([3_200.0])) + assert sim.calculate("esa_income", YEAR)[0] == 2_368 assert eligible(sim) + sim.set_input("esa_income", YEAR, np.array([3_200.0])) + assert not eligible(sim) + assert sim.calculate("claimant_or_partner_esa_income", YEAR)[0] == 200 + # Without the partner's report, the plain total is the excluded adult's. + alone = simulation( + {"carer": CARER, "other_adult": EXCLUDED_ADULT}, + {"esa_income_assessable_capital": {YEAR: 10_000}}, + ) + alone.set_input("esa_income", YEAR, np.array([3_000.0])) + assert eligible(alone) def test_an_award_equal_to_the_reported_amounts_is_read_through_them(): diff --git a/policyengine_uk/tests/test_income_support_gate_reads_claimant_or_partner_awards.py b/policyengine_uk/tests/test_income_support_gate_reads_claimant_or_partner_awards.py new file mode 100644 index 0000000000..b14da3de7b --- /dev/null +++ b/policyengine_uk/tests/test_income_support_gate_reads_claimant_or_partner_awards.py @@ -0,0 +1,565 @@ +"""income_support_eligible reads the claimant-or-partner awards. + +Its income-related ESA and income-based JSA limbs (SSCBA 1992 s.124(1)(h) +and (f)) used to work out the claimant's and partner's award inline, by the +value rule that claimant_or_partner_esa_income and +claimant_or_partner_jsa_income also apply. They now read those variables, so +every reader shares one award, including one entered directly or set by the +disable_simulated_benefits reform. This file checks the switch against the +code it replaced (policyengine-uk main at e89b8abcf, with #2013, #2025 +and #2027 merged; comments removed). + +The two agree in formula mode, for awards entered directly, set later, set on +a branch, and recalculated after deletion. They differ, by design, in one +case: a stored award equal to the plain total of everyone's reports but not +to the formula's award. The old limbs then read the claimant's and partner's +award after the benefit's screen. The variables, and now the limbs, read the +plain total of their reports, the reports paid in full, as the reform that +stores plain totals pays them. That case is checked against its own +expectation. + +The replaced code also ignored a claimant-or-partner award entered directly +(review r3c on #2027). It worked the couple's award out from esa_income and +everyone's reports, so an adult outside the couple could decide whether the +entered award barred the claim. The last tests enter the claimant's and +partner's award and check that it alone decides its limb. +""" + +import numpy as np +import pytest +from hypothesis import event, given +from hypothesis import strategies as st +from policyengine_core.periods import period as as_period + +from policyengine_uk import Simulation +from policyengine_uk.model_api import add +from policyengine_uk.tests.test_income_support_eligibility_properties import ( + DIFFERENTIAL_SETTINGS, + FAMILIES, + input_settings, + situation as property_situation, +) +from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award +from policyengine_uk.variables.gov.dwp.jsa_income import income_related_jsa_award + +YEAR = 2025 +HALF_PENNY = 0.005 +LIMBS = { + "esa_income": ( + "esa_income_reported", + income_related_esa_award, + "claimant_or_partner_esa_income", + ), + "jsa_income": ( + "jsa_income_reported", + income_related_jsa_award, + "claimant_or_partner_jsa_income", + ), +} + + +def pre_switch_income_support_eligible(benunit, period, parameters): + IS = parameters(period).gov.dwp.income_support + person = benunit.members + claimant_or_partner = person("is_claimant_or_partner", period) + has_award = claimant_or_partner & (person("income_support_reported", period) > 0) + youngest_child_5_or_under = ( + benunit("youngest_child_age_for_legacy_benefits", period) + <= IS.eligibility.lone_parent_youngest_child_age_limit + ) + lone_parent_with_young_child = ( + benunit("is_lone_parent", period) & youngest_child_5_or_under + ) + placed_child = ( + person("is_looked_after_by_local_authority", period) + & person("is_child_for_child_benefit", period) + & ~claimant_or_partner + ) + single_with_placed_child = benunit("is_single", period) & benunit.any(placed_child) + prescribed_category = person("is_carer_for_benefits", period) | benunit.project( + lone_parent_with_young_child | single_with_placed_child + ) + under_qualifying_age = ~person( + "has_attained_state_pension_credit_qualifying_age", period + ) + WORK = IS.eligibility.remunerative_work + hours = person("income_support_remunerative_work_hours", period) + not_treated_as_working = person("is_carer_for_benefits", period) + works_as_claimant = ~not_treated_as_working & (hours >= WORK.claimant_hours) + works_as_partner = ( + claimant_or_partner & ~not_treated_as_working & (hours >= WORK.partner_hours) + ) + other_member_works = ( + benunit.project(benunit.sum(works_as_partner)) - works_as_partner + ) > 0 + no_contributory_jsa = person("jsa_contrib", period) <= 0 + no_contributory_esa = person("esa_contrib", period) <= 0 + claimant = ( + has_award + & prescribed_category + & under_qualifying_age + & no_contributory_esa + & no_contributory_jsa + & ~works_as_claimant + & ~other_member_works + ) + esa_income = benunit("esa_income", period) + reported_total = add(benunit, period, ["esa_income_reported"]) + award_on_all_reports = income_related_esa_award(benunit, period, reported_total) + award_on_claimant_or_partner_reports = income_related_esa_award( + benunit, + period, + benunit.sum(person("esa_income_reported", period) * claimant_or_partner), + ) + stored = esa_income.dtype + half_penny = 0.005 + as_reported = np.isclose( + esa_income, award_on_all_reports.astype(stored), rtol=0, atol=half_penny + ) | np.isclose(esa_income, reported_total.astype(stored), rtol=0, atol=half_penny) + income_related_esa = (esa_income > half_penny) & ( + ~as_reported | (award_on_claimant_or_partner_reports > half_penny) + ) + jsa_income = benunit("jsa_income", period) + jsa_reported_total = add(benunit, period, ["jsa_income_reported"]) + jsa_award_on_all_reports = income_related_jsa_award( + benunit, period, jsa_reported_total + ) + jsa_award_on_claimant_or_partner_reports = income_related_jsa_award( + benunit, + period, + benunit.sum(person("jsa_income_reported", period) * claimant_or_partner), + ) + jsa_as_reported = np.isclose( + jsa_income, + jsa_award_on_all_reports.astype(jsa_income.dtype), + rtol=0, + atol=half_penny, + ) | np.isclose( + jsa_income, + jsa_reported_total.astype(jsa_income.dtype), + rtol=0, + atol=half_penny, + ) + income_based_jsa = (jsa_income > half_penny) & ( + ~jsa_as_reported | (jsa_award_on_claimant_or_partner_reports > half_penny) + ) + on_pension_credit = benunit( + "meets_pension_credit_age_conditions", period + ) & benunit("would_claim_pc", period) + capital = benunit("income_support_assessable_capital", period) + return ( + benunit.any(claimant) + & ~on_pension_credit + & ~income_related_esa + & ~income_based_jsa + & (capital <= IS.means_test.capital.limit) + ) + + +def pre_switch_limb(benunit, period, award): + """One of the replaced inline limbs. The ESA and JSA blocks were the same + code with the names swapped; returns the limb and whether the stored award + equals only the plain total (the case that differs by design).""" + reported, award_on, _ = LIMBS[award] + person = benunit.members + claimant_or_partner = person("is_claimant_or_partner", period) + stored = benunit(award, period) + reported_total = add(benunit, period, [reported]) + on_all_reports = award_on(benunit, period, reported_total) + claimant_or_partner_reported = benunit.sum( + person(reported, period) * claimant_or_partner + ) + on_claimant_or_partner_reports = award_on( + benunit, period, claimant_or_partner_reported + ) + as_formula = np.isclose( + stored, on_all_reports.astype(stored.dtype), rtol=0, atol=HALF_PENNY + ) + as_reported_total = np.isclose( + stored, reported_total.astype(stored.dtype), rtol=0, atol=HALF_PENNY + ) + limb = (stored > HALF_PENNY) & ( + ~(as_formula | as_reported_total) + | (on_claimant_or_partner_reports > HALF_PENNY) + ) + # The intended reading in the plain-total case: the claimant's and + # partner's reports paid in full. + plain_total_only = as_reported_total & ~as_formula + intended = np.where( + plain_total_only, + (stored > HALF_PENNY) & (claimant_or_partner_reported > HALF_PENNY), + limb, + ) + return limb, intended, plain_total_only + + +def fresh(simulation, variables): + for variable in variables: + simulation.delete_arrays(variable) + + +def compare(simulation, year=YEAR): + """Assert the switched gate and limbs against the replaced code.""" + period = as_period(year) + benunit = simulation.populations["benunit"] + fresh( + simulation, + ["income_support_eligible", *[limb[2] for limb in LIMBS.values()]], + ) + differs = np.zeros(benunit.count, dtype=bool) + for award, (_, _, variable) in LIMBS.items(): + old, intended, plain_total_only = pre_switch_limb(benunit, period, award) + new = simulation.calculate(variable, year) > 0 + np.testing.assert_array_equal(new, intended, err_msg=award) + np.testing.assert_array_equal( + new[~plain_total_only], old[~plain_total_only], err_msg=award + ) + differs |= plain_total_only & (new != old) + old_gate = pre_switch_income_support_eligible( + benunit, period, simulation.tax_benefit_system.parameters + ) + new_gate = simulation.calculate("income_support_eligible", year) + np.testing.assert_array_equal(new_gate[~differs], old_gate[~differs]) + return differs + + +CARER = { + "age": 40, + "is_claimant_or_partner": True, + "receives_carer_benefit": True, + "income_support_reported": 1_000, +} +PARTNER = {"age": 42, "is_claimant_or_partner": True} +OTHER_ADULT = { + "age": 30, + "is_claimant_or_partner": False, + "current_education": "NOT_IN_EDUCATION", +} + +# Each family is (members, capital). The carer satisfies every other +# condition, so only the income-related limbs decide eligibility. +EXAMPLES = [ + ([CARER], 0), + ([CARER, {**OTHER_ADULT, "esa_income_reported": 3_000}], 0), + ([CARER, {**OTHER_ADULT, "jsa_income_reported": 3_000}], 0), + ([CARER, {**PARTNER, "esa_income_reported": 3_000}], 0), + ([CARER, {**PARTNER, "jsa_income_reported": 3_000}], 0), + # £10,000 of capital: tariff income of £832 a year extinguishes the + # partner's own £200 on the formula's reading. + ( + [ + CARER, + {**PARTNER, "esa_income_reported": 200, "jsa_income_reported": 200}, + { + **OTHER_ADULT, + "esa_income_reported": 3_000, + "jsa_income_reported": 3_000, + }, + ], + 10_000, + ), +] +OVER_TARIFF = len(EXAMPLES) - 1 + + +def simulation(families, entered=None): + people, benunits, households = {}, {}, {} + for i, (members, capital) in enumerate(families): + names = [] + for j, inputs in enumerate(members): + name = f"p{i}_{j}" + people[name] = {k: {YEAR: v} for k, v in inputs.items()} + names.append(name) + benunits[f"b{i}"] = { + "members": names, + "income_support_assessable_capital": {YEAR: capital}, + "esa_income_assessable_capital": {YEAR: capital}, + "jsa_income_assessable_capital": {YEAR: capital}, + } + for award, values in (entered or {}).items(): + benunits[f"b{i}"][award] = {YEAR: values[i]} + households[f"h{i}"] = {"members": names} + return Simulation( + situation={"people": people, "benunits": benunits, "households": households} + ) + + +def gate(simulation): + fresh( + simulation, + ["income_support_eligible", *[limb[2] for limb in LIMBS.values()]], + ) + return simulation.calculate("income_support_eligible", YEAR).tolist() + + +ENTERED = [4_000, 0, 4_000, 0, 3_000, 4_000] + + +def test_formula_mode_agrees(): + sim = simulation(EXAMPLES) + assert not compare(sim).any() + # Only the claimant's and partner's own awards bar the claim. + assert gate(sim) == [True, True, True, False, False, True] + + +def test_awards_entered_in_the_situation_agree(): + for award in LIMBS: + assert not compare(simulation(EXAMPLES, {award: ENTERED})).any(), award + + +def test_awards_set_after_calculating_agree(): + sim = simulation(EXAMPLES) + compare(sim) + for award in LIMBS: + sim.set_input(award, YEAR, np.array(ENTERED, dtype=float)) + assert not compare(sim).any(), award + + +def test_awards_set_on_a_branch_agree_there_and_leave_the_parent_alone(): + sim = simulation(EXAMPLES) + before = gate(sim) + branch = sim.get_branch("entered", clone_system=False) + for award in LIMBS: + branch.set_input(award, YEAR, np.array(ENTERED, dtype=float)) + assert not compare(branch).any() + assert not compare(sim).any() + assert gate(sim) == before + nested = branch.get_branch("zero", clone_system=False) + nested.set_input("esa_income", YEAR, np.zeros(len(EXAMPLES))) + assert not compare(nested).any() + + +def test_a_deleted_award_is_recalculated_and_agrees(): + sim = simulation(EXAMPLES) + before = gate(sim) + for award in LIMBS: + sim.set_input(award, YEAR, np.array(ENTERED, dtype=float)) + assert not compare(sim).any() + for award in LIMBS: + sim.delete_arrays(award) + assert not compare(sim).any() + assert gate(sim) == before + + +def test_a_plain_total_reads_the_claimant_and_partner_reports_in_full(): + # Intended difference. The stored award is the plain total of everyone's + # reports. In the last family the partner's £200 is extinguished by + # tariff income on the formula's reading; the old limb read that, and did + # not bar the claim. The plain total pays the £200 in full, so it does. + for award, (reported, _, variable) in LIMBS.items(): + sim = simulation(EXAMPLES) + plain_total = sim.calculate(reported, YEAR, map_to="benunit") + sim.set_input(award, YEAR, plain_total) + differs = compare(sim) + assert differs.tolist() == [i == OVER_TARIFF for i in range(len(EXAMPLES))] + assert sim.calculate(variable, YEAR)[OVER_TARIFF] == 200 + old_gate = pre_switch_income_support_eligible( + sim.populations["benunit"], + as_period(YEAR), + sim.tax_benefit_system.parameters, + ) + assert old_gate[OVER_TARIFF] + assert not gate(sim)[OVER_TARIFF] + + +# Stored values at and around both readings, half a penny either side. +OFFSETS = [0, 0, 0.004, -0.004, 0.006, -0.006] + + +@st.composite +def stored_award_families(draw): + """A claimant, an optional partner and an optional adult outside the + couple, each reporting ESA and JSA, and how each award is stored.""" + amounts = st.sampled_from([0, 0, 0.004, 200, 3_000, 65_536.01]) + members = [ + {**CARER, "esa_income_reported": draw(amounts), "jsa_income_reported": 0} + ] + members[0]["jsa_income_reported"] = draw(amounts) + if draw(st.booleans()): + members.append( + { + **PARTNER, + "esa_income_reported": draw(amounts), + "jsa_income_reported": draw(amounts), + } + ) + if draw(st.booleans()): + members.append( + { + **OTHER_ADULT, + "esa_income_reported": draw(amounts), + "jsa_income_reported": draw(amounts), + } + ) + capital = draw(st.sampled_from([0, 6_250, 10_000, 20_000])) + storage = { + award: ( + draw(st.sampled_from(["formula", "plain_total", "entered", "zero"])), + draw(st.sampled_from(OFFSETS)), + ) + for award in LIMBS + } + return (members, capital), storage + + +@DIFFERENTIAL_SETTINGS +@given(st.lists(stored_award_families(), min_size=1, max_size=10)) +def test_the_limbs_read_every_stored_value_as_intended(drawn): + families = [family for family, _ in drawn] + sim = simulation(families) + for award, (reported, _, _) in LIMBS.items(): + formula = sim.calculate(award, YEAR).astype(float) + plain_total = sim.calculate(reported, YEAR, map_to="benunit") + stored = [] + for i, (_, storage) in enumerate(drawn): + mode, offset = storage[award] + base = { + "formula": formula[i], + "plain_total": plain_total[i], + "entered": 4_000, + "zero": 0, + }[mode] + stored.append(max(0, base + offset)) + event(f"{award} stored as {mode}") + sim.set_input(award, YEAR, np.array(stored)) + if compare(sim).any(): + event("the plain-total reading changes a family's eligibility") + # The claimant's and partner's award is never negative and never more + # than the stored award it is part of (to the half penny of the rule). + for award, (_, _, variable) in LIMBS.items(): + scoped = sim.calculate(variable, YEAR) + assert (scoped >= 0).all(), award + assert (scoped <= sim.calculate(award, YEAR) + 0.005).all(), award + + +@DIFFERENTIAL_SETTINGS +@given(FAMILIES, st.data()) +def test_the_switched_gate_matches_the_replaced_code(drawn, data): + # Families from the eligibility properties, with adults outside the + # couple who report awards, and awards calculated or entered directly + # (£0 or £4,000, never a plain total). The gates agree exactly. + capital_as_savings, esa_income, jsa_income = data.draw(input_settings(len(drawn))) + units = [(*family, extra) for family, extra in drawn] + sim = Simulation( + situation=property_situation(units, capital_as_savings, esa_income, jsa_income) + ) + assert not compare(sim).any() + + +# What the adult outside the couple reports in the entered-award cases. With +# £3,000 of the award stored for the benefit unit, a report of £3,000 is what +# the reports give, so the replaced code read the stored award through them +# and took it to be the outside adult's; £2,000 and £5,000 are not, so it +# took the stored award to be the couple's. Neither reading looked at the +# claimant-or-partner award entered beside it. +OUTSIDE_REPORTS = [2_000, 3_000, 5_000] + + +def entered_directly(award, scoped): + """A carer otherwise eligible for Income Support and an adult outside the + couple who reports each of OUTSIDE_REPORTS of the award, one family each. + £3,000 of the award is entered for the benefit unit and `scoped` as the + claimant's and partner's.""" + reported, _, variable = LIMBS[award] + families = [ + ([CARER, {**OTHER_ADULT, reported: other}], 0) for other in OUTSIDE_REPORTS + ] + n = len(families) + sim = simulation(families, {award: [3_000] * n, variable: [scoped] * n}) + # Income-based JSA closed on 1 April 2026; the cases need a year in which + # it still pays awards. + assert sim.tax_benefit_system.parameters(YEAR).gov.dwp.JSA.income.active + assert sim.calculate(award, YEAR).tolist() == [3_000] * n + assert sim.calculate(variable, YEAR).tolist() == [scoped] * n + return sim + + +@pytest.mark.parametrize("award", LIMBS) +def test_an_entered_claimant_or_partner_award_bars_the_claim(award): + # The couple are on the income-related award (s.124(1)(h) for ESA, (f) + # for JSA), so Income Support is barred whatever the outside adult + # reports. The replaced code let the claim through where that report was + # £3,000; #2013's gate, which had no JSA limb, let every JSA case through. + sim = entered_directly(award, 3_000) + assert sim.calculate("income_support_eligible", YEAR).tolist() == [False] * 3 + assert sim.calculate("income_support", YEAR).tolist() == [0] * 3 + + +@pytest.mark.parametrize("award", LIMBS) +def test_an_entered_zero_claimant_or_partner_award_never_bars_the_claim(award): + # The £3,000 stored for the benefit unit is all the outside adult's, so it + # never bars the claim. The replaced code barred it where that adult's + # report was not £3,000. + sim = entered_directly(award, 0) + assert sim.calculate("income_support_eligible", YEAR).tolist() == [True] * 3 + assert (sim.calculate("income_support", YEAR) > 0).all() + + +@st.composite +def entered_awards(draw, n): + """What is entered for each of n families. esa_income and jsa_income are + calculated, or entered for every family as £0, £3,000 (what an outside + adult's report can give) or £4,000. The claimant-or-partner awards are + always entered, for every family.""" + entries = {} + for award, (_, _, variable) in LIMBS.items(): + if draw(st.booleans()): + entries[award] = draw( + st.lists(st.sampled_from([0, 3_000, 4_000]), min_size=n, max_size=n) + ) + entries[variable] = draw( + st.lists(st.sampled_from([0, 0, 3_000, 4_000]), min_size=n, max_size=n) + ) + return entries + + +@DIFFERENTIAL_SETTINGS +@given(FAMILIES, st.data()) +def test_an_entered_claimant_or_partner_award_alone_decides_its_limb(drawn, data): + # Families from the eligibility properties, each with an adult outside + # the couple, in three copies: + # - as drawn, with the claimant-or-partner awards entered; + # - the same, with the outside adult's ESA and JSA reports redrawn; + # - as drawn, with the claimant-or-partner awards entered as zero. + # The outside adult's reports never change the gate, and an entered award + # bars the claim exactly when it is positive (s.124(1)(f), (h)). + k = len(drawn) + capital_as_savings = data.draw(st.booleans()) + entries = data.draw(entered_awards(k)) + outside = st.sampled_from([0, 2_000, 3_000, 5_000]) + redrawn = data.draw(st.lists(st.tuples(outside, outside), min_size=k, max_size=k)) + as_drawn = [(*family, extra) for family, extra in drawn] + moved = [ + (*family, {**extra, "esa_income_reported": esa, "jsa_income_reported": jsa}) + for (family, extra), (esa, jsa) in zip(drawn, redrawn) + ] + situation = property_situation( + as_drawn + moved + as_drawn, + capital_as_savings, + entries.get("esa_income"), + entries.get("jsa_income"), + ) + variables = [variable for _, _, variable in LIMBS.values()] + for i in range(3 * k): + for variable in variables: + value = entries[variable][i % k] if i < 2 * k else 0 + situation["benunits"][f"b{i}"][variable] = {YEAR: value} + sim = Simulation(situation=situation) + for variable in variables: + expected = entries[variable] * 2 + [0] * k + assert sim.calculate(variable, YEAR).tolist() == expected, variable + eligible = sim.calculate("income_support_eligible", YEAR) + entered, redrawn_eligible, unbarred = ( + eligible[:k], + eligible[k : 2 * k], + eligible[2 * k :], + ) + np.testing.assert_array_equal(redrawn_eligible, entered) + barred = np.zeros(k, dtype=bool) + for variable in variables: + barred |= np.array(entries[variable]) > 0 + np.testing.assert_array_equal(entered, unbarred & ~barred) + if (unbarred & barred).any(): + event("an entered award bars an otherwise eligible family") + if (unbarred & ~barred).any(): + event("an entered zero leaves a family eligible") diff --git a/policyengine_uk/tests/test_reform_inputs_survive_reset.py b/policyengine_uk/tests/test_reform_inputs_survive_reset.py new file mode 100644 index 0000000000..cc4e3b0ec8 --- /dev/null +++ b/policyengine_uk/tests/test_reform_inputs_survive_reset.py @@ -0,0 +1,404 @@ +"""A reform applied to a built simulation keeps the inputs it stores. + +input_variables is taken when a simulation is built, and reset_calculations +deletes the values of every variable it does not name (as derivative did +before policyengine-core 3.33.0, from which it keeps every input). So +Simulation.apply_reform names each variable the reform stored an input for +that the simulation still holds afterwards, however the reform stored it, and +no other: an input set since the build that the reform did not store is still +deleted by a reset, as before, unless the reform names its variable. +input_variables names whole variables, so a reset keeps every input of a named +variable. See _InputKeyLog in policyengine_uk/simulation.py. + +Each simulation is one adult, the claimant, with £3,000 of income-related ESA +entered for 2026, so their claimant-or-partner award that year is £3,000 +unless an input replaces it. +""" + +import copy +import pickle +import threading +from functools import cache + +import numpy as np +import pytest +from hypothesis import HealthCheck, given, settings +from hypothesis import strategies as st + +from policyengine_core.periods import period + +from policyengine_uk import Simulation +from policyengine_uk.model_api import ( + MONTH, + YEAR, + BenUnit, + Reform, + Variable, + set_input_dispatch_by_period, +) + +SCOPED = "claimant_or_partner_esa_income" +AWARD_YEAR = 2026 +SITUATION = { + "people": { + "claimant": { + "age": {2025: 40, 2026: 40, 2027: 40}, + "is_claimant_or_partner": {2025: True, 2026: True, 2027: True}, + } + }, + "benunits": { + "benunit": {"members": ["claimant"], "esa_income": {AWARD_YEAR: 3_000}} + }, + "households": {"household": {"members": ["claimant"]}}, +} + + +def as_reform(apply): + return type("TestReform", (Reform,), {"apply": apply}) + + +def built(): + # An input set after the build: recorded as an input, not named. + simulation = Simulation(situation=SITUATION) + simulation.set_input(SCOPED, AWARD_YEAR, [100]) + assert SCOPED not in simulation.input_variables + return simulation + + +def through_the_simulation(reform): + reform.simulation.set_input(SCOPED, AWARD_YEAR, [0]) + + +def through_the_holder(reform): + simulation = reform.simulation + holder = simulation.get_holder(SCOPED) + holder.set_input(period(AWARD_YEAR), [0], simulation.branch_name) + + +def from_another_thread(reform): + worker = threading.Thread( + target=reform.simulation.set_input, args=(SCOPED, AWARD_YEAR, [0]) + ) + worker.start() + worker.join() + + +def while_another_simulation_applies_a_reform(reform): + simulation = reform.simulation + other = simulation.clone() + other.apply_reform( + as_reform(lambda _: simulation.set_input(SCOPED, AWARD_YEAR, [0])) + ) + + +def after_applying_another_reform_to_itself(reform): + simulation = reform.simulation + simulation.apply_reform(as_reform(lambda _: None)) + simulation.set_input(SCOPED, AWARD_YEAR, [0]) + + +def after_its_record_is_rebuilt(reform): + # Core replaces the record when it rebinds the simulation to its system. + simulation = reform.simulation + simulation._bind_to_tax_benefit_system() + simulation.set_input(SCOPED, AWARD_YEAR, [0]) + + +STORING = { + "the simulation's set_input": as_reform(through_the_simulation), + "a holder's set_input": as_reform(through_the_holder), + "another thread": as_reform(from_another_thread), + "while another simulation applies a reform": as_reform( + while_another_simulation_applies_a_reform + ), + "after applying another reform to itself": as_reform( + after_applying_another_reform_to_itself + ), + "after its record is rebuilt": as_reform(after_its_record_is_rebuilt), + "a tuple's second reform": ( + as_reform(lambda _: None), + as_reform(through_the_simulation), + ), +} + + +@pytest.mark.parametrize("route", STORING) +def test_a_reform_keeps_what_it_stores_through_a_reset(route): + simulation = built() + names = simulation.input_variables + recorded = list(names) + simulation.apply_reform(STORING[route]) + assert simulation.input_variables == recorded + [SCOPED] + assert names == recorded + assert type(simulation._user_input_keys) is set + assert simulation.tax_benefit_system.simulation is simulation + assert simulation.calculate(SCOPED, AWARD_YEAR).tolist() == [0] + # From policyengine-core 3.33.0 derivative keeps every input, named or not. + assert not simulation.derivative(SCOPED, "employment_income", AWARD_YEAR).any() + simulation.reset_calculations() + assert simulation.calculate(SCOPED, AWARD_YEAR).tolist() == [0] + + +class ended_award(Variable): + value_type = float + entity = BenUnit + label = "An award that ends in 2026" + definition_period = YEAR + end = "2026-12-31" + + def formula(benunit, period, parameters): + return 42 + + +def stores_then_deletes_another_year(simulation): + def apply(reform): + reform.simulation.set_input(SCOPED, AWARD_YEAR + 1, [0]) + reform.simulation.delete_arrays(SCOPED, AWARD_YEAR + 1) + + simulation.apply_reform(as_reform(apply)) + return SCOPED, AWARD_YEAR, 3_000 + + +def sets_a_year_after_the_variables_end(simulation): + simulation.tax_benefit_system.add_variable(ended_award) + simulation.set_input("ended_award", AWARD_YEAR, [100]) + simulation.apply_reform( + as_reform( + lambda reform: reform.simulation.set_input( + "ended_award", AWARD_YEAR + 1, [0] + ) + ) + ) + return "ended_award", AWARD_YEAR, 42 + + +class monthly_flag(Variable): + value_type = bool + entity = BenUnit + label = "A monthly flag a yearly input sets in each month" + definition_period = MONTH + set_input = set_input_dispatch_by_period + + def formula(benunit, period, parameters): + return False + + +def sets_a_year_whose_months_a_helper_keeps(simulation): + # The helper sets a yearly input in each month, skipping a month that + # already holds an input. + simulation.tax_benefit_system.add_variable(monthly_flag) + for month in range(1, 13): + simulation.set_input("monthly_flag", f"{AWARD_YEAR}-{month:02}", [True]) + simulation.apply_reform( + as_reform( + lambda reform: reform.simulation.set_input( + "monthly_flag", AWARD_YEAR, [False] + ) + ) + ) + january = f"{AWARD_YEAR}-01" + assert simulation.calculate("monthly_flag", january).tolist() == [True] + return "monthly_flag", january, False + + +NOT_STORING = { + "stores and deletes another year": stores_then_deletes_another_year, + "sets a year after the variable's end": sets_a_year_after_the_variables_end, + "sets a year whose months a helper keeps": sets_a_year_whose_months_a_helper_keeps, +} + + +@pytest.mark.parametrize("case", NOT_STORING) +def test_a_reform_names_no_input_it_did_not_store(case): + # The reform tries to set the variable but leaves no input of its own, + # while one set since the build survives. That one is not the reform's, + # so it is not named, and a reset deletes it as it would have anyway. + simulation = built() + recorded = list(simulation.input_variables) + variable, year, calculated = NOT_STORING[case](simulation) + assert simulation.input_variables == recorded + simulation.reset_calculations() + assert simulation.calculate(variable, year).tolist() == [calculated] + + +class added_award(Variable): + value_type = float + entity = BenUnit + label = "An award added after the build" + definition_period = YEAR + + def formula(benunit, period, parameters): + return 42 + + +def test_a_reform_names_no_input_of_a_variable_it_neutralizes(): + # Core stores nothing for a variable neutralized before the simulation + # makes its holder. (The build makes a holder for every variable it has + # then, and a holder keeps the definition it was made with, so this + # variable is added after the build.) + def apply(reform): + reform.neutralize_variable("added_award") + reform.simulation.set_input("added_award", AWARD_YEAR, [0]) + + simulation = built() + simulation.tax_benefit_system.add_variable(added_award) + recorded = list(simulation.input_variables) + with pytest.warns(Warning, match="neutralized"): + simulation.apply_reform(as_reform(apply)) + assert simulation.input_variables == recorded + assert simulation.calculate("added_award", AWARD_YEAR).tolist() == [0] + + +@pytest.mark.parametrize("route", ["clone", "branch"]) +def test_a_reform_names_nothing_it_stores_on_a_clone_or_branch(route): + def apply(reform): + simulation = reform.simulation + other = ( + simulation.clone() if route == "clone" else simulation.get_branch("other") + ) + other.set_input(SCOPED, AWARD_YEAR, [0]) + + simulation = built() + names = simulation.input_variables + recorded = list(names) + simulation.apply_reform(as_reform(apply)) + assert simulation.input_variables is names + assert names == recorded + assert simulation.calculate(SCOPED, AWARD_YEAR).tolist() == [100] + + +def test_a_branchs_reform_keeps_what_it_stores_after_its_parent_applies_one(): + # A reform applied to a simulation rebuilds its branches' records too. + parent = built() + branch = parent.get_branch("reformed") + + def apply(reform): + parent.apply_reform(as_reform(lambda _: None)) + reform.simulation.set_input(SCOPED, AWARD_YEAR, [0]) + + branch.apply_reform(as_reform(apply)) + assert SCOPED in branch.input_variables + assert SCOPED not in parent.input_variables + branch.reset_calculations() + assert branch.calculate(SCOPED, AWARD_YEAR).tolist() == [0] + + +def test_a_record_copied_while_a_reform_runs_is_a_plain_set(): + copies = [] + + def apply(reform): + simulation = reform.simulation + simulation.set_input(SCOPED, AWARD_YEAR, [0]) + keys = simulation._user_input_keys + copies.extend( + [copy.copy(keys), copy.deepcopy(keys), pickle.loads(pickle.dumps(keys))] + ) + + simulation = built() + simulation.apply_reform(as_reform(apply)) + assert SCOPED in simulation.input_variables + for keys in copies: + assert type(keys) is set + assert (SCOPED, "default", period(AWARD_YEAR)) in keys + + +def test_a_reform_that_raises_leaves_the_record_as_it_was(): + def apply(reform): + reform.simulation.set_input(SCOPED, AWARD_YEAR, [0]) + raise ValueError("raised by the reform") + + simulation = built() + recorded = list(simulation.input_variables) + with pytest.raises(ValueError, match="raised by the reform"): + simulation.apply_reform(as_reform(apply)) + # A failed reform names nothing, and logs nothing afterwards. + assert simulation.input_variables == recorded + assert type(simulation._user_input_keys) is set + assert simulation.tax_benefit_system.simulation is simulation + simulation.set_input(SCOPED, AWARD_YEAR, [5]) + assert simulation.calculate(SCOPED, AWARD_YEAR).tolist() == [5] + + +# Property: drawn inputs set since the build, then a reform that sets and +# deletes inputs by any route. The reform names exactly the variables it +# leaves an input of its own for. A reset keeps every input of those +# variables (its own, and any set since the build that it left in place), +# and every variable it does not name recalculates as if nothing had been +# set since the build. +VARIABLES = [SCOPED, "claimant_or_partner_jsa_income"] +YEARS = [AWARD_YEAR - 1, AWARD_YEAR, AWARD_YEAR + 1] +ROUTES = ["simulation", "holder", "thread", "delete"] +EARLIER = 7 + + +@cache +def base(): + return Simulation(situation=SITUATION) + + +def write(simulation, route, variable, year, value): + if route == "simulation": + simulation.set_input(variable, year, [value]) + elif route == "holder": + holder = simulation.get_holder(variable) + holder.set_input(period(year), [value], simulation.branch_name) + elif route == "thread": + worker = threading.Thread( + target=simulation.set_input, args=(variable, year, [value]) + ) + worker.start() + worker.join() + else: + simulation.delete_arrays(variable, year) + + +cells = st.tuples(st.sampled_from(VARIABLES), st.sampled_from(YEARS)) + + +@settings( + max_examples=20, + deadline=None, + derandomize=True, + suppress_health_check=[HealthCheck.too_slow], +) +@given( + st.sets(cells), + st.lists(st.tuples(st.sampled_from(ROUTES), cells), max_size=6), +) +def test_a_reform_names_exactly_the_variables_it_leaves_inputs_for(earlier, operations): + parent = base() + names = parent.input_variables + recorded = list(names) + simulation = parent.clone() + for variable, year in earlier: + simulation.set_input(variable, year, [EARLIER]) + # The inputs left after the reform, by variable and year: those set since + # the build, then each write replacing and each deletion removing one. + left = dict.fromkeys(earlier, EARLIER) + for value, (route, (variable, year)) in enumerate(operations, start=100): + if route == "delete": + left.pop((variable, year), None) + else: + left[(variable, year)] = value + + def apply(reform): + for value, (route, (variable, year)) in enumerate(operations, start=100): + write(reform.simulation, route, variable, year, value) + + simulation.apply_reform(as_reform(apply)) + named = {variable for (variable, _), value in left.items() if value != EARLIER} + assert simulation.input_variables == recorded + sorted(named) + assert parent.input_variables is names + assert names == recorded + simulation.reset_calculations() + for (variable, year), value in left.items(): + if variable in named: + assert simulation.calculate(variable, year).tolist() == [value] + pristine = parent.clone() + for variable in set(VARIABLES) - named: + for year in YEARS: + np.testing.assert_array_equal( + simulation.calculate(variable, year), + pristine.calculate(variable, year), + err_msg=f"{variable} {year}", + ) diff --git a/policyengine_uk/utils/benefit_unit.py b/policyengine_uk/utils/benefit_unit.py index 2938336cb8..bf136eaf90 100644 --- a/policyengine_uk/utils/benefit_unit.py +++ b/policyengine_uk/utils/benefit_unit.py @@ -2,6 +2,10 @@ import numpy as np +# The tolerance for reading a stored award: values within half a penny of +# each other are the same, and an award within half a penny of zero is none. +HALF_PENNY = 0.005 + def add_for_members(benunit, period, variables, members): """Sum variables over the benefit-unit members a means test counts. @@ -57,12 +61,19 @@ def award_of_members(benunit, period, award, report, award_on_reports, members): ``award`` gives on a reported amount. Which awards ``award`` holds is decided by value: when it holds the award its formula gives on all reported awards, this is the same award on the reports of ``members``; - when it holds the plain total of the reports (the - disable_simulated_benefits reform), this is the plain total of their - reports; when it holds anything else (an award entered directly, or a - reform that replaces it), that value is taken to be theirs. Values are - compared to within half a penny after rounding to the precision - ``award`` is stored in. A stored zero is zero. + when it holds the plain total of the reports (as the + disable_simulated_benefits reform sets it in the dataset year), this is + the plain total of their reports; when it holds anything else (an award + entered directly, or a reform that replaces it), that value is taken to + be theirs. Values are compared to within half a penny after rounding to + the precision ``award`` is stored in. A stored zero is zero. + + In a later year the reform's held award is neither, so this takes all of + it, another member's award included, to be the members'. So readers of + the claimant's and partner's award read the claimant_or_partner_* + variables, which the reform sets: the means tests, the passports and the + Income Support gate directly, and the benefit cap through + award_of_couple. """ person = benunit.members stored_award = benunit(award, period) @@ -75,10 +86,10 @@ def award_of_members(benunit, period, award, report, award_on_reports, members): # formula's own award always matches the award recomputed here. stored = stored_award.dtype as_formula = np.isclose( - stored_award, award_on_all_reports.astype(stored), rtol=0, atol=0.005 + stored_award, award_on_all_reports.astype(stored), rtol=0, atol=HALF_PENNY ) as_reported_total = np.isclose( - stored_award, reported_total.astype(stored), rtol=0, atol=0.005 + stored_award, reported_total.astype(stored), rtol=0, atol=HALF_PENNY ) scoped = np.where( as_formula, @@ -86,3 +97,86 @@ def award_of_members(benunit, period, award, report, award_on_reports, members): np.where(as_reported_total, members_reported, stored_award), ) return np.where(stored_award > 0, scoped, 0) + + +def claimant_or_partner_award(benunit, period, award, reported, award_on_reports): + """The claimant's and partner's part of a stored benefit-unit award. + + ``award`` names the stored award (esa_income or jsa_income), which covers + every member of the benefit unit, and ``reported`` the person-level + reported amount it is built from. ``award_on_reports(benunit, period, + amount)`` is the award paid on a reported amount: the benefit's screen + and tariff income (income_related_esa_award, income_related_jsa_award). + + This is award_of_members for the claimant and partner, so which awards + the stored value holds is decided by its value, compared in the + precision it is stored in, to within half a penny: + + - equal to the award on everyone's reports (its formula's result), the + claimant's and partner's part is the award on their own reports; + - otherwise, equal to the plain total of everyone's reports, the + claimant's and partner's part is the plain total of their own reports; + - anything else (an award entered directly, or a reform that replaces or + scales it) is taken to be wholly the claimant's or partner's. + + An award within half a penny of zero is no award, whether stored or + worked out here: a stored zero gives zero, and so does the float residual + of about £0.000002 left by reports that exactly match the tariff income. + + The claimant_or_partner_* variables use this as their formula, and every + reader of the claimant's or partner's award reads those variables: the + means tests, the passports and the Income Support gate directly, and the + benefit cap through award_of_couple. So an award set on them directly (as + disable_simulated_benefits does for each year) is read the same way + everywhere. The one exception is a Universal Credit couple that is not + the claimant and partner (award_of_couple). + """ + stored = benunit(award, period) + scoped = award_of_members( + benunit, + period, + award, + reported, + award_on_reports, + benunit.members("is_claimant_or_partner", period), + ) + return np.where((stored > HALF_PENNY) & (scoped > HALF_PENNY), scoped, 0) + + +# The variable holding the claimant's and partner's part of each benefit-unit +# award; claimant_or_partner_award is its formula. +CLAIMANT_OR_PARTNER_AWARDS = { + "esa_income": "claimant_or_partner_esa_income", + "jsa_income": "claimant_or_partner_jsa_income", +} + + +def award_of_couple(benunit, period, award, reported, award_on_reports, couple): + """The part of the benefit-unit award ``award`` that ``couple`` claim. + + ``award`` is esa_income or jsa_income, and ``couple`` a person-level + boolean array: the single person or couple a rule reads (the benefit + cap's couple, for example). Where ``couple`` is the claimant and partner + (is_claimant_or_partner) for every member of the benefit unit, this is + their award, the claimant_or_partner_* variable for ``award``, so a value + set on it (as disable_simulated_benefits does for each year) or entered + directly is read here as everywhere else. In formula mode that equals + award_of_members for ``couple``, except that an award within half a + penny of zero is no award. Otherwise it is award_of_members for + ``couple``, read from the stored award by value. That is a Universal + Credit couple that is not the claimant and partner, which takes an + entered is_uc_claimant or is_uc_assessed_claimant, or more than two + members entered as claimant or partner; under disable_simulated_benefits + it still takes another member's held award to be the couple's after the + dataset year. + """ + person = benunit.members + same_couple = ~benunit.any(couple != person("is_claimant_or_partner", period)) + claimant_or_partner = benunit(CLAIMANT_OR_PARTNER_AWARDS[award], period) + if np.all(same_couple): + return claimant_or_partner + return np.where( + same_couple, + claimant_or_partner, + award_of_members(benunit, period, award, reported, award_on_reports, couple), + ) diff --git a/policyengine_uk/variables/gov/dwp/benefit_cap_welfare_benefits.py b/policyengine_uk/variables/gov/dwp/benefit_cap_welfare_benefits.py index b17e0c6644..03f20ded9c 100644 --- a/policyengine_uk/variables/gov/dwp/benefit_cap_welfare_benefits.py +++ b/policyengine_uk/variables/gov/dwp/benefit_cap_welfare_benefits.py @@ -1,6 +1,6 @@ from policyengine_uk.model_api import * from policyengine_uk.utils.benefit_cap import benefit_cap_couple -from policyengine_uk.utils.benefit_unit import award_of_members +from policyengine_uk.utils.benefit_unit import award_of_couple from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award from policyengine_uk.variables.gov.dwp.jsa_income import income_related_jsa_award @@ -72,16 +72,17 @@ def formula(benunit, period, parameters): # (benefit_cap_couple). couple = benefit_cap_couple(benunit, period) # Income-based JSA and income-related ESA are awarded to the benefit - # unit; the couple's part is read from their reports, as in - # claimant_or_partner_jsa_income and claimant_or_partner_esa_income. - income_based = award_of_members( + # unit; the couple's part is claimant_or_partner_jsa_income and + # claimant_or_partner_esa_income where the couple is the claimant and + # partner, and is read from their reports otherwise (award_of_couple). + income_based = award_of_couple( benunit, period, "jsa_income", "jsa_income_reported", income_related_jsa_award, couple, - ) + award_of_members( + ) + award_of_couple( benunit, period, "esa_income", diff --git a/policyengine_uk/variables/gov/dwp/claimant_or_partner_esa_income.py b/policyengine_uk/variables/gov/dwp/claimant_or_partner_esa_income.py index 6a8c8c3ff1..bda90a24b7 100644 --- a/policyengine_uk/variables/gov/dwp/claimant_or_partner_esa_income.py +++ b/policyengine_uk/variables/gov/dwp/claimant_or_partner_esa_income.py @@ -1,5 +1,5 @@ from policyengine_uk.model_api import * -from policyengine_uk.utils.benefit_unit import award_of_members +from policyengine_uk.utils.benefit_unit import claimant_or_partner_award from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award @@ -13,20 +13,24 @@ class claimant_or_partner_esa_income(Variable): "Anyone else in the benefit unit claims in their own right, so their " "award is left out here: a non-dependent adult, or a young person " "with an award of their own, which is payable to them. It still " - "counts in esa_income and so in household income. Which awards " - "esa_income holds is decided by value: " + "counts in esa_income and so in household income. Income Support's " + "gate (income_support_eligible) reads this too (s.124(1)(h)). " + "Which awards esa_income holds is decided by value: " "when it holds the award its formula gives on all reported awards, " "this is the same award on the claimant's and partner's reports; " - "when it holds the plain total of the reports (the " - "disable_simulated_benefits reform), this is the plain total of the " - "claimant's and partner's reports; when it holds anything else (an " - "award entered directly, or a reform that replaces it), that value " - "is taken to be the claimant's or partner's. Values are compared to " - "within half a penny after rounding to the precision esa_income is stored " - "in. A stored zero is zero. " - "An entered esa_income equal to either reported amount is read through " - "the reports, so to enter the claimant's or partner's award whatever " - "other members report, enter this variable directly." + "when it holds the plain total of the reports, this is the plain " + "total of the claimant's and partner's reports; when it holds " + "anything else (an award entered directly, or a reform that replaces " + "it), that value is taken to be the claimant's or partner's. Values " + "are compared to within half a penny after rounding to the precision " + "esa_income is stored in, and an award within half a penny of zero, " + "stored or worked out, is no award: a stored zero is zero, and so is " + "the float residual left by reports that exactly match the tariff " + "income. An entered esa_income equal to either reported amount is read " + "through the reports, so to enter the claimant's or partner's award " + "whatever other members report, enter this variable directly. The " + "disable_simulated_benefits reform does that for each year, from the " + "claimant's and partner's reports in the dataset year." ) definition_period = YEAR unit = GBP @@ -37,11 +41,10 @@ class claimant_or_partner_esa_income(Variable): ) def formula(benunit, period, parameters): - return award_of_members( + return claimant_or_partner_award( benunit, period, "esa_income", "esa_income_reported", income_related_esa_award, - benunit.members("is_claimant_or_partner", period), ) diff --git a/policyengine_uk/variables/gov/dwp/claimant_or_partner_jsa_income.py b/policyengine_uk/variables/gov/dwp/claimant_or_partner_jsa_income.py index 9efb7b710d..7571840388 100644 --- a/policyengine_uk/variables/gov/dwp/claimant_or_partner_jsa_income.py +++ b/policyengine_uk/variables/gov/dwp/claimant_or_partner_jsa_income.py @@ -1,5 +1,5 @@ from policyengine_uk.model_api import * -from policyengine_uk.utils.benefit_unit import award_of_members +from policyengine_uk.utils.benefit_unit import claimant_or_partner_award from policyengine_uk.variables.gov.dwp.jsa_income import income_related_jsa_award @@ -13,20 +13,24 @@ class claimant_or_partner_jsa_income(Variable): "Anyone else in the benefit unit claims in their own right, so their " "award is left out here: a non-dependent adult, or a young person " "with an award of their own, which is payable to them. It still " - "counts in jsa_income and so in household income. Which awards " - "jsa_income holds is decided by value: " + "counts in jsa_income and so in household income. Income Support's " + "gate (income_support_eligible) reads this too (s.124(1)(f)). " + "Which awards jsa_income holds is decided by value: " "when it holds the award its formula gives on all reported awards, " "this is the same award on the claimant's and partner's reports; " - "when it holds the plain total of the reports (the " - "disable_simulated_benefits reform), this is the plain total of the " - "claimant's and partner's reports; when it holds anything else (an " - "award entered directly, or a reform that replaces it), that value " - "is taken to be the claimant's or partner's. Values are compared to " - "within half a penny after rounding to the precision jsa_income is stored " - "in. A stored zero is zero. " - "An entered jsa_income equal to either reported amount is read through " - "the reports, so to enter the claimant's or partner's award whatever " - "other members report, enter this variable directly." + "when it holds the plain total of the reports, this is the plain " + "total of the claimant's and partner's reports; when it holds " + "anything else (an award entered directly, or a reform that replaces " + "it), that value is taken to be the claimant's or partner's. Values " + "are compared to within half a penny after rounding to the precision " + "jsa_income is stored in, and an award within half a penny of zero, " + "stored or worked out, is no award: a stored zero is zero, and so is " + "the float residual left by reports that exactly match the tariff " + "income. An entered jsa_income equal to either reported amount is read " + "through the reports, so to enter the claimant's or partner's award " + "whatever other members report, enter this variable directly. The " + "disable_simulated_benefits reform does that for each year, from the " + "claimant's and partner's reports in the dataset year." ) definition_period = YEAR unit = GBP @@ -37,11 +41,10 @@ class claimant_or_partner_jsa_income(Variable): ) def formula(benunit, period, parameters): - return award_of_members( + return claimant_or_partner_award( benunit, period, "jsa_income", "jsa_income_reported", income_related_jsa_award, - benunit.members("is_claimant_or_partner", period), ) diff --git a/policyengine_uk/variables/gov/dwp/housing_benefit/benefit_cap/is_housing_benefit_benefit_cap_exempt_specified_benefit.py b/policyengine_uk/variables/gov/dwp/housing_benefit/benefit_cap/is_housing_benefit_benefit_cap_exempt_specified_benefit.py index 08bb32bb18..6ebc6992ed 100644 --- a/policyengine_uk/variables/gov/dwp/housing_benefit/benefit_cap/is_housing_benefit_benefit_cap_exempt_specified_benefit.py +++ b/policyengine_uk/variables/gov/dwp/housing_benefit/benefit_cap/is_housing_benefit_benefit_cap_exempt_specified_benefit.py @@ -1,5 +1,5 @@ from policyengine_uk.model_api import * -from policyengine_uk.utils.benefit_unit import award_of_members +from policyengine_uk.utils.benefit_unit import award_of_couple from policyengine_uk.variables.gov.dwp._legacy_award_payee import ( is_payee_of_couple_award, ) @@ -104,11 +104,10 @@ def received_by(benefits, members): # an employment and support allowance ... which includes a support # component". Contributory and income-related allowances both carry # it; another member's allowance does not count. The couple's - # income-related award is the part of the family's award on the - # couple's reports, as in claimant_or_partner_esa_income, and it is - # received by its payee (reg 2(3A); is_on_income_related_esa reads - # the same). - couple_esa_award = award_of_members( + # income-related award is claimant_or_partner_esa_income + # (award_of_couple), and it is received by its payee (reg 2(3A); + # is_on_income_related_esa reads the same). + couple_esa_award = award_of_couple( benunit, period, "esa_income", diff --git a/policyengine_uk/variables/gov/dwp/income_support_eligible.py b/policyengine_uk/variables/gov/dwp/income_support_eligible.py index be1f79c4e4..480ae5f2bf 100644 --- a/policyengine_uk/variables/gov/dwp/income_support_eligible.py +++ b/policyengine_uk/variables/gov/dwp/income_support_eligible.py @@ -1,6 +1,4 @@ from policyengine_uk.model_api import * -from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award -from policyengine_uk.variables.gov.dwp.jsa_income import income_related_jsa_award class income_support_eligible(Variable): @@ -26,21 +24,26 @@ class income_support_eligible(Variable): "Jobseeker's Allowance. The partner must not be engaged in " "remunerative work (24 hours a week or more, unless a carer). Neither " "the claimant nor the partner may be entitled to income-related ESA " - "or income-based JSA, and the other member of a couple may not be " - "entitled to State Pension Credit, which the model takes to be so " - "where the couple meets the Pension Credit age conditions and would " - "claim it. An adult in the benefit unit who is neither the " + "or income-based JSA: the model reads their award from " + "claimant_or_partner_esa_income and claimant_or_partner_jsa_income, " + "as the means tests and passports do. The other member of a couple " + "may not be entitled to State Pension Credit, which the model takes " + "to be so where the couple meets the Pension Credit age conditions " + "and would claim it. An adult in the benefit unit who is neither the " "claimant nor the partner (such as a non-dependent adult) does not " "affect eligibility, with a declared exception for a stored " - "esa_income or jsa_income that is not the formula's own result. " - "Whether entered directly or replaced by a reform (including one that " - "scales it), such a value is read through the reported awards when it " - "equals what they give (the award after the benefit's screen, or " - "their plain total, to within half a penny after rounding to the " - "precision it is stored in), and is otherwise taken to be the " - "claimant's or partner's. So another member's report can change how " - "such a value is read. A stored award within half a penny of zero " - "never bars the claim." + "esa_income or jsa_income that is not the formula's own result and " + "for which no claimant-or-partner award is entered. Whether entered " + "directly or replaced by a reform (including one that scales it), " + "such a value is read through the reported awards when it equals " + "what they give (the award after the benefit's screen, or their " + "plain total, to within half a penny after rounding to the precision " + "it is stored in), and is otherwise taken to be the claimant's or " + "partner's. So another member's report can change how such a value " + "is read. A claimant-or-partner award entered directly, or set by the " + "disable_simulated_benefits reform, is read as it stands. Worked out " + "from a stored award, an award within half a penny of zero never " + "bars the claim." ) definition_period = YEAR reference = ( @@ -144,76 +147,25 @@ def formula(benunit, period, parameters): ) # ... and the other member of a couple is not entitled to an # income-related allowance. An income-related allowance covers the - # couple, so it bars Income Support whichever of them has it: the - # award on the claimant's and partner's reported amounts, after the - # same screen as esa_income (esa_income_eligible). When esa_income - # holds what the reported amounts give, either after that screen (the - # formula) or as their plain total (the disable_simulated_benefits - # reform), the reports say whose award it is. When it holds anything - # else (an award entered directly, or a reform that replaces, scales - # or removes it), they do not, and it is taken to be the claimant's - # or partner's. A stored value equal to either amount, to within half - # a penny after rounding to its stored precision, is read through the - # reports. Either way, no income-related ESA is paid when esa_income - # is zero, so a zero never bars the claim. An amount within half a - # penny of zero counts as zero, the same tolerance as the comparisons: - # reports that exactly match the tariff income can leave a float - # residual of about £0.000002. - esa_income = benunit("esa_income", period) - reported_total = add(benunit, period, ["esa_income_reported"]) - award_on_all_reports = income_related_esa_award(benunit, period, reported_total) - award_on_claimant_or_partner_reports = income_related_esa_award( - benunit, - period, - benunit.sum(person("esa_income_reported", period) * claimant_or_partner), - ) - # Compare in the precision esa_income is stored in (float32), so the - # formula's own award always matches the award recomputed here. - stored = esa_income.dtype - half_penny = 0.005 - as_reported = np.isclose( - esa_income, award_on_all_reports.astype(stored), rtol=0, atol=half_penny - ) | np.isclose( - esa_income, reported_total.astype(stored), rtol=0, atol=half_penny - ) - income_related_esa = (esa_income > half_penny) & ( - ~as_reported | (award_on_claimant_or_partner_reports > half_penny) - ) + # couple, so it bars Income Support whichever of them has it. The + # claimant's and partner's award is claimant_or_partner_esa_income, + # which every reader of it shares (the benefit cap through + # award_of_couple). On the formula's award it is the + # award on their own reports after esa_income's screen + # (esa_income_eligible); on the plain total of the reports it is the + # plain total of theirs; on any other stored esa_income it is that + # value. An award within half a penny of zero, stored or worked out, + # is no award (claimant_or_partner_award): a stored zero never bars + # the claim, and nor does the float residual of about £0.000002 left + # by reports that exactly match the tariff income. A value entered for + # it directly, or set for each year by disable_simulated_benefits, is + # read as it stands. + income_related_esa = benunit("claimant_or_partner_esa_income", period) > 0 # s.124(1)(f): neither the claimant nor the other member of a couple # is, and the couple are not, entitled to an income-based jobseeker's - # allowance, read the same way: the award on the claimant's and - # partner's reported amounts after the screen jsa_income applies - # (jsa_income_eligible), unless jsa_income holds something the - # reported amounts do not give. - jsa_income = benunit("jsa_income", period) - jsa_reported_total = add(benunit, period, ["jsa_income_reported"]) - jsa_award_on_all_reports = income_related_jsa_award( - benunit, period, jsa_reported_total - ) - jsa_award_on_claimant_or_partner_reports = income_related_jsa_award( - benunit, - period, - benunit.sum(person("jsa_income_reported", period) * claimant_or_partner), - ) - # Compare in the stored award's precision: the holder keeps it as - # float32, while the sums are float64. - jsa_as_reported = np.isclose( - jsa_income, - jsa_award_on_all_reports.astype(jsa_income.dtype), - rtol=0, - atol=half_penny, - ) | np.isclose( - jsa_income, - jsa_reported_total.astype(jsa_income.dtype), - rtol=0, - atol=half_penny, - ) - # As for ESA, an amount within half a penny of zero is no award: a - # stored zero never bars the claim, and nor does the float residual - # left by reports that exactly match the tariff income. - income_based_jsa = (jsa_income > half_penny) & ( - ~jsa_as_reported | (jsa_award_on_claimant_or_partner_reports > half_penny) - ) + # allowance, read the same way through claimant_or_partner_jsa_income + # (the screen is jsa_income_eligible). + income_based_jsa = benunit("claimant_or_partner_jsa_income", period) > 0 # s.124(1)(g): the other member of a couple is not entitled to State # Pension Credit. Entitlement needs a claim (SSAA 1992 s.1). Reading the # Pension Credit amount here would be circular (Pension Credit income diff --git a/policyengine_uk/variables/gov/dwp/universal_credit/benefit_cap/is_uc_benefit_cap_exempt_specified_benefit.py b/policyengine_uk/variables/gov/dwp/universal_credit/benefit_cap/is_uc_benefit_cap_exempt_specified_benefit.py index 72d1b3d395..531c812ba6 100644 --- a/policyengine_uk/variables/gov/dwp/universal_credit/benefit_cap/is_uc_benefit_cap_exempt_specified_benefit.py +++ b/policyengine_uk/variables/gov/dwp/universal_credit/benefit_cap/is_uc_benefit_cap_exempt_specified_benefit.py @@ -1,7 +1,7 @@ import os from policyengine_uk.model_api import * -from policyengine_uk.utils.benefit_unit import award_of_members +from policyengine_uk.utils.benefit_unit import award_of_couple from policyengine_uk.utils.supplied_inputs import supplied_input from policyengine_uk.utils.uc_work_related_requirements import ( other_member_of_single_claim, @@ -132,11 +132,12 @@ def received_by(benefits, members): # Reg 83(1)(a): "the claimant is receiving an employment and support # allowance ... which includes a support component". Contributory and # income-related allowances both carry it; another member's allowance - # does not count. The couple's income-related award is the part of - # the family's award on the couple's reports, as in - # claimant_or_partner_esa_income, and it is received by its payee + # does not count. The couple's income-related award is + # claimant_or_partner_esa_income where the couple is the claimant and + # partner, and is read from their reports otherwise + # (award_of_couple); it is received by its payee # (is_payee_of_couple_award). - couple_esa_award = award_of_members( + couple_esa_award = award_of_couple( benunit, period, "esa_income",