Read calendar-year series' bare years as calendar years, and make generated-series reforms take effect

changelog.d/reform-dict-process-time-params.fixed.md

@@ -0,0 +1 @@
+A bare-year reform key, or a key dated 1 January, on the economic growth rates, the indices built from them and the household parameters now names that calendar year. A bare year used to name the fiscal year from 6 April, which moved the next year: a 2027 earnings growth rate showed as 2027's but moved the 2028 index and the April 2029 triple lock. A 1 January key used to override that one day after processing, which nothing built from the parameter read. A change to an uprated household parameter past its last stated value now leaves the years around it on their projected path instead of freezing them. Reforms to series built during processing (the triple lock rate, the indices, the lagged series) now uprate what is built from them. Deprecated `labor_supply_responses` paths in reform dicts reach the parameters formulas read, and changes to a parameter given by both its path and its alias are merged in order instead of one being silently dropped.

docs/book/programs/gov/dwp/state-pension.md

@@ -225,20 +225,20 @@ hand, run it with `--gaps-only`.
 
 ### Scenarios
 
-A macro scenario applied before the data load that edits calendar-year
-growth moves the forecast inputs one for one, and so the triple lock.
-Calendar-year series are keyed to 1 January, so key the change
-`year:YYYY-01-01:1`: a bare year names the fiscal year from 6 April, which
-lands in the next calendar-year value and so moves the following year's
-inputs.
+A macro scenario that edits calendar-year growth moves the forecast inputs
+one for one, and so the triple lock. Calendar-year series are keyed to
+1 January, and a bare year (or `year:YYYY-01-01:1`) names that calendar
+year: 5% earnings growth in 2027 sets the May to July 2027 input and so the
+April 2028 rise. Apply it before the data load to project the data's
+incomes with it too.
 
 ```python
 from policyengine_uk.model_api import Scenario
 
 scenario = Scenario(
     parameter_changes={
         "gov.economic_assumptions.yoy_growth.obr.average_earnings": {
-            "year:2027-01-01:1": 0.05,
+            "2027": 0.05,
         },
     },
     applied_before_data_load=True,
@@ -265,10 +265,12 @@ Scenario(
 )
 ```
 
-These changes, and the reform levers below, must go through
-`Scenario(parameter_changes=...)`. The rates are built when parameters are
-processed, and a `reform=` dictionary edits parameters after that, so it
-changes the parameter but not the uprating.
+These changes, and the reform levers below, work through
+`Scenario(parameter_changes=...)` or a `reform=` dictionary alike: both are
+written onto the parameters before the rates are built. A rate can also be
+set directly, as `gov.economic_assumptions.yoy_growth.triple_lock` or the
+statutory override `triple_lock/outturn.yaml`; either uprates the pension
+from that April.
 
 ### Reform levers
 

docs/book/usage/scenarios.md

@@ -390,13 +390,17 @@ Each way of passing a reform reads period keys as it always has:
 | Key | `reform=` dict | `parameter_changes` | `Reform.from_dict` |
 |---|---|---|---|
 | `"2027-04-06.2028-04-05"` | those dates | those dates | those dates |
-| `"2027"` or `2027` | fiscal year 2027 only | fiscal year 2027 only | from 1 January 2027 onwards |
+| `"2027"` or `2027` | fiscal year 2027 only (calendar year 2027 on a calendar-year series) | fiscal year 2027 only (calendar year 2027 on a calendar-year series) | from 1 January 2027 onwards |
 | `"2027-04-06"` | that one day of the annualised value | that one day | from that day onwards |
 | a value with no period | from 1 January 2023 onwards | 2000 to 2099 | 2000 to 2099 |
 
-A single-date key in a `reform=` dict overrides the annualised value on that day only. Formulas read annual values on 1 January, so `"2027-01-01"` sets fiscal year 2027 and `"2027-04-06"` changes nothing. The model logs a warning when a key can change nothing. Use `"start.stop"` keys for dated measures.
+A single-date key in a `reform=` dict overrides the annualised value on that day only. Formulas read annual values on 1 January, so `"2027-01-01"` sets fiscal year 2027 and `"2027-04-06"` changes nothing. On a calendar-year series (below), a 1 January key names that calendar year instead. The model logs a warning when a key can change nothing. Use `"start.stop"` keys for dated measures.
 
-Changes applied after the data is loaded (the default) leave the data as built, so a changed economic assumption does not re-uprate incomes. Dated changes (`parameter_changes` and `"start.stop"` or bare-year keys) still come before the steps that read parameters while the simulation is built: the Universal Credit rebalancing awards and the structural reforms that parameters switch on. Single-day keys and a scenario's own simulation modifier run after those steps, as they always have. Set `applied_before_data_load=True` on a scenario to project the data with changed economic assumptions too.
+Some parameters hold one value per calendar year, dated 1 January, and are read there: the growth rates under `gov.economic_assumptions.yoy_growth`, the indices built from them under `gov.economic_assumptions.indices`, and the `household` parameters, which are not annualised. On these a bare year, or its 1 January, names the calendar year. So `{"gov.economic_assumptions.yoy_growth.obr.average_earnings": {"2027": 0.05}}` sets earnings growth in 2027: the 2027 earnings index, the May to July 2027 earnings input and the April 2028 triple lock all follow it.
+
+Processing also builds some parameters from others: the triple lock rate (`gov.economic_assumptions.yoy_growth.triple_lock`), the lagged and private pension series, the September CPI uprating rate and the indices. A change to one is written as soon as it is built, so everything built from it follows. `{"gov.economic_assumptions.yoy_growth.triple_lock": {"2028": 0.05}}` uprates the State Pension by 5% in April 2028, exactly as `gov.dwp.state_pension.triple_lock.outturn` set for that April does. A change to an index uprates whatever is uprated by it. The deprecated `gov.simulation.labor_supply_responses` paths change the `labour_supply_responses` parameters they copy.
+
+Changes applied after the data is loaded (the default) leave the data as built, so a changed economic assumption does not re-uprate incomes. Dated changes (`parameter_changes` and `"start.stop"` or bare-year keys) still come before the steps that read parameters while the simulation is built: the Universal Credit rebalancing awards and the structural reforms that parameters switch on. Single-day keys (other than a 1 January key on a calendar-year series, which is a dated change) and a scenario's own simulation modifier run after those steps, as they always have. Set `applied_before_data_load=True` on a scenario to project the data with changed economic assumptions too.
 
 All of this applies when the scenario or reform is passed to `Simulation(...)`. `Scenario.apply(simulation)` on a simulation that is already built edits its annualised values instead, as it always has: a `"2027-04-06.2028-04-05"` key there overrides those dates of the fiscal-year values, so it does not set fiscal year 2027, and the model logs a warning saying so. Pass the scenario when constructing the simulation for dated measures.
 

policyengine_uk/parameters/gov/economic_assumptions/create_economic_assumption_indices.py

@@ -10,9 +10,11 @@ def create_economic_assumption_indices(
 ) -> ParameterNode:
     econ_assumptions: ParameterNode = parameters.gov.economic_assumptions
     yoy_growth: ParameterNode = econ_assumptions.yoy_growth
+    # Index levels dated 1 January, one per calendar year, so a bare-year
+    # reform key names the calendar year (policyengine_uk.utils.dated_reforms).
     indices = ParameterNode(
         name="gov.economic_assumptions.indices",
-        data={},
+        data={"metadata": {"calendar_year_series": True}},
     )
     econ_assumptions.add_child(
         "indices",

policyengine_uk/parameters/gov/economic_assumptions/yoy_growth.yaml

@@ -33,6 +33,12 @@
 #   https://obr.uk/box/the-long-run-difference-between-rpi-and-cpi-inflation/
 #   https://www.bankofengland.co.uk/monetary-policy/inflation
 
+metadata:
+  # Each value is the growth over a calendar year, dated 1 January, and
+  # processing reads it there, so a bare-year reform key names the calendar
+  # year (policyengine_uk/utils/dated_reforms.py).
+  calendar_year_series: true
+
 obr:
   rpi:
     description: Retail price index year-on-year growth.

policyengine_uk/parameters/gov/hmrc/create_threshold_indexation.py

@@ -65,13 +65,17 @@
 parameters carry no ``uprating`` metadata, so core uprating does not extend
 them.
 
-The values are computed when the parameters are processed. A reform passed as
-a dict edits the processed parameters, so changing one of these parameters
-does not move the others. A scenario whose ``parameter_changes`` edit the raw
-parameters, or a simulation modifier that resets and reprocesses them (as
-``no_economic_assumptions`` does), rebuilds the values. So a changed CPI
-forecast moves all seven, and a raw edit to the personal allowance or basic
-rate limit after 2030-31 also moves the aligned NICs thresholds.
+The values are computed when the parameters are processed. Every reform route
+(a reform dict, ``parameter_changes``, ``Reform.from_dict``) writes its changes
+onto the parameters before they are processed, as does a simulation modifier
+that resets and reprocesses them (as ``no_economic_assumptions`` does), so the
+values are rebuilt with the change: a changed CPI forecast moves all seven,
+and a change to the personal allowance or basic rate limit after 2030-31 also
+moves the aligned NICs thresholds (see ``policyengine_uk.utils.dated_reforms``).
+The exceptions edit the processed parameters, so they do not move the others:
+``Scenario.apply`` on an already-built simulation, and a reform-dict key naming
+a single day or month (other than 1 January on a calendar-year series, such as
+a CPI growth rate, which is a dated change and so does move them).
 """
 
 from decimal import ROUND_CEILING, ROUND_HALF_UP, Decimal

policyengine_uk/parameters/gov/simulation/labour_supply_responses/aliases.py

@@ -24,6 +24,27 @@ def canonicalize_lsr_parameter_path(path: str) -> str:
     return path
 
 
+def canonicalize_parameter_changes(changes: dict) -> dict:
+    """Changes keyed by canonical parameter paths.
+
+    A deprecated alias path names the parameter it copies, the one formulas
+    read. Where a path and its alias both appear (two scenarios spelling it
+    differently, say), their changes are merged in order, as two changes to
+    one path are: dated keys from both are kept, a later key for the same
+    period wins, and a value with no period replaces what came before.
+    """
+    canonical = {}
+    for path, value in changes.items():
+        name = canonicalize_lsr_parameter_path(path)
+        if name not in canonical:
+            canonical[name] = value
+        elif isinstance(canonical[name], dict) and isinstance(value, dict):
+            canonical[name] = {**canonical[name], **value}
+        else:
+            canonical[name] = value
+    return canonical
+
+
 def add_lsr_deprecation_aliases(parameters: ParameterNode) -> ParameterNode:
     simulation = parameters.gov.simulation
     canonical = simulation.labour_supply_responses

policyengine_uk/simulation.py

@@ -61,17 +61,19 @@
 )
 from policyengine_uk.reforms import create_structural_reforms_from_parameters
 from policyengine_uk.parameters.gov.simulation.labour_supply_responses.aliases import (
-    canonicalize_lsr_parameter_path,
+    canonicalize_parameter_changes,
 )
 
-from .tax_benefit_system import CountryTaxBenefitSystem, DEFAULT_DATASET_ENV_VAR
+from .tax_benefit_system import (
+    CountryTaxBenefitSystem,
+    DEFAULT_DATASET_ENV_VAR,
+    parameter_reads_calendar_years,
+)
 
 
 def _canonical_parameter_paths(changes: dict) -> dict:
     """Changes keyed by canonical parameter paths (labour supply aliases)."""
-    return {
-        canonicalize_lsr_parameter_path(path): value for path, value in changes.items()
-    }
+    return canonicalize_parameter_changes(changes)
 
 
 from microdf import MicroDataFrame
@@ -620,7 +622,10 @@ def apply_parameter_changes(self, changes: dict):
         read) and the parameters are processed as current law is.
         """
         self.apply_parameter_assignments(
-            read_parameter_changes(_canonical_parameter_paths(changes))
+            read_parameter_changes(
+                _canonical_parameter_paths(changes),
+                calendar_years=parameter_reads_calendar_years(),
+            )
         )
 
     def apply_scenario_parameters(self, scenario: Scenario) -> None:
@@ -635,7 +640,8 @@ def apply_scenario_parameters(self, scenario: Scenario) -> None:
         assignments = []
         if scenario.parameter_changes:
             assignments += read_parameter_changes(
-                _canonical_parameter_paths(scenario.parameter_changes)
+                _canonical_parameter_paths(scenario.parameter_changes),
+                calendar_years=parameter_reads_calendar_years(),
             )
         assignments += scenario.dated_assignments(order_start=len(assignments))
         self.apply_parameter_assignments(assignments)