You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
LA Band D target is silently dropped now, and will compute as a sum-vs-level mismatch once council_tax_band_d_amount exists #483
targets/sources/la_council_tax.py:182 emits a target whose variable is council_tax_band_d_amount. That variable does not currently exist in policyengine-uk, so the target is silently dropped. Once it exists, the target will activate and be computed incorrectly — as a sum where the target value is a level.
This is currently latent. It becomes a live calibration defect the moment policyengine-uk gains the variable and the pin here is bumped.
Current behaviour: silent drop
build_loss_matrix.py::_compute_column routes voa/council_tax/ and obr/council_tax, but has no route for the ons/council_tax_band_d/ prefix. So it falls through to _compute_simple_gbp, which does:
KeyError → None → the caller continues. The logger.debug("No compute logic...") branch never fires, so there is no diagnostic either. ~360 LA targets are being discarded with no signal.
The defect that activates on merge
_compute_simple_gbp's docstring is "Sum a variable at household level", and for a household-entity variable it returns ctx.pe(variable) — the weighted sum across households.
But the target value is a level: band_d_amount is a single LA's Band D council tax, on the order of £2,500.
So the loss matrix would compare a national weighted sum (order 1e10) against ~2.5e3. That will distort calibration rather than improve it.
Related architectural note
council_tax_band_d_amount is inherently an LA-level attribute, not a household-level quantity. Note that LA analysis in this repo works by reweighting the national sample per LA (local_authority_weights.h5, one weight vector per code in local_authorities_2021.csv) rather than by tagging households with an LA — so "the Band D amount for this household" is not well-defined outside a specific LA weighting context. Whatever fix is chosen should account for that.
Options
Drop the target. If Band D amount is an input to liability rather than something to calibrate towards, it may not belong in the loss matrix at all. Band counts (voa/council_tax/{code}/{band}) and the net requirement are the calibratable quantities.
Add a compute route for ons/council_tax_band_d/ returning a weighted mean rather than a sum.
Retarget onto total council tax revenue per LA, which is a sum and already available as total_council_tax_net in storage/la_council_tax.csv.
Option 1 or 3 look more principled than 2, but this is a judgement call for whoever owns the calibration design.
Why now
PolicyEngine/policyengine-uk#1855 adds a council_tax_band_d_amount variable. If that merges and the policyengine-uk pin here is bumped without addressing this, the defect goes live silently — it will not raise, it will just calibrate against a wrong comparison.
Microcosm has already adjudicated this exact question, and its verdict resolves the open design choice above.
packages/microcosm-build/src/microcosm/build/uk/uk_data_target_parity.json carries a concern covering policyengine_uk_data.targets.sources.la_council_tax:
{
"concern_id": "local_council_tax_band_d_rate",
"classification": "non_linear",
"status": "reviewed_exclusion",
"reason": "Band D currency amounts are per-rate, not additive household controls.",
"fence": {
"origin": "uk-data published Band D rates but did not wire them into the additive LA loss matrix.",
"purpose": "Avoid treating a rate as an additive target.",
"verdict_basis": "Keep excluded until a rate-aware non-linear objective exists."
}
}
So Microcosm reached the same conclusion independently — a Band D currency amount is a rate, not an additive household control — and deliberately declined to port it, with a signed fence. Its status vocabulary defines reviewed_exclusion as "Deliberately not ported after review, with a complete fence."
That makes option 1 (drop the target) the right resolution here, not option 2 (a mean-returning compute route). A weighted mean would make the number stop being wrong, but it would still be binding a rate into an additive objective, which is the thing Microcosm fenced against. Option 3 (retarget onto total_council_tax_net) is separately blocked on their side anyway — the local_council_tax_net concern sits at blocked_source, "The pinned feed has no comparable 2025 local-authority council-tax net series across the 361-area roster."
Two further things from that register worth carrying into any fix:
The band-count family is ported_local_declared and actively bound — local_council_tax_band_counts says the Microcosm council_tax/band_a..i contract rows "bind 2,511 authority cells on the taxbase basis with the 296 English band-H cells and Shetland's band-H cell signed deferred (microcosm#929)". So voa/council_tax/{code}/{band} is the council tax target surface that legitimately calibrates. That one is a count, and counts are additive.
This is a live constraint on Extend LA council tax to all eight bands (MHCLG Table 9, 2026-27) #482. That PR adds band_A_amount..band_H_amount columns. Those are amounts, i.e. rates — so they must not be wired into the loss matrix as additive targets. Extend LA council tax to all eight bands (MHCLG Table 9, 2026-27) #482 deliberately emits no targets from the new columns and flags that as a follow-up; per this register, the correct follow-up is "no additive target", not "add eight more". Worth stating explicitly so a later contributor doesn't helpfully wire them in.
Proposing to fix this by dropping the ons/council_tax_band_d/{code} emission from la_council_tax.py and leaving a comment recording why, so the rate stays available as data for liability computation without entering the objective.
/cc @juaristi22 — flagging since this sits on the Microcosm parity surface and the fence is yours; tell me if dropping rather than fencing-in-place is the wrong shape for your register.
Activity
vahid-ahmadi commentedon Sep 18, 2026
Microcosm has already adjudicated this exact question, and its verdict resolves the open design choice above.
packages/microcosm-build/src/microcosm/build/uk/uk_data_target_parity.jsoncarries a concern coveringpolicyengine_uk_data.targets.sources.la_council_tax:{ "concern_id": "local_council_tax_band_d_rate", "classification": "non_linear", "status": "reviewed_exclusion", "reason": "Band D currency amounts are per-rate, not additive household controls.", "fence": { "origin": "uk-data published Band D rates but did not wire them into the additive LA loss matrix.", "purpose": "Avoid treating a rate as an additive target.", "verdict_basis": "Keep excluded until a rate-aware non-linear objective exists." } }So Microcosm reached the same conclusion independently — a Band D currency amount is a rate, not an additive household control — and deliberately declined to port it, with a signed fence. Its status vocabulary defines
reviewed_exclusionas "Deliberately not ported after review, with a complete fence."That makes option 1 (drop the target) the right resolution here, not option 2 (a mean-returning compute route). A weighted mean would make the number stop being wrong, but it would still be binding a rate into an additive objective, which is the thing Microcosm fenced against. Option 3 (retarget onto
total_council_tax_net) is separately blocked on their side anyway — thelocal_council_tax_netconcern sits atblocked_source, "The pinned feed has no comparable 2025 local-authority council-tax net series across the 361-area roster."Two further things from that register worth carrying into any fix:
The band-count family is
ported_local_declaredand actively bound —local_council_tax_band_countssays the Microcosmcouncil_tax/band_a..icontract rows "bind 2,511 authority cells on the taxbase basis with the 296 English band-H cells and Shetland's band-H cell signed deferred (microcosm#929)". Sovoa/council_tax/{code}/{band}is the council tax target surface that legitimately calibrates. That one is a count, and counts are additive.This is a live constraint on Extend LA council tax to all eight bands (MHCLG Table 9, 2026-27) #482. That PR adds
band_A_amount..band_H_amountcolumns. Those are amounts, i.e. rates — so they must not be wired into the loss matrix as additive targets. Extend LA council tax to all eight bands (MHCLG Table 9, 2026-27) #482 deliberately emits no targets from the new columns and flags that as a follow-up; per this register, the correct follow-up is "no additive target", not "add eight more". Worth stating explicitly so a later contributor doesn't helpfully wire them in.Proposing to fix this by dropping the
ons/council_tax_band_d/{code}emission fromla_council_tax.pyand leaving a comment recording why, so the rate stays available as data for liability computation without entering the objective./cc @juaristi22 — flagging since this sits on the Microcosm parity surface and the fence is yours; tell me if dropping rather than fencing-in-place is the wrong shape for your register.
🤖 Generated with Claude Code
Drop ons/council_tax_band_d/{code} target emission (#483)