UK local surface: band domain, signed small-cell rules, renewals, support flag (#1278)

changelog.d/1278-communal-receipt-bands.changed.md

@@ -0,0 +1 @@
+`tools/uk_target_surface_receipt.py --communal` sizes the age-universe change planned on microcosm#1278. Beside the 18-24 student rule it now gives, for every local ten-year age band and 80 and over, each area's census communal-establishment share and how many areas reach 20% and 25%: the amount a private-household target would sit below the total-resident row the incumbent evaluation gates at 25%. It also composes the region-tier shares from local authorities and reads NISRA CT0105 for Northern Ireland's districts, marked approximate (CT0105 counts communal residents only, so its bands are spread evenly over single years against NISRA's mid-2025 district population). `--communal-only` writes that receipt without compiling the target surface. The tool's first tests use synthetic facts. An age label such as "Aged 2 years and under" now reads as 0-2.

changelog.d/1278-council-tax-band-domain-terminal.added.md

@@ -0,0 +1 @@
+A terminal, release-blocking gate, `uk_council_tax_band_domain_terminal`, checks each household's council-tax band against its nation's bands on the frame the release ships, with the spine gate's parameters (microcosm#1278 review). The spine gate stops at the transferred boundary, so a later step that writes the band, such as a full build's band redraw after geography derivation, was checked by nothing. The release-cut scope owns it.

changelog.d/1278-council-tax-band-domain.fixed.md

@@ -0,0 +1 @@
+The UK SPI housing shell now keeps every council-tax band inside the bands its nation uses (microcosm#1278). Its classifier conditions on region but had no nation mask, so SPI-channel households in England and Scotland could draw band I, and Northern Ireland households a band other than the FRS placeholder D. No target counts those rows, and a computed council tax would charge a band missing from a nation's ratio table at the Band D ratio. A drawn band outside the nation's bands is now drawn again from the classifier's probabilities over those bands, with its own identity-keyed uniform: every valid draw is unchanged, the redrawn rows follow the classifier conditioned on the nation, and the stage receipt counts the redraws by nation. The bands live in the new `uk_runtime.council_tax_constants`, which `frs_spine` and `local_targets` now import, and a new release-blocking spine gate, `uk_council_tax_band_domain`, checks every household at the transferred boundary. For it, the shared `enum_domain_gate` takes optional per-partition domains, and a declared domain must equal the shared constant.

changelog.d/1278-frs-ni-placeholder-band.fixed.md

@@ -0,0 +1 @@
+Northern Ireland FRS households take the placeholder council-tax band D whatever `ctband` holds (`frs_spine.frs_council_tax_band`), so the band no longer relies on the FRS leaving `ctband` unset there (microcosm#1278 review). Every Northern Ireland household on the licensed data already holds D, so no band moves.

changelog.d/1278-local-binding-adjudications-renewed.changed.md

@@ -0,0 +1 @@
+The six UK local binding adjudications in `uk/local_binding_adjudications.json` are renewed for three months, to 2027-01-14, approved by juaristi22 on 2026-10-14 (microcosm#1278). Each adjudication records the renewal and why the basis still holds. Four reasons are re-worded where the basis moved: the census household cells now reconcile to the nation household controls (microcosm#1198); the HMRC area surface also binds higher rate taxpayer counts, which HMRC classes by the marginal rate paid rather than by a total-income band; the age cells bind mid-2025 for local authorities and mid-2024 for constituencies, and the private-household entry now points to the age-universe change microcosm#1278 plans, which retires it; and the UC area series still count every claim, nil awards included, bridged to the paid caseload until PolicyEngine/chronicle#339 lands.

changelog.d/1278-parity-register-council-tax-coverage.fixed.md

@@ -0,0 +1 @@
+The UK data-target parity register now credits the 25 local council-tax contract rows (`mhclg`, `welshgov` and `scotgov` `council_tax_stock.by_area`) with covering uk-data's `la_council_tax` source; the coverage key still named the retired `voa.council_tax_stock.by_area.` prefix (microcosm#1278). Three stale texts are corrected: the English region concern names the councils' taxbase rows and the nation grain of the cross-grain rule, and the two private-rent concerns describe the pinned feed's calendar-2025 PIPR series rather than a June 2026 snapshot.

changelog.d/1278-parity-small-cell-rationale.fixed.md

@@ -0,0 +1 @@
+In the local compile-parity receipt, a cell the incumbent carries but a signed small-cell rule defers now carries its deferral's signed rationale, which names the rule, instead of the generic taxbase reason (microcosm#1278 review).

changelog.d/1278-small-cell-deferrals-narrowed-runs.fixed.md

@@ -0,0 +1 @@
+A UK small-cell deferral is now placed in the reconciliation frame of a run with its own ladder as an off-surface cell is (microcosm#1278). When the run's ladder cannot place the deferral's English area but the membership binds cells for that area, the run is a narrowed one (a sample or a test ladder), so the deferral is left out and named in the cross-grain receipt's `small_cell_deferrals.areas_outside_the_ladder`; an area the membership binds nothing for still refuses. While both small-cell rules were unsigned no cell took this path; signing them made every narrowed run refuse. `uk_small_cell_deferred_cells` carries `area_has_bound_cells`, and `tools/uk_target_surface_receipt.py` reports the block.

changelog.d/1278-small-cell-rules-signed.changed.md

@@ -0,0 +1 @@
+Both UK local small-cell rules in `uk/local_small_cell_rule.json` are signed, approved by juaristi22 on 2026-10-14 and expiring 2027-04-01 (microcosm#1278). The regenerated local references now defer the cells the rules name: the 16 council-tax band cells of fewer than 50 dwellings (band G 3, band H 7, band I 6) and the 192 higher rate taxpayer cells HMRC publishes at 5,000 or fewer, where its rounding to the thousand can reach 10% (155 constituencies, 37 local authorities). Each deferred cell still counts in its region's or nation's control through reconciliation. The local surface binds 24,269 references, down from 24,477; the local scorer's frozen count, the target-surface battery pin, the assembly runbook and the local compile-parity receipt move with it. New tests require every committed rule to be signed, pin the deferred cells, and fail 14 days before a rule lapses.

changelog.d/1278-support-below-floor.added.md

@@ -0,0 +1 @@
+UK local builds again flag the areas below the support floor (microcosm#1278). The graph path's area-support table (the candidate manifest's `area_support_summary`) carries `support_below_floor`, set under the floors the `uk_local_area_support` gate declares, and a reviewed exclusion does not clear it. The dense release manifest gains an `area_support` block that names the areas below the floor by grain, as codes only, with the floors; the data contract checks the block when present, and each listed area must carry its reviewed `area_support_exclusion` limitation. The support floors live in one shared helper that both the gate and the flag use. The two support-floor exclusions say where the flag is now recorded; it went with the old driver on 2026-09-25.

changelog.d/1278-support-floor-reasons-no-figures.changed.md

@@ -0,0 +1 @@
+The two UK local-authority support-floor exclusions (Isles of Scilly, City of London) and the generated deferral rationale for their own cells now say which part of the 50-row / ESS-50 / 50-source floor each area misses, without per-area support figures (microcosm#1278 review).

docs/evidence/uk-local-surface-1278/README.md

@@ -0,0 +1,16 @@
+# UK local surface, first #1278 PR: measurement scripts
+
+Evidence for microcosm#1278's first PR. The results are in [experiments/1278-uk-local-surface-receipts.md](../../../experiments/1278-uk-local-surface-receipts.md); this folder holds the scripts that produced the measured ones.
+
+## Scripts
+
+- `scripts/compare_twin_bands.py` (C1). Compares the council-tax bands of two spines built from the licensed inputs with the same recipe and seed, one on main and one on the branch. It aligns the base households (those that are not CGT copies) by lineage and reports, by nation, the households whose band lies outside the nation's bands, the bands that changed, and the columns that differ, with the households each moves on at all and by more than 0.1% and 1%. It writes counts only as bands (0, <10, 10-29, 30+) and no weighted figure.
+- `scripts/cgt_stage_residual.py` (C8). Reads one built spine at its design weights and compares the liable gainers in each of the CGT stage's own age and region groups with the HMRC margins the stage rakes onto, folded by the stage's own functions. A group below 30 distinct survey source households, or whose weighted gains fail either dominance rule (the largest source household holds at most half of the group's gains; the p% rule at p = 10), is pooled with its neighbour, so no group can be recovered by subtraction and no group's gains reveal one household's. Its output stays outside the repository for now.
+
+`tools/uk_target_surface_receipt.py --communal-only` produces C7's census sizing; it reads only public census facts from the pinned Chronicle feed.
+
+## What is not here
+
+- **Licensed inputs and unit records.** The spines, the survey files and the twin comparison's raw output stay outside the repository.
+- **C8's output.** The script was run on main's spine, and its output stays outside the repository for now. A later report should use that one spine: the same figures on the branch's spine would differ only through the households the fix moved, so the two could be subtracted.
+- **Weighted households by band before and after the fix.** The difference between the two spines' weighted band totals would isolate the households the fix moved, so neither spine's weighted band totals are reported.

docs/evidence/uk-local-surface-1278/scripts/cgt_stage_residual.py

@@ -0,0 +1,195 @@
+"""Stage-level CGT residual at design weights, by the stage's own groups (microcosm#1278, #725).
+
+Usage: cgt_stage_residual.py <spine.h5> <out.json>
+
+The CGT stage rakes liable gainers onto HMRC Table 6 (taxpayers and gains by age), folded onto
+seven age groups with 75 and over pooled, and onto Table 5 (taxpayers by region), folded onto four
+region groups. This script reads a built spine at its design weights and compares the liable
+gainers in each group (gains above the annual exempt amount) with those folded margins, through the
+stage's own folding functions. A reported group rests on 30 or more distinct survey source
+households (clones and copies count once), and a group that reports gains also passes two
+dominance rules on its weighted gains, counted by source household: the largest holds at most half
+of the group's gains, and what the group holds beyond its two largest is at least 10% of the
+largest (the p% rule at p = 10). A group that fails is pooled with its neighbour (the next older
+age group, or the next region group in the stage's order; the last group with the one before it)
+until every reported group passes, so no group can be recovered by subtracting the others from a
+total, and no group's gains reveal one household's.
+"""
+
+from __future__ import annotations
+
+import json
+import sys
+
+import numpy as np
+import pandas as pd
+
+from microcosm.build.uk_runtime.cgt_imputation import (
+    UK_CGT_AGE_GROUP_LOWER_BOUNDS,
+    UK_CGT_REGION_GROUP_LABELS,
+    _fold_age_gains_margin,
+    _fold_age_margin,
+    _fold_region_margin,
+    _person_conditioning_cells,
+    uk_cgt_policy_parameters,
+)
+from microcosm.build.uk_runtime.hmrc_capital_gains import (
+    load_hmrc_cgt_conditioning_facts,
+)
+
+MINIMUM_SOURCES = 30
+#: Dominance rules on a group's weighted gains, by source household.
+TOP_SOURCE_SHARE_MAX = 0.5
+P_PERCENT = 0.10
+
+
+def passes_dominance(contributions: pd.Series) -> bool:
+    """The largest source holds at most half, and the p% rule holds."""
+
+    ordered = contributions.sort_values(ascending=False).to_numpy(float)
+    total = float(ordered.sum())
+    if total <= 0.0:
+        return False
+    largest = ordered[0]
+    second = ordered[1] if len(ordered) > 1 else 0.0
+    remainder = total - largest - second
+    return largest <= TOP_SOURCE_SHARE_MAX * total and remainder >= P_PERCENT * largest
+
+
+def pooled_groups(
+    group_sources: list[set], group_contributions: list[pd.Series] | None = None
+) -> list[list[int]]:
+    """Adjacent groups pooled until each pool passes the floor and, with gains, dominance."""
+
+    def passes(pool: list[int]) -> bool:
+        sources = set().union(*(group_sources[index] for index in pool))
+        if len(sources) < MINIMUM_SOURCES:
+            return False
+        if group_contributions is None:
+            return True
+        joined = pd.concat([group_contributions[index] for index in pool])
+        return passes_dominance(joined.groupby(level=0).sum())
+
+    pools: list[list[int]] = []
+    current: list[int] = []
+    for index in range(len(group_sources)):
+        current.append(index)
+        if passes(current):
+            pools.append(current)
+            current = []
+    if current:
+        if not pools:
+            raise SystemExit("The liable gainers cannot be grouped to pass the rules.")
+        pools[-1].extend(current)
+    # Joining the tail can break the pool it joins; join backwards until it passes.
+    while not passes(pools[-1]):
+        if len(pools) == 1:
+            raise SystemExit("The liable gainers cannot be grouped to pass the rules.")
+        tail = pools.pop()
+        pools[-1].extend(tail)
+    return pools
+
+
+def age_labels() -> list[str]:
+    bounds = UK_CGT_AGE_GROUP_LOWER_BOUNDS
+    return [
+        f"{low}-{bounds[index + 1] - 1}" if index + 1 < len(bounds) else f"{low}+"
+        for index, low in enumerate(bounds)
+    ]
+
+
+def main(spine_path: str, out_path: str) -> None:
+    with pd.HDFStore(spine_path, mode="r") as store:
+        person = store["person"]
+        household = store["household"]
+    weight = (
+        person["person_household_id"]
+        .map(
+            pd.Series(
+                household["household_weight"].to_numpy(float),
+                index=household["household_id"],
+            )
+        )
+        .to_numpy(float)
+    )
+    source = (
+        person["person_household_id"]
+        .map(
+            pd.Series(
+                household["household_source_id"].to_numpy(),
+                index=household["household_id"],
+            )
+        )
+        .to_numpy()
+    )
+    _, age_group, _, region_group = _person_conditioning_cells(person, household)
+    exempt = uk_cgt_policy_parameters(2024).annual_exempt_amount
+    gains = person["capital_gains"].to_numpy(float)
+    liable = gains > exempt
+    conditioning = load_hmrc_cgt_conditioning_facts()
+    targets = {
+        "age_taxpayers": _fold_age_margin(conditioning),
+        "age_gains": _fold_age_gains_margin(conditioning),
+        "region_taxpayers": _fold_region_margin(conditioning),
+    }
+
+    def report(labels, groups, target_counts, target_gains=None):
+        masks = [liable & (groups == index) for index in range(len(labels))]
+        contributions = (
+            None
+            if target_gains is None
+            else [
+                pd.Series((weight * gains)[mask]).groupby(source[mask]).sum()
+                for mask in masks
+            ]
+        )
+        rows = []
+        sources = [set(pd.unique(source[mask])) for mask in masks]
+        for pool in pooled_groups(sources, contributions):
+            mask = np.logical_or.reduce([masks[index] for index in pool])
+            target_count = float(sum(target_counts[index] for index in pool))
+            achieved = float(weight[mask].sum())
+            row = {
+                "group": " + ".join(labels[index] for index in pool),
+                "target_taxpayers": round(target_count),
+                "achieved_taxpayers": round(achieved),
+                "taxpayers_residual": achieved / target_count - 1.0,
+            }
+            if target_gains is not None:
+                target = float(sum(target_gains[index] for index in pool))
+                achieved_gains = float((weight * gains)[mask].sum())
+                row["target_gains"] = round(target)
+                row["achieved_gains"] = round(achieved_gains)
+                row["gains_residual"] = achieved_gains / target - 1.0
+            rows.append(row)
+        return rows
+
+    out = {
+        "spine": spine_path.rsplit("/", 1)[-1],
+        "annual_exempt_amount": exempt,
+        "weights": "design (spine household_weight)",
+        "minimum_distinct_sources_per_group": MINIMUM_SOURCES,
+        "gains_dominance": {
+            "top_source_share_max": TOP_SOURCE_SHARE_MAX,
+            "p_percent": P_PERCENT,
+        },
+        "age_groups": report(
+            age_labels(),
+            age_group,
+            targets["age_taxpayers"],
+            targets["age_gains"],
+        ),
+        "region_groups": report(
+            list(UK_CGT_REGION_GROUP_LABELS),
+            region_group,
+            targets["region_taxpayers"],
+        ),
+    }
+    with open(out_path, "w", encoding="utf-8") as handle:
+        json.dump(out, handle, indent=1)
+        handle.write("\n")
+    print(json.dumps(out, indent=1))
+
+
+if __name__ == "__main__":
+    main(*sys.argv[1:3])