Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Concept mappings now refuse a `product` binding unless it reads exactly one usual rate, held as a float, and one annual flow counted in weeks, in either order. Before, `Product(liquid_financial_assets, weeks_worked)` validated, although the first is a stock; so did a product of usual weekly hours and a year's income. Concepts now also refuse an `annual_flow` temporal basis with any period but `year`. Every committed binding and concept already meets both rules, so no binding, coverage report, golden or pinned hash changes; policyengine-uk's `hours_worked`, usual weekly hours times weeks worked, still validates.
64 changes: 50 additions & 14 deletions docs/concept-schema-transport-adr.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,17 @@ schema version 2 adds `liquid_financial_assets`, the first stock concept, for
New Zealand's Accommodation Supplement cash-asset test. Only the Axiom New
Zealand mapping binds it, as a `Family` binding that `encode_groups` executes
on built benefit units; the other three mappings list it as unmapped with a
reason. A scale, sum, scaled sum, share or fraction now computes from annual
flows only (invariant 11), so none of them can rescale the stock, add it to a
year's income, or split it.
reason. A scale, sum, scaled sum, signed part, share or fraction now computes
from annual flows only (invariant 11), so none of them can rescale the stock,
add it to a year's income, or split it.
**Amended 2026-10-10:** schema version 3 adds original
`household.source_household_id` lineage for donor support diagnostics. The
reader scopes explicit raw household identifiers by source year and carries
the same key across support clones and their benefit units. Rules mappings
leave this provenance concept unmapped.
**Amended 2026-11-11:** a product multiplies one usual rate by one annual flow
counted in weeks, and every annual flow has period `year` (invariant 11 states
both).

## Context

Expand Down Expand Up @@ -75,7 +78,8 @@ Each concept declares:
person id, source id or unit interval.
- **Period semantics.** The kernel's `year`, `month` or `point`, plus what the
value measures over it: an annual flow, a state at the reference date, a usual
rate over the weeks worked, or persistent state.
rate over the weeks worked, or persistent state. An annual flow's period is
`year`.
- **Currency and price level.** Every amount is nominal, in whole units of the
frame's single declared currency (`ConceptFrameDeclaration.currency`, ISO
4217), at the prices of its own period. Each amount names the index family
Expand Down Expand Up @@ -152,7 +156,9 @@ binds engine inputs to concepts. Each binding states:
positivity test, sum, a sum then a fixed scale, product, allocation to the
reference person, a predicate, a relationship role, a co-resident child
count, a take-up threshold, or a test of a built unit's composition. A scale,
sum, scaled sum, share or fraction computes from annual flows only;
sum, scaled sum, signed part, share or fraction computes from annual flows
only, and a product multiplies one usual rate by one annual flow counted in
weeks;
- the relation, in the same vocabulary;
- the evidence, quoting the engine's own definition or how builds populate the
input.
Expand Down Expand Up @@ -318,9 +324,10 @@ These hold for every input and are tested (Hypothesis properties unless noted):
another exception, whatever is corrupted; a missing, unexpected or
wrongly typed field is named. So do unit rules, state bindings and input
closures.
11. **Arithmetic reads annual flows.** A scale, sum, scaled sum, share or
fraction binding computes only from concepts whose declared temporal basis
is `annual_flow`: every concept in its `concepts`. A stock such as
11. **Arithmetic reads annual flows, and a product multiplies a usual rate by
weeks.** A scale, sum, scaled sum, signed part, share or fraction binding
computes only from concepts whose declared temporal basis is
`annual_flow`: every concept in its `concepts`. A stock such as
`liquid_financial_assets` has no weekly value and cannot be added to a
year's income. The rule also refuses a scale of a usual rate or a
persistent draw. The household reference person that a binding allocated
Expand All @@ -333,12 +340,41 @@ These hold for every input and are tested (Hypothesis properties unless noted):
were an annual flow. Two checks are exhaustive rather than sampled.
Redeclaring any operand of a committed arithmetic binding under another
basis refuses that binding, while redeclaring its placement pointer
leaves it valid. Redeclaring anything any other committed binding reads
leaves that binding valid. Every committed mapping validates. Other
transforms are outside the rule. The New Zealand binding reads the stock
through an identity whose group rule sums it over a family's members, and
policyengine-uk's hours input is a product of usual weekly hours and
weeks worked.
leaves it valid. Redeclaring anything any other committed binding (other
than a product) reads, under any basis, leaves that binding valid. Every
committed mapping validates.

A product has a rule of its own. It reads exactly two concepts, in either
order: one usual rate held as a float, and one annual flow counted in weeks
(unit `weeks`, an int or a float). A usual rate holds over the reference
year's weeks of activity, so only a number of weeks multiplies it to a
year's total. The one committed product is policyengine-uk's hours input,
usual weekly hours times weeks worked. The rule refuses a product of any
number of concepts but two, and one that reads a concept of any other basis
(a stock or a persistent draw, for example), two usual rates, two annual
flows, a usual rate that is not a float, or an annual flow that is not a
number of weeks, such as a year's income. The rule leaves the usual rate's
unit unchecked. A refusal of two concepts names only operands at fault. A Hypothesis property
swaps either operand of the committed product for any other concept and redeclares
its basis, and, for a numeric quantity that is neither an amount nor a
pointer, its dtype and unit. In either order, the binding validates exactly
when this rule says it should. Each way to break the rule is also refused by
name. Three checks are exhaustive: of all ordered pairs of committed
concepts, only usual weekly hours and weeks worked multiply; redeclaring
either operand of the committed product under any other basis refuses it;
and in policyengine-uk's JSON form, putting any other person concept in
either operand's place is refused.

Every annual flow has period `year`. A concept declared an annual flow
with another period is refused at construction. The tie runs one way:
other bases take any period, and usual weekly hours, a usual rate, has
period `year`. Every committed annual flow already had period `year`, so
the schema digest does not move. The test is exhaustive over every
committed concept, basis and period.

Other transforms are outside both rules. The New Zealand binding reads
the stock through an identity whose group rule sums it over a family's
members.
12. **Units.** Benefit units partition persons: every person is in exactly
one unit, every unit has a head and at most one partner, partners share a
unit and the partner role goes to the head's partner alone, every
Expand Down
61 changes: 52 additions & 9 deletions packages/microcosm-frame/src/microcosm/frame/concept_mapping.py
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,13 @@ def __post_init__(self) -> None:

@dataclass(frozen=True)
class Product:
"""The engine input is the product of two concepts (hours x weeks)."""
"""A usual rate times the weeks it held over, in either order.

Reads exactly two concepts: one usual rate, a float, and one annual flow
counted in weeks. Usual weekly hours times weeks worked is the year's
hours. A usual rate holds over the reference year's weeks of activity, so
nothing else multiplies it to a year's total.
"""


@dataclass(frozen=True)
Expand Down Expand Up @@ -924,17 +930,21 @@ def bind(


#: Transforms that do arithmetic on a year's amount: a scale restates it per
#: week or month, a sum adds amounts (and a scaled sum scales the total), and
#: a share or fraction splits one. Each reads annual flows only, so none of
#: them takes a stock (a value at the reference date), a usual rate or a
#: persistent draw by accident; a binding that needs one needs a new transform.
#: week or month, a sum adds amounts (a scaled sum scales the total, and a
#: signed part keeps one sign of it), and a share or fraction splits one. Each
#: reads annual flows only, so none of them takes a stock (a value at the
#: reference date), a usual rate or a persistent draw by accident; a binding
#: that needs one needs a new transform.
#: The rule checks the concepts a
#: binding computes from (``concepts``), not the household reference person a
#: binding allocated to the reference unit also reads to place its value. A
#: product is outside the rule: it multiplies a usual rate (weekly hours) by
#: weeks.
#: product has its own rule (:data:`_PRODUCT_BASES`).
_FLOW_TRANSFORMS = (Scale, Sum, ScaledSum, SignedPart, Share, Fraction)

#: A product multiplies one usual rate by one annual flow (the weeks the rate
#: held over), so it reads exactly these two bases, once each.
_PRODUCT_BASES = (TemporalBasis.USUAL_RATE, TemporalBasis.ANNUAL_FLOW)


def _check_transform_arity(binding: InputBinding) -> None:
transform = binding.transform
Expand Down Expand Up @@ -999,8 +1009,8 @@ def _check_transform_arity(binding: InputBinding) -> None:
f"reads annual flows only, and {item.id} has temporal "
f"basis {item.temporal_basis.value}."
)
if isinstance(transform, Product) and len(items) != 2:
raise ValueError(f"Binding for {name!r}: a product reads two concepts.")
if isinstance(transform, Product):
_check_product(name, items)
if isinstance(transform, AllocateToReferencePerson) and items[0].entity != (
"household"
):
Expand Down Expand Up @@ -1057,6 +1067,39 @@ def _check_transform_arity(binding: InputBinding) -> None:
raise ValueError(f"Binding for {name!r}: take-up names its program.")


def _check_product(name: str, items: list[Concept]) -> None:
if len(items) != 2:
raise ValueError(f"Binding for {name!r}: a product reads two concepts.")
for item in items:
if item.temporal_basis not in _PRODUCT_BASES:
raise ValueError(
f"Binding for {name!r}: a product reads a usual rate and an "
f"annual flow, and {item.id} has temporal basis "
f"{item.temporal_basis.value}."
)
first, second = items
if first.temporal_basis is second.temporal_basis:
raise ValueError(
f"Binding for {name!r}: a product reads one usual rate and one "
f"annual flow, and {first.id} and {second.id} both have temporal "
f"basis {first.temporal_basis.value}."
)
rate, weeks = sorted(
items, key=lambda item: _PRODUCT_BASES.index(item.temporal_basis)
)
if rate.dtype != "float":
raise ValueError(
f"Binding for {name!r}: a product's usual rate is a float, and "
f"{rate.id} has dtype {rate.dtype}."
)
if weeks.unit is not Unit.WEEKS or weeks.dtype not in ("int", "float"):
raise ValueError(
f"Binding for {name!r}: a product multiplies a usual rate by a "
f"number of weeks, and {weeks.id} has unit {weeks.unit.value} and "
f"dtype {weeks.dtype}."
)


def _check_group_rule(engine: str, binding: InputBinding) -> None:
rule = binding.group_rule
if isinstance(binding.transform, UnitComposition):
Expand Down
11 changes: 9 additions & 2 deletions packages/microcosm-frame/src/microcosm/frame/concepts.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,8 @@ class TemporalBasis(StrEnum):
``annual_flow`` matches ``microcosm.build.monetary_targets``.
"""

#: An amount or count accumulated over the reference year.
#: An amount or count accumulated over the reference year. Its period is
#: ``year``.
ANNUAL_FLOW = "annual_flow"
#: The record's state at the survey reference date.
REFERENCE_STATE = "reference_state"
Expand Down Expand Up @@ -524,7 +525,8 @@ class Concept:
dtype: Kernel dtype kind: ``float``, ``int``, ``bool`` or ``str``.
unit: The unit values are expressed in.
period: Kernel period semantics: ``year``, ``month`` or ``point``.
temporal_basis: What the value measures over its period.
temporal_basis: What the value measures over its period. An annual
flow's period is ``year``.
provenance: How values come to exist.
transport: What transport does with the column.
monetary: Currency and price-level handling; set exactly when the
Expand Down Expand Up @@ -582,6 +584,11 @@ def __post_init__(self) -> None:
raise ValueError(
f"Concept {self.id!r} period must be one of {_PERIOD_SEMANTICS}."
)
if self.temporal_basis is TemporalBasis.ANNUAL_FLOW and self.period != "year":
raise ValueError(
f"Concept {self.id!r}: an annual flow accumulates over the year, "
f"so its period is year, not {self.period}."
)
self._validate_unit_contract()
self._validate_bounds()
self._validate_transport()
Expand Down
Loading
Loading