Repository navigation
Build benefit units from concept pointers and execute group encoding (NZ v0, G4) - #1122
Conversation
…(NZ G4)
Unit construction (microcosm.frame.unit_construction):
- build_benefit_units(person, household, rule) -> (family table, person_family_id).
A unit is an adult, their co-resident partner and their dependent children;
every other adult is their own unit (method card MC7). Who counts as a
dependent child is a declared BenefitUnitRule (format
microcosm.benefit_unit_rule.v1): maximum age, an optional employment-based
financial-independence test, where a child with no co-resident parent goes
(reference person's unit, own unit, or refuse) and what happens to a child
whose two co-resident parents are in different units (refuse, or parent 1).
rulespec-nz 8dc2507 encodes no dependent-child definition, so the rule is
spec data (amendment MC11a). Invalid pointers are refused through
validate_concept_tables. Unit ids are the head's person id, the table is
sorted by id (a Frame group table requirement), and units carry no weight.
- benefit_unit_roles, benefit_unit_attributes, benefit_unit_membership and
units_per_household for the unit-attributes node and group encoding.
Group encoding (ConceptMapping.encode_groups):
- Executes sum_over_members, any_member, reference_member, household_value
and allocate_to_reference_unit on a checked GroupMembership; person_role
bindings and entities with no membership stay deferred. encode() is
unchanged and still defers every group binding.
- Executes declared StateBindings: group inputs fed from model state (receipt
flags, an assigned area), not concepts.
- GroupKnobs: allocation reference_unit | per_adult_share (MC8), zeroed
inputs (asset test off, MC10), state-column substitution (area column, MC9)
and factors (rent factor). A knob naming an input the call does not produce
or cannot change is refused.
- New transforms: scaled_sum (a sum, then a fixed scale) and
unit_composition (has_partner, has_dependent_child,
two_or_more_dependent_children, sole_parent on the built unit).
Input closure (microcosm.frame.input_closure, build/nz/axiom_input_closure.json):
- Every root input of the four modules the NZ v0 graph binds is closed exactly
once against the committed input surface (entitlement 60, rates 28, NZS core
13, AS core 39): 41 concept bindings (one awaiting G3a), 4 state bindings (area flags,
non-beneficiary), 2 graph-node inputs (the reg 17/18 bridge values, not
computed here), 93 defaults with reasons and citations, all listed for D1.
No input is classed engine-optional: axiom-rules-engine 04315d94 has no
optional inputs (an absent input that a row reaches fails the request). The
closure is for transport frames, so a binding that reads a concept
transport drops cannot encode an input there.
- Undetermined judgments: takeup.assign@1 refuses code 0 (none can arise at
04315d94 with every input fed); resolve_judgments implements all three
actions. Receipt-layer requirements (one main benefit per person, never NZS
with a main benefit, a positive income-tested net payment) are declared.
- The cash-asset entry awaits G3a's liquid-asset binding; delete its
"awaiting" note when G3a lands (the closure check reports it until then).
axiom:nz mapping: Accommodation Supplement has_dependent_children (now from
the built unit), has_two_or_more_dependent_children, sole_parent,
owner_weekly_required_payments ((interest + principal) / 52 on the reference
family), owns_premises_and_not_joint_owner_with_resident and
relevant_weekly_income; main-benefit rates income inputs. Coverage golden and
docs/concept-coverage/axiom-nz.md regenerated (88 inputs fed by a concept).
Invariants (Hypothesis property tests):
- partition: every person is in exactly one unit; every unit has members
- every unit has one head (an adult) and at most one partner
- partners share a unit
- every dependent child shares a unit with a co-resident parent, or with the
household reference person when it has none
- units nest in households
- sum of family weights = sum over households of household weight x units in
the household (family weights inherited through Frame.resolve_weights)
- unit ids are deterministic and independent of row order
- household_value is constant within a household
- allocate_to_reference_unit puts each household amount on exactly the unit
holding the reference person and conserves it; per_adult_share conserves it
- sum_over_members conserves member totals
- the ADR round-trip, idempotence and conservation properties still hold for
every mapping (existing suite, unchanged pins)
- the closure partitions each bound module's surface exactly
Differential tests: encode_groups vs a row-by-row reference with independent
roles; unit_composition vs benefit_unit_attributes; golden-08 encoded inputs vs
the oracle harness's 28 AS inputs (engine-free); golden-08 through the real AS
module vs the oracle's weekly amount within $0.01 (engine-present, skipped
without axiom_rules_engine and POPULACE_RULESPEC_NZ).
Depends on G2's build/nz/country_package.json: add the row
{"path": "axiom_input_closure.json", "kind": "legacy_json", "schema_id":
"legacy_json"}. Until then test__given_country_like_directories__then_each_has_a_manifest
fails (build/nz/ has no manifest on main).
axiom: n/a: microsim-side unit construction, group encoding and input closure;
no policy is encoded in a country model.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ing (G4) Applies the independent review of #1122 (APPROVE_WITH_NITS at 11521ae). F1. benefit_unit_roles marked "no partner" with -1 and compared that marker with person ids, so a person whose id is -1 was read as the partner of a head who has none (a sole parent and child -1 came out as a couple with no children). Partner presence is now a mask and the head is found by row position, so no id can pass for a missing partner. The concept-frame contract accepts any unique integer id. - Regression: TestPlacement.test_a_person_id_of_minus_one_is_a_person_not_a_missing_partner (and negative ids in every role). - concept_frames takes min_id; when it is negative it also mixes in ids near zero, since a draw from +/-10**9 never hit -1 in 300 examples. The unit-construction, group-encode and NZ closure properties now draw negative ids (default draws for every other caller are unchanged). - New unit-construction invariant: the partner role is held by the head's partner and nobody else. On the old code it and eight group-encode properties fail. F2. The membership check that a partner is its head's partner had no test of its own (mutant M13, `if False:`, survived). New refusal case: a child in a sole parent's unit relabelled "partner". The swapped-roles case now matches that check's message rather than any message containing "partner". F3. The new axiom:nz mapping content is pinned against hand-computed households (a couple with two children, wages and a mortgage plus an adult boarder; a renting sole parent with one child; every income concept non-zero somewhere), with weekly values worked by hand from annual amounts, plus a check that every NZ Scale and ScaledSum reads annual-flow concepts, says 1/52 in its note and divides by 52. Mutants M28, M29, M30, M31 and M34 pass the PR's original tests and fail these. Twelve sibling mutants (feature swaps, other factors, group rules, dropped concepts) also fail the new tests: ten on their assertions, two already at mapping load. F4. The changelog said the closure closes all 140 root inputs; it closes 139, and the cash-asset input awaits G3a. F6. The closure test cited build/nz benefit_unit_rule.json, which G2 (#1119) adds; the comment now says so. The ADR's units invariant now states the partner-role rule and that it holds for negative ids. Not changed here: F5 (two dependent-child definitions in axiom:nz, a D1 methodology question) and F7 (the engine-present golden-08 run, which needs an axiom_rules_engine build). axiom: n/a: microsim-side unit construction and tests; no policy is encoded in a country model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts: - docs/concept-schema-transport-adr.md: kept G4's "three pieces beyond the schema and benefit-unit construction" and main's Belgian count (46 of its 105 module bindings stay deferred). - docs/concept-coverage/axiom-nz.md and the axiom-nz coverage golden: regenerated with tools/refresh_concept_coverage.py --engine axiom-nz on the merged tree (--check exits 0). Fed by a concept 88 -> 89: G3a's fact:person.liquid_financial_assets now feeds accommodation_supplement_cash_assets. Also lists build/nz/axiom_input_closure.json in G2's build/nz/country_package.json as a legacy_json resource. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merging main (G2 #1119, G3a #1120) into G4 left seven engine-free tests red in four files. 1. test_nz_spec_package.py::TestRulespecPin (two tests). G2's rulespec commit scanner read the closure's axiom-rules-engine commit (04315d94) as an unreviewed rulespec commit: input_surface held it under engine_commit in a mapping with no repository, and engine_optional_evidence put the hex in prose that cites the engine's src/rulespec.rs. The closure now records the engine as a foreign record, input_surface.engine = {repository: TheAxiomFoundation/axiom-rules-engine, commit}, the shape the committed surface fixture and axiom_rules_bindings.json already use. The evidence, and for consistency the undetermined-judgment reason (which the scanner did not flag), name "the axiom-rules-engine commit named in input_surface.engine" instead of repeating the hex. The scanner and the reviewed-commit set are unchanged. InputClosure gains surface_engine_repository; surface_engine_commit is kept, and from_dict refuses the old flat engine_commit shape. - New: a rulespec commit planted in the closure is still caught (beside the engine record, inside it under a rulespec key, in an entry reason, with the record relabelled as rulespec, and the engine hex back in the evidence prose). - New: the engine record's commit is pinned to the surface's engine commit by InputClosure.check, so the foreign record cannot carry a rulespec commit unnoticed. 2. test_nz_axiom_input_closure.py (three tests). G3a bound accommodation_supplement_cash_assets (fact:person.liquid_financial_assets, summed over the family), so its closure entry drops "awaiting" and is an ordinary concept-encoded input; the closure closes all 140 root inputs (60 + 28 + 13 + 39). The test that allowed only the cash-asset entry to await now requires that none does and that the entry is the mapping's summed liquid-asset binding. A new property checks that group encoding gives each family the sum of its members' liquid assets and that scenario S1 (MC10) zeroes it. 3. test_country_spec.py golden [nz] and test_spec_engine_country_bundles.py::test_nz_generation_zero_views_come_from_the_country_spec_seam. The merge listed axiom_input_closure.json in build/nz/country_package.json (the loader refuses undeclared files) without updating G2's pins. The nz golden is regenerated from the loaded spec after the last closure edit: the diff is only the new resource and its hash, country_package.json's hash and the fingerprint. The legacy_json resource set now includes the closure. Stale claims: the changelog fragment said the closure closes 139 of 140 inputs with the 140th awaiting G3a; it now says all 140. The axiom:nz cash-asset note said "no code applies it until a future unit-construction step builds Family units"; it now says encode_groups applies it on the units the unit-construction step builds, keeping the phrase G3a's note test pins. A closure-test comment said G2 "adds" benefit_unit_rule.json; G2 merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve the concept-mapping and transport ADR conflicts while retaining benefit-unit construction, group encoding, and the annual-flow transform rule. Apply the rule to ScaledSum and extend the existing arithmetic regressions. Regenerate and verify the axiom-nz coverage and NZ country-spec artifacts. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
#1122 adds an NZ resource whose format id is microcosm.axiom_input_closure.v1, which the dotted-symbol scan read as a Python symbol and failed to import. A dotted name now counts as a symbol citation only when its second component is a shard in this checkout (build, calibrate, data, diagnostics, fit, frame, graph). Every dotted name the packages carry today is under a shard, so the scanned sets are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The closure cites axiom-rules-engine source lines (src/rulespec.rs, src/dense.rs, src/formula.rs, python-ext/src/lib.rs) in engine_optional_evidence. A new top-level engine_source_pins records each cited file with the repository and commit of input_surface.engine, so every line citation names one file version. This is what the cross-country citation guard in microcosm#1146 accepts (a mapping with path and commit for the file). Putting the sha inline in the evidence string instead would trip G2's rulespec-commit scanner, which reads hex runs in any string that mentions RuleSpec; the pins are foreign records (repository is axiom-rules-engine), which that scanner skips by design. InputClosure.from_dict allows unknown top-level fields, so the loader is unchanged. A new test requires every cited .rs file to be pinned at the surface engine's repository and commit, and every pin to be cited; it fails without the pins. The NZ country-spec golden is regenerated (closure hash and fingerprint only). Tests: test_nz_axiom_input_closure.py 17 passed 1 skipped, test_nz_spec_package.py 71, test_spec_only_country_packages.py 8, test_country_spec.py 135. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1716120 to
8cdc884
Compare
Benefit-unit construction converted every id and pointer to int64 with a
bare cast. The concept-frame contract accepts unsigned ids, so a household
id of 2**63 became -2**63: its unit nested in no household and the household
counted no unit. The validator matches pointers as int64 too, so a partner
pointer of 2**64 - 1 passed as person -1. build_benefit_units now refuses,
before validation, any id or pointer column holding an unsigned id above
2**63 - 1, naming the columns. Every other id conversion in the module
(roles, attributes, membership) goes through one exact helper that refuses
such ids and non-integer ids. Ids that fit are kept exactly, and the unit
outputs stay int64.
encode_groups compares membership ids as int64 too. A uint64 membership
naming unit 2**64 - 1 was read as unit -1 and accepted, and so was a person
renamed 2**64 - 1 under a head column of -1. It now refuses unsigned ids
above 2**63 - 1 in the membership columns and the frame's person id
columns. The bound check is shared with unit construction.
encode_groups iterated state_bindings twice: once to encode and again to
find the columns a knob may replace. A one-shot iterator was used up by the
first pass, so a valid area knob was refused ("Knobs replace state columns
no binding reads"). The bindings are now materialized once on entry.
Tests: boundary regressions at 2**63 - 1 (kept exactly: nesting, identity
and unit counts), at 2**63 and at 2**64 - 1 (refused), plus the wrapped
partner pointer, the wrapped membership and the renamed person. Hypothesis
properties: ids of every accepted integer dtype, signed and unsigned mixed,
are kept exactly or refused, and build the same units, roles and attributes
as the same ids typed int64. Membership ids retyped as uint64, UInt64 or
Int64 encode exactly as the int64 ones, or are refused. State bindings given
as a list, iterator or generator encode, or are refused, exactly as their
tuple.
Follow-ups from the independent verification of this fix (APPROVE_WITH_NITS):
- the membership check also refuses a partner pointer int64 cannot hold
(person.partner_person_id of 2**64 - 1 was read as person -1 and passed as
that head's partner); a regression covers it and the int64 dangling-pointer
control;
- the closure's notes say takeup.assign@1 is declared but not yet implemented
(three sentences read as if it ran); the NZ country-spec golden is
regenerated for that text.
Still open, outside this PR: concepts._check_pointers.positions and
_Context.person_rows on main cast uint64 pointers to int64 the same way.
Final targeted runs: test_unit_construction.py 38, test_group_encode.py 78,
test_input_closure.py 52, test_concept_mapping.py 168,
test_nz_axiom_input_closure.py 17 passed 1 skipped, test_nz_spec_package.py 71,
test_country_spec.py 135, test_spec_engine_country_bundles.py 21,
test_spec_only_country_packages.py 8.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Heads-up from #1158, which fixes the unsigned→int64 wrap at its source in #1158 makes Whichever PR lands second needs this change. With it, - # The concept-frame contract accepts this frame. Cast to int64, 2**63
- # became household -2**63: the unit nested in no household and its
- # household counted no unit.
+ # Cast to int64, 2**63 became household -2**63: the unit nested in no
+ # household and its household counted no unit. Validation refuses the
+ # same columns (microcosm#1158).
person, household = self._one_adult(household_id)
- assert not validate_concept_tables({"person": person, "household": household})
columns = "['person.person_household_id', 'household.household_id']"
+ with pytest.raises(ValueError, match=re.escape(columns)):
+ validate_concept_tables({"person": person, "household": household})
with pytest.raises(ValueError, match=re.escape(columns)):
build_benefit_units(person, household, self.RULE)#1158 also adds |
Two conflicts: - docs/concept-schema-transport-adr.md: keep both sides. Main's status text (#1130) describes the transport reader; G4 adds that benefit-unit construction from the concept pointers is built for New Zealand. Main's closing bullet listed 13 of New Zealand's 80 module bindings as deferred until unit construction exists; G4 builds it, so the bullet now says New Zealand's group bindings execute on built units, Belgium's and the US/UK group bindings stay deferred, and a country pack is still needed. - packages/microcosm-build/tests/golden/nz_country_spec.json: regenerated from the loaded spec (canonical_json_bytes(_loaded_spec_summary("nz"))), not hand-merged. Resource hashes are the union of both sides (G4's axiom_input_closure.json and country_package.json, #1138's gates.json); fingerprint 09ed486f... Tests (one file at a time, after `uv sync --all-packages --locked --inexact` for main's new sqlalchemy dependency): country spec 135, NZ input closure 17 (1 skipped), NZ spec package 71, transport gate bindings 110, group encode 78, unit construction 38, input closure 52, concept mapping 168; all passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merged by the NZ hub (session local_e407a527) at 3f66897 with --merge. Gates: |
After merging main (with #1122): - test_unit_construction: TestIdRange's unsigned-household case asserted validate_concept_tables accepts a uint64 household id of 2**63, which this PR refuses; it now expects validation to refuse the same columns (the 4-line change agreed with the NZ hub). - _INT64_MAX / _exceeds_int64 were defined twice with the same bodies, once in concepts (this PR) and once in concept_mapping (#1122); concept_mapping and unit_construction now import them from concepts. - _Units' wide-id list dropped its person.* entries: _Units is only built from a _Context, which has already refused them, so they could not fire. - _Units matched unit households against the household ids in their native dtype, so under uint8 household ids a family nesting in household 261 was placed in household 5 (UInt8 raised a TypeError). It now matches through _matched_ids, as _Context does; a regression covers uint8, UInt8 and the int64 control. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unit construction (microcosm.frame.unit_construction):
A unit is an adult, their co-resident partner and their dependent children;
every other adult is their own unit (method card MC7). Who counts as a
dependent child is a declared BenefitUnitRule (format
microcosm.benefit_unit_rule.v1): maximum age, an optional employment-based
financial-independence test, where a child with no co-resident parent goes
(reference person's unit, own unit, or refuse) and what happens to a child
whose two co-resident parents are in different units (refuse, or parent 1).
rulespec-nz 8dc2507 encodes no dependent-child definition, so the rule is
spec data (amendment MC11a). Invalid pointers are refused through
validate_concept_tables. Unit ids are the head's person id, the table is
sorted by id (a Frame group table requirement), and units carry no weight.
units_per_household for the unit-attributes node and group encoding.
Group encoding (ConceptMapping.encode_groups):
and allocate_to_reference_unit on a checked GroupMembership; person_role
bindings and entities with no membership stay deferred. encode() is
unchanged and still defers every group binding.
flags, an assigned area), not concepts.
inputs (asset test off, MC10), state-column substitution (area column, MC9)
and factors (rent factor). A knob naming an input the call does not produce
or cannot change is refused.
unit_composition (has_partner, has_dependent_child,
two_or_more_dependent_children, sole_parent on the built unit).
Input closure (microcosm.frame.input_closure, build/nz/axiom_input_closure.json):
once against the committed input surface (entitlement 60, rates 28, NZS core
13, AS core 39): 41 concept bindings (the Accommodation Supplement cash-asset input among them, through the liquid-asset binding; no entry awaits a binding), 4 state bindings (area flags,
non-beneficiary), 2 graph-node inputs (the reg 17/18 bridge values, not
computed here), 93 defaults with reasons and citations, all listed for D1.
No input is classed engine-optional: axiom-rules-engine 04315d94 has no
optional inputs (an absent input that a row reaches fails the request). The
closure is for transport frames, so a binding that reads a concept
transport drops cannot encode an input there.
resolve_judgmentsis a helper implementing the three actions (refuse, not eligible, eligible); no production code calls it yet. The receipt-layer requirements (one main benefit per person, never NZS with a main benefit, a positive income-tested net payment) are declared.axiom:nz mapping: Accommodation Supplement has_dependent_children (now from
the built unit), has_two_or_more_dependent_children, sole_parent,
owner_weekly_required_payments ((interest + principal) / 52 on the reference
family), owns_premises_and_not_joint_owner_with_resident and
relevant_weekly_income; main-benefit rates income inputs. Coverage golden and
docs/concept-coverage/axiom-nz.md regenerated (89 inputs fed by a concept).
Invariants (Hypothesis property tests):
household reference person when it has none
the household (family weights inherited through Frame.resolve_weights)
holding the reference person and conserves it; per_adult_share conserves it
every mapping (existing suite, unchanged pins)
Differential tests: encode_groups vs a row-by-row reference with independent
roles; unit_composition vs benefit_unit_attributes; golden-08 encoded inputs vs
the oracle harness's 28 AS inputs (engine-free); golden-08 through the real AS
module vs the oracle's weekly amount within $0.01 (engine-present, skipped
without axiom_rules_engine and POPULACE_RULESPEC_NZ).
Depends on G2's build/nz/country_package.json: add the row
{"path": "axiom_input_closure.json", "kind": "legacy_json", "schema_id":
"legacy_json"}. Until then test__given_country_like_directories__then_each_has_a_manifest
fails (build/nz/ has no manifest on main).
axiom: n/a: microsim-side unit construction, group encoding and input closure;
no policy is encoded in a country model.
Invariants, acceptance and tests (from the build lane report)
See commit message.
Shared files and owners
test__given_country_like_directories__then_each_has_a_manifestneeded G2'sbuild/nz/country_package.jsonto listaxiom_input_closure.json; the hub merged main and added that row.concept_mapping.pyandconcepts.py(frame concepts; Max's sessions),test_support/microcosm_frame/*anddocs/agent-guide.md. No adapter or kernel hash moves;frame/kernels.pyis untouched.Part of the Microcosm NZ v0 graph plan (Max ruling d962). Built by the NZ hub on an Opus build lane; the hub verified scope (no frozen graph files, uk_runtime/us_runtime or frame/kernels.py touched) before committing.
Review history and fixes after the first review (head a545fd7, before the merge of main at 3f66897)
build/nz/country_package.json), then 581d52c (Refuse scale, sum, share and fraction on concepts that are not annual flows #1128's annual-flow rule; both sides' behaviour kept). f1024b0: the closure records the axiom-rules-engine commit as a foreign record (input_surface.engine), so G2's rulespec-commit scanner is unchanged, and the cash-asset input is bound, not awaited (140 of 140 root inputs closed).engine_source_pinspins each engine source file the closure cites by line (repository, commit, path), which the cross-country citation guard in Cite US and UK spec sources by symbol or at a commit, never by a drifting line #1146 accepts.build_benefit_unitsand the unit readers now refuse such ids before any conversion and keep every id that fits exactly;encode_groups' membership check refuses them too, including a wrapped partner pointer.encode_groupsiteratedstate_bindingstwice, so a one-shot iterator made a valid area knob fail. The bindings are materialized once.test_unit_construction.py38,test_group_encode.py78,test_input_closure.py52,test_concept_mapping.py168,test_nz_axiom_input_closure.py17 passed and 1 skipped (it needs a local Axiom engine),test_nz_spec_package.py71,test_country_spec.py135,test_spec_engine_country_bundles.py21,test_spec_only_country_packages.py8. The new regressions fail on 8cdc884.concepts._check_pointers.positionsand_Context.person_rowson main cast uint64 pointers to int64 the same way (a follow-up); the NZ legal values in the closure's defaults are proposals awaiting the D1 method card, with no source-text receipts in this checkout.Reviews
partner_person_idrefused in_Units' wide check, with a regression test; closure notes say takeup.assign@1 is declared but not yet implemented; golden regenerated). An independent delta review of those follow-ups returned APPROVE_WITH_NITS, Needs Max: no: the probe is now refused, a mutant without the hunk makes the regression test fail, frames without the column or with fitting ids encode unchanged, and the golden diff is limited to the closure hash and fingerprint.household.reference_person_idtyped UInt64 can wrap through the shared_Context.person_rowson main; it is handed to the session fixing uint64 pointers on main.🤖 Generated with Claude Code