Skip to content

Give each dimension value one Chronicle-declared label, keep the package's own wording beside it (consumer_fact.v5) - #377

Draft
anth-volk wants to merge 12 commits into
mainfrom
fix/one-label-per-dimension-value
Draft

anth-volk wants to merge 12 commits into
mainfrom
fix/one-label-per-dimension-value

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #376

Summary

This PR was first a register of accepted label variants (6d74e42). It now implements the design the maintainer chose on 2026-10-21: one Chronicle-declared label per dimension value, with the package's own wording kept in its own field, and no list of exceptions. It applies to dimension values what chronicle#281 and #282 did for geography names (geography.name and geography.publisher_name, consumer_fact.v4), and it reverses the rule of #263 and #267 that "Chronicle keeps each publisher's text; a value worded two ways only warns".

The design

Where the label is declared. Chronicle's register, chronicle/dimension_values/canonical_labels.json, gives one label to each (dimension id, value id) that two or more default-bundle packages emit, and to each value its packages word more than one way: 4,824 values in 147 dimensions (4,801 shared values, plus 23 values that only one package emits but words two ways). It sits next to the geography register and is read the same way (chronicle.dimension_labels.canonical_value_label).

  • Why a register plus the existing package declarations, not one file of every value. Packages emit about 24,000 distinct values, and 19,200 of them come from one package only. Those need no entry: a value one package emits gets its label from that package, as before (its dimension_value_labels declaration, else its row label, else its source-row text, else the number or date itself). For them, one label everywhere follows from a rule: only one package emits them, and that package must not label them two ways (checked below). The register is needed only where two packages could disagree.
  • How it was built. From the consumer facts of every default-bundle package at main aa288c3, each built at a year it validates: the 405 that build at 2023, and the 10 that do not (ACS S0101 ×3, ACS S2201, CMS ACA ×5, JCT ×2) at their own years. A value with one wording keeps it. The 43 values with more than one wording take the labels in the table below.

When it applies. label_facts (inside build_facts) gives every listed value the register's label in dimension_value_labels and, for the groupby value, in layout.groupby_value_label. That happens whenever a package is built, at any year, so a value's label no longer depends on which packages one build compares. This closes the gap the ledger-contract review found: Microcosm merges packages built for different years into one feed, and week 47 (built at 2024) never met the 2022 and 2023 IRS packages in one bundle.

The package's own wording is kept in a new optional field, package_dimension_value_labels (dimension id → value id → text, like dimension_value_labels), present only where it differs from the exported label. It holds the row's label:, else the package's declaration, else its source-row text. That is the package's text, and Chronicle does not check it against the publisher's file. It is the publisher's wording only where a row's guard compares the same text with a source cell, as for week 47's IRS column header. Elsewhere it is often text Chronicle wrote: Statbel's value id "65_plus", Census B01001's parser constants "Female 20 years" and "female", or Eurostat's "First income decile cut-point", whose guard checks the code D1. The publisher's cells stay reachable through each fact's lineage. For a groupby value whose axis is also a filter with a different value (Eurostat quant_inc: filter D1, row d1), the field is keyed by layout.groupby_value_id.

  • Why the field is not limited to publisher text (the reviews' option (a)). The guards compare expected_row_header, expected_column_header and guard cells with the cells Chronicle parses, and those cells are not always the publisher's text. The B01001 parser builds its header cells from Chronicle's own constants (B01001_FEMALE_AGE_COLUMNS), and the Statbel and Belgian country-total packages read extracts Chronicle prepared. A label that passes its guard can therefore still be Chronicle's text. Keeping only publisher text would need a new record, for each artifact and parser, of whether its cells are the publisher's text. So the field is named and described as the package's wording (option (b)): the v5 schema, the code comments, the README, the changelog and docs/agent-source-package-harness.md say so.

No exceptions. REVIEWED_VALUE_LABEL_VARIANTS, is_reviewed_value_label_variant and _cross_package_value_label_warnings are removed. A value labelled two ways is an error wherever Chronicle can see both labels:

Check Command What it refuses Code
Package file, any year validate-package a groupby value the package's rows word two ways, when the register does not list it and the package does not declare it conflicting_value_label
Built facts build-suite (agent acceptance) any value the package's facts label two ways conflicting_value_label
One package in a bundle build-bundle the same conflicting_groupby_value_label, conflicting_dimension_value_label
Two packages in a bundle build-bundle a value two packages label differently conflicting_value_label_across_packages

Each message gives the one remedy that applies:

  • Two packages: add the value to chronicle/dimension_values/canonical_labels.json with one label, chosen by the rule in docs/agent-source-package-harness.md (the wording most of the packages use, a tie going to the national statistics office's wording), then run scripts/build_dimension_value_register.py --static-check.
  • One package: declare one label in its dimension_value_labels.
  • A value the register already lists (possible only for rows built before the register): the message names the register's label and says to rebuild.

validate-package names the package file too. Facts carry the register's label for every listed value, so the cross-package error can only come from a value the register does not list yet.

The register is the one source of a listed value's label. validate-package refuses a dimension_value_labels declaration that the register contradicts (declared_value_label_differs_from_register), and the message gives the register's label. The register had overridden 13 declarations in six packages without any message. Those declarations now equal its labels:

  • sex ("Females"/"Males" → "Female"/"Male") in nrs-mye-2025-council-area, nrs-pcon24-population-by-age-2024-revised-2026 and nisra-mye-2025-lgd-single-year-age;
  • property_type in ons-hpssa-pcon24-year-ending-march-2026 ("Detached houses" → "Detached" and so on) and ons-pipr-rents-by-area-august-2026 ("detached" → "Detached", "terraced" → "Terraced");
  • age_band in dft-nts0601-age-mode-trips-2025 ("Age 30 to 39" → "30 to 39", "Age 40 to 49" → "40 to 49").

No exported label changes. Those facts no longer carry the old declaration text in package_dimension_value_labels. NRS's guard cells still compare its "Females" and "Males" with the source.

How the register is maintained. scripts/build_dimension_value_register.py generates the register:

  • How it builds the register: it counts each package's own wording of each value over every default-bundle package, each built at its build year. A value with one wording keeps it. A value worded more than one way takes the label in the script's CHOSEN_LABELS.
  • Reproduced: run over the stored main aa288c3 builds the register was first made from (405 packages at 2023, 10 at their own years), the script's output is byte-identical to the committed register: all 4,824 entries and the sources block. (coicop_02 differs only from the e29da32 register, because its new label is in CHOSEN_LABELS.)
  • Sources: the register's new sources block records the script, that commit, the build years and the chosen values, as Add the UK income facts issue 280 asks for, and one name per UK geography #282's geography register records its ONS artifacts.
  • --static-check: compares the register with the package files without building them. Every value that two packages' files name (groupby row value ids, filters and declarations, compiled at each package's build year) must be listed. Every listed value must be named by a package file. No declaration may contradict the register. It takes about a minute and runs in CI through tests/test_dimension_value_register.py, in the "Chronicle surface" job.

What is still not checked (follow-up). The static check cannot see a value that a package takes only from its source rows at build time. The full comparison, scripts/build_dimension_value_register.py --check, builds every default-bundle package at its build year. It is not in CI because it builds every package, which is likely to cost about as much as the full-bundle build (not timed). Until it is in CI, two packages can share such a value without the register listing it, and their labels are then compared only when both build in one bundle.

The week-47 package

The two under_1 rows go back to the IRS's header, label: No adjusted gross income and deficit, which their column guard checks. Their facts carry the register's "No adjusted gross income" in dimension_value_labels and layout.groupby_value_label, and the IRS text in package_dimension_value_labels. The YAML now differs from main only by comments, so layout.record_set_spec_hash keeps main's value on all 34 facts. The evidence that it is the same class of returns is unchanged from the earlier version of this PR: each IRS file puts deficit returns in it, and every package constrains under_1 to adjusted gross income < 1.

Two other rows change their label:, because their labels were Chronicle's own and not a publisher's (ledger-source-fidelity finding 5). The label now equals the register's, so these two packages put nothing in package_dimension_value_labels:

  • census-pep-2024-national-age-sex, 85_plus: "Age 85 and over" → "Age 85 plus". The file has numeric age codes only, and the row's guard checks the code 85.
  • soi-historic-table-2, all: "All returns" → "All returns, total". The CSV has no text for AGI_STUB 0, and the row's guard checks the code 0.

What changes for consumers

The contract becomes chronicle.consumer_fact.v5: v4 plus the optional top-level package_dimension_value_labels. v4's row schema has additionalProperties: false, so the field needs a new contract, as #282 needed v4 for geography.publisher_name. v4 restamped as v5 is valid. The schema is in docs/schemas/consumer_fact.v5.schema.json and policyengine_chronicle/schemas/ (byte-identical), with sha256 acd33a091cd8d192be5d264e3e94c13e00afb6fd1972e57d4576ac55a54f6970. (At e29da32 it was c4d2579c…, with the field named publisher_dimension_value_labels and described as the publisher's text. No consumer had pinned that digest.)

Fields, on facts built from this branch:

Field Change
schema_version chronicle.consumer_fact.v4 → chronicle.consumer_fact.v5, on every row
dimension_value_labels[dimension][value] the register's label, on facts whose package worded one of the 43 values differently (183,946 facts over the 2023 bundle and the ten other packages; 155,088 of them are sex in the NRS and NISRA packages). At e29da32 this was 183,977: coicop_02's 31 overview facts now keep ONS's own "Alcohol, tobacco and narcotics".
layout.groupby_value_label the same label, on those of the facts above where the value is the groupby value
package_dimension_value_labels new; holds the package's own wording where it differs from the exported label. It is on fewer facts than the row above, because the six packages whose declarations now equal the register put nothing in it for those values (not counted over the whole bundle).
layout.record_set_spec_hash changes only on the two record sets whose row text changed: Census PEP national (19 facts at 2023) and Historic Table 2 US (605 facts at 2023). Week 47's is main's again.
layout.record_set_spec_id, every key (aggregate_fact_key, semantic_fact_key, legacy_fact_key, dimension_set_key and the others), values, constraints, periods, lineage unchanged

The validator's recomputed key. _recompute_aggregate_fact_key never reads labels (it reads the declared nested keys, the period, the geography without its display names, and the like), so the new field cannot move a key. A test exports week 47's under_1 facts, builds a consumer artifact and loads it back, which runs that check, as #284's regression test does for publisher_name. The two keys equal main's.

What a Microcosm re-pin onto v5 needs (the ledger-contract re-review's list for e29da32, with the schema digest of this head). Microcosm does not validate rows against a Chronicle schema: it carries each row's schema_version without checking it, and no Microcosm file reads package_dimension_value_labels. What fails until it moves is the feed pin, which validate_chronicle_feed_pin compares exactly.

For every country (UK, NZ and US), in the feed pin JSON (packages/microcosm-build/src/microcosm/build/<country>/chronicle_feed.json):

  • consumer_fact_schema_versions: ["chronicle.consumer_fact.v5"];
  • consumer_fact_schema_sha256: "acd33a091cd8d192be5d264e3e94c13e00afb6fd1972e57d4576ac55a54f6970";
  • the new source_commit, facts_sha256 and fact_row_count;
  • manifest_sha256, for the UK and NZ artifact feeds.

For every country, in the tests that pin those fields:

  • tests/engine_free/uk/test_uk_chronicle_feed.py, which asserts the version tuple and the schema sha at about lines 28–31;
  • tests/engine_free/us/test_us_chronicle_feed.py, which has _FACTS_SHA256 and _SCHEMA_SHA256 at lines 36–37 and the version tuple at 61 and 70;
  • for NZ, tests/engine_free/shared/test_ledger_artifact.py at about 501–513. It checks the real NZ artifact against nz/chronicle_feed.json, but only when NZ_CHRONICLE_ARTIFACT is set.

For every country, by convention (no loader needs it):

  • declare v5 in chronicle_epoch.py, as CHRONICLE_CONSUMER_FACT_SCHEMA_VERSION_V5 in DECLARED_IDENTITIES;
  • extend the version tests: test_chronicle_epoch.py's exact accepted-set assertion (about line 400) and its per-row parametrisation (about line 598);
  • add a re-pin history entry to docs/uk-chronicle-feed-repin.md or docs/us-chronicle-feed-repin.md.

UK feed (the --suite uk bundle):

  • All the relabelled UK packages are in the UK suite: NRS and NISRA population (sex), PIPR and HPSSA (property_type), ONS families and households (household_type), consumer trends (ons.coicop), ASHE and NTS0601 (age_band), and SLC.
  • The UK national register version will move on display metadata alone: ons.household_composition.couple_3_plus_children_households takes the new ledger_layout_groupby_value_label and hierarchy value_label. The local register version may move too. The re-pin record should say so, and the frozen scoring register in uk-candidate-eval needs a re-freeze before the next national solve, as after the v4 re-pin.
  • The other relabelled UK packages are not selected by any target: PIPR targets select monthly averages by area, not property type; HPSSA is read only by tools/build_uk_high_value_home_stock.py, by the property_type id; the coicop targets select 04.5, 04.5.1, 04.5.2 and 07.2.2, none of which moves; ASHE age-band rows are not vendored.
  • Files to regenerate as in any UK re-pin (docs/uk-chronicle-feed-repin.md, about lines 17–94): the vendored resource headers; the national and local references and membership; the local census; the source_feed block of uk/local_validation_levels.json; the compile-parity receipts; uk/release_input_coverage_manifest.json; tests/engine_free/uk/test_uk_full_targets.py (test_current_national_and_local_pins_share_one_reviewed_identity).
  • One condition the re-pin relies on: in every selected row, dimension_value_labels[d][v] must equal layout.groupby_value_label for the groupby value, or _validate_chronicle_hierarchy_labels raises. Chronicle sets both from the same resolution (_groupby_value_label), so this holds.

US feed:

  • The feed is still v3 at 51aa40f. A re-pin onto this PR also moves it past v4, which Microcosm already accepts.
  • Follow docs/us-chronicle-feed-repin.md: rebuild with tools/build_us_chronicle_feed.py; move facts_sha256, _FACTS_SHA256 and the target-parity digests; check scope_sha256.
  • Label changes to expect in the US scope (us/chronicle_feed_scope.json includes every one of these packages): adjusted gross income under_1 on week 47 ("No adjusted gross income"); all on Table 1.1 and Historic Table 2 ("All returns, total"); age=85_plus on PEP national; the B01001 female ages ("Female 20 years" → "Age 20", and so on) and its sex=female ("female" → "Female").
  • tests/engine_free/us/test_us_fiscal_targets.py:914 pins surface.version == "3fad14f85468". It will very likely need updating, because the selected Table 1.1, Historic Table 2, PEP and B01001 facts carry new labels in spec metadata.
  • tests/fixtures/us_compiled_ledger_filter_specs.json holds ledger_layout_groupby_value_label: "All returns" for Historic Table 2 (lines 1383 and 1444). It is a frozen capture; test_pinned_chronicle_feed_compiles_no_unsupported_ledger_filters checks only filter keys, so it does not fail, but the text becomes stale.

NZ feed:

  • No labels move. The person.age_band=65_plus register label "65+" is Stats NZ's own.
  • The digest moves from cc9efb90… to acd33a09….

The separate Microcosm US feed re-pin being prepared on its own Chronicle branch does not include this PR, and this PR pushes nothing to feed/*.

Labels for the 43 values with more than one wording

Default rule: the wording the most packages use; a tie goes to the national statistics office's wording. Where that rule would give a wrong label, the proposal deviates and says why. Two kinds of deviation recur: a wording that names the measure rather than the value (Eurostat's decile "cut-point" and "share", Groeipakket's "child" and "family caseload"), and a footnote marker or typo. The three US labels are the maintainer's choice; the user approved all 40 others on 2026-10-21. coicop_02 was approved with ONS's overview wording, "Alcohol, tobacco and narcotics", after the ledger-source-fidelity re-review. Counting facts instead of packages would pick "Females" and "Males" (NRS and NISRA, 77,544 facts each), ONS PIPR's lower-case "detached" and "terraced", and "Female 20 years" for the Census projections' age 20. That is why the rule counts packages.

Dimension = value Label Status Wordings today Facts whose label changes Why
age = 85_plus "Age 85 plus" chosen "Age 85 plus": 1 pkg, 51 facts (census-pep-2024-state-age-sex)
"Age 85 and over": 1 pkg, 1 facts (census-pep-2024-national-age-sex)
1 Chosen by the maintainer (a 1–1 tie; both labels are Chronicle's own, the Census files carry numeric codes only).
age = age_20 "Age 20" approved (2026-10-21) "Female 20 years": 1 pkg, 52 facts (census-b01001-female-age-2023)
"Age 20": 1 pkg, 1 facts (census-population-projections-2023)
52 Tie (1–1), both Census Bureau. "Female …" names the sex dimension, not the age, so the form of the other age values ("Age N") is used. Deviation from the default rule.
age = age_21 "Age 21" approved (2026-10-21) "Female 21 years": 1 pkg, 52 facts (census-b01001-female-age-2023)
"Age 21": 1 pkg, 1 facts (census-population-projections-2023)
52 Tie (1–1), both Census Bureau. "Female …" names the sex dimension, not the age, so the form of the other age values ("Age N") is used. Deviation from the default rule.
age = age_25_to_29 "Age 25 to 29" approved (2026-10-21) "Age 25 to 29": 3 pkg, 490 facts (census-acs-s0101-congressional-district-age-2024, census-acs-s0101-national-age-2024, census-acs-s0101-state-age-2024)
"Female 25 to 29 years": 1 pkg, 52 facts (census-b01001-female-age-2023)
52 Default rule: 3 packages to 1.
age = age_30_to_34 "Age 30 to 34" approved (2026-10-21) "Age 30 to 34": 3 pkg, 490 facts (census-acs-s0101-congressional-district-age-2024, census-acs-s0101-national-age-2024, census-acs-s0101-state-age-2024)
"Female 30 to 34 years": 1 pkg, 52 facts (census-b01001-female-age-2023)
52 Default rule: 3 packages to 1.
age = age_35_to_39 "Age 35 to 39" approved (2026-10-21) "Age 35 to 39": 3 pkg, 490 facts (census-acs-s0101-congressional-district-age-2024, census-acs-s0101-national-age-2024, census-acs-s0101-state-age-2024)
"Female 35 to 39 years": 1 pkg, 52 facts (census-b01001-female-age-2023)
52 Default rule: 3 packages to 1.
age = age_40_to_44 "Age 40 to 44" approved (2026-10-21) "Age 40 to 44": 3 pkg, 490 facts (census-acs-s0101-congressional-district-age-2024, census-acs-s0101-national-age-2024, census-acs-s0101-state-age-2024)
"Female 40 to 44 years": 1 pkg, 52 facts (census-b01001-female-age-2023)
52 Default rule: 3 packages to 1.
age_band = 30 to 39 "30 to 39" approved (2026-10-21) "30 to 39": 5 pkg, 419 facts (ons-ashe-employee-contribution-bands-by-age-2024-full-pay, ons-ashe-employee-contribution-bands-by-age-2024-qualifying-earnings, ons-ashe-employer-contribution-bands-by-age-2024-full-pay, ons-ashe-employer-contribution-bands-by-age-2024-qualifying-earnings, ons-ashe-pension-membership-by-age-earnings-2024)
"Age 30 to 39": 1 pkg, 9 facts (dft-nts0601-age-mode-trips-2025)
9 Default rule: 5 packages to 1.
age_band = 40 to 49 "40 to 49" approved (2026-10-21) "40 to 49": 5 pkg, 424 facts (ons-ashe-employee-contribution-bands-by-age-2024-full-pay, ons-ashe-employee-contribution-bands-by-age-2024-qualifying-earnings, ons-ashe-employer-contribution-bands-by-age-2024-full-pay, ons-ashe-employer-contribution-bands-by-age-2024-qualifying-earnings, ons-ashe-pension-membership-by-age-earnings-2024)
"Age 40 to 49": 1 pkg, 9 facts (dft-nts0601-age-mode-trips-2025)
9 Default rule: 5 packages to 1.
early_year_award_line = new_returning_total_returning_31_33_35 "Total returning [31][33][35]" approved (2026-10-21) "Total returning [31[[33][35]": 1 pkg, 1 facts (slc-student-support-england-2025-provisional-2025-26)
"Total returning [31][33][35]": 1 pkg, 1 facts (slc-student-support-england-2025-provisional-2025-26)
1 One package, two spellings; "[31[[33]" is a typo in the workbook. Deviation: not a count.
groeipakket.component = pleegzorgtoeslag "Foster care supplement" approved (2026-10-21) "Foster care supplement child caseload": 1 pkg, 1 facts (opgroeien-groeipakket-caseload-2025)
"Foster care supplement family caseload": 1 pkg, 1 facts (opgroeien-groeipakket-caseload-2025)
2 One package; the two wordings name the count (per child or per family), not the supplement, so the common part is used. Deviation: neither existing wording.
groeipakket.component = sociale_toeslag "Social supplement" approved (2026-10-21) "Social supplement child caseload": 1 pkg, 1 facts (opgroeien-groeipakket-caseload-2025)
"Social supplement family caseload": 1 pkg, 1 facts (opgroeien-groeipakket-caseload-2025)
2 One package; the two wordings name the count (per child or per family), not the supplement, so the common part is used. Deviation: neither existing wording.
groeipakket.component = wezentoeslag "Orphan supplement" approved (2026-10-21) "Orphan supplement child caseload": 1 pkg, 1 facts (opgroeien-groeipakket-caseload-2025)
"Orphan supplement family caseload": 1 pkg, 1 facts (opgroeien-groeipakket-caseload-2025)
2 One package; the two wordings name the count (per child or per family), not the supplement, so the common part is used. Deviation: neither existing wording.
groeipakket.component = zorgtoeslag "Care supplement" approved (2026-10-21) "Care supplement child caseload": 1 pkg, 1 facts (opgroeien-groeipakket-caseload-2025)
"Care supplement family caseload": 1 pkg, 1 facts (opgroeien-groeipakket-caseload-2025)
2 One package; the two wordings name the count (per child or per family), not the supplement, so the common part is used. Deviation: neither existing wording.
household_type = couple_3_plus_children_households "3 or more dependent children" approved (2026-10-21) "3 or more dependent children [note 5]": 1 pkg, 8 facts (ons-families-households-2025)
"3 or more dependent children [note 3]": 1 pkg, 1 facts (ons-households-by-type-country-2025)
9 Tie (1–1), both ONS; only the table's own footnote number differs, so it is dropped. Deviation: neither existing wording.
measure = country_total "Country total" approved (2026-10-21) "Belgium PIT federal and local tax before withholding": 1 pkg, 1 facts (spf-finances-pit-2023)
"Complete unemployment benefit recipients": 1 pkg, 1 facts (onem-rva-unemployment-2024)
"Household disposable income gross": 1 pkg, 1 facts (nbb-national-accounts-household-disposable-income-2024)
"Worker-borne personal social security contributions": 1 pkg, 1 facts (onss-contributions-2024)
4 Four-way tie; each package names its own measure, which the fact's measure already carries. Deviation: neither existing wording.
na_item = belgium_household_disposable_income_gross "Disposable income, gross (B.6G)" approved (2026-10-21) "Disposable income, gross (B.6G), paid-direction cell": 1 pkg, 3 facts (eurostat-nasa-10-nf-tr)
"Disposable income, gross (B.6G), received-direction cell": 1 pkg, 3 facts (eurostat-nasa-10-nf-tr)
6 One package; "paid-direction"/"received-direction cell" names the cell, not the item, so the item's name is used. Deviation: neither existing wording.
na_item = belgium_household_mixed_income_gross "Mixed income, gross (B.3G)" approved (2026-10-21) "Mixed income, gross (B.3G), paid-direction cell": 1 pkg, 3 facts (eurostat-nasa-10-nf-tr)
"Mixed income, gross (B.3G), received-direction cell": 1 pkg, 3 facts (eurostat-nasa-10-nf-tr)
6 One package; "paid-direction"/"received-direction cell" names the cell, not the item, so the item's name is used. Deviation: neither existing wording.
na_item = belgium_household_operating_surplus_gross "Operating surplus, gross (B.2G)" approved (2026-10-21) "Operating surplus, gross (B.2G), paid-direction cell": 1 pkg, 3 facts (eurostat-nasa-10-nf-tr)
"Operating surplus, gross (B.2G), received-direction cell": 1 pkg, 3 facts (eurostat-nasa-10-nf-tr)
6 One package; "paid-direction"/"received-direction cell" names the cell, not the item, so the item's name is used. Deviation: neither existing wording.
ons.coicop = coicop_01 "Food and non-alcoholic beverages" approved (2026-10-21) "Food and drink": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
"Food and non-alcoholic beverages": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
31 One package, tie (overview and detailed worksheets, both ONS); the fuller detailed-worksheet wording is used, without its footnote marker. Deviation: tie broken by a second rule.
ons.coicop = coicop_02 "Alcohol, tobacco and narcotics" approved (2026-10-21) "Alcohol, tobacco and narcotics": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
"Alcoholic beverages and tobacco [note 1]": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
31 One package, tie (overview and detailed worksheets, both ONS). The overview (0CN) wording is used: the total, CDID ADFL, equals ADFM (alcoholic beverages) + ADFN (tobacco) + MNC2 (narcotics) in every year, and the detailed 02CN header leaves narcotics out. Deviation: tie broken by what the total contains.
ons.coicop = coicop_04 "Housing, water, electricity, gas and other fuel" approved (2026-10-21) "Housing": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
"Housing, water, electricity, gas and other fuel": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
31 One package, tie (overview and detailed worksheets, both ONS); the fuller detailed-worksheet wording is used, without its footnote marker. Deviation: tie broken by a second rule.
ons.coicop = coicop_05 "Furnishings, household equipment and routine maintenance of the house" approved (2026-10-21) "Furnishing, household equipment and routine HH maintenance": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
"Furnishings, household equipment and routine maintenance of the house": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
31 One package, tie (overview and detailed worksheets, both ONS); the overview (0CN) wording is used, because the detailed 05CN header abbreviates it. Deviation: tie broken by a second rule.
ons.coicop = coicop_11 "Restaurants and hotels" approved (2026-10-21) "Restaurant and hotels": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
"Restaurants and hotels": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
31 One package, tie (overview and detailed worksheets, both ONS); the overview (0CN) wording is used, because the detailed 11CN header writes "Restaurant and hotels". Deviation: tie broken by a second rule.
ons.coicop = coicop_12 "Miscellaneous goods and services" approved (2026-10-21) "Miscellaneous": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
"Miscellaneous goods and services": 1 pkg, 31 facts (ons-consumer-trends-current-price-2026)
31 One package, tie (overview and detailed worksheets, both ONS); the fuller detailed-worksheet wording is used, without its footnote marker. Deviation: tie broken by a second rule.
ons.household_type = couple_3_plus_children_households "3 or more dependent children" approved (2026-10-21) "3 or more dependent children [note 5]": 1 pkg, 8 facts (ons-families-households-2025)
"3 or more dependent children [note 3]": 1 pkg, 1 facts (ons-households-by-type-country-2025)
9 Tie (1–1), both ONS; only the table's own footnote number differs, so it is dropped. Deviation: neither existing wording.
person.age_band = 65_plus "65+" approved (2026-10-21) "65_plus": 2 pkg, 12 facts (statbel-population-structure-2025, statbel-population-structure-2026)
"65+": 1 pkg, 17 facts (stats-nz-subnational-population-estimates-2025)
12 Default rule would give "65_plus" (2 packages to 1), which is the value id, not a wording; Stats NZ's "65+" is used. Deviation.
property_type = detached "Detached" approved (2026-10-21) "Detached": 4 pkg, 16 facts (desnz-need-england-wales-2023, desnz-need-england-wales-2024, desnz-need-scotland-2023, desnz-need-scotland-2024)
"detached": 1 pkg, 6,978 facts (ons-pipr-rents-by-area-august-2026)
"Detached houses": 1 pkg, 4,545 facts (ons-hpssa-pcon24-year-ending-march-2026)
11,523 Default rule: 4 packages to 1 and 1 (by facts, ONS PIPR's "detached" would win).
property_type = semi_detached "Semi detached" approved (2026-10-21) "Semi detached": 4 pkg, 16 facts (desnz-need-england-wales-2023, desnz-need-england-wales-2024, desnz-need-scotland-2023, desnz-need-scotland-2024)
"Semi-detached houses": 1 pkg, 4,596 facts (ons-hpssa-pcon24-year-ending-march-2026)
4,596 Default rule: 4 packages to 1 (alternative: "Semi-detached").
property_type = terraced "Terraced" approved (2026-10-21) "Terraced": 2 pkg, 8 facts (desnz-need-scotland-2023, desnz-need-scotland-2024)
"terraced": 1 pkg, 6,978 facts (ons-pipr-rents-by-area-august-2026)
"Terraced houses": 1 pkg, 4,600 facts (ons-hpssa-pcon24-year-ending-march-2026)
11,578 Default rule: 2 packages to 1 and 1 (by facts, "terraced" would win).
quant_inc = d1 "First income decile" approved (2026-10-21) "First income decile cut-point": 1 pkg, 3 facts (eurostat-ilc-di01)
"First income decile share": 1 pkg, 3 facts (eurostat-ilc-di01)
6 One package; "cut-point"/"share" names the measure, not the decile, so the common part is used. Deviation: neither existing wording.
quant_inc = d2 "Second income decile" approved (2026-10-21) "Second income decile cut-point": 1 pkg, 3 facts (eurostat-ilc-di01)
"Second income decile share": 1 pkg, 3 facts (eurostat-ilc-di01)
6 One package; "cut-point"/"share" names the measure, not the decile, so the common part is used. Deviation: neither existing wording.
quant_inc = d3 "Third income decile" approved (2026-10-21) "Third income decile cut-point": 1 pkg, 3 facts (eurostat-ilc-di01)
"Third income decile share": 1 pkg, 3 facts (eurostat-ilc-di01)
6 One package; "cut-point"/"share" names the measure, not the decile, so the common part is used. Deviation: neither existing wording.
quant_inc = d4 "Fourth income decile" approved (2026-10-21) "Fourth income decile cut-point": 1 pkg, 3 facts (eurostat-ilc-di01)
"Fourth income decile share": 1 pkg, 3 facts (eurostat-ilc-di01)
6 One package; "cut-point"/"share" names the measure, not the decile, so the common part is used. Deviation: neither existing wording.
quant_inc = d5 "Fifth income decile" approved (2026-10-21) "Fifth income decile cut-point": 1 pkg, 3 facts (eurostat-ilc-di01)
"Fifth income decile share": 1 pkg, 3 facts (eurostat-ilc-di01)
6 One package; "cut-point"/"share" names the measure, not the decile, so the common part is used. Deviation: neither existing wording.
quant_inc = d6 "Sixth income decile" approved (2026-10-21) "Sixth income decile cut-point": 1 pkg, 3 facts (eurostat-ilc-di01)
"Sixth income decile share": 1 pkg, 3 facts (eurostat-ilc-di01)
6 One package; "cut-point"/"share" names the measure, not the decile, so the common part is used. Deviation: neither existing wording.
quant_inc = d7 "Seventh income decile" approved (2026-10-21) "Seventh income decile cut-point": 1 pkg, 3 facts (eurostat-ilc-di01)
"Seventh income decile share": 1 pkg, 3 facts (eurostat-ilc-di01)
6 One package; "cut-point"/"share" names the measure, not the decile, so the common part is used. Deviation: neither existing wording.
quant_inc = d8 "Eighth income decile" approved (2026-10-21) "Eighth income decile cut-point": 1 pkg, 3 facts (eurostat-ilc-di01)
"Eighth income decile share": 1 pkg, 3 facts (eurostat-ilc-di01)
6 One package; "cut-point"/"share" names the measure, not the decile, so the common part is used. Deviation: neither existing wording.
quant_inc = d9 "Ninth income decile" approved (2026-10-21) "Ninth income decile cut-point": 1 pkg, 3 facts (eurostat-ilc-di01)
"Ninth income decile share": 1 pkg, 3 facts (eurostat-ilc-di01)
6 One package; "cut-point"/"share" names the measure, not the decile, so the common part is used. Deviation: neither existing wording.
sex = female "Female" approved (2026-10-21) "Female": 10 pkg, 6,561 facts (hmrc-cgt-carried-interest-2026, hmrc-income-tax-liabilities-july-2026, hmrc-spi-income-bands-2023-24, hmrc-spi-income-by-region-2023-24, ons-employee-workplace-pensions-summary-2024, ons-mye-2023-england-regions, ons-mye-2023-uk-countries, ons-mye-2024-uk, ons-mye-2025-uk, ons-uk-population-projections-2024)
"Females": 3 pkg, 77,544 facts (nisra-mye-2025-lgd-single-year-age, nrs-mye-2025-council-area, nrs-pcon24-population-by-age-2024-revised-2026)
"female": 1 pkg, 468 facts (census-b01001-female-age-2023)
78,012 Default rule: 10 packages to 3 to 1 (by facts, NRS and NISRA's "Females" would win: 77,544 facts).
sex = male "Male" approved (2026-10-21) "Male": 10 pkg, 6,572 facts (hmrc-cgt-carried-interest-2026, hmrc-income-tax-liabilities-july-2026, hmrc-spi-income-bands-2023-24, hmrc-spi-income-by-region-2023-24, ons-employee-workplace-pensions-summary-2024, ons-mye-2023-england-regions, ons-mye-2023-uk-countries, ons-mye-2024-uk, ons-mye-2025-uk, ons-uk-population-projections-2024)
"Males": 3 pkg, 77,544 facts (nisra-mye-2025-lgd-single-year-age, nrs-mye-2025-council-area, nrs-pcon24-population-by-age-2024-revised-2026)
77,544 Default rule: 10 packages to 3 (by facts, "Males" would win: 77,544 facts).
us:statutes/26/62#adjusted_gross_income = all "All returns, total" chosen "All returns, total": 4 pkg, 93 facts (soi-table-1-2, soi-table-1-4, soi-table-2-1, soi-table-4-3)
"All returns": 2 pkg, 59 facts (soi-historic-table-2, soi-table-1-1)
59 Chosen by the maintainer (also the default rule: 4 packages to 2).
us:statutes/26/62#adjusted_gross_income = under_1 "No adjusted gross income" chosen "No adjusted gross income": 7 pkg, 410 facts (soi-historic-table-2, soi-historic-table-2-state-agi-2022, soi-historic-table-2-state-agi-2023, soi-table-1-1, soi-table-1-4, soi-table-2-5-eitc-agi-children-2022, soi-table-2-5-eitc-agi-children-2023)
"No adjusted gross income and deficit": 1 pkg, 2 facts (soi-filing-season-week47-2024-eitc-total)
2 Chosen by the maintainer (also the default rule: 7 packages to 1).

Facts whose exported label changes, over the 2023 default bundle plus the ten packages built at their own years: 183,946 (183,977 at e29da32, less coicop_02's 31 overview facts).

Review findings from the 6d74e42 reviews, and where they stand

  • ledger-contract, medium (packages built at different years are never compared): addressed by the design. Facts take the register's label when they are built, at any year. The four ACS S0101 (2024) against B01001 female (2023) values, age_25_to_29 to age_40_to_44, are in the register, and a test builds those packages and week 47 against Table 1.4 at their own years.
  • ledger-contract, low: the register no longer has keys without labels or labels without packages, because it is not a list of accepted variants. A new package cannot adopt a listed variant: a listed value takes one label. The error messages name the register's path and the package declaration. The within-package check also runs in validate-package and build-suite. The dimension_label_issues docstring and the README sentence are rewritten.
  • ledger-source-fidelity, low (the register's comments called Chronicle's own labels publisher wording): the register's comments are gone with it, and the two rows whose labels were Chronicle's own now carry the register's label, so package_dimension_value_labels holds nothing for them.
  • ledger-boundary, governance text: kept and updated below for the new files and roles.

Re-review findings on e29da32, and where they stand

  • publisher_dimension_value_labels held Chronicle's text (ledger-source-fidelity High; ledger-contract and ledger-boundary Medium). Renamed package_dimension_value_labels and described as the package's own, unchecked wording (option (b); the reason is under "The design"). Tests build each example the reviews found (Statbel 65_plus, B01001 age_20 and sex=female, Eurostat d1, the four Belgian country_total rows, eurostat-nasa-10-nf-tr, the Groeipakket caseload) and check that the text goes to a field that says it may be Chronicle's. NTS0601's "Age 30 to 39" came from its declaration, which now equals the register's "30 to 39".
  • The Eurostat key (d1 in the field, D1 in the fact's dimensions): it does not follow from option (b), so it is documented. The field is keyed by layout.groupby_value_id for a groupby value whose axis is also a filter with another value, and the v5 description says so.
  • coicop_02 left out narcotics (ledger-source-fidelity Medium). It now takes "Alcohol, tobacco and narcotics", and a test checks ADFL = ADFM + ADFN + MNC2 in the stored cpnsa.xlsx. The table also says correctly that the coicop_05 and coicop_11 labels are the overview sheet's (ledger-source-fidelity Low).
  • 13 declarations overridden without a message (ledger-contract finding 5). validate-package now refuses a declaration the register contradicts, and the 13 declarations (in six packages, not the seven the review counted) now equal the register's labels.
  • Messages gave a remedy that could not apply, and no label (ledger-contract). Each message now gives the one remedy for its case, the rule for choosing a register label, or the register's own label.
  • The register's script and membership check (ledger-contract finding 4, ledger-boundary finding 2). The script is committed with --check and --static-check. The static check runs in CI, and the register records its sources. The full per-year build check is a follow-up (above).
  • The guards check expected_row_header and expected_column_header, not label: (ledger-source-fidelity Low). The harness doc says so, including that a row's label: is compared with the row-header cell only when the row sets no expected_row_header.
  • Re-pin guidance (ledger-contract finding 2). It is replaced by the reviewer's list above.
  • Not changed: ledger-contract finding 3. A v5 row restamped from facts built before this PR carries the package's label, not the register's; this matches Add the UK income facts issue 280 asks for, and one name per UK geography #282's restamp behaviour, and Microcosm's feeds rebuild their facts.

Re-check findings on d24819c, and where they stand

All three judges passed d24819c. The fix round (1cd9ce1, eaf1612 red, c74db62 green, then 2339091 merging main):

  • N1, CI failure (High, not a judge criterion): tests/test_etl_eurostat.py::test_eurostat_prior_vintage_facts_remain_byte_stable hashes every AggregateFact field of eurostat-gov-10a-taxag and eurostat-spr-exp-func, and the new field is {} on every fact of both. 1cd9ce1 re-pins the full-fact digests (516a28c5…, e0231b78…) and asserts the reason: the field is {} on every fact, and with it removed the facts still hash to the earlier 89b50ea0… and b57eca7e…. It is one commit, not a red and a green: the code was already correct and only the pinned constants were stale, so a test-only commit would fail exactly as d24819c does, on the full-fact digest, while the new strip-and-compare assertions already pass there.
  • N2, the cross-package remedy (Low): the message for a value two packages label differently now also says to add the chosen label to CHOSEN_LABELS in scripts/build_dimension_value_register.py, so the script still reproduces the register. Test first in eaf1612.
  • N3, package text called the publisher's (Low): the chronicle.dimension_labels docstring's label-source list now names the package's row label (the record-set row's label:) and the source-row text as Chronicle's parser read it, and the local variable publisher_labels is now package_wordings. A docstring test came first in eaf1612.
  • N4, the reproduction wording (Info): corrected above. With the current script the output is byte-identical; coicop_02 differs only from the e29da32 register.
  • N5, --check with --output (Info): --check compares with the committed register even when --output names another file. Not changed.
  • coicop_02's wording "Alcohol, tobacco and narcotics" was approved by the maintainer on 2026-10-21, as the table says.
  • main moved to 6d6af42 (Stats NZ regional resident population at 30 June 2023 (NZ, G8-d4) #379) and is merged in with a merge commit (2339091); the reviewed commits are not rebased. Stats NZ regional resident population at 30 June 2023 (NZ, G8-d4) #379 adds a 2023 record set, 17 facts, to stats-nz-subnational-population-estimates-2025. Its groupby values (regc_01 to regc_18 and nz, on geography) are emitted by that package only, with the same labels as its 2025 record sets, so the register needs no entry (--static-check: 0 problems). It moves the merged-bundle pins in tests/test_chronicle_bundle.py (1,579,263 to 1,579,280 facts, and the Stats NZ, period, geography and entity counts), which come from main unchanged. It adds no warning on main, so warning_count stays 83 here.

Review of the fix round (1cd9ce1 to 2339091), and where it stands

The review found the round mergeable on code and CI, with two Low wording findings and two Info findings:

  • F1 (Low): the _dimension_label_reports docstring in chronicle/bundle.py still called the record-set rows' label: values "publisher row labels". 74fb2b8 calls them the package's row labels; a sibling of the N3 docstring test, test_the_bundle_label_report_does_not_call_package_wording_the_publishers, came first in 35514b4. Docstring only.
  • F2 (Low): the PR title said "keep the publisher's wording beside it". It now says "keep the package's own wording beside it", as the field's name and description do.
  • F3 (Info), known and deliberately left: an author who follows the N2 message alone still leaves the register's sources.chosen_labels list and the len(CHOSEN_LABELS) == 43 pin stale. tests/test_dimension_value_register.py (test_the_register_records_what_it_was_built_from, test_the_chosen_labels_are_the_registers_labels) reports both, so neither can reach main unnoticed.
  • F4 (Info), known and deliberately left: the script path is held twice, in chronicle.dimension_labels.REGISTER_SCRIPT_DISPLAY_PATH and the script's own SCRIPT_DISPLAY_PATH.

Chronicle Governance

  • Approved Chronicle agent roles:

    • ledger-source-ingestor: the row labels in packages/irs_soi/filing_season_week47_2024/source_package.yaml, packages/census/pep_2024_national_age_sex/source_package.yaml and packages/irs_soi/historic_table_2/source_package.yaml; the dimension_value_labels declarations that now equal the register in packages/nrs/mye_2025_council_area/source_package.yaml, packages/nrs/pcon24_population_by_age_2024_revised_2026/source_package.yaml, packages/nisra/mye_2025_lgd_single_year_age/source_package.yaml, packages/ons/hpssa_pcon24_year_ending_march_2026/source_package.yaml, packages/ons/pipr_rents_by_area_august_2026/source_package.yaml and packages/dft/nts0601_age_mode_trips_2025/source_package.yaml; the merged-bundle pins in tests/test_chronicle_bundle.py; the v5 pins in tests/test_chronicle_consumer.py; the one-label expectation in tests/test_chronicle_uk_national_2025.py; the re-pinned Eurostat fact digests in tests/test_etl_eurostat.py.
    • ledger-contract-maintainer: the consumer_fact.v5 contract: chronicle/core.py (the package_dimension_value_labels fact field), chronicle/consumer_contract.py (exporting it), policyengine_chronicle/schema.py, policyengine_chronicle/consumer.py (a docstring on the key recompute), policyengine_chronicle/schemas/consumer_fact.v5.schema.json, docs/schemas/consumer_fact.v5.schema.json, tests/test_chronicle_consumer_contract.py and tests/test_chronicle_core.py (the re-pinned sample sha256; also in the ingestor's paths). The role is also named, for its purpose (consumer contracts), for the one-label-per-value rule, whose code is outside its paths.
    • Outside both roles' allowed_paths (19 files): chronicle/dimension_values/canonical_labels.json and chronicle/dimension_labels.py (the label file and label resolution), scripts/build_dimension_value_register.py (the script that builds and checks the label file), chronicle/bundle.py (the build-bundle errors), chronicle/source_package.py (the validate-package checks), chronicle/suite.py (the build-suite check), chronicle/store.py and chronicle/database.py (storing the new field; a fact without it keeps its build hash), chronicle/epoch.py (v5 in CONSUMER_FACT_ROW_CONTRACTS), chronicle/fixtures/consumer_facts.jsonl (sample rows re-pinned to v5), .github/workflows/ci.yml, README.md, docs/agent-source-package-harness.md, changelog.d/376.fixed.md, tests/test_dimension_value_register.py, tests/test_issue_261_dimension_labels.py, tests/test_issue_265_non_uk_labels.py, tests/test_issue_376_one_label_per_value.py and tests/test_policyengine_chronicle_schema.py.
    • The ci.yml change edits only the Python assertions inside the "Test wheel install" step: the wheel must ship the v5 schema and the label file. It adds no environment variable or secret and changes no trigger, permission or job. The label file's static check runs inside the existing "Test Chronicle surface" pytest step (tests/test_dimension_value_register.py), so it needs no workflow change. The script reads no environment variable.
  • Deterministic checks run (at d24819c unless stated; this round's checks at 2339091 are listed first):

    • this round, at 2339091 (after merging main 6d6af42): scripts/build_dimension_value_register.py --static-check reports 0 problems; for stats-nz-subnational-population-estimates-2025, the package Stats NZ regional resident population at 30 June 2023 (NZ, G8-d4) #379 changed, chronicle validate-package is valid with 0 errors and 0 warnings and chronicle build-suite is valid with 0 agent-acceptance errors, at 2023 and at 2025 (102 facts each, all chronicle.consumer_fact.v5, no value labelled two ways, none carrying package_dimension_value_labels; the one acceptance warning is concept_alignment_validation_skipped).
    • source-package validation: chronicle validate-package is valid with 0 errors and 0 warnings, and chronicle build-suite is valid with 0 agent-acceptance errors, for the six packages whose declarations changed (nrs-mye-2025-council-area, nisra-mye-2025-lgd-single-year-age, nrs-pcon24-population-by-age-2024-revised-2026, ons-hpssa-pcon24-year-ending-march-2026, ons-pipr-rents-by-area-august-2026, dft-nts0601-age-mode-trips-2025, all at 2023), for ons-consumer-trends-current-price-2026 (2023, whose coicop_02 label changes) and for week 47 (2024, a comment changed). The only acceptance warnings are about concept alignment, not labels: no_concept_alignments (NRS ×2, NISRA, HPSSA, PIPR, week 47) and concept_alignment_validation_skipped (NTS0601, consumer trends). At e29da32 (ledger-boundary review), the same checks were valid for week 47, Census PEP national and Historic Table 2.
    • source-cell preservation: the package changes are dimension_value_labels declarations and comments only; no row, guard or cell selection changes. Compared field by field with the stored main aa288c3 builds of the same seven 2023 packages (the builds the label file came from), every fact keeps its source record id and every field except schema_version, dimension_value_labels (the register's labels: 155,088 sex facts in the NRS and NISRA packages, 13,741 HPSSA, 13,956 PIPR, 18 NTS0601 and 186 consumer-trends facts) and, on the 186 consumer-trends facts, layout.groupby_value_label and package_dimension_value_labels. Keys, values, constraints, periods, lineage and geography are identical. The six declaration packages carry no package_dimension_value_labels. tests/test_chronicle_source_cells.py passes.
    • consumer-contract validation: tests/test_chronicle_consumer_contract.py and tests/test_chronicle_consumer.py pass, including a v5 artifact round trip; the week-47 export, artifact build and load test passes with main's keys.
    • schema validation: the v5 schema is byte-identical in docs/schemas and policyengine_chronicle/schemas, sha256 acd33a091cd8d192be5d264e3e94c13e00afb6fd1972e57d4576ac55a54f6970; v1 to v4 are unchanged. tests/test_policyengine_chronicle_schema.py passes.
    • package import compatibility: tests/test_policyengine_chronicle_imports.py passes.
    • raw-facts boundary validation: tests/test_chronicle_boundaries.py and tests/test_chronicle_governance.py pass.
    • the label file: scripts/build_dimension_value_register.py --from-consumer-facts over the stored main aa288c3 builds of all 415 default-bundle packages gives output byte-identical to the committed register (all 4,824 entries and the sources block; re-check of d24819c); --static-check reports no problem at 2339091 (tests/test_dimension_value_register.py).
    • ruff check chronicle policyengine_chronicle db scripts tests (ruff 0.14.10): passed.
    • CI at d24819c (run 39685539423): "Chronicle bundle" passed, including tests/test_chronicle_uk_national_2025.py and the merged-bundle test; "Chronicle surface" had 2 failures, both test_eurostat_prior_vintage_facts_remain_byte_stable (the empty package_dimension_value_labels field moved the two Eurostat digests), fixed in 1cd9ce1, and 5,104 passed. CI at 2339091 (run 39786541579): all three jobs passed. "Chronicle surface": 5,135 passed, 4 skipped. "Chronicle bundle": tests/test_chronicle_uk_national_2025.py 50 passed and tests/test_chronicle_bundle.py 25 passed, including the merged-bundle test with main's 1,579,280 facts and warning_count 83 (peak memory 14.8 GB). "Chronicle checks": passed. CI at 74fb2b8 (F1, a docstring and a test): see the runs on this PR.
  • LLM judge verdicts (re-check of the fix commits 7faaa66 and d24819c, 2026-10-22): all three PASS on d24819c.

    • ledger-source-fidelity: PASS. The field is named and described as the package's own wording, not checked against the publisher's file; coicop_02 carries ONS's overview wording "Alcohol, tobacco and narcotics", which matches what the ADFL total contains; no fact value, source record or source cell changes.
    • ledger-contract: PASS. The rename stays inside the unreleased v5 contract, the v5 schema digest acd33a09… is the only one pinned, no key reads a label, and the declaration refusal and the one-remedy messages are what the earlier review asked for.
    • ledger-boundary: PASS. The fix commits change display labels, package declarations, a label-building script and checks that only report errors; no value, key, period, constraint, lineage or geography moves, and reconciliation, aging, imputation, support-aware activation and solver construction stay outside Chronicle.
    • The re-check's other findings (CI's two Eurostat digest failures, two Low wording findings, one wording fix in this description) are addressed in the fix round after d24819c: see "Re-check findings on d24819c". That round changes a test pin, one error message, a docstring and a local variable name; it changes no label, fact or field.

This PR touches the source-data boundary through nine packages' row labels or declarations and through the display label every fact exports for a dimension value. No fact value, key, cell, period, constraint, lineage or geography changes. chronicle/dimension_values/canonical_labels.json holds display labels keyed by dimension id and value id, and no values. For 4,781 of its 4,824 values it repeats the one wording the packages use. For the other 43 it holds a label chosen by the rule in docs/agent-source-package-harness.md or by the maintainer, and 19 of those are wordings no package used. These are Chronicle's display labels, not names read from a publisher file as #282's geography names are; scripts/build_dimension_value_register.py regenerates them from the packages' builds. package_dimension_value_labels keeps each package's own wording for a value where it differs: the row's label:, else the package's declaration, else the source-row text. That is the publisher's text where a row's guard compares the same text, as for week 47's IRS column header, and Chronicle's text elsewhere; the field's name and description say so. The new checks in validate-package, build-suite and build-bundle report errors and change no fact.

Tests

Fix round after the re-check, and the review of that round (F1):

Commit What ran Result
1cd9ce1 (N1 re-pin) tests/test_etl_eurostat.py 20 passed (the 2 digest tests failed at d24819c)
eaf1612 (tests only) the two new or changed tests in test_issue_376_one_label_per_value.py 2 failed
c74db62 the same 2 passed
2339091 (merged main) test_etl_eurostat.py, test_issue_376_one_label_per_value.py, test_issue_261_dimension_labels.py, test_issue_265_non_uk_labels.py, test_chronicle_source_label_authority.py, test_dimension_value_register.py, test_chronicle_boundaries.py, test_chronicle_facts_only.py, test_chronicle_consumer_contract.py, test_chronicle_consumer.py, test_chronicle_source_cells.py, test_policyengine_chronicle_imports.py, test_chronicle_governance.py, test_policyengine_chronicle_schema.py, test_chronicle_core.py, test_chronicle_nz_population.py 399 passed (17 min 37 s)
35514b4 (tests only, F1) the two docstring tests in test_issue_376_one_label_per_value.py 1 failed (the new F1 test), 1 passed
74fb2b8 the same; then the whole of test_issue_376_one_label_per_value.py 2 passed; 46 passed
2339091 test_chronicle_bundle.py (all but the merged-bundle test) and test_chronicle_uk_national_2025.py 74 passed, 1 deselected (12 min)

ruff check chronicle policyengine_chronicle db scripts tests: passed (ruff 0.14.10). Not run locally: the merged-bundle test.

Red and green, round 2 (the re-review fixes):

Commit What ran Result
7faaa66 (tests only) test_issue_376_one_label_per_value.py, the new test_dimension_value_register.py, test_policyengine_chronicle_schema.py, the SLC test in test_chronicle_uk_national_2025.py, the fixture test in test_chronicle_core.py 53 failed, 31 passed
d24819c the same 84 passed

The red failures, by reason: the field is still publisher_dimension_value_labels, and the sample rows and schema still carry that name (25 tests, including the 11 examples of Chronicle-written wording and the schema restamp tests); the 13 declarations still disagree with the register (13); validate-package has no declaration check, and the messages still give both remedies and no label (5); the register script and its sources block do not exist (7); the v5 description (1); coicop_02 (1); the harness doc (1). The ADFL = ADFM + ADFN + MNC2 source test passes at red, as a statement about the source. d24819c replaces one red test: the NTS0601 example's "Age 30 to 39" came from the declaration that commit corrects, so it is now a test that those facts carry DfT's "30 to 39" and no package wording.

Also run at d24819c: the changed and new test files, plus test_issue_261_dimension_labels.py, test_issue_265_non_uk_labels.py, test_chronicle_consumer.py, test_chronicle_consumer_contract.py, test_chronicle_boundaries.py, test_chronicle_source_cells.py, test_policyengine_chronicle_imports.py, test_chronicle_governance.py, test_chronicle_core.py, test_chronicle_bundle.py (all but the merged-bundle test), test_chronicle_uk_national_2025.py, test_chronicle_source_label_authority.py, test_chronicle_source_package.py and test_chronicle_suite.py: 566 passed, 1 deselected (the merged-bundle test), in 24 min 53 s on a shared machine.

Round 1 (601b219 red, 27 failed and 136 passed; e29da32 green, 163 passed; 4,197 passed in the wider set) is unchanged.

CI pins (test_build_bundle_writes_merged_consumer_contract, CI only)

Not run locally: the merged-bundle test (about 84 minutes on CI).

🤖 Generated with Claude Code

anth-volk and others added 2 commits October 21, 2026 23:03
Tests only. They fail until the next commit.

- test_issue_376_one_label_per_value.py (new): the week-47 filing-season
  rows for adjusted gross income `under_1` carry "No adjusted gross income",
  the label the other IRS SOI packages use, while their column guard keeps
  the IRS header "No adjusted gross\nincome and deficit". A bundle of week
  47, Historic Table 2 and Table 2.5 reports nothing for that value.
  build-bundle refuses a value that two packages label differently, or
  that one package's groupby rows label two ways, unless every label is in
  REVIEWED_VALUE_LABEL_VARIANTS; those wordings stay warnings.
- test_issue_265: the week-47 pair it used as its example of a warning is
  now an error.
- test_issue_261: a groupby value whose row labels differ is now an error
  when it is not a reviewed wording.
- test_chronicle_bundle (merged bundle, CI only): 15 cross-package value
  warnings instead of 16, 121 warnings instead of 122, and the register
  lists exactly the value wordings the bundle still warns about.

On this commit the first three files give 10 failed, 37 passed:
`_cross_package_value_label_errors` and REVIEWED_VALUE_LABEL_VARIANTS do
not exist yet, the week-47 label is still the IRS header, and a groupby
drift is still a warning. The merged-bundle test was not run locally.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bels (#376)

The IRS filing-season week-47 package labelled value `under_1` of
`us:statutes/26/62#adjusted_gross_income` "No adjusted gross income and
deficit", the IRS's own column header, where every other IRS SOI package
writes "No adjusted gross income". It is the same class. Column C of
25inweek47.xlsx holds every return with adjusted gross income below $1:
its AGI is -$120,197,225 thousand, and the 16 income columns add up to all
156,221,133 returns. The "No adjusted gross income" rows of Publication
1304 Tables 1.1, 1.4 and 2.5 and Historic Table 2's AGI_STUB 1 are negative
too. Microcosm copied both labels into US release
populace-us-2024-0fb05b6-4b57d15a287c-20260930T150401Z, and the dashboard
refuses a value with two labels when it builds the release's tree.

- The two eitc_by_agi rows now read "No adjusted gross income"; their
  column guard still checks the IRS header. Built at 2024, the package's
  34 facts keep their keys and values. The two under_1 facts' value labels
  change, and layout.record_set_spec_hash changes on the 32 eitc_by_agi
  facts, because that hash covers row labels.
- build-bundle now refuses a dimension value that two packages label
  differently (`_cross_package_value_label_errors`), and a groupby value
  whose row labels differ inside one package, unless every label is listed
  in REVIEWED_VALUE_LABEL_VARIANTS (chronicle/dimension_labels.py). The
  register holds the values the merged bundle warned about at year 2023
  before this change, except under_1: 15 across packages and 23 inside one
  package, each with every label it carries. They stay warnings.
- The source-package harness guide states the rule; changelog fragment.

The tests from the previous commit pass (47 passed in test_issue_376,
test_issue_265 and test_issue_261). Applying this commit's label and
geography-name checks to the facts of all 405 default-bundle packages built
at year 2023 gives 0 errors and 120 warnings (23 within-package and 15
cross-package value labels, 82 geography names); with the unchanged
duplicate-semantic-key warning that is the merged-bundle test's 121. The
value-label warnings are exactly the register's 38 keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk and others added 2 commits October 22, 2026 00:34
Tests only. They fail until the next commit.

The design changes from a register of accepted label variants to one
label per (dimension id, value id), declared in Chronicle's register
chronicle/dimension_values/canonical_labels.json, with each package's own
wording kept in a new field, as chronicle#281 did for geography names.

- test_issue_376_one_label_per_value.py (rewritten):
  - the register holds the three US labels the maintainer chose:
    adjusted gross income under_1 "No adjusted gross income", all "All
    returns, total", and age 85_plus "Age 85 plus";
  - REVIEWED_VALUE_LABEL_VARIANTS and the warning path are gone;
  - built at 2024, week 47's under_1 facts carry the register's label in
    dimension_value_labels and layout.groupby_value_label, and the IRS
    wording in publisher_dimension_value_labels; the package's row label
    is the IRS text again, still checked by its column guard;
  - the 34 week-47 fact keys equal main's, and the two under_1 facts'
    record_set_spec_hash is main's again;
  - an exported row with the new field loads through
    build_consumer_artifact and load_consumer_artifact with its key;
  - packages built at different years (week 47 at 2024 with Table 1.4 at
    2023; ACS S0101 at 2024 with B01001 female at 2023) give each value
    one label;
  - a value two packages label differently is an error with no exception,
    and the message names the register and the new field;
  - a value one package labels two ways is refused by validate-package,
    build-suite and build-bundle, and a package declaration settles it.
- Consumer contract tests: rows are emitted as chronicle.consumer_fact.v5,
  v4 plus the optional publisher_dimension_value_labels; the v5 schema's
  sha256 is pinned.
- test_issue_265, test_issue_261 and the merged-bundle pins in
  test_chronicle_bundle.py: no value-label warnings remain (121 warnings
  become 83) and no value-label error is expected.

On this commit the six changed files give 27 failed, 136 passed: the
register, canonical_value_label, the new fact field and the v5 schema do
not exist yet; the week-47 row label and record-set hash are still the
previous commit's; the S0101 and B01001 age values still differ; the old
exception list still exists; validate-package and build-suite have no
value-label check; and rows are still v4. The keys assertion passes, as it
should. The merged-bundle test was not run locally.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replaces the register of accepted label variants with one label per
(dimension id, value id), as chronicle#281 did for geography names.

- chronicle/dimension_values/canonical_labels.json is the register: 4,824
  values in 147 dimensions. It lists every value that two or more
  default-bundle packages emit (4,801) and every value its packages word
  more than one way (43, of which 23 only one package emits). It was
  generated from every default-bundle package built at its own year: the
  405 that build at 2023, and the ten that do not, at the year each
  validates. Values with one wording keep it; the 43 take the label in
  the PR's table. The three US labels are the maintainer's: adjusted gross
  income under_1 "No adjusted gross income", all "All returns, total", age
  85_plus "Age 85 plus". The other 40 are proposals for review.
- label_facts gives a listed value the register's label in
  dimension_value_labels and layout.groupby_value_label, at every build
  year, and keeps the package's own wording (its row label, else its
  declaration, else its source-row text) in the new
  AggregateFact.publisher_dimension_value_labels where it differs. A value
  the register does not list takes its package's label as before.
- REVIEWED_VALUE_LABEL_VARIANTS, is_reviewed_value_label_variant and
  _cross_package_value_label_warnings are gone. A value labelled two ways
  is an error: validate-package reports conflicting_value_label for a
  groupby value the package's rows word two ways (from the package file,
  at any year), build-suite reports it for any value the facts label two
  ways, and build-bundle reports it inside a package and across packages.
  Each message names the register and the package declaration as the two
  places to fix it.
- consumer_fact.v5: v4 plus the optional top-level
  publisher_dimension_value_labels, in docs/schemas and the package, with
  CONSUMER_FACT_ROW_CONTRACTS, the wheel smoke check and the checked-in
  sample rows updated. The validator's recomputed aggregate key never reads
  labels, so keys do not move; a test exports and reloads a row carrying
  the field.
- Packages: week 47's two under_1 rows go back to the IRS header "No
  adjusted gross income and deficit", which their facts now carry as the
  package's wording, so their record_set_spec_hash is main's again. The
  rows for age 85_plus in Census PEP national and adjusted gross income
  all in Historic Table 2 take the register's label, because neither file
  has text there and the old labels were Chronicle's own.
- Docs: the harness guide, the README and the changelog fragment describe
  the rule; stale docstrings that described warnings are rewritten.

Test changes in this commit, beyond the previous commit's:
- one record id in the new declaration test was wrong;
- the schema test's v2 helper drops the new field;
- two test_issue_261 tests used values the register now labels: one
  switches to an unlisted value id, and one no longer expects missing value
  labels for a package whose values other ISC packages emit;
- test_chronicle_core pins the re-pinned sample's sha256;
- the SLC test in test_chronicle_uk_national_2025 expects one label and the
  footnote typo in publisher_dimension_value_labels, not a warning.

The six files from the previous commit give 163 passed. The related test
files (bundle except the merged-bundle test, source packages, consumer
contract, schema, boundaries, source cells, imports, governance, core,
geography names, SOI targets, the label and NZ files) give 4,197 passed,
1 deselected. Applying the register to the stored consumer rows of all 415
default-bundle packages, each built at a year it validates, and running
the bundle's value label checks over their union gives 0 errors and 0
warnings. ruff check passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk anth-volk changed the title Label week-47 AGI under_1 as the other IRS SOI packages do, and refuse a value labelled two ways Give each dimension value one Chronicle-declared label, keep the publisher's wording beside it (consumer_fact.v5) Oct 21, 2026
anth-volk and others added 8 commits October 22, 2026 01:44
Tests only; they fail at e29da32. From the re-reviews of e29da32
(ledger-source-fidelity FAIL, ledger-contract and ledger-boundary PASS
with findings):

- The new fact field holds each package's own wording, which is often
  text Chronicle wrote (Statbel "65_plus", B01001 "Female 20 years" and
  "female", Eurostat "First income decile cut-point", the four Belgian
  country-total phrases, eurostat-nasa-10-nf-tr "... paid-direction cell",
  Groeipakket "Social supplement child caseload", NTS0601 "Age 30 to 39").
  The tests expect it named package_dimension_value_labels and described
  in the v5 schema as unchecked text Chronicle may have written, and the
  Eurostat groupby key (d1, not the filter's D1) described.
- ons.coicop=coicop_02 takes ONS's "Alcohol, tobacco and narcotics"; a
  source test checks ADFL = ADFM + ADFN + MNC2 in cpnsa.xlsx (it passes
  now, as a statement about the source).
- validate-package refuses a dimension_value_labels declaration the
  register contradicts, and the 13 overridden declarations in six
  packages agree with the register.
- Each one-label error gives only the remedy that applies: a register
  entry, with the rule for choosing its label, for two packages; a
  declaration for one package; the register's label when it lists the
  value.
- scripts/build_dimension_value_register.py regenerates the register and
  its --static-check compares it with the package files; the register
  records what it was built from.
- The harness doc says the guards check expected_row_header and
  expected_column_header, which are fields separate from label:.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#376)

Fixes the re-review findings on e29da32:

- publisher_dimension_value_labels becomes package_dimension_value_labels.
  It held each package's own wording (row label, else declaration, else
  source-row text), which is often text Chronicle wrote, while the v5
  schema called it "the publisher's row or column text". Filling it only
  with guard-checked publisher text is not possible with the existing
  guards: B01001's guard compares the row label with a cell its parser
  builds from Chronicle's own constants, and Statbel's and the Belgian
  country-total guards compare text in extracts Chronicle prepared. The v5
  schema, code comments, README, changelog and harness doc now say the
  field is the package's text, not checked against the publisher's file,
  often Chronicle's, with the publisher's cells reached through lineage;
  the schema also says how a groupby value whose axis is a filter with
  another value is keyed (Eurostat quant_inc: d1, not D1). v5 schema
  sha256: acd33a091cd8d192be5d264e3e94c13e00afb6fd1972e57d4576ac55a54f6970
  (was c4d2579c...). The sample rows' sha256 is re-pinned.
- ons.coicop=coicop_02 takes ONS's overview wording "Alcohol, tobacco and
  narcotics": the total (ADFL) includes narcotics (MNC2).
- validate-package refuses a dimension_value_labels declaration the
  register contradicts (declared_value_label_differs_from_register). The
  13 declarations in six packages that the register silently overrode now
  equal its labels: NRS mid-year estimates and PCON24 and NISRA sex,
  ONS HPSSA and PIPR property_type, DfT NTS0601 age_band. No exported label
  changes; those facts no longer carry the old declaration text in the
  package-wording field.
- Each one-label error gives only the remedy that applies: two packages,
  a register entry with the rule for choosing its label and the static
  check to run; one package, a declaration; a value the register lists,
  its label and a rebuild.
- scripts/build_dimension_value_register.py regenerates the register
  (building every default-bundle package at its build year, or reading
  stored builds) with --check, and --static-check compares it with the
  package files only. Run over the stored main builds the register was
  made from, it reproduces all 4,824 entries; only coicop_02 differs, by
  design. tests/test_dimension_value_register.py runs the static check in
  CI. The register records its sources: the script, the commit and years
  the packages were built at, and the chosen labels.
- The harness doc says the guards compare expected_row_header and
  expected_column_header, fields separate from a row's label:.

Test changed after the red commit: the NTS0601 example in
test_text_chronicle_wrote_is_kept_as_the_packages_wording_not_the_publishers
is replaced by test_nts0601_age_bands_now_carry_dfts_header_and_no_package_wording,
because its "Age 30 to 39" came from the declaration this commit corrects.
Two new files were also formatted with ruff format.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`test_eurostat_prior_vintage_facts_remain_byte_stable` hashes every
`AggregateFact` field of `eurostat-gov-10a-taxag` and `eurostat-spr-exp-func`.
The new `package_dimension_value_labels` field is an empty mapping on every
fact of both packages, and `asdict` includes it, so the digests moved
(CI run 39685539423, "Chronicle surface"). With that key removed, the facts
hash to the old digests exactly.

The test now pins the new full-fact digests (516a28c5..., e0231b78...) and
also checks the reason: the field is `{}` on every fact, and with it removed
the facts still hash to 89b50ea0... and b57eca7e.... #263 recorded the same
kind of strip-and-compare result only in the comment; here it is asserted.

One commit, not a red and a green: the code is already correct and only the
pinned constants were stale. A test-only commit with the new assertions and the
old pins fails exactly as the PR head does, on the full-fact digest, while the
strip-and-compare assertions already pass at `d24819c`; no code change turns
it green, only the re-pin in this file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two Low findings from the re-check of `d24819c`:

- N2: the error for a value two packages label differently tells the author
  to add a register entry and run `--static-check`, but not to add the chosen
  label to `CHOSEN_LABELS` in `scripts/build_dimension_value_register.py`.
  Such a value always has two wordings, so without that entry the script no
  longer reproduces the register. The message test now asserts that step, and
  that a listed value's message does not mention it.
- N3: the `chronicle.dimension_labels` docstring still calls the row label
  "the fact's publisher row label" and the source-row text "the publisher's
  text". A new test asserts it names them as the package's row label and the
  source-row text instead.

Both tests fail here (2 failed) and pass with the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- N2: `cross_package_value_label_remedy` now also tells the author to add the
  chosen label to `CHOSEN_LABELS` in
  `scripts/build_dimension_value_register.py`, so the script still reproduces
  the register, before running `--static-check`. The script path is a new
  constant, `REGISTER_SCRIPT_DISPLAY_PATH`, which `REGISTER_STATIC_CHECK` uses
  too. A listed value's message is unchanged.
- N3: the `chronicle.dimension_labels` docstring's label-source list now names
  item 3 as the package's row label for the fact (the record-set row's
  `label:`) and item 4 as the source-row text as Chronicle's parser read it,
  from the publisher's file or from an extract Chronicle prepared. The local
  variable `publisher_labels` in `label_facts` and `_groupby_value_label`,
  which holds that wording, is renamed `package_wordings`. No label, fact or
  field changes.
- Formatting only: `ruff format` joins one assertion in
  `tests/test_etl_eurostat.py` and one string in
  `tests/test_issue_376_one_label_per_value.py`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #379 (Stats NZ regional estimated resident population at
30 June 2023: 17 facts in `stats-nz-subnational-population-estimates-2025`,
with the merged-bundle count pins in `tests/test_chronicle_bundle.py`). No
textual conflict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

F1 from the review of the fix round: the `_dimension_label_reports`
docstring in `chronicle/bundle.py` still says "publisher row labels that
differ across one groupby value's rows". Those are the record-set rows'
`label:` values, package text that nothing checks against the publisher's
file, the same wording N3 removed from `chronicle.dimension_labels`. The N3
test read only that module's docstring.

A sibling test now asserts that the bundle docstring does not say
"publisher row label" and names the package's row labels. It fails here
(1 failed; the N3 test still passes) and passes with the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
F1: the `_dimension_label_reports` docstring now says "the package's row
labels (its record-set rows' `label:`) that differ across one groupby
value's rows" instead of "publisher row labels". Docstring only; no message,
label, fact or field changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk anth-volk changed the title Give each dimension value one Chronicle-declared label, keep the publisher's wording beside it (consumer_fact.v5) Give each dimension value one Chronicle-declared label, keep the package's own wording beside it (consumer_fact.v5) Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Week-47 filing-season package labels AGI value under_1 differently from every other IRS SOI package

1 participant