Skip to content

Accept Chronicle consumer_fact.v5 facts #1509

Closed

Description

@anth-volk

Problem

PolicyEngine/chronicle#377 merged on 2026-11-03 as 44eebe037. Since then Chronicle main writes every consumer fact as chronicle.consumer_fact.v5: the v4 row plus one optional top-level field, package_dimension_value_labels (dimension id, then value id, then text). That field holds a source package's own wording of a dimension value, and only where it differs from the label the row exports. dimension_value_labels and layout.groupby_value_label now carry the one label Chronicle registers for each value. Keys, values, constraints, periods and lineage do not change. v4's row schema has additionalProperties: false, so the new field needed a new contract version.

Microcosm declares Chronicle's consumer-fact row contracts only up to v4:

  • microcosm.build.chronicle_epoch has CHRONICLE_CONSUMER_FACT_SCHEMA_VERSION_V4 as its newest row contract, so ACCEPTED_CONSUMER_FACT_SCHEMA_VERSIONS and consumer_fact_schema_epoch do not know v5. Chronicle's own table (CONSUMER_FACT_ROW_CONTRACTS in chronicle/epoch.py) now ends at v5.
  • test_chronicle_epoch.py pins the accepted set at v1 to v4, and the shared artifact helper (test_support/microcosm_build/chronicle_feed.py) writes only v4 rows.
  • No Microcosm code checks package_dimension_value_labels.

The loader carries each row's schema_version without checking it, and validate_chronicle_feed_pin compares the pinned versions and schema digest exactly with the artifact. Review round 1 of #1510 found that no code outside the tests reads the declared list, so a v5 feed would already load and the declaration on its own refuses nothing. The maintainer decided to enforce the list: a Chronicle feed pinned on a row contract Microcosm does not declare is refused. With that refusal, the next re-pin of any country's feed (UK, US, NZ or BE) onto Chronicle main needs v5 declared first. The UK feed has been re-pinned four times since 2026-10-24 for new packages.

Change

  • Declare chronicle.consumer_fact.v5 beside v1 to v4. v1 to v4 keep loading.
  • Refuse a feed pin, or a pinned artifact, that names a row contract the declaration table lacks: in the UK and US pin loaders, chronicle_feed_identity and validate_chronicle_feed_pin. A feed loaded without a full pin keeps carrying its per-row ids unchecked.
  • Check the shape of package_dimension_value_labels where Microcosm checks dimension_value_labels: when a target is compiled from a fact, the field, if present, must map each dimension id to an object of value id to non-empty text, as the v5 schema says. Microcosm never reads the field as a label: each value keeps the label Chronicle exports.
  • Tests use a v5 fact fixture validated against the real v5 schema, taken from Chronicle at 44eebe037.

No feed pin moves in this change. Re-pinning the UK feed onto Chronicle main is a separate pull request.

Follow-up

Once the US and Belgian feeds are on v5, the reviewed label choices for those countries in microcosm.calibrate.hierarchy_labels may no longer be needed. This change leaves them alone.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions