Skip to content

Add household calculation contracts and record country capabilities in v2 manifest schema version 2 - #841

Closed
anth-volk wants to merge 4 commits into
mainfrom
apiv2/1c-household-contracts
Closed

anth-volk wants to merge 4 commits into
mainfrom
apiv2/1c-household-contracts

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #839

Summary

First of two stacked pull requests for the household part of the API v2 migration (Step 1, group 1C). This one adds the household calculation contracts and records country capabilities in the v2 version manifest. #842, stacked on it, adds the household calculation and the Modal household functions. Nothing calls the new household models yet.

Household calculation contracts

New module policyengine_simulation_contract.household_execution:

  • HouseholdDocument is a household in the situation shape the Public API's household calculation routes accept: people, group entity collections keyed by plural (each group lists its members), variable values by period, and optional axes. It is a typed Pydantic model (typed extra keys for the group collections and variables), checks structure, types and size limits, and serializes back to the same JSON, with its top-level keys in the caller's order (the Public API walks a household in the order it was sent, so the order of unrecognized inputs and warnings depends on it). That keeps v2 results directly comparable with the Public API's result. Whether an entity collection, variable or policy parameter exists is decided by the country model inside the household function, which reports unknown names with the Public API's error types and messages.
  • HouseholdExecutionRequest (Simulation Entrypoint to Modal household function): schema_version, the durable calculation_id, country, household, one to ten policies with unique policy_keys (so a baseline and its reform run in one invocation), add_missing_variables (the Public API's /calculate-full behavior), an optional SPM selection, spm_selection_source (request or saved household), and the expected PolicyEngine.py bundle (HouseholdBundleProvenance).
  • HouseholdExecutionResult and HouseholdExecutionFailure, discriminated by status. A result holds one HouseholdPolicyResult per policy (calculated household, warnings, compact SPM receipt), the bundle provenance and the worker identity. A failure holds a typed error: invalid_inputs, invalid_household, invalid_spm_selection, bundle_mismatch or calculation_failed, with the Public API's input-error fields.
  • SubmitHouseholdCalculationRequest is the public submission, with execution_request(...) to build the request for the resolved bundle. The poll responses PendingHouseholdCalculation, RunningHouseholdCalculation, SucceededHouseholdCalculation and FailedHouseholdCalculation are discriminated by status.
  • Saved households: the contract's docstring states how to calculate a saved household as the Public API's stored-household route does: set add_missing_variables (that route always adds missing variables), pass the saved SPM selection as spm, and set spm_selection_source to saved_household.
  • SPM omission versus choice matches the Public API: an omitted spm inherits the bundle's certified defaults and stays omitted through serialization, {} is a choice, and an explicit null is rejected.
  • Limits: 1,000 people and groups per collection (as the Public API's stored v2 households), 10,000 axis points per household (the product of the perpendicular axis groups' point counts), 10 policies per request.

v2 version manifest schema version 2

  • Each country entry records society_wide_capable and household_callable. A society-wide capable country, and only such a country, names its single-simulation callable.
  • The worker validation receipt gains household_validation_invocation_ids. Simulation-worker validation must cover exactly the society-wide countries, and every household function must have been validated.
  • The schema version 2 manifest is written under a new key, active-schema-2, in the existing simulation-api-v2-version-manifest Modal Dict. The same manifest in schema version 1 form, without the new fields, is still written under active-with-cache and active, which revisions released before this change read. Schema version 1 cannot describe a country that is not society-wide capable, so such a manifest is written only under the new key.
  • Readers prefer active-schema-2 and upgrade a schema version 1 document, marking every listed country society-wide capable with no household function. The first publish after this change therefore keeps the earlier versions.
  • The publisher marks every bundle country society-wide capable and records no household function yet (Add v2 Modal household functions that calculate one household under several policies #842 records them once they are validated).
  • docs/migration/stage-12-modal-worker-foundation.md describes the keys.

Report coordinator accepts only society-wide capable countries

The Simulation Entrypoint's Stage 12 comparison adapter skips any country the manifest does not mark as society-wide capable, with the existing unsupported_country reason, so the report coordinator only receives such countries. Every bundle country is society-wide capable today, so current behavior is unchanged.

Deployment effects

Merging to main deploys staging and then production:

  • The publish_stage12_v2_manifest job writes the new active-schema-2 key and keeps writing the two schema version 1 keys.
  • The entrypoint revision deployed with this change reads active-schema-2, falling back to the schema version 1 keys until the first schema version 2 publish.
  • No new Modal functions, no workflow, Dockerfile or deploy-script changes.
  • Rollback note: if an older commit is deployed afterwards, its publisher writes only the schema version 1 keys, and active-schema-2 keeps the last schema version 2 manifest. A newer entrypoint revision would read that older document until the next publish from this code; the older entrypoint revision that a rollback deploys reads the schema version 1 keys and is unaffected.

Configuration and environment variables

None added or changed. No operator step is needed before merging.

Coordination

Group 1B changes the report coordinator (ReportExecutionResult and the coordinator's return mode) in parallel. This pull request does not touch the coordinator runtime; it changes stage12_adapter.py, stage12_manifest.py and the manifest publisher, so a textual conflict is possible only if 1B edits those files.

Tests

  • libs/policyengine-simulation-contract: uv run pytest tests → 235 passed, including the new test_household_execution.py and the updated test_stage12_manifest.py (schema version 1 field sets for older readers, upgrade on read, first schema version 2 publish, household-only country, capability and receipt validation).
  • projects/policyengine-simulation-entry: uv run pytest tests → 538 passed, including new adapter and backend tests for a country that is not society-wide capable.
  • projects/policyengine-simulation-executor: publisher tests and the runtime-evaluated models guard passed.
  • ruff check . and ruff format --check projects/*/src libs/*/src (ruff 0.15.22), and scripts/run_quality_guards.py (rules registry, CI scripts) passed.
  • Not run locally: the Docker build and the local integration tests; CI runs them. No public API schema changed, so no client regeneration.

🤖 Generated with Claude Code

anth-volk and others added 2 commits October 20, 2026 01:37
Add the wire models for v2 household calculations to the shared contract
library:

- HouseholdDocument keeps the situation shape the Public API's household
  routes accept (people, group collections with members, variable values by
  period, optional axes), so results can be compared field by field.
- HouseholdExecutionRequest names one or more policies, so a baseline and
  its reform run in one invocation, plus the expected PolicyEngine.py
  bundle, the calculate-full option and the SPM selection. An omitted SPM
  selection inherits the bundle's defaults and survives serialization; an
  explicit null is rejected.
- HouseholdExecutionResult and HouseholdExecutionFailure, told apart by
  status, carry per-policy calculated households, warnings and SPM receipts,
  or a typed error with the Public API's input-error fields.
- SubmitHouseholdCalculationRequest is the public submission, and the
  pending, running, succeeded and failed poll responses form a discriminated
  union.

Nothing uses these models yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The v2 version manifest now records, for each country a worker application
serves, whether it is society-wide capable (it has a single-simulation
worker) and which function calculates its households.

- Schema version 2 is written under a new active-schema-2 key. The same
  manifest is still written in schema version 1 form under active-with-cache
  and active, for readers released before this change, whenever schema
  version 1 can describe it.
- Readers prefer the schema version 2 key and upgrade a schema version 1
  document, marking every listed country society-wide capable with no
  household function, so the first publish keeps earlier versions.
- The worker validation receipt gains household validation invocation IDs;
  every household function must be validated, and simulation-worker
  validation must cover exactly the society-wide countries.
- The publisher marks every bundle country society-wide capable and records
  no household functions yet.
- The Simulation Entrypoint's comparison adapter skips any country the
  manifest does not mark as society-wide capable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk added this pull request to stack #843 October 19, 2026 22:02
@anth-volk anth-volk changed the title apiv2/1c household contracts Add household calculation contracts and record country capabilities in v2 manifest schema version 2 Oct 19, 2026
anth-volk and others added 2 commits October 20, 2026 17:19
Pydantic dumps declared fields before extra ones, so a household document
always came back with `axes` first and `people` before every group
collection, whatever order the caller sent. The Public API walks a
household in the order it was sent, so code that walks the serialized
situation (the unrecognized-input checks, deprecated-input warnings and
requested values) listed its results in a different order. The situation
models now record the caller's top-level key order and serialize in it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Public API's stored-household route always adds missing variables and
treats a saved SPM selection as chosen, while the contract ties
`add_missing_variables` only to `/calculate-full`. State which fields a
saved-household replay sets, so callers built later match that route.

Documentation only: the change is a docstring, so no test can fail first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk anth-volk closed this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Household calculation contracts and country capabilities in the v2 version manifest

1 participant