Repository navigation
Add household calculation contracts and record country capabilities in v2 manifest schema version 2 - #841
Closed
anth-volk wants to merge 4 commits into
Closed
Add household calculation contracts and record country capabilities in v2 manifest schema version 2#841anth-volk wants to merge 4 commits into
anth-volk wants to merge 4 commits into
Conversation
Add the wire models for v2 household calculations to the shared contract library: - HouseholdDocument keeps the situation shape the Public API's household routes accept (people, group collections with members, variable values by period, optional axes), so results can be compared field by field. - HouseholdExecutionRequest names one or more policies, so a baseline and its reform run in one invocation, plus the expected PolicyEngine.py bundle, the calculate-full option and the SPM selection. An omitted SPM selection inherits the bundle's defaults and survives serialization; an explicit null is rejected. - HouseholdExecutionResult and HouseholdExecutionFailure, told apart by status, carry per-policy calculated households, warnings and SPM receipts, or a typed error with the Public API's input-error fields. - SubmitHouseholdCalculationRequest is the public submission, and the pending, running, succeeded and failed poll responses form a discriminated union. Nothing uses these models yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The v2 version manifest now records, for each country a worker application serves, whether it is society-wide capable (it has a single-simulation worker) and which function calculates its households. - Schema version 2 is written under a new active-schema-2 key. The same manifest is still written in schema version 1 form under active-with-cache and active, for readers released before this change, whenever schema version 1 can describe it. - Readers prefer the schema version 2 key and upgrade a schema version 1 document, marking every listed country society-wide capable with no household function, so the first publish keeps earlier versions. - The worker validation receipt gains household validation invocation IDs; every household function must be validated, and simulation-worker validation must cover exactly the society-wide countries. - The publisher marks every bundle country society-wide capable and records no household functions yet. - The Simulation Entrypoint's comparison adapter skips any country the manifest does not mark as society-wide capable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
added this pull request to stack #843
October 19, 2026 22:02
Pydantic dumps declared fields before extra ones, so a household document always came back with `axes` first and `people` before every group collection, whatever order the caller sent. The Public API walks a household in the order it was sent, so code that walks the serialized situation (the unrecognized-input checks, deprecated-input warnings and requested values) listed its results in a different order. The situation models now record the caller's top-level key order and serialize in it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Public API's stored-household route always adds missing variables and treats a saved SPM selection as chosen, while the contract ties `add_missing_variables` only to `/calculate-full`. State which fields a saved-household replay sets, so callers built later match that route. Documentation only: the change is a docstring, so no test can fail first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #839
Summary
First of two stacked pull requests for the household part of the API v2 migration (Step 1, group 1C). This one adds the household calculation contracts and records country capabilities in the v2 version manifest. #842, stacked on it, adds the household calculation and the Modal household functions. Nothing calls the new household models yet.
Household calculation contracts
New module
policyengine_simulation_contract.household_execution:HouseholdDocumentis a household in the situation shape the Public API's household calculation routes accept:people, group entity collections keyed by plural (each group lists itsmembers), variable values by period, and optionalaxes. It is a typed Pydantic model (typed extra keys for the group collections and variables), checks structure, types and size limits, and serializes back to the same JSON, with its top-level keys in the caller's order (the Public API walks a household in the order it was sent, so the order of unrecognized inputs and warnings depends on it). That keeps v2 results directly comparable with the Public API'sresult. Whether an entity collection, variable or policy parameter exists is decided by the country model inside the household function, which reports unknown names with the Public API's error types and messages.HouseholdExecutionRequest(Simulation Entrypoint to Modal household function):schema_version, the durablecalculation_id, country, household, one to ten policies with uniquepolicy_keys (so a baseline and its reform run in one invocation),add_missing_variables(the Public API's/calculate-fullbehavior), an optional SPM selection,spm_selection_source(request or saved household), and the expected PolicyEngine.py bundle (HouseholdBundleProvenance).HouseholdExecutionResultandHouseholdExecutionFailure, discriminated bystatus. A result holds oneHouseholdPolicyResultper policy (calculated household, warnings, compact SPM receipt), the bundle provenance and the worker identity. A failure holds a typed error:invalid_inputs,invalid_household,invalid_spm_selection,bundle_mismatchorcalculation_failed, with the Public API's input-error fields.SubmitHouseholdCalculationRequestis the public submission, withexecution_request(...)to build the request for the resolved bundle. The poll responsesPendingHouseholdCalculation,RunningHouseholdCalculation,SucceededHouseholdCalculationandFailedHouseholdCalculationare discriminated bystatus.add_missing_variables(that route always adds missing variables), pass the saved SPM selection asspm, and setspm_selection_sourcetosaved_household.spminherits the bundle's certified defaults and stays omitted through serialization,{}is a choice, and an explicitnullis rejected.v2 version manifest schema version 2
society_wide_capableandhousehold_callable. A society-wide capable country, and only such a country, names its single-simulation callable.household_validation_invocation_ids. Simulation-worker validation must cover exactly the society-wide countries, and every household function must have been validated.active-schema-2, in the existingsimulation-api-v2-version-manifestModal Dict. The same manifest in schema version 1 form, without the new fields, is still written underactive-with-cacheandactive, which revisions released before this change read. Schema version 1 cannot describe a country that is not society-wide capable, so such a manifest is written only under the new key.active-schema-2and upgrade a schema version 1 document, marking every listed country society-wide capable with no household function. The first publish after this change therefore keeps the earlier versions.docs/migration/stage-12-modal-worker-foundation.mddescribes the keys.Report coordinator accepts only society-wide capable countries
The Simulation Entrypoint's Stage 12 comparison adapter skips any country the manifest does not mark as society-wide capable, with the existing
unsupported_countryreason, so the report coordinator only receives such countries. Every bundle country is society-wide capable today, so current behavior is unchanged.Deployment effects
Merging to
maindeploys staging and then production:publish_stage12_v2_manifestjob writes the newactive-schema-2key and keeps writing the two schema version 1 keys.active-schema-2, falling back to the schema version 1 keys until the first schema version 2 publish.active-schema-2keeps the last schema version 2 manifest. A newer entrypoint revision would read that older document until the next publish from this code; the older entrypoint revision that a rollback deploys reads the schema version 1 keys and is unaffected.Configuration and environment variables
None added or changed. No operator step is needed before merging.
Coordination
Group 1B changes the report coordinator (
ReportExecutionResultand the coordinator's return mode) in parallel. This pull request does not touch the coordinator runtime; it changesstage12_adapter.py,stage12_manifest.pyand the manifest publisher, so a textual conflict is possible only if 1B edits those files.Tests
libs/policyengine-simulation-contract:uv run pytest tests→ 235 passed, including the newtest_household_execution.pyand the updatedtest_stage12_manifest.py(schema version 1 field sets for older readers, upgrade on read, first schema version 2 publish, household-only country, capability and receipt validation).projects/policyengine-simulation-entry:uv run pytest tests→ 538 passed, including new adapter and backend tests for a country that is not society-wide capable.projects/policyengine-simulation-executor: publisher tests and the runtime-evaluated models guard passed.ruff check .andruff format --check projects/*/src libs/*/src(ruff 0.15.22), andscripts/run_quality_guards.py(rules registry, CI scripts) passed.🤖 Generated with Claude Code