[anth-volk]

Fixes #918

Merge requirement: don't merge this until PolicyEngine/policyengine-app-v2#1436 is deployed. The app on main still copies x and y from sim-api's constituency and local-authority rows (keyAreaRowsByName in societyWideOutputNormalizer.ts), and a hex map whose layout lacks any of a result's areas places them all by x and y. PolicyEngine/policyengine-app-v2#1436 removes every read of x and y. This requirement first named PolicyEngine/policyengine-app-v2#1433, whose commits merged as part of PolicyEngine/policyengine-app-v2#1448.

Same-Repository Draft PR Check

• This PR is a draft.
• This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

New results stop carrying x and y (each area's integer map position) on UK constituency and local-authority records, and the executor stops shipping coordinates_lad22.csv and coordinates_lad23.csv. The contract keeps accepting x and y, because apps deployed for older .py versions are never rebuilt and keep sending them.

Three commits, each with one behavioural change, and a fourth that rewords one sentence of the migration doc:

1. Accept UK area records without x and y. x and y on ConstituencyImpactRecord and LocalAuthorityImpactRecord default to None and use pydantic's field-level exclude_if, so a record without them validates and every dump leaves the keys out instead of writing null. Records with integer x and y still validate and keep them. The contract's pydantic floor rises to 2.12, the first release with exclude_if; the contract and the four packages whose locks record its metadata are relocked, and only that specifier line changes in each lock. The gateway's golden OpenAPI no longer lists x and y as required.
2. Stop sending x and y on UK area records. The executor reads only the code and name columns of .py's constituency lookup, attaches only names to local authorities, and no longer refuses an area without a map position (the "missing lookup coordinates" check). The beta integration check of a local-area breakdown now requires records without x or y.
3. Ship local-authority code lists instead of coordinate files. codes_lad22.csv and codes_lad23.csv (one code column) replace the coordinate files, with exactly the same codes in the same order. UKLocalAuthorityResources carries codes per boundary version and names_for(...) replaces metadata_for(...). The Stage 12 static runtime file smoke reports lad22_codes and lad23_codes.

How the JSON loses the keys

• The executor returns output.model_dump(mode="json") from SimulationOutputBuilder.serialize and LocalAreaBreakdownBuilder.serialize; Stage 12 stores the same dump as its aggregate.
• The gateway validates that dict into JobStatusResponse and /jobs/{job_id} serializes it with response_model_exclude_none=True, which already drops null fields.
• With exclude_if, the dump itself has no x or y, so neither the Modal result, the Stage 12 aggregate nor the gateway JSON carries the keys. The gateway tests check the response JSON for a breakdown and a national result whose records omit x and y, and for records that send them as null.

Boundary-version detection

detect_uk_local_authority_boundary_version reads the names file and the two hard-coded distinguishing code sets, which this PR doesn't change. Each version's full code list, which names_for and the "not part of the detected boundary version" check use, now comes from the code files. The load-time checks stay: the names must exactly cover both lists, and each version's distinguishing codes must match its list. A test pins each list to a SHA-256 of the codes the coordinate files listed (374 for LAD22, 361 for LAD23), and new tests check the loader refuses a repeated code, a list with another version's distinguishing code, and a list with x and y columns.

Deployment code and environment variables

The Modal images change content only: add_static_runtime_files copies the whole static_runtime_files directory, which now holds names.csv and the two code lists. src/modal/v2_static_runtime_files_smoke.py changes only the keys it reports. No environment variable is added, removed or read differently. For reference, the smoke's wiring is unchanged:

• MODAL_TOKEN_ID and MODAL_TOKEN_SECRET: GitHub Actions secrets, set in the Run image smokes step's env: in pr-image-smoke.yml.
• PE_UK_PRIVATE_HF_READ_TOKEN: GitHub Actions secret, set in the same step; .github/scripts/modal-image-smoke.sh writes it to the Modal secret pe-uk-private-hf-read-token as HUGGING_FACE_TOKEN, which the smoke function attaches through hf_secret (required_keys=["HUGGING_FACE_TOKEN"]).

Compatibility

Every .py version the gateway already routes to keeps its app and keeps sending x and y; the contract accepts both. Removing the fields from the contract is the stacked PR above this one, which must not merge until no routed version still sends them.

Testing

Written red-green: for each commit the tests were written first and seen failing for the expected reason, then the code was written and both were committed together.

• Commit 1: contract tests failed on Field required for x and y and on a null x and y in the dump; the gateway returned 500 for records without x and y; the generated client did not read such a breakdown as a LocalAreaBreakdownOutput.
• Commit 2: executor tests failed on "missing lookup coordinates" and on x in the constituency and local-authority records; the breakdown check's new cases failed with DID NOT RAISE.
• Commit 3: tests failed because the resources had no codes or names_for, the package still held the coordinate files, and the smoke reported the coordinate keys.

Commands run (focused, locally):

• uv run --frozen pytest on test_macro_output_uk_geography.py, test_local_area_breakdown.py and test_gateway_models.py (contract); test_endpoints.py and test_openapi_golden.py (gateway); test_openapi.py (entry); test_simulation_output_geographic.py, test_simulation_output_builder.py, test_uk_local_authority_metadata.py, test_stage12_modal_app.py, test_smoke_app.py, test_stage12_runtime.py, test_stage12_output_planning.py, test_stage12_worker_validation.py and test_modal_bundle_image.py (executor).
• Golden OpenAPI regenerated with python -m policyengine_simulation_gateway.generate_openapi; the diff removes x and y from both records' required lists and updates the constituency description.
• ./scripts/generate-clients.sh, then uv run --frozen pytest tests/ -m "not requires_gcp and not beta_only" in policyengine-apis-integ: everything passed except test_ping, which needs a gateway running on localhost.
• make check (ruff and Pyright in every package), python -m scripts.rules pyright --sync projects/policyengine-simulation-executor, python scripts/run_quality_guards.py with GITHUB_BASE_REF=main, python -m scripts.rules render (no changes) and .github/scripts/check-uv-locks.sh.
• Not run locally: the full test suites, the Modal image smoke and the beta integration tests, which CI and the deployment run.

🤖 Generated with Claude Code