Skip to content

Add a typed report result and a result delivery field for the v2 report coordinator - #836

Closed
anth-volk wants to merge 2 commits into
mainfrom
apiv2/1b-report-execution-result
Closed

anth-volk wants to merge 2 commits into
mainfrom
apiv2/1b-report-execution-result

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #834

Same-Repository Draft PR Check

  • This PR is a draft.
  • This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

First of two stacked PRs that let the v2 Modal report coordinator return a typed result to its caller instead of writing Stage 12 rows (API v2 migration, Stage 14 economy reports). This PR adds the types and the selecting field. #837, stacked on it, adds the return mode that uses them.

  • ReportExecutionResult (new policyengine_simulation_contract.report_execution): the result a coordinator returns when the report succeeds.
    • aggregate: the report's aggregate inline, validated as the existing SingleYearMacroOutput.
    • aggregate_sha256: SHA-256 of the aggregate's compact, key-sorted JSON (report_aggregate_sha256). The digest is the same before and after a JSON round trip, and NaN or infinity is rejected.
    • output_plan: the Stage12OutputPlan that declares the columns of the simulation artifacts.
    • baseline and reform: the existing SimulationArtifactDescriptor for each Parquet artifact (storage URI, media type, checksum, size, row identity, bundle, SPM receipt).
    • provenance: the PolicyEngine.py bundle, worker version, Modal environment, application and simulation function, version manifest digest, and the coordinator's Modal call ID.
    • The validator rejects artifacts that do not belong to the report, have the wrong roles, use another bundle or output plan, cover different rows, or share a simulation ID, a digest that does not match the aggregate, and requested aggregates that differ from the ones the output plan was built for.
  • ReportExecutionFailure and ReportExecutionError: the redacted error code and summary that a Stage 12 row would record for the same failure. The error's message is fixed. It pickles with its failure, so Modal's FunctionCall.get can raise it in the caller as the same type.
  • Stage12InvocationContext.result_delivery: stage12_database (the default, today's behavior) or return_to_caller. A return_to_caller context cannot name a production comparison job, because the comparison is recorded on the Stage 12 report row.
  • Stage 12 child context unchanged. The Stage 12 coordinator now leaves result_delivery out of the context it sends to its baseline and reform workers. The worker payload has exactly the fields it had before, so a worker deployed without the field still accepts it.

Apart from the Stage 12 coordinator, which reads it only to leave it out of the context it sends to its workers, nothing reads result_delivery in this PR.

What a merge deploys

Merging to main runs the complete staging and then production deployment, as every merge does: the existing simulation stack, the entrypoint Cloud Run service and the versioned v2 Modal application. The contract library is installed in all of them. Behavior does not change: the entrypoint does not send result_delivery, so every context defaults to stage12_database, and the payloads the coordinator sends to its workers are unchanged.

Environment variables

This PR adds, removes or rewires no environment variable, workflow input, GitHub secret, Secret Manager secret or Modal secret.

Coordination with other PRs

Testing

  • New libs/policyengine-simulation-contract/tests/test_report_execution.py (26 tests): the default and return_to_caller delivery, rejection of a production comparison and of an unknown delivery, JSON round trip, aggregate validation against SingleYearMacroOutput, digest mismatch and non-finite values, digest stability (key order, SPM provenance aliases), each artifact consistency check, timezone and unique-aggregate checks, requested aggregates that must match the output plan, unknown fields, failure field bounds, and the error surviving pickling.
  • New test_stage12_children_receive_the_context_without_a_delivery_field in the executor's test_stage12_runtime.py: Stage 12 workers receive the same context fields as before.
  • Contract: 221 passed. Executor: 1,315 passed, 22 skipped (full suite, run on the top of the stack). Entrypoint: 534 passed. Gateway: 243 passed. Stage 12 persistence: 19 passed.
  • ruff check . and ruff format --check projects/*/src libs/*/src with the pinned ruff 0.15.22 pass. scripts/run_quality_guards.py passes.
  • Generated clients are unchanged: no entrypoint or gateway API schema changed.

🤖 Generated with Claude Code

…ontext

Add a typed outcome for a report coordinator that returns its result to
the caller instead of writing Stage 12 rows:

- ReportExecutionResult carries the aggregate inline, validated as
  SingleYearMacroOutput, with its canonical JSON SHA-256, the output plan
  that declares the artifacts' columns, the baseline and reform simulation
  artifact descriptors, and provenance (bundle, worker version, Modal
  environment, application, simulation function, version manifest digest
  and coordinator call ID). Its validator rejects artifacts that do not
  form the report and a digest that does not match the aggregate.
- ReportExecutionFailure and ReportExecutionError carry the redacted error
  code and summary a Stage 12 row would record. The error pickles with its
  failure, so Modal can raise it in the caller.
- Stage12InvocationContext gains result_delivery. It defaults to
  stage12_database, so existing callers keep the Stage 12 behavior; a
  return_to_caller context cannot request a production comparison.

The Stage 12 coordinator leaves the new field out of the context it sends
to its child simulations, so the child payload keeps exactly the fields it
had before and a worker deployed without the field still accepts it.
Nothing else reads the field yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk added this pull request to stack #838 October 19, 2026 21:51
@anth-volk anth-volk changed the title Add ReportExecutionResult and a result delivery field to the report context Add a typed report result and a result delivery field for the v2 report coordinator Oct 19, 2026
…put plan

ReportExecutionResult carries the requested aggregates twice: in
requested_aggregates and in output_plan.requirements.aggregates, which the
coordinator builds from the same report. The validator accepted a result
in which the two disagreed, so a caller storing requested_aggregates could
keep a value the output plan contradicts. The validator now requires them
to be equal, in the same order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a typed report result and a result delivery field for the v2 report coordinator

1 participant