Repository navigation
Record when each policyengine.py route was published and latest changed - #961
Conversation
The routing state gains two optional fields: published_at (when each policyengine.py version's route was first published) and latest_since (when latest.policyengine last changed: a publish that advances it, a forced latest or a rollback). A date is recorded only when the publish is that event, so states from before this change get no wrong, later date for what they already route. The gateway's /versions returns both; the golden OpenAPI and the deploy smoke fixture are regenerated. Fixes #960 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API's GET /deployment (PolicyEngine/policyengine-api#4084) reports policyengine_version_live_since, and the gateway's /versions (PolicyEngine/policyengine-sim-api#961) reports latest_since. The page reads both when present; an API without the route, or a gateway without the field, shows no date as before. A missing gateway worker keeps the API's own date. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
left a comment
There was a problem hiding this comment.
Code-quality notes from a review of this PR; the two inline comments below have the details.
- Test coverage: the date handling added to the merge with the older per-kind version Dicts has no test, and no test runs the path deployments take (
plan_routing_publication, then applying the plan) with dates. - Typing:
published_atandlatest_sinceare plain strings, and nothing checks that they hold the ISO 8601 UTC times the contract describes.
| merged["latest"][kind] = current_latest | ||
|
|
||
| merged["bundles"].update(current["bundles"]) | ||
| if "published_at" in current: |
There was a problem hiding this comment.
No test covers this: nothing merges the older per-kind version Dicts into a current state that already has published_at and latest_since and checks that both survive. A test alongside the ones in test_routing_dates would cover it.
Related: the new tests call build_next_routing_state directly. No test runs plan_routing_publication and then applies the plan with dates, which is the path the deploy workflow uses (the date is taken when the plan is computed).
There was a problem hiding this comment.
Fixed in a93c3c0: test_merging_the_legacy_seed_keeps_the_current_dates merges legacy per-kind Dicts into a dated current state and checks that both dates survive. test_the_deploy_path_records_the_dates_it_plans runs plan_routing_publication, record_routing_plan and apply_routing_plan and checks the live state's dates.
| "wrapper version: read its identity at `bundles[latest]`." | ||
| ), | ||
| ) | ||
| published_at: dict[str, str] = Field( |
There was a problem hiding this comment.
published_at and latest_since are plain strings here and in RoutingState. Neither coerce_routing_state nor the gateway's routing_state_published_at / routing_state_latest_since checks the ISO 8601 UTC format these descriptions promise, so any non-empty string in the routing state reaches clients. Typing them as AwareDatetime (dict[str, AwareDatetime] and AwareDatetime | None) would make the response enforce it.
There was a problem hiding this comment.
Fixed in a93c3c0: VersionsResponse types them as dict[str, AwareDatetime] and AwareDatetime | None (golden OpenAPI now says date-time). The gateway parses the stored text and drops anything that is not an ISO 8601 time with a timezone, and coerce_routing_state keeps only such times, normalized to UTC. The state itself stays JSON text in the Modal Dict. Tests cover malformed and timezone-less values in the state and in /versions.
# Conflicts: # changelog_entry.yaml
Review follow-up on #961: - The routing state keeps a date only when it is an ISO 8601 time with a timezone, normalized to UTC; the gateway parses the stored text and /versions types published_at and latest_since as AwareDatetime, so the response enforces the format (golden OpenAPI: format date-time). - Tests: the deploy path (plan_routing_publication, record, apply) records the dates it planned; merging the legacy per-kind Dicts keeps the current state's dates; malformed or timezone-less dates are dropped in the state and in /versions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@anth-volk thanks — both points are addressed in a93c3c0 (replies inline): tests for the legacy-seed merge and for the deploy path (plan → record → apply), and the dates are now I also merged current |
anth-volk
left a comment
There was a problem hiding this comment.
Thanks, both points are addressed in a93c3c0: the new tests cover the legacy-seed merge and the plan → record → apply path, and the dates are now timezone-aware throughout.
Before merging: changelog_entry.yaml conflicts with main again (it's the only conflicting file), so it needs another merge from main.
One minor, optional point: the publisher normalizes dates to UTC, but the gateway's _aware_datetime keeps whatever offset is stored, and test_versions_reports_only_dates_with_a_timezone expects +02:00 back, while the field descriptions say "ISO 8601 UTC time". Only the publisher writes the state, so /versions shows UTC in practice; converting with .astimezone(UTC) in the gateway would make the description hold everywhere.
# Conflicts: # changelog_entry.yaml
…ce serves (#1450) * Add a Deployments developer tool: which .py and data build each service serves /{country}/dev-tools/deployments shows, for the US and the UK, which policyengine.py version, country model and data build each PolicyEngine service declares in its lock file, and what the running service serves. - Declared: each service's lock on its default branch pins .py; that .py tag's bundle manifest certifies one data build per country. The latest .py on PyPI is the reference. - Served: api.policyengine.org's /v2/tax-benefit-models/by-country/{country} reports the served .py (the full /metadata is ~75 MB for the US), and the simulation gateway's /versions reports the latest route's .py, model and data build. A service whose served versions differ from its lock is marked "Not deployed". Services: the API, app v3 (through the API), the simulation API, the household API (no microdata, so no data build) and API v2 alpha. Moved here from PolicyEngine/calibration-diagnostics#271: this is PolicyEngine infrastructure, not Microcosm calibration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Show each service as a card with declared and served side by side The seven-column table needed about 1,100 px, so on narrower windows the data-build column was cut off and every cell wrapped. Each service is now a card: the service and its links on top, then a Declared and a Served column with the same labeled rows, stacked on narrow screens. The latest-bundle summary uses the same labeled rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Add a drift overview table above the per-service cards The cards show detail but not drift across services at a glance. An overview table now lists every service in one row: deployment status, status against the latest .py, and the .py, model and data-build values. Where a running service serves something other than its lock, the cell turns red and shows "lock X / serving Y". Build IDs are shortened to their date and hash ("2026-10-19 · 6cba5d71"); the full ID is in the tooltip and in the card. Each service name links to its card under "Details". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep only the overview table on the Deployments page Drop the per-service detail cards and the note under the table. The repository, lock and runtime-report links move under each service name, and each status label shows its explanation as a tooltip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Tidy the latest-bundle summary and shorten every Microcosm build ID - The latest-bundle box shows .py, model and data build as labeled values in one row; the "every reporting service is deployed" label is gone (the red "N not deployed" label still appears when a service drifts). - Overview build IDs drop the repeated producer, country and year prefix: "spm · 2026-09-15", "national · 2026-10-19 · 6cba5d71". They wrap only between parts, and the table fits a 1024 px window with a drifted row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Name the latest data build the same way as the table The latest-bundle box showed the full build ID while the table showed the short form, so the two could not be compared by eye. The box now uses the same short form ("national · 2026-10-25 · a6299504", full ID in the tooltip), the same value size as the table, and a larger heading. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Read the API's data build from its gateway worker and flag a missing one The API sends each US and UK economy simulation to the simulation gateway's worker for its own .py. The page now looks up that worker in the gateway's /versions instead of inferring the data build from the .py bundle: - the worker exists and reports the certified build: the API's data build is reported, not inferred; - no worker for the API's .py: "No worker" (economy simulations fail); - the worker reports another build than its .py certifies: "Not deployed", with both builds named. App v3 inherits the API's report, so it gets the same check. When the gateway report cannot be read, the API falls back to the .py bundle as before. The header counts every service that needs attention. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Read the household API's served versions; drop the retired API v2 alpha - Household API: its /versions lists a `current` channel (what users get) and a `frontier` channel (the next rollout). The page compares `current` with the lock and names `frontier` in the explanation. Today it shows real drift: current serves policyengine-us 2.42.4 while the lock declares 2.53.1, which runs only on frontier. - API v2 alpha: the repository is archived and v2.api.policyengine.org no longer resolves, so the service is removed from the list instead of showing a lock file for something that does not run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say "live differs from main" instead of "not deployed" A service whose running version differs from its lock is still deployed and serving; only the version differs. The column is now "Live vs main" with "Same as main" / "Differs from main", drift cells read "main X / live Y", and explanations say "the lock on main declares". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Show newest releases in the header; name household channels; calmer drift - The header is now "Latest versions" with two aligned rows: what the latest .py certifies (.py, country model, Core, data build) and the newest release of each (PyPI for .py, the country model and Core; Microcosm's public releases API for its current data build). - The household API's model cell uses its own channel names, "current" and "frontier", when either differs from main. - Drift cells keep the table's font and size, label each line in grey and colour only the differing value; no red cell background. The "N services need attention" label is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Drop the status columns and align channel lines The overview keeps Service, .py, country model and data build. Where the running service differs from main, a cell lists the values as labeled lines ("main"/"live", or the household API's "current"/"frontier"); labels and values now share the table's monospace font and baseline, without red. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Link table data builds to Microcosm; drop the lock and report links Each Microcosm data build in the overview links to its release page on the Microcosm calibration dashboard, like the header already does. The service cell keeps its repository link and drops the "lock" and "report" links. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Add a "Live since" column from the services' own reports The household API reports `deployed_at` for each channel in /versions; the table now shows when its `current` channel (what users get) went live. Services that report no deploy time show a dash until they do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Read live-since dates from the API and the simulation gateway The API's GET /deployment (PolicyEngine/policyengine-api#4084) reports policyengine_version_live_since, and the gateway's /versions (PolicyEngine/policyengine-sim-api#961) reports latest_since. The page reads both when present; an API without the route, or a gateway without the field, shows no date as before. A missing gateway worker keeps the API's own date. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Fixes #960
Summary
The gateway's
/versionssays which policyengine.py (.py) version each route serves and which islatest, but not since when. This adds two optional fields to the routing state the routing publisher writes, and returns them from/versions:published_at({version: ISO UTC}): when each .py version's route was first published. Republishing a version keeps its date.latest_since(ISO UTC ornull): whenlatest.policyenginelast changed — a publish that advances it, a forced latest, or a rollback (build_restored_routing_state).A date is recorded only when the publish is that event (a route that did not exist, or a change of
latest). The live state was written before this change, so it shows{}/nullfor what it already routes until the next new version is published, rather than a later, wrong date. Both fields areNotRequired,coerce_routing_statekeeps valid values and drops malformed ones, and the legacy-seed merge keeps the current state's dates.The app-v2 Deployments developer tool (PolicyEngine/policyengine-app-v2#1450) will read
latest_sincefor the simulation API's "Live since" column; the household API already reports the equivalent, and PolicyEngine/policyengine-api#4084 does it for the API. GitHubproddeployments are no substitute: they run several times a day, including worker rebuilds that route nothing new.Changes
routing_publisher/state.py:published_at,latest_since,utc_now(), anowparameter onbuild_next_routing_stateandbuild_restored_routing_state.routing_publisher/legacy.py: the legacy-seed merge keeps the current state's dates.VersionsResponse:published_atandlatest_since, documented; gatewayroute_resolution.py/version_routes.pyread them type-safely.tests/golden/openapi.json(two added fields only) and the entry project'sversions_body.jsonsmoke fixture (REGENERATE_VERSIONS_FIXTURE=1).Testing
tests/version_registry/test_routing_dates.py(9 tests): first publish, republish, newer version, older version without moving latest, forced older latest, an undated state getting no wrong date, rollback, coercion,utc_nowformat.tests/version_registry: 290 passed. Gatewaytests: 268 passed. Contracttests: 322 passed. Entrytests: 742 passed.ruff check .,ruff format --check(0.15.22): passed.python -m scripts.rules pyright --sync: passed for the contract, gateway, executor, entry and other libraries;apis-integwas not checked locally because it needs the generated client.python scripts/run_quality_guards.py: all guards passed.🤖 Generated with Claude Code