Skip to content

v3: metadata and saved reforms on the v2 routes, unused until the switch - #1347

Closed
anth-volk wants to merge 5 commits into
apiv2/1e-public-api-clientfrom
apiv2/1e-v3-metadata-reforms-on-v2
Closed

anth-volk wants to merge 5 commits into
apiv2/1e-public-api-clientfrom
apiv2/1e-v3-metadata-reforms-on-v2

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1343

Summary

Adds the v3 workspace's v2 code for model metadata and saved reforms, merged unused (plan item 1E-2). Production behaviour does not change: v3 keeps calling the v1 routes until the switch pull request (plan item 4A-1).

The one place v3 chooses v1 or v2

app/src/libs/flagship/v3ApiRoutes.ts:

export const V3_API_ROUTES: V3ApiRoutes = 'v1';

It is a constant, not a runtime flag or an environment variable, so every build behaves the same. Everything in this PR that can use v2 reads it: the metadata loader for the (workspace) route group, v3's metadata readiness and retry, and the saved-reform store. Later v2 work for v3 (saved households, reports, household calculations) can key off the same constant so the switch stays one line.

What the switch (4A-1) will change

The switch PR changes 'v1' to 'v2' in v3ApiRoutes.ts and the matching expectation in app/src/tests/unit/libs/flagship/v3ApiRoutes.test.ts. From then on:

  1. Metadata, v3 only. In the (workspace) route group, MetadataBootstrap (calculator-app/src/app/[countryId]/providers.tsx) loads model metadata with fetchV2Metadata: GET /v2/tax-benefit-models/by-country/{country} selects the catalog version, then GET /v2/model-catalog and GET /v2/regions read that exact version. The result is converted to the shape GET /{country}/metadata has today (government parameters only, v1's labels and region types, v1's variable and entity field names), plus the catalog's model and parameter UUIDs. v3 counts as ready only once metadata from the v2 routes has loaded, and its retry buttons ask the v2 routes. The legacy calculator, the (calculator) route group, keeps loading GET /{country}/metadata.
  2. Saved reforms, API mode, signed in and signed out. Wherever reforms are saved to the account (US and UK with sign-in configured), saving a reform creates (or finds) a /v2/policies policy for the loaded catalog version and saves a /v2/user-policies row with policy_id. Editing a reform's parameters creates a new policy and repoints the row with PATCH policy_id (the API then clears legacy_policy_id); a rename only patches the name. Values are typed as v1 runs type them, and dates keep v1's whole-day periods (including its 2100-12-31 "forever"), as the dual-write copies of v1 policies do, so identical changes deduplicate to one v2 policy. Policies are cached for the session, as v1 policies are today.
  3. Reforms saved before the switch still open. A row that holds only legacy_policy_id (every reform saved before the switch whose v1 policy has no dual-write copy yet) is listed and opened by reading its parameters through GET /{country}/policy/{id}, until the one-time migration (4B-1) links it to a v2 policy. Editing it saves a v2 policy and repoints the row. 4B-3 removes this fallback.

Unchanged by the switch: reforms kept in this browser (sign-in not configured, or countries without accounts) still make v1 policies, and report runs still create v1 policies and simulations until the v2 report work (3A-3) lands.

Not covered here (known gaps after the switch)

Pieces

  • app/src/api/publicApi/metadata.ts: fetchV2Metadata.
  • app/src/api/reformStore.ts: ApiV2ReformStore, createV2ReformPolicy, parametersFromV2Policy; getReformStore(account, options, routes = V3_API_ROUTES).
  • app/src/reducers/metadataReducer.ts: fetchMetadataThunk accepts a country (v1, unchanged) or {countryId, routes: 'v2'}; the store remembers which routes its metadata came from and the v2 catalog identities. useFetchMetadata(countryId, routes = 'v1') reloads when the routes differ.
  • app/src/hooks/useFlagshipStores.ts: the reform store gets the loaded catalog and a cached v2 policy reader.
  • app/src/types/v2User.ts: policy_id on the user-policy create and patch bodies. A create body must name policy_id, legacy_policy_id or both (a union type), as the API requires.
  • app/src/libs/flagship/v3ApiRoutes.ts: metadataRoutesForRouteGroup(routeGroup, v3Routes = V3_API_ROUTES); the second argument exists so tests can name 'v2'.

Merge requirements

Environment variables

None added, read or changed. The v2 code uses the existing BASE_URL (VITE_API_URL, mapped from NEXT_PUBLIC_API_URL) and the existing saved-work credentials. No workflow changes in this PR.

Testing

  • fetchV2Metadata (src/tests/unit/api/publicApi/metadata.test.ts): requests pin the selected catalog version and page regions; summary fields, parameters and nodes (v1 labels, flags, the earliest-date spelling), variables (v1 field names, enum labels, defaults), regions (v1 types, national first, district state fields) and catalog identities match v1's shape; a catalog published before format 2 falls back to defaults; API errors propagate.
  • Saved reforms (src/tests/unit/api/reformStoreV2.test.ts): the POST /v2/policies body uses catalog UUIDs, typed values and UTC midnights; an unknown parameter sends nothing; v2 policies convert back to dated parameters; create links policy_id; an existing v2 policy is reused; no catalog means no requests; v2 rows read their policy and v1-only rows read the v1 policy; a failed load is listed without its changes; edits repoint policy_id; renames patch only the name; delete; the store choice by routes and account.
  • Store and hooks: metadataReducer routes (v2 request reads v2 and remembers it; a country alone reads v1; overlapping requests run once), useFetchMetadata reloads on a routes change, and with V3_API_ROUTES mocked to 'v2', v3 is not ready on v1 metadata, retry asks v2, and the reform store is the v2 store for signed-in and signed-out users and caches v2 policies.
  • v3ApiRoutes: the constant is 'v1'; the workspace route group follows the v3 routes ('v2' when named, V3_API_ROUTES by default), everything else stays on v1.
  • Calculator providers (src/tests/unit/calculator/CalculatorProviders.test.tsx): with v3 on 'v2', a workspace page fetches metadata from v2 and a legacy calculator page from v1.
  • Saving through useReformStore with v3 on 'v2' sends POST /v2/policies for the catalog in the metadata store (model UUID, parameter UUIDs, policyengine_version).
  • UserPolicyCreateBody (src/tests/unit/types/v2User.test.ts, checked by bun run typecheck): a body naming neither policy does not compile.
  • End to end against real data: the 6.7.5 catalog published to a disposable Postgres, served through the Public API's v2 routes from the API branch, then converted by fetchV2Metadata and compared with v1's GET /{country}/metadata documents. US: parameter labels 100,275 of 100,276 equal, descriptions, units, periods and flags 100% equal, values equal except two parameters with an invalid source date, all node labels equal, variable fields 100% equal except 2 enum labels (integer labels become strings) and 237 parameter-driven adds lists; regions 487 of 488 identical (the national label). Details in Store the metadata v3 reads from /{country}/metadata in the v2 catalog policyengine-api#3970.
  • Results: the tests above, with the existing v1 reform store tests and Generate a typed Public API v2 client, with a CI check that it is current #1346's client tests, pass (11 files, 108 tests). bun run typecheck (app and calculator-app), bun run prettier, bun run eslint --max-warnings 0 and bun run public-api:check pass; bun run build in calculator-app succeeds. Full bun run vitest: all pass except the two ScorecardComparisons tests that fail identically on main in my environment.

Possible conflicts

Groups 1D (v3 households) and 1F (country list) edit v3 in parallel. Likely overlaps: useFlagshipStores.ts and household code (1D-2's saved households should use V3_API_ROUTES too), providers.tsx and libs/countries (1F-3), and the generated openapi.json/schema.d.ts (regenerate with bun run public-api:refresh).

🤖 Generated with Claude Code

@anth-volk
anth-volk added this pull request to stack #1348 October 20, 2026 01:28
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-calculator-next Ready Ready Preview Oct 20, 2026 1:26pm UTC
policyengine-website Ready Ready Preview Oct 20, 2026 1:26pm UTC

Request Review

anth-volk and others added 5 commits October 20, 2026 17:16
…switch

V3_API_ROUTES in libs/flagship/v3ApiRoutes.ts is the one place v3
chooses between the PolicyEngine API's v1 and v2 routes. It is a
constant, 'v1', so production behaves as before; the switch pull request
changes it to 'v2'.

With 'v2':
- the (workspace) route group loads model metadata through
  fetchV2Metadata: /v2/tax-benefit-models/by-country, then
  /v2/model-catalog and /v2/regions for that exact catalog version,
  converted to the v1 metadata shape (government parameters, v1 labels,
  region types and field names), plus the catalog's model and parameter
  UUIDs. The legacy calculator stays on GET /{country}/metadata;
- saved reforms in the account (signed in or signed out) become
  /v2/policies policies, and /v2/user-policies rows point at them with
  policy_id; edits make a new policy and repoint the row. Rows that hold
  only a v1 policy (legacy_policy_id) still open, reading their
  parameters from GET /{country}/policy/{id} until the one-time
  migration links them.

Reforms kept in this browser (no accounts) keep making v1 policies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The v2 catalog stores the earliest start as 0001-01-01 (Python's first
year); v1 metadata wrote 0000-01-01, which the parameter history chart
leaves out. fetchV2Metadata now returns v1's spelling. A comparison of
the real 6.7.5 catalog through the v2 routes with v1's metadata found
this the only systematic difference in parameter values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two parts of the v2 code had no test that fails without them:

- The calculator's provider tree picks the metadata routes from the
  selected route group. Only the selection function was tested, and with
  V3_API_ROUTES at 'v1' its workspace case could not tell v1 from v2.
  metadataRoutesForRouteGroup now takes the v3 routes as an optional
  argument (defaulting to V3_API_ROUTES, so behaviour is unchanged), its
  tests name 'v2', and a new test renders CalculatorProviders on a
  workspace page and on a legacy calculator page.
- useReformStore hands the store the v2 catalog from the metadata store.
  A new test saves a reform through the hook and checks the
  POST /v2/policies request names that catalog.

The workspace case of the selection tests failed first (it returned 'v1'
for 'v2'). The provider test fails when MetadataBootstrap stops passing
the routes to useFetchMetadata, and the hook test fails when
useReformStore stops passing getV2Catalog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Making legacy_policy_id optional for v2 rows left both IDs optional in
UserPolicyCreateBody, so a body naming neither compiled although the API
refuses it. The body is now a union that needs policy_id, legacy_policy_id,
or both. The 'v1' | 'v2' union was also declared twice (MetadataRoutes and
V3ApiRoutes); V3ApiRoutes now reuses MetadataRoutes.

The new type test failed `bun run typecheck` first: its
@ts-expect-error on a body with neither ID was unused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The reform store module's header still said every save makes a v1
  policy. It now describes the browser store, the account store on v1
  policies before the switch, and the account store on /v2/policies after
  it.
- V1_POLICY_JSON_GC_TIME also sets how long cached v2 policies are kept,
  so it is now SAVED_POLICY_GC_TIME.
- Comments in v3ApiRoutes and the v2 metadata adapter call policyengine-api
  the Public API.

Documentation and naming only, so no test could fail first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

2 active deployments
Preview – policyengine-calculator-next — 99be8faf Deployed Oct 20, 2026 by vercel[bot]
Preview – policyengine-website — 99be8faf Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v3: load metadata and save reforms through the v2 routes, unused until the switch

1 participant