From baf019e58195bdb98c9d5df5dbaf4c583ffa5500 Mon Sep 17 00:00:00 2001 From: BADRI NARAYANAN Date: Wed, 23 Sep 2026 22:07:01 -0700 Subject: [PATCH] Say what deploying to Azure actually took PolicyPal now runs on Azure. The runbook that was meant to guide that would have failed four times over, so it is rewritten from what worked, and ADR 0023 records the four choices the subscription forced rather than the ones it originally reasoned its way to. The runbook's gaps: --public-access None while the seed restores from a laptop, an app and job created from a :bootstrap image nothing builds, no AcrPull for the managed identity, and a Static Web App whose token it needs but never creates. It now also names the Free Trial limits that shape the topology and the two client-version traps that cost real time. ADR 0023 keeps its reasoning and gains an amendment: Central US because Postgres is restricted in four regions, a shared Container Apps environment because a Free Trial allows one per subscription, a public endpoint behind a firewall because a consumption environment has no VNet path to a private server, and one always-on replica instead of scale-to-zero at about $70 a month. Its SBC chunk count is corrected to 35,756; the old figure predated the ADR 0020 refresh. Verified rather than asserted: counties in 0.43s, 401 unauthenticated, a cited answer in 13s, the migration job green, and data/ byte-identical before and after at 2,247 files and 1,593 PDFs. --- CHANGELOG.md | 30 ++ docs/decisions/0023-deploying-to-azure.md | 95 ++++++- docs/runbooks/deploy.md | 326 +++++++++++++++++----- 3 files changed, 368 insertions(+), 83 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ad001be..be31ef8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,20 @@ ### Added +- **PolicyPal is deployed to Azure.** The API runs on Container Apps in + Central US on the image digest built from `main`, backed by a PostgreSQL 16 + Flexible Server with `pgvector`, and the browser app is on Static Web Apps. + Verified with real requests rather than assumed: `/api/counties?zip=33101` + returns 200 in 0.43s, an unauthenticated `/api/auth/me` returns 401, and a + question answered through the browser returns citations in 13s — which + proves both models loaded from the image with `HF_HUB_OFFLINE=1` and the + BM25 index loaded from Postgres. The migration job was run once by hand and + succeeded, so CD's migration step is proven before CD runs. `data/` was + checksummed before and after and is byte-identical: 2,247 files, 1,593 PDFs, + none uploaded (ADR 0016). +- The database was seeded once from the local dump — 1,568 chunks, 35,756 SBC + chunks, 3,276 plans, 56,280 ZIP-county rows and the `bm25` index row — with + zero restore errors and every count matching the source. - A `Dockerfile` for the API, and a CI job that builds and scans the image (ADR 0022) — the first of ADR 0002's two deferred items. Multi-stage, runs as a non-root user, `torch` from PyTorch's CPU index on Linux so no CUDA @@ -223,6 +237,22 @@ ### Changed +- `docs/runbooks/deploy.md` rewritten from what actually worked. The previous + version would have failed mid-deploy four times over: it set + `--public-access None` while step 6 restored from a laptop, created the app + and job from a `:bootstrap` image that nothing built, never granted the + managed identity `AcrPull`, and never created the Static Web App whose token + it then required. It now also records the Free Trial limits that shape the + topology — Postgres restricted in four regions, one Container Apps + environment per *subscription*, and ACR Tasks disabled — plus the two client + version traps (`pg_dump` 15 cannot dump a 16 server; an Apple Silicon build + is arm64 and Container Apps refuses it). +- ADR 0023 amended with the four decisions the subscription forced: the region, + a shared rather than dedicated Container Apps environment, a public endpoint + behind a firewall instead of a private server, and one always-on replica + instead of scale-to-zero (~$70/month, stated plainly). Its SBC chunk count + is corrected from 31,359 to 35,756 — the old figure predated the ADR 0020 + robots.txt refresh. - In production the logs also go to stdout as JSON, where a container platform collects them; a container's own filesystem is ephemeral and unread. Development is unchanged, so an ingestion run's log lines do not interleave diff --git a/docs/decisions/0023-deploying-to-azure.md b/docs/decisions/0023-deploying-to-azure.md index fb52481..f93e587 100644 --- a/docs/decisions/0023-deploying-to-azure.md +++ b/docs/decisions/0023-deploying-to-azure.md @@ -2,9 +2,11 @@ ## Status -Accepted. Closes the second of ADR 0002's two deferred items ("CD to Azure -(ACR push, migrations, deploy)"). Depends on ADR 0022 for the image, and on -ADR 0021 without which a deployed container cannot answer. +Accepted, and **amended by the first real deployment** — see "What deploying +it actually changed" at the end. Four of the choices below did not survive +contact with the subscription. Closes the second of ADR 0002's two deferred +items ("CD to Azure (ACR push, migrations, deploy)"). Depends on ADR 0022 for +the image, and on ADR 0021 without which a deployed container cannot answer. ## Context @@ -23,7 +25,8 @@ text already extracted, not the documents it came from. ### Container Apps for the API - **Scale-to-zero** suits a project with no steady traffic, and the revision - model gives rollback for free. + model gives rollback for free. (The deployment runs one replica instead — + see the amendment.) - **Revisions are the rollback.** Reactivating the previous revision is one command, which is what ADR 0002's "keep rollback a single re-run" asks for. - **One replica, for now.** ADR 0022 explains why: Flask-Limiter has no shared @@ -48,8 +51,10 @@ changing it is a rebuild. Azure Database for PostgreSQL Flexible Server with `pgvector`. -- **`pg_dump` → `pg_restore`, once.** 75 MB: the corpus, its 1,568 chunks, the - 31,359 SBC chunks, the plan catalog, and the BM25 index row (ADR 0021). +- **`pg_dump` → `pg_restore`, once.** A 78 MB database, 10 MB as a + custom-format dump: the corpus, its 1,568 chunks, the 35,756 SBC chunks, the + 3,276-plan catalog, and the BM25 index row (ADR 0021). (An earlier draft said + 31,359 SBC chunks — that count predated the ADR 0020 robots.txt refresh.) - **Not a migration and not a workflow step.** Seeding is an operator action with the local database in front of them, because that is where the data lives and where ingestion will keep running. @@ -93,9 +98,9 @@ checked against `^[0-9a-f]{40}$` before it reaches `ref:` or an image tag. ## Consequences -- **Cold starts are slow**, because scale-to-zero means loading two models - from the image on the first request after idling. The trade is cost; the - fix, if it stops being acceptable, is a minimum of one replica. +- **Cold starts are slow** whenever scale-to-zero is used, because the first + request after idling loads two models from the image. The trade is cost; the + deployment took the other side of it and runs one replica always. - **A deploy needs a human.** Deliberate. This application answers questions about people's health coverage, and there is no staging environment to catch a bad revision first. @@ -104,8 +109,74 @@ checked against `^[0-9a-f]{40}$` before it reaches `ref:` or an image tag. - **The deployed data is a snapshot.** Ingestion runs locally, so the cloud database is as current as its last restore. The runbook says how to refresh it, and `checked_at` in `sbc_documents` says how old it is. -- **Costs are real but small**: Container Apps scaled to zero, a free Static - Web App, and a Flexible Server that cannot scale to zero and is therefore - the monthly floor. +- **Costs are real.** Scaled to zero the floor would be the Flexible Server, + which cannot scale to zero. As deployed, with one replica always running, + the API dominates instead — see the amendment for measured figures. - **Still not done:** a shared rate-limit store, a staging environment, and alerting. Named here rather than discovered later. + +## What deploying it actually changed + +The estate above was built on 2026-09-24. Four decisions changed, each forced +by a subscription limit rather than chosen. They are recorded here because the +reasoning above is still right in principle and wrong in fact. + +### The region was chosen for us + +Postgres Flexible Server is **restricted in `eastus`, `eastus2`, `westus2` and +`southcentralus`** on this subscription — `list-skus` returns "Provisioning is +restricted in this region." Only `centralus`, `westus3` and `northcentralus` +were open. Everything now runs in **Central US**, except the registry in West +US 3, where it was created before the constraint was understood. Cross-region +pull costs one transfer per revision and is not worth moving. + +### The Container Apps environment is shared, not dedicated + +A Free Trial subscription permits **one Container Apps environment globally**, +and another project already held it. `policypal-api` therefore runs inside +that environment and shares its Log Analytics workspace. This is real coupling +and the first thing to undo on a paid subscription. + +Worth recording how this was missed: the per-region usage API reports +`ManagedEnvironmentCount limit=1` for every region independently, which reads +as "one per region" and is not. Only the failed create says +`MaxNumberOfGlobalEnvironmentsInSubExceeded`. + +### The database is public behind a firewall, not private + +The original text said `--public-access None` and claimed migrations running +inside the environment made that sufficient. It does not: a consumption +environment has no VNet path to a private server, and the one-off seed +restores from a laptop. The server now has a public endpoint with two firewall +rules — the operator's IP and Azure services. TLS is enforced by Azure either +way. Making this genuinely private needs a VNet-integrated workload profile +environment, which the limit above also forbids. + +### It does not scale to zero + +"Scale-to-zero suits a project with no steady traffic" was the original +reasoning and is still sound; the deployment nonetheless runs +`--min-replicas 1` by explicit choice, trading roughly **$70/month** for the +absence of a cold start. At 1 vCPU / 2 GiB that is the dominant line item — +Postgres B1ms is ~$16, the registry ~$5, Static Web Apps free. Setting +`--min-replicas 0` is a one-word change and the right one if idle cost ever +matters more than first-request latency. + +### Two smaller things worth knowing + +- **ACR Tasks are not permitted** on this subscription, so `az acr build` + fails. The image is built with `docker buildx --platform linux/amd64` — + necessary anyway, since an Apple Silicon Mac otherwise produces an arm64 + image that Container Apps refuses. 270s under emulation, 639 MB compressed. +- **Azure ships pgvector 0.8.2 against 0.8.6 locally.** Harmless today because + the corpus uses only the plain `vector` type, but it bounds which pgvector + features are safe to adopt. + +### What the deployment proved + +Measured, not assumed: `/api/counties?zip=33101` returns 200 in **0.43s**; an +unauthenticated `/api/auth/me` returns **401**; a full RAG answer with +citations takes **13s**, which demonstrates both models loading from the image +under `HF_HUB_OFFLINE=1` and the BM25 index loading from Postgres. The +migration job was run once by hand and succeeded. `data/` was checksummed +before and after and is byte-identical — 2,247 files, 1,593 PDFs (ADR 0016). diff --git a/docs/runbooks/deploy.md b/docs/runbooks/deploy.md index 0e50959..19b3060 100644 --- a/docs/runbooks/deploy.md +++ b/docs/runbooks/deploy.md @@ -4,138 +4,309 @@ What to create once, what runs on every merge, and what to do when a deploy goes wrong. The decision behind it is [ADR 0023](../decisions/0023-deploying-to-azure.md). **The issuer PDFs never leave this machine** (ADR 0016). Nothing below uploads -`data/sbc/`, and ingestion stays local. +`data/sbc/`, and ingestion stays local. The first deployment verified this by +checksumming `data/` before and after: 2,247 files, 1,593 PDFs, identical. -## What gets deployed +This runbook describes a deployment that was actually performed, not a +proposal. Where the first attempt hit a wall, the wall is documented rather +than smoothed over. -| Piece | Where | Built by | +## What is deployed + +| Piece | Where | Name | | --- | --- | --- | -| API | Container Apps, one replica | `Dockerfile` (ADR 0022) | -| Browser app | Static Web Apps | `npm run build` in CI | -| Database | PostgreSQL Flexible Server + `pgvector` | seeded once, by hand | -| Migrations | Container Apps job, same image | `alembic upgrade head` | +| API | Container Apps, 1 replica, Central US | `policypal-api` | +| Browser app | Static Web Apps (Free), Central US | `policypal-ui` | +| Database | PostgreSQL Flexible Server 16 + `pgvector`, Central US | `policypal-db` | +| Migrations | Container Apps job, same image | `policypal-migrate` | +| Registry | Container Registry, Basic, West US 3 | `policypalcr` | + +## Before you start: four subscription limits that shape all of this + +These were found by hitting them. On a **Free Trial** subscription +(`quotaId: FreeTrial_2014-09-01`), check each before choosing a region. + +1. **Postgres Flexible Server is restricted in many regions.** `eastus`, + `eastus2`, `westus2` and `southcentralus` all return *"Provisioning is + restricted in this region."* Only `centralus`, `westus3` and + `northcentralus` worked. Check with: + ```sh + az postgres flexible-server list-skus -l -o json | head -5 + ``` +2. **One Container Apps environment per subscription — globally.** Not per + region. The per-region `ManagedEnvironmentCount` usage API reports + `limit=1` for each region independently, which is true and misleading; the + create fails with `MaxNumberOfGlobalEnvironmentsInSubExceeded`. **Try the + create before planning around a region.** +3. **ACR Tasks are not permitted.** `az acr build` fails with + `TasksOperationsNotAllowed`, so the image must be built elsewhere. +4. **The spending limit is ON**, so exhausting credit deallocates resources + rather than billing. Budget accordingly. ## One-time setup -Names below are placeholders; use your own and set them as repository -variables. - ### 1. Resource group, registry, database ```sh -az group create --name policypal-rg --location eastus +az group create --name policypal-rg --location westus3 -az acr create --resource-group policypal-rg --name policypalacr --sku Basic +az acr create --resource-group policypal-rg --name policypalcr \ + --sku Basic --location westus3 +# Central US, because Postgres is restricted elsewhere (see limit 1) and the +# shared Container Apps environment lives there. az postgres flexible-server create \ - --resource-group policypal-rg --name policypal-db \ + --resource-group policypal-rg --name policypal-db --location centralus \ + --admin-user ppadmin --admin-password "$PGPASS" \ --tier Burstable --sku-name Standard_B1ms --version 16 \ - --storage-size 32 --public-access None + --storage-size 32 --public-access --yes + az postgres flexible-server parameter set \ --resource-group policypal-rg --server-name policypal-db \ --name azure.extensions --value VECTOR + +az postgres flexible-server firewall-rule create \ + --resource-group policypal-rg --server-name policypal-db \ + --name AllowAzureServices \ + --start-ip-address 0.0.0.0 --end-ip-address 0.0.0.0 +``` + +**Generate the password as hex** (`openssl rand -hex 24`). Alembic passes +`DATABASE_URL` through a ConfigParser that interpolates `%`, so a password +containing one breaks migrations in a way that is tedious to diagnose. + +**The server is public, behind a firewall — not `--public-access None`.** An +earlier version of this runbook said otherwise and was self-contradictory: the +seed in step 5 restores from your laptop, and a consumption Container Apps +environment has no VNet path to a private server. Public plus two firewall +rules is what actually works. Azure enforces TLS regardless. + +**`az postgres flexible-server db create` rejected every flag spelling** in +CLI 2.90.0. Create the database with psql instead: + +```sh +psql "host=policypal-db.postgres.database.azure.com user=ppadmin \ + dbname=postgres sslmode=require" -c "CREATE DATABASE policypal;" ``` -`--public-access None` keeps the server off the internet. Migrations run -inside the environment (ADR 0023), so nothing needs to reach it from outside. +### 2. Build and push the image -### 2. Container Apps environment, the app, and the migration job +`az acr build` is unavailable (limit 3), and a Mac builds `arm64` while +Container Apps needs `amd64`. Build locally for the right platform: ```sh -az containerapp env create --resource-group policypal-rg \ - --name policypal-env --location eastus +az acr login -n policypalcr # AAD token; admin user stays disabled + +docker buildx build --platform linux/amd64 --file Dockerfile \ + --tag policypalcr.azurecr.io/policypal-api:"$(git rev-parse HEAD)" \ + --tag policypalcr.azurecr.io/policypal-api:latest \ + --push . +``` + +Measured on an M-series Mac under QEMU: **270s total** — `uv sync` 118s, both +models baked 56s, push 158s. Emulation overhead was far smaller than expected, +because the build is dominated by network and disk rather than CPU. The result +is **639 MB compressed** in the registry. Confirm the platform before +deploying — an arm64 image fails at pull time with an unhelpful message: + +```sh +az acr manifest list-metadata -r policypalcr -n policypal-api -o json \ + | python3 -c "import sys,json;[print(m.get('architecture'),m.get('os')) for m in json.load(sys.stdin)]" +``` + +### 3. The app and the migration job + +There is no `:bootstrap` image — build the real one first (step 2) and create +the app directly on its digest. + +```sh +ENVID=$(az containerapp env show -g -n --query id -o tsv) +IMG=policypalcr.azurecr.io/policypal-api@sha256: az containerapp create --resource-group policypal-rg --name policypal-api \ - --environment policypal-env --target-port 8000 --ingress external \ - --min-replicas 0 --max-replicas 1 \ - --registry-server policypalacr.azurecr.io --system-assigned \ - --image policypalacr.azurecr.io/policypal-api:bootstrap + --environment "$ENVID" --image "$IMG" \ + --target-port 8000 --ingress external \ + --cpu 1.0 --memory 2.0Gi --min-replicas 1 --max-replicas 1 \ + --system-assigned \ + --registry-server policypalcr.azurecr.io --registry-identity system \ + --secrets database-url="$DBURL" openai-key="$OPENAI" \ + jwt-secret="$JWT" cms-key="$CMS" \ + --env-vars ENVIRONMENT=production LOG_DIR=/tmp/logs \ + DATABASE_URL=secretref:database-url \ + OPENAI_API_KEY=secretref:openai-key \ + JWT_SECRET_KEY=secretref:jwt-secret \ + CMS_MARKETPLACE_API_KEY=secretref:cms-key +``` + +**`--registry-identity system` is what grants `AcrPull`.** Without it the +identity exists but cannot pull, and the revision never starts. Verify: +```sh +PID=$(az containerapp show -g policypal-rg -n policypal-api --query identity.principalId -o tsv) +az role assignment list --assignee "$PID" --all -o tsv # expect AcrPull on policypalcr +``` + +**`--max-replicas 1` is load-bearing.** Flask-Limiter keeps its counters in +process, so a second replica doubles every rate limit — including 5/min on +register and 10/min on login (ADR 0022). Raise it only together with a shared +store. `--min-replicas 1` avoids a cold start and is the more expensive +choice; `0` is correct if idle cost matters more than first-request latency. + +`LOG_DIR=/tmp/logs` is required, not cosmetic: uid 10001 cannot write `/app`, +and `setup_logging()` creates the directory at import, so a bad value crashes +startup before the app serves anything. + +The migration job takes the same image and the same three required settings — +`Settings` is constructed at import, so the job crashes without them even +though a migration uses none of them: + +```sh az containerapp job create --resource-group policypal-rg \ - --name policypal-migrate --environment policypal-env \ + --name policypal-migrate --environment "$ENVID" --image "$IMG" \ --trigger-type Manual --replica-timeout 600 \ - --registry-server policypalacr.azurecr.io --system-assigned \ - --image policypalacr.azurecr.io/policypal-api:bootstrap \ + --cpu 0.5 --memory 1.0Gi --mi-system-assigned \ + --registry-server policypalcr.azurecr.io --registry-identity system \ + --secrets database-url="$DBURL" openai-key="$OPENAI" jwt-secret="$JWT" \ + --env-vars DATABASE_URL=secretref:database-url \ + OPENAI_API_KEY=secretref:openai-key \ + JWT_SECRET_KEY=secretref:jwt-secret \ + LOG_DIR=/tmp/logs ENVIRONMENT=production \ --command "/app/.venv/bin/alembic" --args "upgrade" "head" ``` -**`--max-replicas 1` is load-bearing.** Flask-Limiter keeps its counters in -process, so a second replica doubles every rate limit — including the 5/min on -register and 10/min on login (ADR 0022). Raise it only together with a shared -store for the limiter. +### 4. The browser app + +```sh +az staticwebapp create --name policypal-ui --resource-group policypal-rg \ + --location centralus --sku Free +``` -### 3. Settings and secrets on the app +Static Web Apps exists only in Central US, East US 2, West US 2, West Europe +and East Asia. It is CDN-served, so the region is metadata. ```sh -az containerapp secret set --resource-group policypal-rg --name policypal-api \ - --secrets database-url="postgresql+psycopg://..." \ - openai-key="sk-..." jwt-secret="$(python -c 'import secrets;print(secrets.token_urlsafe(48))')" +TOKEN=$(az staticwebapp secrets list --name policypal-ui \ + --resource-group policypal-rg --query properties.apiKey -o tsv) + +cd frontend +VITE_API_BASE_URL="https://" npm run build +grep -r "127.0.0.1" dist/ && echo "STOP: localhost leaked into the bundle" + +npx @azure/static-web-apps-cli deploy ./dist \ + --deployment-token "$TOKEN" --env production +``` + +**Grep the bundle.** `frontend/.env` pins `VITE_API_BASE_URL` to +`http://127.0.0.1:5000`. A shell variable does take precedence, but verifying +the artifact costs one command and catches the case where it does not. +Then point CORS at it — exact match, **no trailing slash**: + +```sh az containerapp update --resource-group policypal-rg --name policypal-api \ - --set-env-vars ENVIRONMENT=production LOG_DIR=/tmp/logs \ - FRONTEND_ORIGIN="https://.azurestaticapps.net" \ - DATABASE_URL=secretref:database-url \ - OPENAI_API_KEY=secretref:openai-key \ - JWT_SECRET_KEY=secretref:jwt-secret + --set-env-vars FRONTEND_ORIGIN="https://" +``` + +A mismatch breaks the browser app silently while every curl check still +passes, because curl ignores CORS. Verify with an `Origin` header: + +```sh +curl -si -H "Origin: https://" "https:///api/counties?zip=33101" \ + | grep -i access-control-allow-origin ``` -The migration job needs `DATABASE_URL` and the two settings with no default -(`OPENAI_API_KEY`, `JWT_SECRET_KEY`) as well — `Settings` will not construct -without them, even though a migration uses neither. +### 5. Seed the database, once -`.env.example` lists every setting. Never put one in the repo. +The schema comes from the dump along with the data, so `alembic upgrade head` +afterwards is a no-op — correct, not skipped. -### 4. Sign-in without a stored credential +**Use the container's `pg_dump`.** A v15 client refuses a v16 server, and +macOS Postgres installs are often older than the dev container: + +```sh +docker exec policypal_db pg_dump --no-owner --no-privileges \ + --format=custom -U policypal -d policypal > policypal.dump # ~10 MB + +docker exec -i -e PGPASSWORD="$PGPASS" policypal_db pg_restore \ + --no-owner --no-privileges \ + --dbname "host=policypal-db.postgres.database.azure.com user=ppadmin \ + dbname=policypal sslmode=require" < policypal.dump +``` + +Verify against the source, not against expectations: + +```sql +select count(*) from chunks; -- 1568 +select count(*) from sbc_chunks; -- 35756 +select count(*) from plans; -- 3276 +select count(*) from zip_counties; -- 56280 +select name, chunks from search_indexes; -- bm25 | 1568 +select version_num from alembic_version; -- f6e50c4aec26 +``` + +If `search_indexes` is missing, run `make build-index` against the Azure +database rather than re-ingesting: the index is derived from the chunks +(ADR 0021). + +**Azure ships pgvector 0.8.2; the dev container has 0.8.6.** The restore is +clean because the corpus uses only the plain `vector` type and cosine +distance, both stable across those versions. Worth knowing before adopting a +newer pgvector feature locally. + +### 6. Sign-in without a stored credential Create an app registration, give it `AcrPush` on the registry and `Contributor` on the resource group, then add **federated credentials** for -this repository — one for `environment:production`, one for `ref:refs/heads/main`. -No client secret is created (ADR 0023). +this repository — one for `environment:production`, one for +`ref:refs/heads/main`. No client secret is created (ADR 0023). -### 5. GitHub Environment and variables +### 7. GitHub Environment and variables Create an Environment named `production` with **yourself as a required -reviewer** — that is the manual approval gate. +reviewer** — that is the manual approval gate. Two jobs reference it, so +**approval is requested twice per deploy**. | Secret | | | --- | --- | | `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID` | from the app registration | -| `AZURE_STATIC_WEB_APPS_API_TOKEN` | from the Static Web App | +| `AZURE_STATIC_WEB_APPS_API_TOKEN` | `az staticwebapp secrets list` | -| Variable | Example | +| Variable | Value | | --- | --- | -| `ACR_NAME` | `policypalacr` | +| `ACR_NAME` | `policypalcr` | | `AZURE_RESOURCE_GROUP` | `policypal-rg` | | `CONTAINERAPP_NAME` | `policypal-api` | | `MIGRATION_JOB_NAME` | `policypal-migrate` | -| `API_URL` | `https://policypal-api..azurecontainerapps.io` | -| `FRONTEND_URL` | `https://.azurestaticapps.net` | +| `API_URL` | the API origin, **no trailing slash** | +| `FRONTEND_URL` | the Static Web App origin | -### 6. Seed the database, once +## Verifying a deployment -The schema comes from migrations; the contents come from here, because -ingestion runs here (ADR 0019). +Creation succeeding is not the same as the app working. In order: ```sh -pg_dump --no-owner --no-privileges --format=custom \ - -h localhost -U policypal policypal > policypal.dump # ~75 MB +az containerapp revision list -g policypal-rg -n policypal-api \ + --query "[].{rev:name,state:properties.runningState}" -o table # RunningAtMaxScale -pg_restore --no-owner --no-privileges --clean --if-exists \ - -d "postgresql://...@policypal-db.postgres.database.azure.com/policypal" policypal.dump +curl -s -w '%{http_code}\n' "$API/api/counties?zip=33101" # 200, Miami-Dade +curl -s -o /dev/null -w '%{http_code}\n' "$API/api/auth/me" # 401, not 500 ``` -This carries the corpus, its chunks, the SBC text, the plan catalog and the -BM25 index row. It does **not** carry the PDFs, and must not. +Then ask a real question through the API or the UI and confirm the answer +carries **citations**. That is the only check that proves all three of: both +models loaded from the image under `HF_HUB_OFFLINE=1`, the BM25 index came +from Postgres, and pgvector search ran. Measured on the first deployment: +`/api/counties` **0.43s**, a full cited RAG answer **13s**. -Afterwards, check the index arrived: +Finally, run the migration job once by hand. It should succeed as a no-op, and +it proves CD's migration step works before CD ever runs: -```sql -select name, chunks, built_at from search_indexes; -- bm25 | 1568 | ... +```sh +az containerapp job start -g policypal-rg -n policypal-migrate +az containerapp job execution list -g policypal-rg -n policypal-migrate \ + --query "[0].properties.status" -o tsv # Succeeded ``` -If it is missing, run `make build-index` against the Azure database rather -than re-ingesting: the index is derived from the chunks (ADR 0021). - ## Every merge to main 1. CI runs. A failed run never reaches CD. @@ -145,6 +316,10 @@ than re-ingesting: the index is derived from the chunks (ADR 0021). `/api/counties?zip=33101` until it returns 200. 4. **frontend** — builds with `VITE_API_BASE_URL` and uploads to Static Web Apps. +Note that the health poll is **data-dependent**: `/api/counties?zip=33101` +returns 404 on a migrated-but-unseeded database, so step 5 must have run or +the deploy job fails against a perfectly healthy app. + Deploying a specific commit by hand: **Actions → CD → Run workflow**, with a full 40-character commit id. @@ -175,13 +350,22 @@ already dropped something does not bring it back. restore has not happened, or `search_indexes` is empty. Retrieval says which — `MissingSearchIndexError` names `make build-index`. +**The revision will not start.** Check `AcrPull` first (step 3), then whether +the image is amd64 (step 2), then whether all three required settings are +present — a missing one fails at import, before any log line is written. + +**The browser app shows nothing but the API answers curl.** CORS. Compare +`FRONTEND_ORIGIN` to the Static Web App origin character for character. + **Cold start feels broken.** The first request after idle loads two models from -the image. It is slow, not stuck. If that stops being acceptable, set -`--min-replicas 1` and pay for it. +the image. It is slow, not stuck. At `--min-replicas 1` this does not arise. ## What this does not do -- **No staging environment.** One environment, one reviewer. +- **No staging environment.** One environment, one reviewer — and on a Free + Trial, one environment is the hard limit anyway. +- **No dedicated environment.** The app shares a Container Apps environment, + and therefore a Log Analytics workspace, with another project. - **No alerting.** Failures are seen in the Actions run, not pushed anywhere. - **No shared rate-limit store**, which is what pins the app at one replica. - **No scheduled ingestion.** The deployed data is as current as the last