[anth-volk]

Fixes #232

Merge requirements (all three must hold)

This PR stays a draft until they do.

1. The collector has deployed from PolicyEngine/microcosm-collector, and that deployment is verified (step 6 of the switch-over runbook in calibration-diagnostics#233).
   • microcosm-collector#11 merged on 2026-10-20 at 23:12 UTC (f59d94c). Its release commit's Push run deploys staging, then production.
   • Verification checks Add stable diagnostics API layer #11's release commit against the image, the production backup, the service address, the database and the mounted secrets.
2. A few normal builds have deployed from microcosm-collector after that (step 7 of the runbook). This PR deletes .github/workflows/telemetry-service.yml, including the confirmed manual redeploy that calibration-diagnostics#233 kept as the rollback path. Merging it before deployment from microcosm-collector is proven would leave no fallback.
3. This PR has been re-reviewed since its earlier fix and its rebases onto main (re-reviewed at 566a708; its fixes are listed under "Re-review (2026-10-20)"), and the review's findings are resolved.

After merging, do the cleanup under "Cleanup after merging" below.

Summary

PolicyEngine/microcosm-collector now holds the collector's history, source, tests and deployment. This PR:

• Replaces the dashboard deploy's read-secret check. .github/workflows/deploy.yml confirmed that the collector read secret exists by running telemetry-service/scripts/check-secrets-exist.sh "$READ_SECRET_NAME". It now runs bash .github/scripts/deploy-dashboard.sh check-read-secret-exists, a new command in the script that already holds the deploy's shell logic. It makes the same metadata-only call, gcloud secrets describe "$READ_SECRET_NAME" --project "$PROJECT_ID" --format=none, with the same PROJECT_ID and READ_SECRET_NAME from the Production environment, after the same Google Cloud authentication. It never reads the secret's value, and it fails before anything is deployed if the secret or either setting is missing.
• Removes telemetry-service/ (35 files, including scripts/require-collector-push-runs-finished.sh from calibration-diagnostics#233 and the two test files calibration-diagnostics#219 added), .github/workflows/telemetry-service.yml (including the manual redeploy that calibration-diagnostics#233 kept for rollback), and frontend/tests/test_telemetry_service_workflow.py, which tested that workflow.
• Keeps the dashboard's documentation for reading the collector. The removed telemetry-service/README.md held the only description of the dashboard's own settings for it. docs/build-monitor.md now has it, in "Reading the collector": MICROCOSM_TELEMETRY_COLLECTOR_URL, MICROCOSM_TELEMETRY_READ_SECRET_VERSION and MICROCOSM_TELEMETRY_GOOGLE_IDENTITY_PROVIDER and where production and preview get each one, the Vercel OIDC trust, the per-environment secret access, local development with Application Default Credentials, and what the production deploy's read-secret check does and doesn't show. It also corrects the claim that the read credential comes from the deployment environment: the server fetches a Secret Manager version itself.
• Points docs/build-monitor.md at microcosm-collector.

A search of every tracked file finds no other reference to telemetry-service/, telemetry-service.yml, telemetry_collector or check-secrets-exist. New tests keep it that way. The test that the directory is gone asks git ls-files, because switching an existing clone to this branch leaves ignored files such as .venv/ and __pycache__/ in telemetry-service/.

Deployment configuration

Merging deploys the dashboard to production through deploy.yml, as every merge does, and that deployment runs the new check first.

Name	Provider	Wiring	Verified
PROJECT_ID ← vars.TELEMETRY_GCP_PROJECT_ID	calibration-diagnostics Production environment variable	step env: → check-read-secret-exists	Present; unchanged from today
READ_SECRET_NAME ← vars.TELEMETRY_READ_SECRET_NAME	Production environment variable naming a Google Secret Manager secret	step env: → gcloud secrets describe	Present; the secret exists (checked read-only)
TELEMETRY_GCP_WIF_PROVIDER, TELEMETRY_GCP_DEPLOY_SERVICE_ACCOUNT	Production environment secrets	google-github-actions/auth in the step before	Present; unchanged; they authenticate today's identical call. The service account is the collector's deploy service account, which has much more access than this check needs; see the follow-up below
Provider github/calibration-diagnostics	Google Cloud Workload Identity	accepts assertion.repository=='PolicyEngine/calibration-diagnostics'	Exists, and the deploy service account grants this repo roles/iam.workloadIdentityUser. Keep both until the follow-up below
MICROCOSM_TELEMETRY_COLLECTOR_URL (documented, not changed)	Production environment variable	deploy.yml step env: → deploy-dashboard.sh deploy-frontend → vercel deploy --env → server runtime	Present (name listed 2026-10-20)
MICROCOSM_TELEMETRY_READ_SECRET_VERSION, MICROCOSM_TELEMETRY_GOOGLE_IDENTITY_PROVIDER (documented, not changed)	Vercel project environment variables	server runtime → telemetry-collector-client.ts, secret-manager.ts	Unverified: Vercel's settings can't be listed from here. This PR only documents them

No variable or secret is added. After this merges, the staging environment and most TELEMETRY_* variables in Production are unused; see "Cleanup after merging".

Merging

• See "Merge requirements" above.
• calibration-diagnostics#233 merged on 2026-10-20 (a45aaf1, a merge commit). GitHub then moved this PR onto main and rebased it, so it needs no base change. Its diff against main is its own changes: six changed files and 37 removals.
• calibration-diagnostics#219 merged on 2026-10-20, and its changes reach production through microcosm-collector#11. This branch keeps the deletions of Apply each telemetry event batch incrementally instead of re-materializing the run #219's telemetry-service/ files.
• Merging deploys the dashboard to production. Merge only when no other main CI or "Deploy production" run is unfinished, because a merge cancels an unfinished main CI run and skips its deployment.

Rebase onto main (2026-10-20, 2026-10-22)

This branch and calibration-diagnostics#233 were rebased onto main at 19be8dc on 2026-10-20, with no change to their own content. This branch's five commits were replayed onto #233's new head fc660c4 (0d69ba5 → e102651, 9ccbfd8 → 79436b6, d015902 → 036550e, 82101cd → 9f07958, 257e953 → df4447f), and the merge commit 269c3fa, which only brought #233 in, was dropped. main now includes calibration-diagnostics#219, so 79436b6 keeps the deletions where #233 or #219 changed a file, and also deletes scripts/require-collector-push-runs-finished.sh (previously deleted by 269c3fa) and #219's new tests/event_schedules.py and tests/test_incremental_batch_apply.py; git ls-files telemetry-service prints nothing. Every file this PR adds or changes has the same added and removed text as before. The three test files: 28 passed (main added 6 tests to test_deploy_workflow.py); shellcheck and actionlint report no findings.

After #233 and #245 merged, GitHub rebased the branch again, onto 1548d93 (#245's merge commit): e102651 → ab182c7, 79436b6 → 3b9581a, 036550e → ca540f0, 9f07958 → 30f9fec, df4447f → 566a708. Each new commit has the same subject, files and number of added and removed lines as the one it replaced.

On 2026-10-22 the branch was rebased onto main at 2f7cdcd (the merge of calibration-diagnostics#261, after #247, #248, #250, #256 and #258) and force-pushed with a lease on 3509e33: ab182c7 → 80a2d1c, 3b9581a → 3386b7c, ca540f0 → 5cceb49, 30f9fec → bd8759e, 566a708 → ff40071, 2822d17 → 9587f51, 8b29c49 → d4cb573, 3509e33 → df1e4ba. The "Tests" and "Re-review" tables below name the new commits.

• One conflict: telemetry-service/uv.lock, which this PR deletes and which Stop trusting the compromised eMudhra CA certificates #256 (a2dbad6) changed to upgrade certifi from 2026.7.22 to 2026.10.19. The deletion is kept: nothing on main uses the file, and the collector deploys from microcosm-collector. microcosm-collector's uv.lock still pins certifi 2026.7.22 (read 2026-10-22), so that upgrade has not reached the collector; that is a follow-up in microcosm-collector, not in this PR.
• git range-diff: seven of the eight commits have identical patches. 3386b7c differs only in the deleted text of telemetry-service/uv.lock, which now contains Stop trusting the compromised eMudhra CA certificates #256's certifi lines.
• Start production deploy runs only for main CI runs #261 changed deploy.yml (the trigger accepts only CI runs for main, and the concurrency group moved to the deploy job) and added tests to test_deploy_workflow.py. This PR's hunks in both files applied without conflict and did not change. The "Check collector read secret exists" step is still in the deploy job (environment: Production), after the same Google Cloud authentication and before "Deploy Modal backend", and reads only PROJECT_ID and READ_SECRET_NAME from the Production variables TELEMETRY_GCP_PROJECT_ID and TELEMETRY_READ_SECRET_NAME (both present; names listed 2026-10-22).
• Diff against main: still this PR's own changes, six changed files and 37 removals, with the same added and removed lines in each of the six files as before. git ls-files telemetry-service prints nothing, and no tracked file refers to telemetry-service, check-secrets-exist or require-collector-push-runs-finished.
• Tests: test_collector_lives_in_its_own_repo.py 3 passed, test_deploy_script.py 9 passed, test_deploy_workflow.py 27 passed (39 in all; Start production deploy runs only for main CI runs #261 added 9 test cases to the last file). shellcheck 0.11.0 on deploy-dashboard.sh and actionlint 1.7.12 on deploy.yml report no findings; ruff check passes on the three files. ruff format --check reports only code that main wrote in test_deploy_workflow.py, including Start production deploy runs only for main CI runs #261's new lines; main's copy fails the same check. No TypeScript file changed, so bun run lint (the TypeScript type check) was not run.

Tests

Commit	Red (before the change)	Green (after)
80a2d1c Check the collector read secret from the dashboard deploy script	5 failed: test_deploy_script.py::test_read_secret_check_reads_only_the_secret_metadata, test_missing_read_secret_stops_deployment, test_read_secret_check_requires_its_configuration[PROJECT_ID] and [READ_SECRET_NAME] (each runs the script with a gcloud stub that records its arguments), and test_deploy_workflow.py::test_dashboard_checks_read_secret_metadata_before_deployment	test_deploy_script.py and test_deploy_workflow.py: 19 passed. shellcheck 0.11.0 on deploy-dashboard.sh and actionlint 1.7.12 on deploy.yml: no findings
3386b7c Remove telemetry-service/	3 failed: test_collector_lives_in_its_own_repo.py::test_the_collector_directory_and_its_workflow_are_removed, test_no_tracked_file_refers_to_the_removed_collector_directory, test_no_workflow_or_workflow_script_deploys_the_collector	3 passed; with the two files above, 22 passed
5cceb49 Check the removed collector directory with git ls-files	1 failed, 2 passed: test_the_collector_directory_and_its_workflow_are_removed, in a clone switched from #233's branch that kept the ignored telemetry-service/.venv/ and __pycache__/ folders	3 passed with those folders still on disk; with the two files above, 22 passed
bd8759e Document how the dashboard reads the collector; ff40071 Say where production gets the collector address	Documentation only, so no red step	22 passed (test_no_tracked_file_refers_to_the_removed_collector_directory also covers the new text)

After each commit, its test files passed again. Ruff lint passes on the three test files, and Ruff format on this PR's changes to them (since the rebase onto 1548d93, ruff format --check also reports tests that main added to test_deploy_workflow.py; main's copy fails the same check); shellcheck 0.11.0 on deploy-dashboard.sh and actionlint 1.7.12 on deploy.yml were re-run with no findings. Locally I ran only these test files. CI runs the rest.

Re-review (2026-10-20)

Re-reviewed at 566a708. The earlier review's findings are still fixed after both rebases, nothing in the repo refers to the removed files, and check-read-secret-exists makes the same call as the old script. The review found no bug in the script, the workflow or the documentation; it found two gaps in the new tests and one imprecise sentence. Fixes:

Commit	Red (before the change)	Green (after)
9587f51 Record every gcloud call in the read-secret check tests	The gcloud stub replaced its record on each call, so a check that also read the value (gcloud secrets versions access latest … before the describe call) passed all four read-secret tests. With the stub recording every call: 2 failed against that temporary, uncommitted change to the script (test_read_secret_check_reads_only_the_secret_metadata, test_missing_read_secret_stops_deployment: the first recorded call was secrets versions access)	test_deploy_script.py: 7 passed against the committed script. Test-only: the script already makes just the describe call
d4cb573 Test the read-secret check with empty settings	GitHub Actions passes an empty string for a missing vars.NAME, and the tests only removed the variable. 2 failed against a temporary, uncommitted change that treats an empty value as present ([empty-PROJECT_ID], [empty-READ_SECRET_NAME]: the check exited 0 after calling gcloud)	test_deploy_script.py: 9 passed. Test-only: the script already treats an empty value as missing
df1e4ba Say that the collector address may also use HTTP on 127.0.0.1	Documentation only, so no red step	—

After the three commits: test_collector_lives_in_its_own_repo.py, test_deploy_script.py and test_deploy_workflow.py gave 30 passed; ruff check passes on them; shellcheck 0.11.0 on deploy-dashboard.sh and actionlint 1.7.12 on deploy.yml report no findings. Locally I ran only these files.

Cleanup after merging (manual; not done by this PR)

All three are needed once this PR has merged and deployed.

• Delete the calibration-diagnostics staging environment: only telemetry-service.yml used it. It still holds the secrets MICROCOSM_TELEMETRY_COLLECTOR_READ_TOKEN and TELEMETRY_HF_QUALIFICATION_TOKEN (names listed 2026-10-20). The plan records both as deleted under its stage 0.1 (2026-10-19), but that deletion removed other copies, not these; they go with the environment. Stage 0.1 also said to rotate the read token if this copy is current, which can't be checked from GitHub because it doesn't return secret values.
• In Production, delete the TELEMETRY_* variables other than TELEMETRY_GCP_PROJECT_ID and TELEMETRY_READ_SECRET_NAME. Keep both TELEMETRY_GCP_* secrets and MICROCOSM_TELEMETRY_COLLECTOR_URL.
• Give the dashboard deploy its own service account. deploy.yml authenticates as the service account in TELEMETRY_GCP_DEPLOY_SERVICE_ACCOUNT, which is the collector's deploy service account, only to run gcloud secrets describe on the read secret. That account has project-level roles/run.admin and a role that can create Cloud SQL backups (found in review; not re-read for this update).
  1. Create a service account with only roles/secretmanager.viewer on the read secret (the secret, not the project).
  2. Let the github/calibration-diagnostics provider's principalSet://…/attribute.repository/PolicyEngine/calibration-diagnostics use it (roles/iam.workloadIdentityUser on the new account).
  3. Set the Production secret TELEMETRY_GCP_DEPLOY_SERVICE_ACCOUNT to the new account, and check that the next dashboard deploy's "Check collector read secret exists" step passes.
  4. Remove calibration-diagnostics' roles/iam.workloadIdentityUser binding from the collector's deploy service account, so this repo can no longer act as it.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
calibration-diagnostics	Ready	Preview	Oct 22, 2026 1:16pm UTC