[anth-volk]

Fixes #7

Based on main. #6 (the submission contract) merged on 2026-10-27 (35680a2) and was released as 0.4.0, now deployed to staging and production; GitHub moved this PR's base to main when #6's branch was deleted. main is merged in again (8741383), bringing #25 (released as 0.4.1), so the diff shows only intake and this PR's merge releases 0.5.0. A merge of this PR deploys staging and production, and its migration job applies 0002_artifact_intake. The intake tables get the runtime user's access from the migration user's default privileges, which both environments already have (checked 2026-10-27). Before it merges, only check each environment with \ddp: see "Default privileges: check before merging" below.

Summary

Implements docs/artifact-submissions.md in the collector, switched off by default.

• Endpoints (mounted only while intake is on):
  • POST /v1/submissions validates the submission, records it under its derived ID (201 new, 200 repeat) and returns a signed upload instruction for each file not yet stored.
  • POST /v1/submissions/{id}/complete lists the submission's objects. Missing files get 409 files_missing. A stored object of another size or recorded SHA-256 rejects the submission (stored_object_mismatch), because stored objects are never replaced. Otherwise it records each object's generation and the round 1 job, and dispatches it through Cloud Tasks. If Cloud Tasks is unavailable it answers 503 worker_queue_unavailable with Retry-After.
  • GET /v1/submissions/{id} (status, with the current job) and GET /v1/submissions/{id}/receipt (the final record with each file's generation and its own SHA-256; 409 submission_not_final until final).
  • Intake errors carry a code beside detail: submission_not_found, submission_owned_by_another_user, files_missing, submission_not_final, worker_queue_unavailable, intake_not_enabled.
• Signed uploads are V4 PUT URLs signed through IAM signBlob with the runtime's own credentials (no key file). Each instruction lists every header the signature covers: Content-Type, Host (from the URL), the declared SHA-256 as object metadata, an exact-size x-goog-content-length-range and x-goog-if-generation-match: 0. URLs are signed 16 at a time so a 256-file response stays inside the 30 s request timeout.
• Bytes never pass through the API. Completion reads only object metadata. The worker recomputes every SHA-256.
• Worker framework (python -m telemetry_collector.worker, a Cloud Run Job). Each dispatch is a Cloud Task named <job_id>-<dispatch number> that calls the Cloud Run Admin API to run the job with COLLECTOR_WORKER_JOB_ID set. The worker leases the job for 65 minutes (longer than the one-hour task timeout) under CLOUD_RUN_EXECUTION, streams each file at its recorded generation, checks size and SHA-256, then calls the handler registered for the kind. Both kinds start with a handler that only records the check ({"processed_by": "intake-only", ...}). Handlers get the disclosure rule from JobContext.output_visibility.
• Recovery and reprocessing (python -m telemetry_collector.operations):
  • recover, every 10 minutes: fails a job whose lease expired after its third attempt (attempts_exhausted) or that would need a fourth dispatch (worker_unavailable), and dispatches again a job whose attempt ended early or that stayed queued for 30 minutes.
  • reprocess SUBMISSION_ID... or reprocess --kind <kind> --intake-only [--dry-run]: records the next processing round of final submissions and dispatches it.
• PostgreSQL: Alembic revision 0002_artifact_intake adds artifact_submissions and artifact_jobs (status check constraints, one job per submission and processing round, attempts, dispatch count, lease, result). The migration grants nothing: the runtime database user's access to the two tables comes from the migration user's default privileges, set once by hand per environment and already in place in staging and production (see "Default privileges: check before merging"). Downgrading drops only these two tables.
• Fakes in tests: Google Cloud Storage and Cloud Tasks sit behind two interfaces (IntakeObjectStore, WorkerQueue). Endpoint and worker tests use in-memory fakes; the in-memory submission store shares the PostgreSQL store's decision functions and runs the same contract tests. One adapter test signs with the real google-cloud-storage library, standing in only for the IAM signBlob request.

Auth: one scope per token

• The exchange accepts an optional body, {"scope": "artifacts:submit"}. No body, or {}, still yields a telemetry:write token with the same claims as before.
• The token response now has a scope field. A collector released before intake ignores the body and returns no scope, so a client can tell it has no intake.
• Submission endpoints require artifacts:submit, and telemetry endpoints still require telemetry:write.
• C5 (ES256 and JWKS) edits the same token functions in app.py (_issue_session_token, _decode_session_token), so whichever merges second resolves a small conflict. This PR's only change there since the last review is scope=scope in the returned CollectorToken.

Switched off by default

• Intake is on only when COLLECTOR_INTAKE_ENABLED is exactly 1. While it is off, the collector reads no other intake setting and doesn't import the Google clients, and the exchange for artifacts:submit and every /v1/submissions path answer 404 with {"code": "intake_not_enabled"} (the paths are absent from OpenAPI).
• scripts/deploy.sh is unchanged. Its --set-env-vars sets only TELEMETRY_MAINTENANCE_MODE, which replaces the service's variables on every deploy, so the switch can't be on in any deployed revision.
• What a deploy does change: the migration job applies 0002_artifact_intake (two empty tables, whose access comes from the default privileges), and token responses gain scope.

Commits (red, then green)

Each commit's test files were run before the change (red, against stubs for names that didn't exist yet, so failures counted individually) and after it (green), then again after committing. PostgreSQL cases ran against a disposable local PostgreSQL 16 container.

Earlier commits:

1. Validate artifact submissions and derive their identifiers: red 39 failed against a stub; green 42 passed.
2. Issue artifacts:submit credentials only while intake is switched on: red 22 failed; green 22 passed.
3. Record artifact submissions and their worker jobs in PostgreSQL: red 36 of 37; green 37 passed.
4. Sign intake uploads and enqueue worker jobs on Google Cloud: red 10 failed; green 10 passed.
5. Accept submissions through signed uploads and enqueue completed ones: red 16 of 18; green 18 passed.
6. Run artifact jobs in a worker framework on Cloud Run Jobs: red 17 failed; green 27 passed.
7. Give the token scope tests the intake services create_app requires: red 5 failed; green 7 passed.
8. Keep the repo's vocabulary and list the intake tables in the ORM test: red 2 failed in CI; green 19 passed.

Review fixes (2026-10-20):

9. Merge claude/submission-contract into claude/submission-intake (1396a98): carries Document the artifact submission contract #6's revised contract. Before it, the intake test files passed (151).
10. Check provenance syntax and name the files digest files_sha256 (d1799f3): build_id, release and folder segments use the emitter's safe_identifier rule, repo parts are at most 96 characters, folders at most 512; files_sha256; SubmissionRecord.status and kind use SubmissionStatus and SubmissionKind; the status sets are defined once in submissions.py; the intake prefix comes from intake_object_prefix. Red 16 failed, 108 passed; green 124 passed. During green, the 512-character folder test itself built 513 characters, and was corrected. The new test that the migration's status constraints equal the models' passed in red too, as it guards an equality that already held.
11. Let a client recognise a collector without intake (2866df3): scope in token responses, intake_not_enabled on the exchange and every submission path, codes on intake errors. Red 10 failed, 15 passed; green 25 passed.
12. Reject a submission whose stored object can never match (b5bb54f): stored_object_mismatch at completion, create signs only missing files, the status document reports the job. Red 10 failed, 49 passed; green 59 passed.
13. List every signed header in each upload instruction (ad876ef): red 2 failed, 31 passed; green 33 passed. The new test that signs with the real library passed in red as well: google-cloud-storage 3.17.1 adds Content-Type and Host to the headers dict it is given, which leaked into the instructions unlisted and untested. The adapter now passes the library a copy and lists all five headers itself.
14. Recover jobs whose attempt ended early, and reprocess final submissions (92fa3b7): processing rounds, numbered dispatches, recovery, reprocessing, the operator commands, the 65-minute lease, the README's worker settings, and PostgreSQL tests that run completions, claims and recoveries concurrently. One test claims the job at its third attempt and ends that attempt without finishing or releasing it. Red 65 failed, 134 passed (the two concurrency tests for completions and for claims passed, as the row locks already held; the two for recoveries failed); green 210 passed.
15. Give handlers the disclosure rule from decision D10 (9991bd8): red 7 failed; green 26 passed.
16. Give the runtime database user the same access to the intake tables (1372cab): red 1 failed, 5 passed; green 12 passed. Replaced by 19 and 20.
17. Report every timestamp in UTC, and describe the intake changes (30dbd1b): red 1 failed; green 22 passed.

Update from main (2026-10-27):

18. Merge main into claude/submission-intake (e57c569): brings in Document the artifact submission contract #6 (released as 0.4.0), Refuse extraction records that squash or rebase merges dropped from main #4, Carry over calibration-diagnostics#219: apply each event batch incrementally #11 and Lock certifi 2026.10.19 to stop trusting the compromised eMudhra CA certificates #23, with no conflicts. README.md, pyproject.toml and uv.lock merged automatically: the README keeps both sides' sections, pyproject.toml has version 0.4.0 with this PR's dependencies, and uv.lock has both this PR's Google packages and certifi 2026.10.19 from Lock certifi 2026.10.19 to stop trusting the compromised eMudhra CA certificates #23 (uv lock --check passes). Document the artifact submission contract #6's consumed fragment changelog.d/artifact-submission-contract.added.md is removed, so artifact-intake.added.md is the only fragment. The contract change Document the artifact submission contract #6 made after this PR's last merge of its branch (393b948: a receipt for a submission rejected at completion has no generation in files) matches the code: reject_submission records no generations (checked by reading the code; no test asserts it).

Default privileges instead of copied grants (2026-10-27):

19. Test that intake tables get their access from default privileges (b6071ed): replaces the test of copying telemetry_runs' grants with three tests that upgrade as a deployment does. A migration role owns the tables, the telemetry tables are granted by hand to the runtime role and to one other role, and the migration role's default privileges are set as the statement below sets them. The tests check that the migration issues no GRANT; that the runtime role has SELECT, INSERT, UPDATE and DELETE on both intake tables and a role with no grants has none; and that the other role, granted telemetry_runs but not named in the default privileges, gets nothing. Red (tests/test_migrations_postgres.py): 2 failed, 6 passed. The default-privileges test passed in red too, as PostgreSQL applies default privileges whatever the migration does; the other two failed because the migration issued 8 GRANTs and gave the other role all four privileges.
20. Stop copying telemetry_runs' grants onto the intake tables (16f1f88): removes grant_telemetry_access_to_intake_tables() and its catalog query. Green: 8 passed.
21. Document the default privileges the intake tables rely on (f8b0895): the README's "Database access" gives the statement, who can run it, how to check it and when sequences would need it; the changelog fragment says the migration grants nothing.

Review of commits 19 to 21, and main merged in (2026-10-27):

22. Merge main into claude/submission-intake (8741383): brings in Test the Hugging Face check's own trusted roots by fingerprint, and name certifi 2026.10.19 as the advisory's first fixed version #25 (released as 0.4.1): test_certificate_authorities.py now checks the trusted roots of the connection HuggingFaceAuthenticator opens, by fingerprint, and CHANGELOG.md gains its 0.4.1 entry. No conflicts. pyproject.toml and uv.lock take version 0.4.1 and keep this PR's Google dependencies (uv lock --check passes), and artifact-intake.added.md is still the only fragment, so this PR's merge releases 0.5.0.
23. Say both environments already have the default privileges (b1495a6): the README's "Database access" said each environment sets them before 0002_artifact_intake is applied. It now says staging and production both have them (checked 2026-10-27), so before a merge there is only a \ddp check, and that the statement applies again when Record the project and hostname decisions (D2, D3) and the runbook for the hostname (C2) and the move (C5) #21's project move recreates the database users. The intake checklist now lists all four privileges the default privileges grant. Documentation only.

Testing

• After commits 22 and 23 (b1495a6), on Python 3.13 with TEST_DATABASE_URL pointing at a disposable local PostgreSQL 16.15: the test files for what changed (test_migrations_postgres.py, test_migrate.py, test_database.py, main's test_certificate_authorities.py, test_auth.py, the changelog's test_release_versioning.py, the lock file's test_packaging.py, the README's test_quality_check.py, test_extraction.py and the vocabulary test's test_deployment_workflow.py), together: 75 passed, none skipped. The review also reran test_migrations_postgres.py against commit 19's migration: 2 failed (test_artifact_intake_migration_issues_no_grant, test_artifact_intake_gives_telemetry_grantees_no_intake_access), 6 passed; and at f8b0895: 8 passed. python -m telemetry_collector.migrate then alembic check reports "No new upgrade operations detected."; alembic upgrade 0001_initial_schema:0002_artifact_intake --sql prints no GRANT; tools/check-quality.sh (Ruff check, format check, bash -n) passes; uv lock --check passes; tools/check-changelog.sh reports 0.4.1 to 0.5.0.
• After commits 19 to 21 (f8b0895), on Python 3.13 with TEST_DATABASE_URL pointing at a disposable local PostgreSQL 16.15: the test files for what changed (test_migrations_postgres.py, test_migrate.py, test_database.py, the vocabulary test's test_deployment_workflow.py, the README's test_quality_check.py, the changelog's test_release_versioning.py), together: 53 passed. alembic upgrade 0001_initial_schema:0002_artifact_intake --sql prints no GRANT; python -m telemetry_collector.migrate then alembic check reports "No new upgrade operations detected."; tools/check-quality.sh passes; tools/check-changelog.sh reports 0.4.0 to 0.5.0 (the branch doesn't contain Test the Hugging Face check's own trusted roots by fingerprint, and name certifi 2026.10.19 as the advisory's first fixed version #25 yet, which released 0.4.1). CI passes on f8b0895.
• After the merge of main (e57c569), on Python 3.13 with TEST_DATABASE_URL pointing at a disposable local PostgreSQL 16: this PR's 11 test files, the test files for what main changed or what both sides changed (test_app.py, test_storage.py, test_storage_postgres.py, test_incremental_batch_apply.py, test_certificate_authorities.py, test_packaging.py, test_release_versioning.py, test_quality_check.py) and test_deployment_workflow.py, together: 337 passed, none skipped. python -m telemetry_collector.migrate on an empty database reaches 0002_artifact_intake, and alembic check reports "No new upgrade operations detected." tools/check-quality.sh passes, and tools/check-changelog.sh reports 0.4.0 to 0.5.0: this PR's one added fragment raises the minor version, so its merge releases 0.5.0. The repository defines no type checker.
• Before the merge, every test file this branch adds or changes, together, with TEST_DATABASE_URL pointing at a disposable local PostgreSQL 16: 241 passed.
• python -m telemetry_collector.migrate then alembic check on that database: "No new upgrade operations detected."
• tools/check-quality.sh (Ruff check and format check, bash -n on scripts) passes. tools/check-changelog.sh passed then too (0.3.0 to 0.4.0, while Document the artifact submission contract #6's fragment was still pending).
• I didn't run the rest of the suite locally, including test_app.py and the vocabulary test in test_deployment_workflow.py, although app.py changed. CI covers them.

Deployment verification

This PR changes no workflow, Dockerfile or deploy script (checked against main after the merge). uv.lock gains google-cloud-storage, google-api-core, google-auth and requests (no gRPC), which the existing Docker build installs.

Variable	Read by	Provider	Wiring
COLLECTOR_INTAKE_ENABLED	service	none; unset means off	Not passed by deploy.sh, so off in every revision (checked in deploy_maintenance and deploy_candidate).
COLLECTOR_INTAKE_BUCKET	service, worker	none yet	Unverified. To be a GitHub staging/production environment variable passed by deploy.sh, and set on the worker job.
COLLECTOR_INTAKE_SIGNING_SERVICE_ACCOUNT	service	none yet; meant to be the existing TELEMETRY_RUNTIME_SERVICE_ACCOUNT value	Unverified, as above. The setting accepts only user-managed accounts (<name>@<project>.iam.gserviceaccount.com) and refuses the default Compute Engine account. I couldn't confirm which account TELEMETRY_RUNTIME_SERVICE_ACCOUNT names: reading the environment variable was refused.
COLLECTOR_WORKER_QUEUE	service, operations	none yet	Unverified. To be passed by deploy.sh to the service and set on the recovery job.
COLLECTOR_WORKER_JOB	service, operations	none yet	Unverified, as above.
COLLECTOR_WORKER_INVOKER_SERVICE_ACCOUNT	service, operations	none yet	Unverified, as above; the same service-account pattern applies.
COLLECTOR_WORKER_JOB_ID	worker	each Cloud Task's jobs.run environment override	Unverified: no queue or job exists.
DATABASE_URL	worker, operations	Google Secret Manager, the runtime database secret (TELEMETRY_DATABASE_SECRET_NAME)	Unverified: to be attached to the worker and recovery jobs with --set-secrets.
CLOUD_RUN_EXECUTION	worker	set by Cloud Run itself	Falls back to a random holder name outside Cloud Run.

On 2026-10-27 the GitHub staging and production environments still had no COLLECTOR_INTAKE_* or COLLECTOR_WORKER_* variable (only the TELEMETRY_* ones push.yaml passes to deploy.sh), and deploy.sh on main still passes only TELEMETRY_MAINTENANCE_MODE with --set-env-vars in both deploy_maintenance and deploy_candidate. None of these is a secret value. The switch-on PR adds the deployment wiring: deploy.sh passing the five service settings with COLLECTOR_INTAKE_ENABLED=1, and updating both jobs' images each release, as it does for the migration job.

Database connections. The migration job connects with the migration database secret, so its user owns the tables it creates; the service, worker and operator commands connect as the runtime user. The migration grants nothing; the runtime user's access to the intake tables comes from the migration user's default privileges, which both environments already have (see "Default privileges: check before merging").

Setting	Read by	Provider	How it reaches the code	Verified
DATABASE_URL in the migration job	telemetry_collector.migrate.main, migrations/env.py	Google Secret Manager, the migration database secret	GitHub staging/production environment variable TELEMETRY_MIGRATION_DATABASE_SECRET_NAME → push.yaml job env MIGRATION_DATABASE_SECRET_NAME → deploy.sh configure_migration_job, --set-secrets DATABASE_URL=<secret>:latest on the job, which runs as TELEMETRY_MIGRATION_SERVICE_ACCOUNT with the Cloud SQL instance TELEMETRY_CLOUD_SQL_CONNECTION	Yes. The variables exist in both environments (names read through the GitHub API, values not read). The release run of 2026-10-27 12:45 UTC (40506286831) passed "Check required Secret Manager entries exist", "Configure the database migration job" and "Apply database migrations" in staging and production, so the secret exists, the job's account can read it, and its user can connect and migrate.
DATABASE_URL in the service	create_app_from_environment (app.py)	Secret Manager, the runtime database secret	TELEMETRY_DATABASE_SECRET_NAME → DATABASE_SECRET_NAME → deploy.sh deploy_maintenance and deploy_candidate, --set-secrets DATABASE_URL=<secret>:latest on the service, which runs as TELEMETRY_RUNTIME_SERVICE_ACCOUNT	Yes, by the same run: the candidate revisions in both environments passed /ready, which connects to the database and reads the Alembic revision.
DATABASE_URL in the worker and recovery jobs	worker.py, operations.py	Secret Manager, the runtime database secret	Not wired yet: manual steps 6 and 8 below.	Unverified, as in the table above.
<migration_role> and <runtime_role>	the person checking the default privileges, or running the statement where an entry is missing	the database user names inside the migration and runtime database secrets	Not read by any code.	Yes, in both environments, by a read-only run of each migration job on 2026-10-27: <migration user> grants <runtime user> arwd on new tables in public.

Default privileges: check before merging (per environment)

Default privileges apply only to tables created after they are set, and this PR's merge applies 0002_artifact_intake in staging and then production. Both environments already have them (see "Status"), so before this PR merges, only check each one with \ddp; there is nothing to run. Where an entry is missing, this sets it:

ALTER DEFAULT PRIVILEGES FOR ROLE <migration_role> IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO <runtime_role>;

• The roles: <migration_role> is the database user in the migration database secret, the Secret Manager secret named by the environment variable TELEMETRY_MIGRATION_DATABASE_SECRET_NAME, which the migration job mounts as DATABASE_URL. <runtime_role> is the user in the runtime database secret, named by TELEMETRY_DATABASE_SECRET_NAME, which the service mounts as DATABASE_URL.

• Status: staging and production both have it. A read-only run of each environment's migration job on 2026-10-27 found that <migration user>, which owns all four tables in public, grants <runtime user> SELECT, INSERT, UPDATE and DELETE on new tables in public ({<runtime user>=arwd/<migration user>}), and that neither database has any sequences. An older third database user in production had no grants on telemetry_runs and no connections, so the grant copying this PR removed would have given it nothing either; it was deleted from production later on 2026-10-27. Check each environment with \ddp before this PR merges; run the statement only where the entry is missing (running it again changes nothing).

• When it applies again: Record the project and hostname decisions (D2, D3) and the runbook for the hostname (C2) and the move (C5) #21's project move recreates the database users. A default-privileges entry belongs to the role itself, not its name, so a recreated <migration user> has one only if the import or the statement creates it. A dump carries the entry as an ALTER DEFAULT PRIVILEGES FOR ROLE statement, and if the import runs as a user without INHERIT membership in the migration user, PostgreSQL 16 refuses it (see "Who can run it"), so the entry can be missing; how Cloud SQL's import treats that error isn't verified. Check \ddp in each new database before a migration job runs there (Record the project and hostname decisions (D2, D3) and the runbook for the hostname (C2) and the move (C5) #21's step 6.4 compares it with the step 3.2 record), and run the statement if the entry is missing.

• Who can run it: connected to the collector's database as the migration user itself, or as a member of it with the INHERIT option. On Cloud SQL that is typically postgres, if it is such a member. PostgreSQL 16 refuses the statement ("permission denied to change default privileges") for a member with only ADMIN or SET, and postgres holds a role it created in SQL with only ADMIN. In that case either connect as the migration user, or have postgres run GRANT <migration_role> TO postgres WITH INHERIT TRUE; first and REVOKE <migration_role> FROM postgres; after, which leaves its original ADMIN membership. postgres isn't a member of a user that gcloud sql users create made, so for that user connect as the migration user. (All checked on PostgreSQL 16.15. The review confirmed the rule in PostgreSQL 16's source: ExecAlterDefaultPrivilegesStmt requires has_privs_of_role, which follows only memberships with INHERIT, since commit 48a257d "Make ALTER DEFAULT PRIVILEGES require privileges, not membership"; PostgreSQL 15 required membership only. PostgreSQL 16's "Role Attributes" page says a role made by a CREATEROLE user that isn't a superuser is granted back to it with ADMIN but "INHERIT FALSE, SET FALSE", and that the creator "can gain access to the created role by simply granting that role back to themselves with the INHERIT and/or SET options".)

• How to check it: before the merge, \ddp shows a row with owner <migration_role>, schema public, type table, and access privileges <runtime_role>=arwd/<migration_role>. After each environment's migration, every row of this is true:

  SELECT table_name, privilege, has_table_privilege('<runtime_role>', table_name, privilege)
  FROM unnest(ARRAY['artifact_submissions', 'artifact_jobs']) AS table_name,
       unnest(ARRAY['SELECT', 'INSERT', 'UPDATE', 'DELETE']) AS privilege;

• If the migration runs first: the two tables exist without the runtime user's access. Nothing reads them while intake is off. Grant them once, as the migration user, with GRANT SELECT, INSERT, UPDATE, DELETE ON artifact_submissions, artifact_jobs TO <runtime_role>;, then set the default privileges for later tables.

• Sequences: neither migration has a serial or identity column (checked on a database migrated to head: no sequences, identity columns or nextval defaults), and neither staging nor production has any sequences (checked 2026-10-27), so USAGE, SELECT ON SEQUENCES isn't needed. A later migration that adds a serial column would need it: on PostgreSQL 16, the runtime user's insert into a serial column is refused without USAGE on its sequence, while an identity column needs no sequence privilege.

• The telemetry tables: their grants to the runtime user were made by hand when each database was set up, outside this repo; collector#21's runbook records them (\dp public.* and \ddp, step 3.2).

Manual infrastructure steps (not done; run per environment, staging first)

The repo keeps no infrastructure templates, so these are manual. PROJECT, REGION, CLOUD_SQL_CONNECTION, RUNTIME_SA and DATABASE_SECRET are the existing TELEMETRY_* environment values.

Which project these steps use depends on decision D2. Taking PROJECT from the existing TELEMETRY_* values puts the bucket, queue and jobs in today's shared project. If intake is switched on before the move, collector#21's runbook keeps the bucket in the current project. A copy would get new generation numbers, and the worker rejects copied files with object_missing the next time it processes them, for example on reprocessing. If D2 is B or C, the memo recommends switching intake on only after the move. The code reads every resource name from the environment, so only the order of these steps and the project move matters.

# 0. Enable the Cloud Tasks and Cloud Scheduler APIs (neither is enabled in the current project; Cloud Run is)
gcloud services enable cloudtasks.googleapis.com cloudscheduler.googleapis.com --project PROJECT
# 1. Private intake bucket (no lifecycle deletion: files are kept for back-building)
gcloud storage buckets create gs://BUCKET --project PROJECT --location REGION \
  --uniform-bucket-level-access --public-access-prevention
# 2. Queue for worker jobs
gcloud tasks queues create QUEUE --project PROJECT --location REGION \
  --max-attempts=20 --min-backoff=10s --max-backoff=600s
# 3. Service accounts: the worker, the account Cloud Tasks uses to start it, the recovery job, and the account Cloud Scheduler uses to start the recovery job (SCHEDULER_SA in step 8)
gcloud iam service-accounts create collector-worker --project PROJECT
gcloud iam service-accounts create collector-worker-invoker --project PROJECT
gcloud iam service-accounts create collector-recovery --project PROJECT
gcloud iam service-accounts create collector-recovery-scheduler --project PROJECT
# 4. Runtime service account: create and list objects, sign as itself, enqueue, act as the invoker
gcloud storage buckets add-iam-policy-binding gs://BUCKET --member serviceAccount:RUNTIME_SA --role roles/storage.objectCreator
gcloud storage buckets add-iam-policy-binding gs://BUCKET --member serviceAccount:RUNTIME_SA --role roles/storage.objectViewer
gcloud iam service-accounts add-iam-policy-binding RUNTIME_SA --member serviceAccount:RUNTIME_SA --role roles/iam.serviceAccountTokenCreator
gcloud tasks queues add-iam-policy-binding QUEUE --project PROJECT --location REGION --member serviceAccount:RUNTIME_SA --role roles/cloudtasks.enqueuer
gcloud iam service-accounts add-iam-policy-binding INVOKER_SA --member serviceAccount:RUNTIME_SA --role roles/iam.serviceAccountUser
# 5. Worker service account: read the bucket, connect to Cloud SQL, read the runtime database secret
gcloud storage buckets add-iam-policy-binding gs://BUCKET --member serviceAccount:WORKER_SA --role roles/storage.objectViewer
gcloud projects add-iam-policy-binding PROJECT --member serviceAccount:WORKER_SA --role roles/cloudsql.client
gcloud secrets add-iam-policy-binding DATABASE_SECRET --project PROJECT --member serviceAccount:WORKER_SA --role roles/secretmanager.secretAccessor
# 6. Worker job: three attempts in all (MAX_JOB_ATTEMPTS) and a one-hour task timeout (WORKER_TASK_TIMEOUT_SECONDS)
gcloud run jobs deploy WORKER_JOB --project PROJECT --region REGION --image IMAGE \
  --service-account WORKER_SA --set-cloudsql-instances CLOUD_SQL_CONNECTION \
  --set-secrets DATABASE_URL=DATABASE_SECRET:latest --set-env-vars COLLECTOR_INTAKE_BUCKET=BUCKET \
  --command uv --args run,--no-sync,python,-m,telemetry_collector.worker \
  --tasks 1 --max-retries 2 --task-timeout 1h --memory 2Gi
gcloud run jobs add-iam-policy-binding WORKER_JOB --project PROJECT --region REGION \
  --member serviceAccount:INVOKER_SA --role roles/run.jobsExecutorWithOverrides
# 7. Recovery service account: connect to Cloud SQL, read the runtime database secret, enqueue, act as the invoker
gcloud projects add-iam-policy-binding PROJECT --member serviceAccount:RECOVERY_SA --role roles/cloudsql.client
gcloud secrets add-iam-policy-binding DATABASE_SECRET --project PROJECT --member serviceAccount:RECOVERY_SA --role roles/secretmanager.secretAccessor
gcloud tasks queues add-iam-policy-binding QUEUE --project PROJECT --location REGION --member serviceAccount:RECOVERY_SA --role roles/cloudtasks.enqueuer
gcloud iam service-accounts add-iam-policy-binding INVOKER_SA --member serviceAccount:RECOVERY_SA --role roles/iam.serviceAccountUser
# 8. Recovery job, and a schedule that runs it every 10 minutes
gcloud run jobs deploy RECOVERY_JOB --project PROJECT --region REGION --image IMAGE \
  --service-account RECOVERY_SA --set-cloudsql-instances CLOUD_SQL_CONNECTION \
  --set-secrets DATABASE_URL=DATABASE_SECRET:latest \
  --set-env-vars COLLECTOR_WORKER_QUEUE=QUEUE_RESOURCE,COLLECTOR_WORKER_JOB=WORKER_JOB_RESOURCE,COLLECTOR_WORKER_INVOKER_SERVICE_ACCOUNT=INVOKER_SA \
  --command uv --args run,--no-sync,python,-m,telemetry_collector.operations,recover \
  --tasks 1 --max-retries 0 --task-timeout 10m
gcloud run jobs add-iam-policy-binding RECOVERY_JOB --project PROJECT --region REGION \
  --member serviceAccount:SCHEDULER_SA --role roles/run.invoker
gcloud scheduler jobs create http collector-recovery --project PROJECT --location REGION \
  --schedule "*/10 * * * *" --http-method POST \
  --uri "https://run.googleapis.com/v2/projects/PROJECT/locations/REGION/jobs/RECOVERY_JOB:run" \
  --oauth-service-account-email SCHEDULER_SA
# 9. Let the deployment service account update both jobs each release
gcloud iam service-accounts add-iam-policy-binding WORKER_SA --member serviceAccount:DEPLOY_SA --role roles/iam.serviceAccountUser
gcloud iam service-accounts add-iam-policy-binding RECOVERY_SA --member serviceAccount:DEPLOY_SA --role roles/iam.serviceAccountUser

Then create the GitHub staging and production environment variables for the switch-on PR: the bucket, queue, worker job, recovery job and invoker account names, and the worker and recovery service accounts.

Not verified against Google Cloud

No real bucket, queue, job or schedule exists, so these need checking on staging before switching on:

• that a V4 signed PUT enforces x-goog-content-length-range and x-goog-if-generation-match: 0 as documented, and what status an expired URL gets;
• that the Cloud Run Admin API v2 jobs.run applies a containerOverrides entry without a container name to the job's only container;
• that Cloud Tasks answers HTTP 409 for an existing task name, including a name used by a task that has already run;
• that a Cloud Run task killed for memory or by its timeout counts as a failed task attempt, so --max-retries 2 gives the three attempts recovery counts on;
• that Cloud Scheduler with an OAuth token can start the recovery job through jobs.run, and the exact role names above, especially roles/run.jobsExecutorWithOverrides and roles/run.invoker on a job;
• that the runtime database user has the four privileges on both intake tables after migration 0002: both environments have the default privileges, so this is expected, but it can be confirmed only after the migration has run, with the has_table_privilege query in "Default privileges: check before merging".

Review round 2026-10-22

A re-review found no code bugs and confirmed every 2026-10-20 finding fixed. It found three documentation problems, now fixed:

• Rollback (d6aa908): the README said deploying this code changes nothing. A deployment applies migration 0002_artifact_intake and adds scope to token responses. After that migration, an image without it (an earlier collector release, or the telemetry-service/ copy in calibration-diagnostics) fails its migration job ("Can't locate revision identified by '0002_artifact_intake'") and its /ready check, so calibration-diagnostics' manual redeploy would stop at its staging migration. Rolling back first needs alembic downgrade 0001_initial_schema, run with this release's code and as the migration database user, which drops only the two intake tables (checked locally). The README now says so (d6aa908, 8357b0d), and that the token exchange's answers change too.
• Manual steps: step 0 now enables the Cloud Tasks and Cloud Scheduler APIs, and step 3 creates the scheduler's service account that step 8 uses.
• Project: the manual steps now say they depend on decision D2, as above.
• A second review of these documentation fixes confirmed the rollback failure and the downgrade on a disposable PostgreSQL 16, and that step 0 is needed; its two corrections are applied.

Review round 2026-10-27

A review of commits 19 to 21 found no code bugs. This description and the README were out of date with the 2026-10-27 production check, and are now corrected.

• The tests prove the change. Against commit 19's migration, which still copied grants, two fail and the default-privileges test passes, as commit 19 says; at f8b0895 all eight pass. The statement recorder also sees CREATE TABLE artifact_jobs, so an empty recording can't pass the no-GRANT test. Its pattern matches only statements that begin with GRANT, and it records only what runs after 0001_initial_schema (which has no grants). The test of the role granted only telemetry_runs checks the resulting privileges, not the SQL, so it catches copying by any statement.
• Nothing else relied on the copied grants. The worker, recovery and operator commands connect as the runtime user, whose access now comes from the default privileges both environments have. deploy.sh checks that the secrets exist and that /ready passes, and /ready reads only alembic_version; no code, workflow or test checks table privileges. docs/artifact-submissions.md and CHANGELOG.md don't mention them. The intake code issues SELECT, INSERT, UPDATE and SELECT ... FOR UPDATE, never DELETE.
• The PostgreSQL 16 INHERIT claim is right (sources under "Who can run it"). A throwaway check on PostgreSQL 16.15 also showed that a member with ADMIN only, or SET only, is refused, a self-grant WITH INHERIT TRUE is accepted, and the REVOKE leaves the original ADMIN membership.
• Corrected: this description and the README said production's default privileges hadn't been checked and that each environment must set them before the merge. Both environments already have them, so only a \ddp check is needed; both texts now also say the statement applies again when Record the project and hostname decisions (D2, D3) and the runbook for the hostname (C2) and the move (C5) #21's project move recreates the database users (b1495a6).
• main merged in (8741383) with no conflicts; this PR's merge releases 0.5.0.

🤖 Generated with Claude Code