From 69e95d7d53bd3faf113282d9a76b62aaec2254f0 Mon Sep 17 00:00:00 2001 From: Arul Sharma <31745423+arul28@users.noreply.github.com> Date: Thu, 6 Aug 2026 14:12:01 -0400 Subject: [PATCH] fix(release): guard Worker deployments --- .agents/skills/release/SKILL.md | 32 +++++++++++++++---- .github/workflows/deploy-web.yml | 4 +-- apps/account-directory/README.md | 11 ++++--- apps/account-directory/package.json | 4 +-- .../scripts/verify-deployment-config.d.mts | 1 + .../scripts/verify-deployment-config.mjs | 25 +++++++++++---- .../test/verifyDeploymentConfig.test.ts | 14 ++++++++ apps/webhook-relay/README.md | 5 +-- apps/webhook-relay/package.json | 2 +- 9 files changed, 74 insertions(+), 24 deletions(-) diff --git a/.agents/skills/release/SKILL.md b/.agents/skills/release/SKILL.md index 97e4d5130..690d5da5c 100644 --- a/.agents/skills/release/SKILL.md +++ b/.agents/skills/release/SKILL.md @@ -661,25 +661,45 @@ done git log $LAST_TAG..origin/main --oneline -- apps/desktop/src/renderer/webclient ``` -Deploy each changed surface from the release commit on main (never a lane): +Deploy each changed surface from the release commit on main (never a lane). +Use the package-owned deployment entry point for every Worker, after installing +that app's locked dependencies. These entry points are the release guardrails: +each owns the applicable D1/Durable Object migrations, required +bindings/secrets, and post-deploy health/auth checks for that Worker. If one +stops on a preflight, missing migration, or missing smoke credential, repair +that blocker and rerun the entry point; do not bypass it with `npx wrangler deploy`. ```bash # Hosted web client (Cloudflare Pages project ade-web-client) npm --prefix apps/desktop run build:webclient npx wrangler pages deploy apps/desktop/dist/web-client --project-name ade-web-client -# Workers (npm install first if node_modules is stale in the checkout) -(cd apps/account-directory && npx wrangler deploy --env production) -(cd apps/webhook-relay && npx wrangler deploy) -(cd apps/tunnel-relay && npx wrangler deploy) # only when changed -(cd apps/push-relay && npx wrangler deploy) # only when changed +# Workers (npm ci first if node_modules is stale in the checkout) +(cd apps/account-directory && npm ci && npm run deploy:production) +(cd apps/webhook-relay && npm ci && npm run deploy) +(cd apps/tunnel-relay && npm ci && npm run deploy) # only when changed +(cd apps/push-relay && npm ci && npm run deploy) # only when changed ``` +One-time gotcha: on 2026-08-06, a direct Wrangler deployment published new +account-directory code while production D1 migrations `0004` and `0005` were +still pending and `DIRECTORY_AUTH_SECRET` was not bound. Authenticated machine +list/register requests then returned HTTP 500 across devices even though the +Worker `/health` endpoint was green. The recovery was to restore the shared +secret, apply the pending migrations, and rerun the guarded production deploy. +Treat a guarded-deploy failure as a release blocker, not as a reason to fall +back to raw Wrangler commands. + Verify after deploying: - `curl https://ade-account-directory-production.arulsharma1028.workers.dev/health` → `{"ok":true}` - `curl -s https://app.ade-app.dev | grep -oE 'index-[A-Za-z0-9]+\.js'` matches the freshly built bundle hash in `apps/desktop/dist/web-client/assets/`. +- For every D1-backed Worker, confirm the deploy output reports the expected + migrations applied and no pending migrations remain; `/health` alone is not + sufficient because it can stay green while authenticated routes are broken. +- For authenticated Workers, verify both the required secret bindings and one + authenticated endpoint after deployment. Do not print token or secret values. - Workers with Durable Object migrations (e.g. webhook-relay REPO_EVENTS) apply them on deploy — read the wrangler output for migration errors. diff --git a/.github/workflows/deploy-web.yml b/.github/workflows/deploy-web.yml index baba6ba86..594b57dec 100644 --- a/.github/workflows/deploy-web.yml +++ b/.github/workflows/deploy-web.yml @@ -111,7 +111,7 @@ jobs: node-version: 22 - run: cd apps/webhook-relay && npm ci - name: Deploy Worker - run: cd apps/webhook-relay && npx wrangler deploy + run: cd apps/webhook-relay && npm run deploy tunnel-relay: needs: changes @@ -124,7 +124,7 @@ jobs: node-version: 22 - run: cd apps/tunnel-relay && npm ci - name: Deploy Worker - run: cd apps/tunnel-relay && npx wrangler deploy --tag "$GITHUB_SHA" --message "main $GITHUB_SHA" + run: cd apps/tunnel-relay && npm run deploy -- --tag "$GITHUB_SHA" --message "main $GITHUB_SHA" - name: Verify deployed protocol and Worker version run: | for attempt in {1..18}; do diff --git a/apps/account-directory/README.md b/apps/account-directory/README.md index b4e2f114f..98b2a5ae3 100644 --- a/apps/account-directory/README.md +++ b/apps/account-directory/README.md @@ -116,11 +116,12 @@ deployment: both fail loudly without it. 3. Apply the remote migrations and deploy the Worker. Use `npm run d1:migrate:production` and `npm run deploy:production` for the - production environment. Both deploy scripts run - `npm run verify:deploy-config` first, which refuses to deploy unless - `DIRECTORY_AUTH_SECRET` and `PUSH_RELAY_URL` are configured for the default - AND production environments — without either one, every machine removal - answers 502 and every re-pair 503. Release builds use the production origin; local + production environment. Each deploy script validates only the environment it + is about to publish, so an unconfigured development Worker cannot block a + production deploy. The check refuses to deploy unless + `DIRECTORY_AUTH_SECRET` and `PUSH_RELAY_URL` are configured for that + environment — without either one, every machine removal answers 502 and every + re-pair 503. Release builds use the production origin; local development uses the development origin. Set the machine-level `ADE_ACCOUNT_DIRECTORY_URL=https://` only for a trusted self-hosted override. diff --git a/apps/account-directory/package.json b/apps/account-directory/package.json index 2fe9e0567..ed7d3f301 100644 --- a/apps/account-directory/package.json +++ b/apps/account-directory/package.json @@ -5,8 +5,8 @@ "type": "module", "scripts": { "dev": "wrangler dev", - "deploy": "npm run verify:deploy-config && npm run d1:migrate:remote && wrangler deploy", - "deploy:production": "npm run verify:deploy-config && npm run d1:migrate:production && wrangler deploy --env production", + "deploy": "npm run verify:deploy-config -- default && npm run d1:migrate:remote && wrangler deploy", + "deploy:production": "npm run verify:deploy-config -- production && npm run d1:migrate:production && wrangler deploy --env production", "verify:deploy-config": "node scripts/verify-deployment-config.mjs", "build": "wrangler deploy --dry-run --outdir dist", "build:production": "wrangler deploy --dry-run --env production --outdir dist-production", diff --git a/apps/account-directory/scripts/verify-deployment-config.d.mts b/apps/account-directory/scripts/verify-deployment-config.d.mts index f0c88f506..8d050908a 100644 --- a/apps/account-directory/scripts/verify-deployment-config.d.mts +++ b/apps/account-directory/scripts/verify-deployment-config.d.mts @@ -7,6 +7,7 @@ export declare class DeploymentConfigError extends Error {} export declare function parseJsonc(source: string): unknown; export declare function verifyDirectoryDeploymentConfig(args: { + environments?: readonly string[]; listSecretNames: (environment: string) => Iterable; readConfig: () => unknown; }): void; diff --git a/apps/account-directory/scripts/verify-deployment-config.mjs b/apps/account-directory/scripts/verify-deployment-config.mjs index d6b5b7875..1f298f1ed 100644 --- a/apps/account-directory/scripts/verify-deployment-config.mjs +++ b/apps/account-directory/scripts/verify-deployment-config.mjs @@ -17,9 +17,10 @@ import { dirname, resolve } from "node:path"; * * `DIRECTORY_AUTH_SECRET` is a wrangler SECRET, so it is read from * `wrangler secret list`; `PUSH_RELAY_URL` is a plain var, so it is read from - * the committed `wrangler.jsonc`. Both are checked for the default environment - * AND for `--env production`: wrangler environments do not inherit secrets, and - * a production deploy that inherits nothing is exactly the case that breaks. + * the committed `wrangler.jsonc`. The deploy entry point passes the exact + * environment it is about to publish: wrangler environments do not inherit + * secrets, and a production deploy must not be blocked by an unrelated local + * development environment that is intentionally unconfigured. */ export const REQUIRED_SECRETS = ["DIRECTORY_AUTH_SECRET"]; @@ -84,7 +85,16 @@ function varsForEnvironment(config, environment) { * @param {() => object} args.readConfig */ export function verifyDirectoryDeploymentConfig(args) { - for (const environment of ENVIRONMENTS) { + const environments = args.environments ?? ENVIRONMENTS; + const unknownEnvironments = environments.filter( + (environment) => !ENVIRONMENTS.includes(environment), + ); + if (unknownEnvironments.length > 0) { + throw new DeploymentConfigError( + `unknown Worker environment(s): ${unknownEnvironments.join(", ")}`, + ); + } + for (const environment of environments) { const secretNames = new Set(args.listSecretNames(environment)); const missingSecrets = REQUIRED_SECRETS.filter((name) => !secretNames.has(name)); if (missingSecrets.length > 0) { @@ -94,7 +104,7 @@ export function verifyDirectoryDeploymentConfig(args) { } } const config = args.readConfig(); - for (const environment of ENVIRONMENTS) { + for (const environment of environments) { const vars = varsForEnvironment(config, environment); const missingVars = REQUIRED_VARS.filter( (name) => typeof vars[name] !== "string" || !vars[name].trim(), @@ -175,6 +185,8 @@ export function wranglerSecretNames(environment, overrides = {}) { } function main() { + const requestedEnvironment = process.argv[2]; + const environments = requestedEnvironment ? [requestedEnvironment] : ENVIRONMENTS; const configPath = resolve( dirname(fileURLToPath(import.meta.url)), "..", @@ -182,6 +194,7 @@ function main() { ); try { verifyDirectoryDeploymentConfig({ + environments, listSecretNames: wranglerSecretNames, readConfig: () => parseJsonc(readFileSync(configPath, "utf8")), }); @@ -194,7 +207,7 @@ function main() { process.exit(1); } console.log( - "Account directory relay hand-off configuration is complete for the default and production environments.", + `Account directory relay hand-off configuration is complete for the ${environments.join(" and ")} environment${environments.length === 1 ? "" : "s"}.`, ); } diff --git a/apps/account-directory/test/verifyDeploymentConfig.test.ts b/apps/account-directory/test/verifyDeploymentConfig.test.ts index 5f6fd5fb5..70d7c00c6 100644 --- a/apps/account-directory/test/verifyDeploymentConfig.test.ts +++ b/apps/account-directory/test/verifyDeploymentConfig.test.ts @@ -23,10 +23,12 @@ const completeConfig = { }; function verify(args: { + environments?: string[]; secretsByEnvironment?: Record; config?: unknown; }): void { verifyDirectoryDeploymentConfig({ + environments: args.environments, listSecretNames: (environment) => args.secretsByEnvironment?.[environment] ?? ["DIRECTORY_AUTH_SECRET"], readConfig: () => args.config ?? completeConfig, @@ -60,6 +62,18 @@ describe("account directory deployment preflight", () => { ).toThrow(/production environment: DIRECTORY_AUTH_SECRET/); }); + it("does not require the unused default environment for a production deploy", () => { + expect(() => verify({ + environments: ["production"], + secretsByEnvironment: { default: [], production: ["DIRECTORY_AUTH_SECRET"] }, + })).not.toThrow(); + }); + + it("rejects an unknown deployment environment", () => { + expect(() => verify({ environments: ["staging"] })) + .toThrow(/unknown Worker environment\(s\): staging/); + }); + it.each([ [ "the default environment", diff --git a/apps/webhook-relay/README.md b/apps/webhook-relay/README.md index 08629d8d0..bb2349724 100644 --- a/apps/webhook-relay/README.md +++ b/apps/webhook-relay/README.md @@ -59,12 +59,13 @@ done. Create the D1 database, paste the returned database id into `wrangler.jsonc`, then apply D1 migrations. The repo-events Durable Object binding and its SQLite class migration live in `wrangler.jsonc` and are applied by the coordinator's -normal Worker deployment: +normal Worker deployment. The package-owned deploy command applies the remote +D1 migrations before publishing the Worker, so use it for every deployment: ```sh cd apps/webhook-relay npx wrangler d1 create ade-github-relay -npm run d1:migrate:remote +npm run deploy ``` Set Worker secrets. Do not commit these values: diff --git a/apps/webhook-relay/package.json b/apps/webhook-relay/package.json index 56d5ca8b3..1e65552dd 100644 --- a/apps/webhook-relay/package.json +++ b/apps/webhook-relay/package.json @@ -5,7 +5,7 @@ "type": "module", "scripts": { "dev": "wrangler dev", - "deploy": "wrangler deploy", + "deploy": "npm run d1:migrate:remote && wrangler deploy", "d1:migrate:local": "wrangler d1 migrations apply ade-github-relay --local", "d1:migrate:remote": "wrangler d1 migrations apply ade-github-relay --remote", "build": "wrangler deploy --dry-run --outdir dist",