Skip to content

feat(build): nodebench-build-sha meta + post-deploy readback - #626

Closed
HomenShum wants to merge 1 commit into
mainfrom
foyer/identity-nodebench-build-sha
Closed

feat(build): nodebench-build-sha meta + post-deploy readback#626
HomenShum wants to merge 1 commit into
mainfrom
foyer/identity-nodebench-build-sha

Conversation

@HomenShum

Copy link
Copy Markdown
Owner

Summary

  • Adds a Vite plugin (copies NodeVoice PR feat: research API + eval harness + premium home (2026-04-23) #10's pattern, itself adapted from node-foyer/vite.config.ts's foyer-build-sha) that stamps exactly one <meta name="nodebench-build-sha" content="<sha>" data-provenance="commit"> into the built apps/web/index.html. Precedence: VERCEL_GIT_COMMIT_SHA, then GITHUB_SHA, then git rev-parse HEAD. Non-strict — falls back to content="unavailable" data-provenance="unavailable" rather than throwing when no signal is available.
  • Adds .github/workflows/deploy-verify.yml (on: deployment_status, nothing else) that polls https://www.nodebenchai.com/ up to 3 minutes after a Production deploy succeeds and fails the check if the live meta never equals the deployed commit sha.

This is the FOYER-V3 identity-nodebench worker task (Node Foyer's deployGate: live-dom requires a product's own deploy workflow to read its live identity back; NodeBenchAI currently publishes none — see docs/campaign/MANIFEST.md's FOYER-V2.1 backlog: "NodeBenchAI, NodeProof and agent-workspace-template publish no build identity").

Deploy topology (gh api + vercel evidence)

gh api repos/HomenShum/NodeBenchAI/deployments?per_page=5 shows this repo has one Vercel project deploying it, with the production environment name as the bare string "Production" (unlike NodeVoice's two-sibling-project case, no environment-name disambiguation beyond the state/environment check is needed):

{"environment":"Production", ...}

and the matching status:

{"state":"success","environment":"Production","environment_url":"https://nodebench-cojqx8ock-hshum2018-gmailcoms-projects.vercel.app"}

vercel project ls / vercel project inspect nodebench-ai confirm the Vercel project is nodebench-ai (Framework Preset: Vite, Build Command bash scripts/vercel-build.sh), whose latest-production alias is https://scratchnode.live — but deployment_status.environment_url is always a unique preview-style URL, never the canonical hostname. .github/workflows/post-deploy-verify.yml (pre-existing) already resolves the canonical Production hostname as the fixed string https://www.nodebenchai.com rather than trusting environment_url, so deploy-verify.yml does the same and polls https://www.nodebenchai.com/ directly, matching the task's stated frontend.

KNOWN HAZARD — why the rollback path cannot fire from this check

.github/workflows/post-deploy-verify.yml in this repo runs npx vercel@latest rollback --yes when its own Verify deployed app step fails on a Production deployment_status event, and it is also triggered by a schedule: cron: "0 */6 * * *". Read in full before writing anything: the rollback step's own if: already requires github.event_name == 'deployment_status', so a scheduled run of that workflow never rolls back on its own. The risk this task was asked to close is a new check accidentally sharing or extending that trigger surface.

deploy-verify.yml avoids it structurally, not by relying on post-deploy-verify.yml's existing guard:

  • It is a separate workflow file — GitHub Actions jobs in different files never share job outcomes, so nothing in deploy-verify.yml can be the failure() that post-deploy-verify.yml's rollback step checks.
  • Its trigger is on: deployment_status only — no schedule:, no workflow_dispatch:. It is structurally incapable of running from the 6-hourly cron.
  • It has no rollback step at all. A failure here just fails this one check; nothing it does calls vercel rollback.

So the two workflows run independently off the same GitHub event with no shared state, and this PR's check cannot reach post-deploy-verify.yml's rollback path in either direction.

Local proof (observed, this session)

$ git worktree add .../nodebench-ai-foyer-identity -b foyer/identity-nodebench-build-sha origin/main
HEAD is now at 0afa46c4 fix(convex-mcp): resolve the Convex functions directory from convex.json (#625)

$ npm install --no-audit --no-fund
added 2203 packages in 3m

$ npm run build
✓ built in 1m 7s
PWA v1.3.0 ... files generated

$ grep -o '<meta name="nodebench-build-sha"[^>]*>' dist/index.html
<meta name="nodebench-build-sha" content="0afa46c4a353f1276a3c7f15d0132a05d9cae544" data-provenance="commit">

$ grep -c '<meta name="nodebench-build-sha"' dist/index.html
1

$ git rev-parse HEAD
0afa46c4a353f1276a3c7f15d0132a05d9cae544

Exactly one tag, content equal to the checkout sha (no VERCEL_GIT_COMMIT_SHA/GITHUB_SHA set locally, so it fell through to git rev-parse).

Served the built output locally and curled it (not just grepped the file on disk):

$ npx vite preview --port 4319 --strictPort &
$ curl -s http://127.0.0.1:4319/ | grep -o '<meta name="nodebench-build-sha"[^>]*>'
<meta name="nodebench-build-sha" content="0afa46c4a353f1276a3c7f15d0132a05d9cae544" data-provenance="commit">
$ npx tsc --noEmit --pretty false
(exit 0, no output)

This is the repo's own CI Typecheck job's "App typecheck" step (.github/workflows/ci.yml). The CI job's other two steps — npx convex codegen and npx tsc -p backend/convex --noEmit — were not run locally: convex codegen performs a real network push against a Convex deployment and needs CONVEX_DEPLOY_KEY, which this worker does not hold and should not exercise outside CI; this diff touches no convex//backend/convex files, so the Convex typecheck is unaffected. Both steps run in CI on this PR.

Secret scan (before push)

$ git diff --cached | grep -inE 'sk-[a-z0-9]|api[_-]?key|BEGIN PRIVATE KEY|secret|\.env|\.codex/config\.toml|\.vercel/|token'
(3 benign prose/code matches, no secret values — see command output in the worker's report)

Test plan

  • Fresh git worktree add ... origin/main (this local clone was on a codex recovery branch and was excluded per instruction)
  • npm install (fresh worktree)
  • npm run build then grep the meta tag in dist/index.html
  • Served dist/ with vite preview and curled the live meta tag
  • npx tsc --noEmit --pretty false (CI's App typecheck step) green
  • Secret grep on the staged diff, clean
  • CI (ci.yml) green on this PR
  • After merge + a real Vercel Production deploy, confirm deploy-verify.yml runs on the deployment_status event and passes

🤖 Generated with Claude Code

Copies NodeVoice PR #10's pattern for the Node Foyer identity lane
(FOYER-V3, worker identity-nodebench): a Vite plugin stamps exactly one
<meta name="nodebench-build-sha" content="<sha>" data-provenance="commit">
into the built apps/web/index.html, with precedence
VERCEL_GIT_COMMIT_SHA -> GITHUB_SHA -> `git rev-parse HEAD`, falling back
to content="unavailable" data-provenance="unavailable" rather than
throwing.

Adds .github/workflows/deploy-verify.yml (on: deployment_status only,
no schedule, no workflow_dispatch, no rollback step) that polls
https://www.nodebenchai.com/ for up to 3 minutes after a Production
deployment_status success and fails if the live meta never equals
github.event.deployment.sha.

KNOWN HAZARD: this repo's existing .github/workflows/post-deploy-verify.yml
runs `npx vercel rollback --yes` on a failed Production deployment_status
event and also runs on a 6-hourly schedule. deploy-verify.yml is a
separate workflow file triggered only by deployment_status (never the
schedule) and carries no rollback step, so it cannot reach that path in
either direction.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
nodebench-ai Building Building Preview Sep 13, 2026 1:09am UTC

Request Review

@HomenShum

Copy link
Copy Markdown
Owner Author

Closing in favor of a correctly-named branch: this repo's Branch-name CI check requires <type>/<short-slug> from a fixed type list (feat/fix/cleanup/docs/chore/refactor/test/perf/ci/build/style/hotfix/dependabot); foyer/... isn't one of them. Same commit, reopened as a new PR from feat/identity-nodebench-build-sha.

@HomenShum HomenShum closed this Sep 13, 2026
@HomenShum
HomenShum deleted the foyer/identity-nodebench-build-sha branch September 13, 2026 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants