Skip to content

fix(ci): restore test.yml on main — hashFiles() is illegal in a job-level if: (3 sites) - #2449

Merged
dcccrypto merged 2 commits into
mainfrom
fix/launch-testyml-hashfiles
Jul 21, 2026
Merged

fix(ci): restore test.yml on main — hashFiles() is illegal in a job-level if: (3 sites)#2449
dcccrypto merged 2 commits into
mainfrom
fix/launch-testyml-hashfiles

Conversation

@dcccrypto

Copy link
Copy Markdown
Owner

Same defect as deploy.yml, bigger blast radius

ecef0545 (#2164, 2026-05-15) added this guard to both deploy.yml and test.yml.
#2444 fixed deploy.yml on playground; #2448 ports that to main. This PR is test.yml,
which carries the same bug in three places — unit-tests, integration-tests and
security-tests:

if: hashFiles('packages/shared/package.json') != ''

hashFiles() is only available in jobs.<job_id>.steps.*, never in a job-level if:, so
GitHub rejects the file at startup and creates zero jobs.

$ actionlint .github/workflows/test.yml
main:   3 hashFiles context errors
branch: 0

Why this matters more than deploy.yml

test.yml is main's entire test workflow: Unit Tests, Integration Tests, Security Tests,
Coverage Gate, E2E Tests, Type Check and the ✅ Merge Gate. While the file fails to parse,
none of them run for main — every main run is a startup failure, 1540 runs deep.

The workflow looks healthy in aggregate (2350 lifetime successes, 82/100 recent) purely
because most active branches are playground-based, and playground already carries a fixed
copy. Filtering to branch=main shows nothing but failures.

This also explains a confusing symptom: a startup failure can't read on:, so GitHub
attributed bogus test.yml runs to pushes on branches that branches: [main] should have
excluded entirely.

Ported surgically, not copied

playground's test.yml is fixed but also carries changes that must not come to main:

  • adds playground to the push/PR branch triggers
  • replaces pnpm run build (and its NEXT_PUBLIC_* env) with
    pnpm --filter app exec tsc --noEmit — swapping a real build for a typecheck, weakening the gate

So I converted only the three existing guards and added the detect-packages job. Branch
triggers, the build step and its env are untouched, and the e2e-tests job is left unguarded
exactly as main has it (playground adds a guard there; that's a playground behaviour choice).

What to expect when this runs

packages/ is not tracked in this repo, so detect-packages reports has_packages=false and
unit-tests / integration-tests / security-tests skip — which is the guard's original
intent. merge-gate already handles that correctly:

if: ${{ always() && github.event_name == 'pull_request'
        && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled') }}

coverage-check needs the skipped jobs and has no always(), so it skips too.

Flagging honestly: e2e-tests and type-check are unguarded and will therefore execute
for the first time since 2026-05-15. If they fail, that is ~2 months of latent breakage
becoming visible — not a regression introduced here. I would rather surface that than keep it
hidden behind a workflow that never starts. See the checks on this PR for the real answer.

🤖 Generated with Claude Code

…evel if

Same defect as deploy.yml (#2444, and #2448 porting it to main), but test.yml
has it in three places — unit-tests, integration-tests and security-tests each
carry:

    if: hashFiles('packages/shared/package.json') != ''

hashFiles() is only available in `jobs.<job_id>.steps.*`, never in a job-level
`if:`, so GitHub rejects the whole file at startup and creates zero jobs.

Impact is larger here than for deploy.yml: test.yml is main's entire test
workflow — Unit Tests, Integration Tests, Security Tests, Coverage Gate, E2E,
Type Check and the Merge Gate. While the file fails to parse, none of them run
for main-based branches. The workflow looks healthy in aggregate only because
most active branches are playground-based, and playground already carries a
fixed copy.

Ported surgically rather than by copying playground's file, which also:
  - adds `playground` to the push/PR branch triggers, and
  - replaces `pnpm run build` (plus its NEXT_PUBLIC_* env) with
    `pnpm --filter app exec tsc --noEmit`

Neither belongs on main — the second would swap a real build for a typecheck
and weaken the gate. Only the three existing guards are converted; the e2e job
is left unguarded exactly as main has it.

Verified: actionlint reports 3 hashFiles context errors on main and 0 here;
build step, env and branch triggers are untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
percolator-launch Ready Ready Preview, Comment Jul 21, 2026 8:10am
percolator-mainnet Ready Ready Preview, Comment Jul 21, 2026 8:10am
percolator-playground Ready Ready Preview, Comment Jul 21, 2026 8:10am

Request Review

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@dcccrypto, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 48 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b54c48c9-4fb8-4d92-a672-dba17bfa323d

📥 Commits

Reviewing files that changed from the base of the PR and between 775a126 and 7b2f254.

📒 Files selected for processing (2)
  • .github/workflows/test.yml
  • e2e/helpers.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/launch-testyml-hashfiles

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…pp errors

Restoring test.yml (previous commit) runs the e2e suite for the first time
since 2026-05-15. Result: 56 passed, 5 skipped, 1 failed.

The single failure is "Homepage has no console errors", and neither console
error is an app defect — both are third-party analytics that only work on a
real deployment:

  - cloudflareinsights.com/cdn-cgi/rum — blocked by CORS from localhost:3000
  - /_vercel/insights/script.js — 404s to an HTML page, so strict MIME
    checking refuses to execute it

collectConsoleErrors already filters exactly this class of environment noise
(Privy, Supabase, WalletConnect, Sentry, hydration warnings...). These two
belong in that list; they were simply never reachable while the workflow could
not start.

Verified by extracting the 31 filter patterns from the real source and applying
them to the two strings CI actually produced: both are filtered, while genuine
app errors (TypeError/ReferenceError/invalid public key) are still reported, so
the assertion keeps its teeth.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dcccrypto

Copy link
Copy Markdown
Owner Author

Outcome: the predicted e2e failure happened, and is fixed — full green

I flagged above that e2e-tests and type-check would execute for the first time since
2026-05-15 and might fail. They did run. Result of the first pass:

Type Check   pass
E2E Tests    FAIL — 56 passed, 5 skipped, 1 failed

The one failure was page-loads.spec.ts › Homepage has no console errors, and neither
console error was an app defect — both are third-party analytics that only work on a real
deployment:

  • cloudflareinsights.com/cdn-cgi/rum — blocked by CORS from localhost:3000
  • /_vercel/insights/script.js — 404s to an HTML page, so strict MIME checking refuses it

collectConsoleErrors already filters exactly this class of noise (Privy, Supabase,
WalletConnect, Sentry, hydration…). These two belong in that list and were simply never
reachable while the workflow could not start. Added in 7b2f254.

Verified the filter keeps its teeth: extracted all 31 patterns from the real source and
applied them to the two strings CI actually emitted — both filtered, while genuine app errors
(TypeError, ReferenceError, invalid public key) are still reported.

Current state — all green

check result
Detect packages pass
Unit / Integration / Security Tests skipping (packages/ absent — guard's intent)
Coverage Gate skipping (needs the skipped jobs)
Type Check pass
E2E Tests pass (3m56s)
✅ Merge Gate pass

That Merge Gate pass is the first on a main-targeting PR since 2026-05-15.

Scope note: this PR now carries two commits — the workflow port (3dc7c1a) and the e2e
noise filter (7b2f254). I kept them together deliberately: restoring the workflow without
the filter would trade an invisible failure for a permanently red main, which is the same
alarm-fatigue problem this is meant to fix. They are separable if you'd prefer them split.

@dcccrypto

Copy link
Copy Markdown
Owner Author

Correction: scoping the real gap — narrower than my summary implied

The PR body is accurate about test.yml itself, but I described the impact elsewhere as main
having "no test coverage" and merging "with zero test signal." That overstated it. The
accurate picture for main, 2026-05-15 → today:

gate what actually happened on main
pr-check.ymlBuild frontend (next build) ran on every PR
pr-check.yml → packages build/test steps skipped — packages/* absent (verified in a real run)
test.yml → Unit / Integration / Security dead — but would have skipped anyway, same packages/* reason
test.yml → Type Check (tsc --noEmit) dead — but largely redundant on main, see below
test.ymlE2E dead — this is the real loss
test.yml → Merge Gate dead

The load-bearing detail I initially got wrong: playground's test.yml carries a comment saying
next build no longer type-checks because ignoreBuildErrors is on. That is true of
playground, not main
:

app/next.config.ts   main:        ignoreBuildErrors = 0 matches, ignoreDuringBuilds = 0
                     playground:  eslint: { ignoreDuringBuilds: true }
                                  typescript: { ignoreBuildErrors: true }

So on main, next build did type-check and lint the whole time. Type safety was never
actually unguarded — which is also why playground needed a separate tsc --noEmit gate and main
did not.

What this changes

The genuine, non-overlapping loss is E2E (8 spec files, 57 tests once green) plus the Merge
Gate. That is a real gap worth closing — it is why this PR exists — but it is not "two months
of untested merges," and it does not justify a retroactive audit of everything that landed
since May. I had suggested that; I am withdrawing it.

Also worth knowing for this PR's value

Even with this merged, Unit / Integration / Security still skip on main, because packages/*
is not in this repo. This PR buys back E2E + Merge Gate, not the full suite. Restoring those
three would need the packages/* workspaces present, which is a separate question.

@dcccrypto
dcccrypto merged commit bd0e7d8 into main Jul 21, 2026
13 of 14 checks passed
@dcccrypto
dcccrypto deleted the fix/launch-testyml-hashfiles branch July 21, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant