Skip to content

ci: serialize Fly deploys and cap turbo concurrency to stop OOM failures - #15

Merged
JasonPaff merged 1 commit into
mainfrom
fix/pipeline-resource-limits
Jul 3, 2026
Merged

JasonPaff merged 1 commit into
mainfrom
fix/pipeline-resource-limits

Conversation

@JasonPaff

Copy link
Copy Markdown
Owner

Why

The PR #14 merge produced two pipeline failures on main that never appeared on the PR itself — both resource exhaustion, not code bugs:

  • Deploy: all three Fly apps (api, match, bots) started building at the same second on the shared Fly remote builder. bots won the race; api and match were OOM-killed (exit code: 137) during pnpm deploy --prod. First main deploy since the consolidated root Dockerfile put all three builds on one builder concurrently.
  • CI: the lockfile change invalidated every turbo task hash, so the merge commit ran all 25 tasks fully cold at turbo's default concurrency of 10 on a 4-core runner. A vitest worker was killed (Worker exited unexpectedly). The identical commit passed on the PR because the branch had warm caches.

What

  • deploy.yml: max-parallel: 1 on the deploy matrix — one app builds on the Fly builder at a time.
  • ci.yml: quality gate runs with --concurrency=4 — warm cached runs are unaffected; cold runs no longer stack multiple vitest/tsc/next-build processes.

🤖 Generated with Claude Code

The PR #14 merge exposed two resource-exhaustion failures that never hit
PR runs:

- Deploy: all three Fly apps built concurrently on the shared remote
  builder and api/match were OOM-killed (exit 137) in pnpm deploy.
  Serialize the matrix with max-parallel: 1.
- CI: the lockfile change invalidated every turbo hash, so main ran all
  25 tasks cold at default concurrency 10 and a vitest worker was killed.
  Cap the quality gate at --concurrency=4 to match runner cores.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
meld-rank-web Ready Ready Preview, Comment Jul 3, 2026 4:09pm

@JasonPaff
JasonPaff merged commit a4a68a8 into main Jul 3, 2026
4 checks passed
@JasonPaff
JasonPaff deleted the fix/pipeline-resource-limits branch July 3, 2026 16:14

This branch was successfully deployed

1 active deployment
Preview — f6dc45c3 Deployed Jul 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant