fix(tests): stop exhausting test-db connections and migrate once - #98
Merged
Merged
Conversation
`pnpm test` at vitest's default concurrency reported ~137 failures that
were not real. Each integration file opens its own pg.Pool against the
single shared container, and pg pools default to 10 connections. At 14
workers that demands ~140 against a server whose max_connections is 100,
so Postgres terminates connections mid-run (57P01) and the failures land
on whichever tests happened to be in flight. That also explains the
suite's flakiness: consecutive runs on identical code gave 5, then 1,
then 0 failures.
Raise max_connections to 300 and cap each pool at 4.
While here, cut the dominant cost: every one of the 52 integration files
replayed the full migration chain. Migrate once into a template database
in global setup and have each file clone it with CREATE DATABASE ...
TEMPLATE. Concurrent clones briefly conflict (55006), so retry. Falls
back to replaying migrations when no template is present, keeping a bare
DATABASE_URL working. Durability settings are also disabled — the server
is destroyed at teardown.
pnpm test, default concurrency
before: 339s, 137 failed / 479 passed
after: 21s, 727 passed
Five consecutive integration runs are now green where the suite
previously varied run to run.
Also ignore .stryker-tmp/** in ESLint. Stryker leaves sandbox copies of
the tree behind, and linting them reported 981 of 1024 errors, failing
`pnpm lint` on code that is not ours. Lint is now 0 errors.
KenTaniguchi-R
added a commit
that referenced
this pull request
Aug 29, 2026
…ions (#101) #98 replaced per-file migrations with a template database that each test file cloned via CREATE DATABASE ... TEMPLATE. That broke CI: every integration file now times out in beforeAll at createTestDb(). Test Files 57 failed | 56 passed (113) Error: Hook timed out in 10000ms. 185| beforeAll(async () => { 186| ({ db, close } = await createTestDb()); CREATE DATABASE ... TEMPLATE takes a lock on the source database, so concurrent clones serialize rather than run in parallel. On the CI runner they block past the 10s hook timeout. The change was measured only on a 14-core dev machine, where the suite went 339s -> 21s. That speedup was real but local: the problem it solved — pool connections (14 workers x pg's default 10) overrunning max_connections=100 — never occurred on CI, which has fewer cores and ran the same suite in 27s beforehand. Restore tests/global-setup.ts and tests/integration/setup.ts to their pre-#98 state. The .stryker-tmp ESLint ignore from #98 is unrelated and stays; it takes `pnpm lint` from 1024 errors to 0. The local exhaustion is better addressed with a maxWorkers cap, which costs CI nothing. Not included here — this commit only restores green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
pnpm testat vitest's default concurrency reported 137 failures that were not real:Errors were
57P01 ProcessInterrupts— Postgres terminating connections — attributed to whichever tests were in flight, not to any actual defect.Root cause
Each integration file opens its own
pg.Poolagainst the single shared testcontainer.pgpools default to 10 connections. On a 14-core machine vitest runs ~14 workers, demanding up to 140 connections against a server whosemax_connectionsis 100 (verified againstpostgres:17-alpine).Postgres starts dropping connections, and the failures land wherever they land. That also explains the flakiness: three runs on identical code gave 5 → 1 → 0 failures.
The fix
Connections.
max_connections=300on the container; each pool capped at 4. The suite needs a couple of concurrent queries per file, not ten.Migrations — the dominant cost. All 52 integration files replayed the full 9-migration chain: 468 migration runs per suite. Now migrated once into a template database in global setup, with each file cloning it via
CREATE DATABASE ... TEMPLATE. Concurrent clones briefly conflict (55006), so that specific error retries with backoff. When no template is present the old path still replays migrations, so a bareDATABASE_URLkeeps working.Durability.
fsync,synchronous_commit, andfull_page_writesoff — the server is destroyed at teardown.Results
pnpm test, default concurrency, no worker cap:16x faster, and green. Five consecutive integration runs all passed 301/301, where the suite previously varied run to run.
This should also help the CI backlog: there is one self-hosted runner (
macmini-ledgr) and every branch's jobs queue behind it serially, so shortening each job compounds.Also: ESLint
.stryker-tmp/**added toglobalIgnores. Stryker leaves sandbox copies of the entire tree behind, and linting them produced 981 of 1024 errors, failingpnpm linton duplicated copies of our own code. Lint is now 0 errors (1 pre-existing warning,_accountTypeinsrc/lib/money.ts).Verification
Run in an isolated worktree off
main:pnpm install --frozen-lockfile— cleanpnpm typecheck— cleanpnpm lint— 0 errors (verified with a sandbox present that previously errored)pnpm test— 727 passed / 112 files🤖 Generated with Claude Code