Skip to content

test(node-integration-tests): Reuse Docker Compose containers across tests#22128

Merged
mydea merged 8 commits into
developfrom
perf/docker-reuse-prototype
Jul 10, 2026
Merged

test(node-integration-tests): Reuse Docker Compose containers across tests#22128
mydea merged 8 commits into
developfrom
perf/docker-reuse-prototype

Conversation

@mydea

@mydea mydea commented Jul 9, 2026

Copy link
Copy Markdown
Member

Speeds up the Docker-backed node-integration-tests suites by bringing each Compose service up once per suite instead of once per test, and by removing redundant per-test setup work.

describeWithDockerCompose

Previously every test spun its own container up and down via runner.withDockerCompose(...), so a suite with N tests paid N × (docker compose down + up --wait healthcheck) — the dominant cost of these suites. The new describeWithDockerCompose(name, { workingDirectory }, () => { ... }) wrapper starts the container in a single beforeAll and tears it down once in afterAll; the inner tests no longer call withDockerCompose() themselves. withDockerCompose() (and the now-unused DockerOptions.setupCommand) has been removed from createRunner.

Migrated suites: amqplib, ioredis-dc, kafkajs, knex (mysql2/pg), mysql2, mysql2-tracing-channel, postgres, postgres-streamed, postgresjs, prisma v5/v6/v7, redis, redis-cache, redis-dc, tedious.

Prisma: stop running prisma generate/migrate on every test

  • migrate → Postgres init script. Each Prisma docker-compose.yml now bind-mounts the committed migration.sql into /docker-entrypoint-initdb.d/, so the schema is created on container init. The runtime prisma migrate dev step is gone entirely.
  • generate → once per suite. New afterSetupCommand option on createEsmAndCjsTests' CommonTestOptions runs a command once in the tmp dir after additionalDependencies are installed (with node_modules/.bin on PATH). The Prisma suites use it for prisma generate (v7 also runs tsc), instead of running it inside the per-test Docker setup command.

Flaky tests fixed

These Docker-backed suites flaked with an opaque Test timed out: on the previous per-test model, docker compose up --wait (DB healthcheck / cold start — slowest for MSSQL) ran inside the per-test timeout, so on a loaded runner the budget was exhausted before the scenario could emit its envelope. Moving Docker startup into beforeAll removes it from the per-test budget, fixing the whole class.

Fixes #22122
Fixes #22117
Fixes #22114
Fixes #22109
Fixes #22110
Fixes #22082
Fixes #22135
Fixes #22078
Fixes #22031
Fixes #22030
Fixes #21698
Fixes #21724

Other test changes

Some tests needed adjustments to make sure we drop tables that have been created again (as multiple tests share a DB instance now). While at this I also hardended the tests a bit by using random emails instead of fixed ones - this alone is not enough because we also check the table creation spans, but IMHO that is fine either way (we could revisit this later).

🤖 Generated with Claude Code

Comment thread dev-packages/node-integration-tests/suites/tracing/postgres/test.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d5ba5d2. Configure here.

@mydea mydea marked this pull request as ready for review July 9, 2026 09:02
@mydea mydea requested a review from a team as a code owner July 9, 2026 09:02
@mydea mydea requested review from JPeer264 and andreiborza and removed request for a team July 9, 2026 09:02
@mydea mydea force-pushed the perf/docker-reuse-prototype branch from acc254b to 9ab3366 Compare July 9, 2026 09:21

@andreiborza andreiborza left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit concerned about having to remember to clean up the db, could we maybe also add a bugbot rule to point this out?

@mydea

mydea commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

Bit concerned about having to remember to clean up the db, could we maybe also add a bugbot rule to point this out?

This would simply fail pretty much consistently in CI, as tests are run sequentially and if a test relies on that it will fail every time. Which I think should be good enough?

@mydea mydea force-pushed the perf/docker-reuse-prototype branch from 9ab3366 to ac1a73c Compare July 10, 2026 07:13
@andreiborza

Copy link
Copy Markdown
Member

Bit concerned about having to remember to clean up the db, could we maybe also add a bugbot rule to point this out?

This would simply fail pretty much consistently in CI, as tests are run sequentially and if a test relies on that it will fail every time. Which I think should be good enough?

My concern was around tests asserting on things that happen to be in the db from the previous test that didn't clear the db and passing accidentally. How would we see that?

@mydea

mydea commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

Bit concerned about having to remember to clean up the db, could we maybe also add a bugbot rule to point this out?

This would simply fail pretty much consistently in CI, as tests are run sequentially and if a test relies on that it will fail every time. Which I think should be good enough?

My concern was around tests asserting on things that happen to be in the db from the previous test that didn't clear the db and passing accidentally. How would we see that?

The tests I looked at would all fail at CREATE TABLE time if the table was not destroyed beforehand I think. Plus, making sure to use randomized IDs for inserts/reads to avoid cross pollution as well as much as possible, although due to us always creating a new table in all of these tests this should not really be relevant at the end of the day because that would trip up a following test immediately - but we can be double sure :D

@JPeer264 JPeer264 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, 1 test is failing, but kinda seems unrelated

@mydea mydea force-pushed the perf/docker-reuse-prototype branch from ac1a73c to f8b9912 Compare July 10, 2026 09:05
Comment on lines +26 to +28
await client.query(
'CREATE TABLE "User" ("id" SERIAL NOT NULL,"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,"email" TEXT NOT NULL,"name" TEXT,CONSTRAINT "User_pkey" PRIMARY KEY ("id"));',
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Test database cleanup is not guaranteed in the new shared Docker container setup, leading to cascading test failures if a test exits prematurely.
Severity: MEDIUM

Suggested Fix

For the postgres test, re-add a .catch(() => {}) to the CREATE TABLE query to gracefully handle cases where the table already exists. For the knex test, move the pgClient.schema.dropTable('User') call into a finally block to ensure it always executes, even if the test fails.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location:
dev-packages/node-integration-tests/suites/tracing/postgres/scenario.mjs#L26-L28

Potential issue: The test suite now uses a shared Docker container, but the database
cleanup logic is not robust enough for this environment. In the `postgres` test, the
removal of a `.catch(() => {})` block on the `CREATE TABLE` statement means that if a
previous test run fails to clean up the `"User"` table, subsequent tests will fail with
a "relation 'User' already exists" error. Similarly, in the `knex` test, the
`dropTable('User')` call is inside the `try` block instead of a `finally` block. If any
error occurs during the test after table creation, the cleanup is skipped, causing the
next test run to fail.

Also affects:

  • dev-packages/node-integration-tests/suites/tracing/knex/pg/scenario.mjs:37

Did we get this right? 👍 / 👎 to inform future reviews.

@mydea mydea merged commit 9379bad into develop Jul 10, 2026
56 checks passed
@mydea mydea deleted the perf/docker-reuse-prototype branch July 10, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Flaky CI]: Node (Orchestrion) Integration Tests - suites/tracing/tedious/test.ts > tedious auto instrumentation > should auto-instrument tedious package [esm] [Flaky CI]: Node Integration Tests - suites/tracing/postgresjs/test.ts > postgresjs auto instrumentation > basic > should auto-instrument postgres package [esm] [Flaky CI]: Node Integration Tests - suites/tracing/redis-cache/test.ts > redis cache auto instrumentation > ioredis non-cache keys > should not add cache spans when key is not prefixed [esm] [Flaky CI]: Node Integration Tests - suites/tracing/amqplib-v1/test.ts > amqplib v1 auto-instrumentation > should be able to send and receive messages with amqplib v1 [esm] [Flaky CI]: Node Integration Tests - suites/tracing/redis-cache/test.ts > redis cache auto instrumentation > ioredis non-cache keys > esm/cjs > should not add cache spans when key is not prefixed

3 participants