Skip to content

Speed up the Python test suites: per-worker Postgres DBs, schema reuse, lazy route resolution - #577

Merged
antosubash merged 6 commits into
mainfrom
claude/keen-pasteur-op0eox
Oct 22, 2026
Merged

antosubash merged 6 commits into
mainfrom
claude/keen-pasteur-op0eox

Conversation

@antosubash

@antosubash antosubash commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Why

The Postgres tests workflow spent ~29 min in make test-py-pg (5418 tests, 1745 s). Each cause below was measured before it was changed.

  1. Forced serial on Postgres. Every test dropped and recreated public in one database, so the plugin refused -n.
  2. Full schema rebuild per test on Postgres, about 0.25 s.
  3. Every create_app resolved all ~400 routes, about 0.5 s and ~70% of a test's setup, on SQLite and Postgres alike. Once 1 and 2 were fixed, Postgres was at parity with the SQLite job (both ~15 min on CI). This was the remaining cost, and it is the critical path of pr-checks too.

Durability is not a bottleneck (~0.5 s of sync per checkpoint in the CI log; fsync=off changed nothing locally), so no server config is added.

What

1. One database per xdist worker (simple_module_test)

  • isolate_worker_database() creates <db>_gw<N> and rewrites SM_TEST_DATABASE_URL, so direct env readers and subprocesses follow. It runs in pytest_load_initial_conftests, before conftests capture the URL at import time.
  • make test-py-pg runs -n $(PYTEST_WORKERS) --dist loadgroup. Behaviour change: the role needs CREATEDB.

2. Schema reuse (prepare_schema_once)

  • Build once per process; between tests delete every row (children first) and restart sequences (~0.05 s).
  • The snapshot is keyed on the table set plus a catalog fingerprint (tables, columns, constraints, types, extensions). A direct reset_schema() call or a test's own DDL triggers a full rebuild. alembic_version is created inside the build so the fingerprint covers it.
  • A template database (CREATE DATABASE … TEMPLATE) measured 0.05–0.15 s per test with a 3.3 s spike, so row deletion was chosen.

3. Lazy route resolution (simple_module_hosting)
FastAPI ≥0.140 resolves an included router's routes (dependants, signatures, pydantic adapters) on first use, but three path walks forced all of them on every app build:

  • FastAPI's empty-path check, run only when include_router gets no prefix=. wire_module_routes now passes the prefix on include instead of on the APIRouter. The effective route table (192 routes: path, methods, name, schema flag, dependency count) and the full OpenAPI document diff byte-identical before and after. _clone_bare_prefix_route now compares router-relative paths.
  • The route prefix guard and the SPA overlap check, which read paths through effective_route_contexts(). Both now use the new _route_paths, which composes paths with FastAPI's own include_context.combine / path_for and falls back to FastAPI's per-route build for any other route type.
  • Outside environment="testing" the lifespan still warms every route at startup (guard_included_routers(warm=True)), so production's first requests are unchanged.
  • New tests check _route_paths against the resolved paths for every router of the real app, and check that it resolves nothing. The private FastAPI members it uses are added to test_route_guard_fastapi_contract.py.

Verification (local, 4 cores)

Suite Before this PR After
Postgres, full serial (CI: 29 min) 5632 passed, 4:33 (-n 4)
SQLite, full (make test-py) 7:29 4:15
modules/background_tasks under -n 4 ~50 s ~25 s
one fixture-heavy file (test_tenant_settings.py) 18.3 s 6.8 s

On GitHub's runner, steps 1 and 2 alone took the Postgres step from 30.5 min to 13.6–17 min. This push adds step 3, which also speeds up the main Python tests job.

Also clean: ci-python-lint, ci-python-typecheck, ci-check-file-size, ci-api-types-fresh, ci-doctor.

Known: one full SQLite run had test_bg_inprocess_sweep.py::test_eager_sweep_recovers_without_schedule[False] time out (a 10 s wait_for). It passed 5/5 alone and 3/3 under -n 4 on this branch. The committed code also had one unexplained failure in seven loaded runs of the same module, which I could not reproduce. The test exercises the outbox sweep, not routing. Not fixed here.

https://claude.ai/code/session_01MeLZyxkDD4sq75m1Siw7om

…worker

make test-py-pg ran serially (~29 min in CI) because every test drops and
recreates public in the one database. Each xdist worker now creates and
uses <db>_gw<N> and points SM_TEST_DATABASE_URL at it; the switch happens
in pytest_load_initial_conftests so helpers that read the URL at import
time (framework/db/tests/_models.py) see the worker's database.

Claude-Session: https://claude.ai/code/session_01MeLZyxkDD4sq75m1Siw7om
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Deploying simple-module-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: fd09449
Status: ✅  Deploy successful!
Preview URL: https://9159060c.simple-module-python.pages.dev
Branch Preview URL: https://claude-keen-pasteur-op0eox.simple-module-python.pages.dev

View logs

Rebuilding public per test cost ~0.25 s; now each process builds it once
and later tests delete every row and restart the sequences (~0.05 s). The
snapshot is keyed on the built tables and a catalog fingerprint (tables,
columns, constraints, types, extensions), so a direct reset_schema() call
or a test's own DDL falls back to the full rebuild. alembic_version is
created inside the build so the fingerprint covers it.

Claude-Session: https://claude.ai/code/session_01MeLZyxkDD4sq75m1Siw7om
@antosubash antosubash changed the title Speed up the Postgres test suite: one database per xdist worker Speed up the Postgres test suite: per-worker databases and schema reuse Oct 22, 2026
@antosubash
antosubash marked this pull request as ready for review October 22, 2026 12:42
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ✅ Completed 2026-10-22T12:46:10.591865Z fd09449 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Each create_app resolved all ~400 routes (dependants, signatures, pydantic
adapters; ~0.5 s, ~70% of a test's setup) through three path walks:

- FastAPI's empty-path check on include_router without prefix=: module
  routers now get their prefix on include, so effective paths are the
  same (route table and OpenAPI diffed identical) and the check is skipped;
- the route prefix guard and the SPA overlap check read effective paths
  via effective_route_contexts(): both now use _route_paths, which composes
  paths with FastAPI's include_context.combine/path_for and resolves nothing
  (tests compare it with the resolved paths for every router of a real app).

The lifespan still warms all routes at startup outside environment=testing.
Locally: SQLite suite 7:29 -> 4:15, Postgres suite 8:15 -> 4:33.

Claude-Session: https://claude.ai/code/session_01MeLZyxkDD4sq75m1Siw7om
@antosubash antosubash changed the title Speed up the Postgres test suite: per-worker databases and schema reuse Speed up the Python test suites: per-worker Postgres DBs, schema reuse, lazy route resolution Oct 22, 2026
@antosubash
antosubash merged commit 1c4270a into main Oct 22, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants