Skip to content

Python suite takes ~24 min serially; no parallelism, and per-test app setup costs up to ~10 s #524

Closed

Description

@antosubash

Tier 3: feedback loop

Measured

make test-py on main (ec46dc4) in a cloud dev container:

4470 passed, 11 skipped, 65 deselected, 1 xfailed, 15 warnings in 1452.50s (0:24:12)
  • Neither pyproject.toml, the Makefile nor pr.yml uses pytest-xdist, so the suite runs on a single core.
  • The slowest entries are mostly fixture setup, not test bodies:
    • tenants/tests/test_resolver_source.py::test_header_source_and_vary: 9.8 s setup
    • api_tokens/tests/test_api_tokens_service_keys.py::...: 9.8 s setup
    • branding/tests/test_tenant_branding_cost.py::...: 9.2 s setup
  • Those setups build a fresh create_app with its lifespan, and create every module's tables, for each test.

Cost

A 24-minute local run means contributors stop running the full suite and rely on CI. That defeats the "prove it before pushing" discipline the project asks for.

Fix

  1. Add pytest-xdist and run -n auto in make test-py and CI. Fixtures already use per-test in-memory SQLite, and Postgres resets its schema per test, so isolation should mostly hold. Audit any process-globals first, for example Audit capture is a process-global; writes made in Celery tasks are never audited #498's _db_state.
  2. Profile create_app and table creation under the app / db_session fixtures. Candidates:
    • Cache discovered modules and built metadata per session.
    • Create the schema once per worker and use a transaction or savepoint per test where tests don't need a fresh app.
  3. Track total suite time in CI, so regressions get noticed.

Activity

  1. antosubash commented on Oct 21, 2026

    @antosubash
    OwnerAuthor

    Main fix landed in #550 (f524a52): pytest-xdist with -n auto --dist loadgroup in make test-py and CI, per-session caching of discovered modules/model bases, and minimum-cost password hashing in tests — the suite went from ~24 min to ~5.5 min on 4 cores. CI writes the pytest totals line to the job summary, so run time is tracked. Per-worker schema reuse / a savepoint fixture is not pursued for now; reopen if setup time becomes a problem again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions