test(fastapi): add pytest-benchmark suite for the FastAPI integration layer

[tmgbedu]

Summary

• Adds pytest-benchmark as a dev dependency for fastapi-startkit.
• Benchmarks the FastAPI integration layer's hot paths, each paired with a raw-FastAPI/APIRouter baseline so overhead shows as a ratio:
  • Application boot + provider register/boot (default providers, with FastAPIProvider, vs raw FastAPI())
  • Router.get() and Router.resource() CRUD expansion (vs hand-registering the same routes on APIRouter)
  • Request handling via TestClient on trivial JSON GET endpoints, including a path-param resource route
• Deterministic and CI-safe: no network, no live DB, all fixtures are in-memory/tmp_path-scoped.
• New tests carry a benchmark marker and are deselected by default (addopts = "-m 'not benchmark'" in pyproject.toml), so they don't slow down or affect the coverage-gated CI run. Run them explicitly with -m benchmark.

Run command

cd fastapi_startkit
uv run pytest -m benchmark tests/fastapi/benchmarks

Sample results table

------------------------------------------------------------------------------------- benchmark 'application-boot': 3 tests -------------------------------------------------------------------------------------
Name (time in us)                       Min                    Max                Mean                StdDev             Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_boot_raw_fastapi_baseline       9.2080 (1.0)         745.6669 (1.0)       15.2707 (1.0)         30.4177 (1.0)      10.2080 (1.0)      0.2082 (1.0)     1471;5364       65.4851 (1.0)       54301           1
test_boot_default_providers         27.6659 (3.00)     21,346.8750 (28.63)     34.5106 (2.26)       283.7424 (9.33)     29.0829 (2.85)     1.1679 (5.61)        7;375       28.9766 (0.44)       5678           1
test_boot_with_fastapi_provider     77.7498 (8.44)     56,685.8330 (76.02)    126.8814 (8.31)     1,131.7811 (37.21)    88.3330 (8.65)     4.0829 (19.62)       2;420        7.8814 (0.12)       3582           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'request-handling-json-get': 3 tests ----------------------------------------------------------------------------------
Name (time in us)                            Min                   Max                Mean             StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_raw_fastapi_request_baseline       219.3339 (1.0)      1,059.9580 (1.80)     244.7352 (1.0)      30.9015 (1.20)     238.0831 (1.0)      12.7079 (1.0)        72;118        4.0860 (1.0)        2314           1
test_router_based_request               227.3340 (1.04)       629.2921 (1.07)     246.1968 (1.01)     26.4277 (1.03)     239.5420 (1.01)     12.8339 (1.01)        46;54        4.0618 (0.99)        930           1
test_resource_route_with_path_param     236.2910 (1.08)       590.4590 (1.0)      258.7016 (1.06)     25.7737 (1.0)      254.2295 (1.07)     15.1249 (1.19)        60;50        3.8655 (0.95)       2286           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------- benchmark 'router-resource-expansion': 2 tests ----------------------------------------------------------------------------------------
Name (time in us)                                    Min                    Max                Mean                StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_raw_apirouter_equivalent_crud_baseline     523.0829 (1.0)      68,672.7080 (50.72)    641.7178 (1.04)     1,675.0025 (16.06)    561.5414 (1.0)      12.8327 (1.0)         1;319        1.5583 (0.96)       1658           1
test_resource_full_crud                         539.5419 (1.03)      1,354.0001 (1.0)      615.4041 (1.0)        104.3191 (1.0)      574.4172 (1.02)     23.6667 (1.84)      167;176        1.6249 (1.0)         937           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'router-single-route': 2 tests --------------------------------------------------------------------------------
Name (time in us)                       Min                 Max               Mean             StdDev             Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_raw_apirouter_get_baseline     17.7911 (1.0)      663.1250 (2.55)     23.7981 (1.0)      26.0061 (1.02)     19.4579 (1.0)      0.5411 (1.0)      710;2385       42.0201 (1.0)       27058           1
test_router_get                     18.2502 (1.03)     259.9589 (1.0)      24.4320 (1.03)     25.5663 (1.0)      20.0421 (1.03)     0.6682 (1.23)     284;1127       40.9299 (0.97)      11050           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10 passed in 6.67s

Test plan

• uv run pytest tests/fastapi -q — 117 passed, 10 deselected (benchmarks excluded by default)
• uv run pytest -m benchmark tests/fastapi/benchmarks — 10 passed, results table generated
• uv run pytest --ignore=tests/masoniteorm/postgres --cov --cov-report=term-missing — 1833 passed, coverage 79.87% (gate: 68%), unaffected
• uv run ruff check — clean

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[tmgbedu]

Code Review — APPROVE ✅

Reviewed the full diff and independently reproduced every claim in the test plan. Clean, well-scoped benchmark suite with fair baselines and correct gating. No blockers.

Benchmark quality & fair baselines ✔

Every framework hot path is paired with an apples-to-apples raw baseline so overhead reads as a ratio:

• Boot — Application(default providers) and Application(+FastAPIProvider) measured against a raw FastAPI() floor, so the framework's register/boot overhead is isolated from FastAPI's own cost.
• Router single route — Router().get() vs APIRouter().add_api_route(); both include object construction, so it's a fair delta.
• resource() expansion — Router().resource("posts", PostController) vs hand-registering the same 7 CRUD routes on APIRouter — measures exactly the expansion overhead.
• Request handling — in-process TestClient GET on trivial JSON endpoints, router-based vs raw-FastAPI, plus a resource() path-param route (/posts/1) exercising Starlette path matching. Client fixtures are module-scoped so setup cost isn't folded into the per-request timing.

Gating — correct, does not touch normal CI ✔

• benchmark marker registered in [tool.pytest.ini_options].markers (no strict-marker warnings).
• addopts = "-m 'not benchmark'" deselects benchmarks by default; a command-line -m benchmark overrides it. Verified:
  • pytest tests/fastapi → 117 passed, 10 deselected
  • pytest -m benchmark tests/fastapi/benchmarks → 10 passed
  • pytest --ignore=.../postgres --cov → 1833 passed, 7 skipped, 10 deselected, coverage 79.87% (gate 68%) — unaffected.

Determinism / no flakiness ✔

No sleeps, no network (TestClient is in-process ASGI), no live DB, no wall-clock/timing assertions (pytest-benchmark only records). The boot file's restore_container_instance fixture correctly restores the Container singleton that Application() mutates.

Scope & conventions ✔

• No core framework abstractions modified — changes limited to pyproject.toml (dev dep + marker/addopts), uv.lock, and new tests/fastapi/benchmarks/ files.
• asyncio_mode = "auto" preserved; benchmark tests are sync (correct — the benchmark fixture is sync-only), clean structure.
• ruff check clean on the new files and pyproject.

Non-blocking observations (optional)

• Request-handling absolute numbers include TestClient/ASGI transport overhead, but since both baseline and framework paths share it, the ~1.01–1.08x ratios correctly show negligible wrapper overhead — which is the intent. Fine as-is; a one-line comment noting that the ratio (not absolute μs) is the signal would help future readers.
• test_boot_default_providers includes .env disk reads each round; consistent and part of real boot cost, so not a flakiness concern.

Approving. (Posted as a comment — GitHub blocks self-approve on the authoring account; verdict is APPROVE.)