From 083c2680a67f99d2305bc1cb720e365ebc472a72 Mon Sep 17 00:00:00 2001 From: Bedram Tamang Date: Mon, 6 Jul 2026 10:14:46 -0700 Subject: [PATCH 1/7] Align fastapi_startkit to Python 3.14 to match other Python frameworks The fastapi_startkit config pinned language.version: 3.13 while every other Python framework (fastapi, django, flask, ...) uses the repo-wide default of 3.14. That runtime mismatch made head-to-head comparisons unfair: benchmarking against fastapi was really comparing Python 3.13 vs 3.14 (and older Starlette), not the framework itself. Removing the override lets fastapi_startkit inherit the shared 3.14 default, so it runs on the same interpreter as fastapi and the rest. QA evidence: - Correctness: real .spec/route_spec.rb rspec suite -> 6/6 pass on 3.14. - Builds cleanly on python:3.14-slim (fastapi-startkit 0.46.0 from PyPI). - Matched-Python (both 3.14) throughput delta vs fastapi collapses to +/-noise around zero, confirming the wrapper adds no meaningful overhead; the earlier apparent advantage was solely the 3.13-vs-3.14 skew. Co-Authored-By: Claude Opus 4.8 --- python/fastapi_startkit/config.yaml | 3 --- 1 file changed, 3 deletions(-) diff --git a/python/fastapi_startkit/config.yaml b/python/fastapi_startkit/config.yaml index e38ef6e238c..05c2dece30c 100644 --- a/python/fastapi_startkit/config.yaml +++ b/python/fastapi_startkit/config.yaml @@ -8,6 +8,3 @@ framework: - hypercorn - daphne - granian - -language: - version: 3.13 From 6fdb7197051e3bff7ba77fffd08b2ac8b2c5feac Mon Sep 17 00:00:00 2001 From: Bedram Tamang Date: Mon, 6 Jul 2026 10:16:08 -0700 Subject: [PATCH 2/7] Pin fastapi to Python 3.14 explicitly to match fastapi_startkit fastapi already inherited the repo-wide 3.14 default, but declaring it explicitly makes the fastapi vs fastapi_startkit comparison unambiguous: both configs now state Python 3.14, so any benchmark delta reflects the frameworks, not the runtime. Co-Authored-By: Claude Opus 4.8 --- python/fastapi/config.yaml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/python/fastapi/config.yaml b/python/fastapi/config.yaml index c75ac93f135..f2f2816c751 100644 --- a/python/fastapi/config.yaml +++ b/python/fastapi/config.yaml @@ -7,3 +7,6 @@ framework: - hypercorn - daphne - granian + +language: + version: 3.14 From 11e0165a6526d9dfb66d99dda7826b58d3e1b646 Mon Sep 17 00:00:00 2001 From: Bedram Tamang Date: Tue, 7 Jul 2026 09:06:06 -0700 Subject: [PATCH 3/7] Pin fastapi_startkit benchmark to fastapi-startkit==0.46.0 Pin the exact known-good release so rebuilds resolve a reproducible dependency instead of drifting within the 0.46.x range. --- python/fastapi_startkit/pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/fastapi_startkit/pyproject.toml b/python/fastapi_startkit/pyproject.toml index 6a799a91c42..4b99df00749 100644 --- a/python/fastapi_startkit/pyproject.toml +++ b/python/fastapi_startkit/pyproject.toml @@ -7,4 +7,4 @@ requires = ["flit_core"] name = 'server' version = '1.0.0' description = "Simple benchmark implementation" -dependencies = ["fastapi-startkit[fastapi]>=0.46,<0.47"] +dependencies = ["fastapi-startkit[fastapi]==0.46.0"] From b49433a797333c0f06baf2556c5217ba557c35c1 Mon Sep 17 00:00:00 2001 From: Bedram Tamang Date: Tue, 7 Jul 2026 09:13:36 -0700 Subject: [PATCH 4/7] Ignore Claude Code config and per-run container id/ip files Stop local Claude Code config (.claude/, .mcp.json) from cluttering git status. Also ignore the per-run cid-.txt / ip-.txt files the docker harness generates (the existing rules only covered the older cid.txt / ip.txt names). --- .gitignore | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.gitignore b/.gitignore index fb0ebdd34d2..2094f0a8bae 100644 --- a/.gitignore +++ b/.gitignore @@ -16,8 +16,10 @@ FRAMEWORKS.yaml */*/.Dockerfile* */*/.Makefile */*/ip.txt +*/*/ip-*.txt venv */*/cid.txt +*/*/cid-*.txt */.Makefile /lib/**/*.cr @@ -62,3 +64,7 @@ __pycache__/ .vscode .idea .zed + +# Claude Code local config +.claude/ +.mcp.json From 11e17bf5c00b70a63735b55a28df3457be38f879 Mon Sep 17 00:00:00 2001 From: Bedram Tamang Date: Wed, 8 Jul 2026 10:17:05 -0700 Subject: [PATCH 5/7] python/fastapi_startkit: pin to 0.47.0 (FastAPI 0.139) to match baseline The previous range resolved fastapi-startkit to 0.46.x -> FastAPI 0.124.4, while the fastapi baseline entry runs FastAPI 0.139. Pin to 0.47.0 so both Python entries benchmark on the same FastAPI/Starlette (0.139.0 / 1.3.1), making the comparison apples-to-apples. Idiomatic include_router registration is kept unchanged. route_spec.rb passes 6/6 on the 0.139 Docker build. --- python/fastapi_startkit/pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/fastapi_startkit/pyproject.toml b/python/fastapi_startkit/pyproject.toml index 4b99df00749..318cbfe6b46 100644 --- a/python/fastapi_startkit/pyproject.toml +++ b/python/fastapi_startkit/pyproject.toml @@ -7,4 +7,4 @@ requires = ["flit_core"] name = 'server' version = '1.0.0' description = "Simple benchmark implementation" -dependencies = ["fastapi-startkit[fastapi]==0.46.0"] +dependencies = ["fastapi-startkit[fastapi]==0.47.0"] From efb2b4dc2d32272fb51edd1b207cc5a8e7a6de35 Mon Sep 17 00:00:00 2001 From: Bedram Tamang Date: Wed, 8 Jul 2026 10:21:19 -0700 Subject: [PATCH 6/7] python/fastapi_startkit: bump displayed framework version to 0.47 Keep the benchmark's reported version label in sync with the 0.47.0 pin. --- python/fastapi_startkit/config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/fastapi_startkit/config.yaml b/python/fastapi_startkit/config.yaml index 05c2dece30c..702fd9cfd71 100644 --- a/python/fastapi_startkit/config.yaml +++ b/python/fastapi_startkit/config.yaml @@ -1,7 +1,7 @@ framework: website: fastapi-startkit.github.io github: fastapi-startkit/fastapi-startkit-framework - version: 0.46 + version: 0.47 engines: - uvicorn From 278a9f2d2ba5111a80fcd4b2495aa63832272dab Mon Sep 17 00:00:00 2001 From: Bedram Tamang Date: Wed, 15 Jul 2026 12:22:34 -0700 Subject: [PATCH 7/7] Fix FastAPI benchmark: match baseline routing to fastapi-startkit The raw-FastAPI baseline registered routes with direct @app.get/@app.post decorators while the fastapi-startkit app registers the same routes via APIRouter + include_router. That mismatch meant the benchmark was measuring include_router's per-request routing cost, not framework overhead. Register the baseline's routes on an APIRouter mounted with app.include_router(router) so both sides use the identical routing mechanism, isolating the comparison to actual framework overhead. --- python/fastapi/server.py | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/python/fastapi/server.py b/python/fastapi/server.py index 7f3141520b9..8572508958c 100644 --- a/python/fastapi/server.py +++ b/python/fastapi/server.py @@ -1,19 +1,23 @@ -from fastapi import FastAPI +from fastapi import FastAPI, APIRouter from starlette.responses import PlainTextResponse -app = FastAPI() +router = APIRouter() -@app.get("/") +@router.get("/") async def index(): return PlainTextResponse(content="") -@app.get("/user/{id}") +@router.get("/user/{id}") async def get_user(id: int): return PlainTextResponse(content=f"{id}".encode()) -@app.post("/user") +@router.post("/user") async def create_user(): return PlainTextResponse(content="") + + +app = FastAPI() +app.include_router(router)