Skip to content

perf: add bench/memray/locust tooling + fix i18n dict-copy allocation - #32

Merged
antosubash merged 1 commit into
mainfrom
feature/pensive-chaum-ef6b58
Apr 17, 2026
Merged

antosubash merged 1 commit into
mainfrom
feature/pensive-chaum-ef6b58

Conversation

@antosubash

Copy link
Copy Markdown
Owner

Summary

  • Adds a performance harness — pytest-benchmark, memray, and locust — wired through the Makefile (make bench, make memray-run, make loadtest, make loadtest-memray) so the project can measure CPU, allocations, and load behavior without ad-hoc scripts.
  • Fixes the top own-code allocation site surfaced by that harness: build_i18n_block was copying the i18n messages dict on every Inertia render. A new I18nRegistry.messages_snapshot(locale) caches the plain-dict copy at load() time.

What changed

Tooling

  • pytest-benchmark>=4.0, memray>=1.14 (non-Windows), locust>=2.31 added to the dev group.
  • perf pytest marker gates benchmarks out of the default pytest run; make bench opts them in. A sample benchmark against forge_session_cookie demonstrates the pattern.
  • make memray-run / make memray-flamegraph record and render allocation profiles of any Python target (defaults to the benchmark suite).
  • make loadtest drives an already-running make dev; make loadtest-memray starts uvicorn under memray, seeds an admin, drives it with locust, shuts down (process-tree-aware) and emits an HTML flamegraph.
  • scripts/loadtest_seed.py mints a signed Starlette session cookie so the locustfile's AuthedUser exercises real product-service code paths (bypasses the login rate limiter under load).
  • .memray/ and .benchmarks/ added to .gitignore.

Fix

Why

Under make loadtest-memray with 100 users / 30s of authenticated traffic, build_i18n_block was the #6 allocation site with ~37K allocations and ~58 MB of churn per run — for data that is immutable within a process. The surrounding code had a comment explaining the copy was required because Inertia's JSON encoder doesn't accept MappingProxyType; moving the copy to load-time is a one-off rather than per-request.

After the fix, the function is no longer in the top-50 sites. Remaining hotspots (starlette.routing.Route.matches, itsdangerous signing, json.iterencode, sqlalchemy.await_only, inertia.templating) are all vendored framework code.

Measurements (100 users, 30s, authenticated scenarios)

metric before after Δ
Requests / failures 9,210 / 0 9,240 / 0 —
Throughput 308 req/s 310 req/s —
p99 latency 25 ms 21 ms flat
Total allocations 1,582,505 1,547,720 −34,785
Total memory churned 2.214 GB 2.155 GB −59 MB
Peak RSS 65.7 MB 65.5 MB —
build_i18n_block rank #6 (37K allocs) not in top-50 removed

Test plan

  • uv run pytest framework/core/tests/test_i18n.py framework/hosting/tests/ → 136 passed
  • make bench runs the sample benchmark and prints timings
  • make loadtest-memray (default args) seeds a user, profiles a full run, and emits a flamegraph; port is freed on exit
  • 100u/30s run before-and-after confirms build_i18n_block drops out of the top allocation sites

…ocation

Introduces a first-class performance harness for the project:

- pytest-benchmark (`make bench`) for micro-benchmarks, gated behind a `perf`
  marker so `make test-py` stays unchanged.
- memray (`make memray-run`, `make memray-flamegraph`) for allocation
  profiling of arbitrary Python targets.
- locust + memray load-profiling (`make loadtest`, `make loadtest-memray`)
  that starts uvicorn under memray, seeds a forged-session-cookie admin so
  traffic exercises real authenticated code paths, and renders a flamegraph.

The load-profile also surfaced a real own-code allocation hotspot:
`build_i18n_block` was calling `dict(registry.messages(locale))` on every
Inertia page render to coerce a `MappingProxyType` into something the JSON
encoder accepts. Across a 100-user / 30s run this was ~37K allocations and
~58 MB churn for data that never changes within a process.

`I18nRegistry.messages_snapshot(locale)` builds the plain-dict copy once at
`load()` time and hands out the same instance on every call. After the fix
the function is no longer in the top-50 allocation sites; total allocations
drop by ~35K and churn by ~59 MB while throughput and latency stay flat.
@antosubash
antosubash merged commit 4d47733 into main Apr 17, 2026
8 checks passed
@antosubash
antosubash deleted the feature/pensive-chaum-ef6b58 branch October 8, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant