docs: document what the observability stack costs in RPS - #234
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #187.
A Performance page under Introduction, sourced from
benchmarks/README.mdrather than copying it,plus a re-measurement of the whole write-up and the harness change that makes its "tuned"
configuration something a reader can actually paste.
The page
docs/introduction/performance.md, in the nav, covering what the issue asked for: headline table,per-instrument table, the four findings that surprised the author, the trade-off table, the
trivial-handler caveat, and an "if you are RPS-constrained, start here" section. Linked from the
Sentry, OpenTelemetry, Prometheus and Structlog sections of the configuration page.
The issue asked for a page listing knobs that did not exist at the time. Three of its four sibling
issues have since shipped, so the page documents
opentelemetry_sampler,opentelemetry_exclude_spans,sentry_auto_session_trackingandsentry_logging_breadcrumb_levelas real settings.
One copy of every number
The page pulls five blocks out of
benchmarks/README.mdthroughpymdownx.snippets, which wasalready enabled. Markers live in HTML comments so the benchmarks file still renders cleanly on
GitHub, and they are stripped from the published page.
benchmarks/is outsidedocs_dir, so this is a deliberate coupling.check_paths: truemakes itfail loudly: I pointed a snippet at a missing file and confirmed the build aborts with
Snippet at path 'benchmarks/MOVED.md' could not be found, then confirmed all five regions actuallyrender rather than silently emitting nothing.
Re-measured, one session, one machine
Every number in
benchmarks/README.mdis from today. The environment block needed no change: samemacOS 26.6.2, CPython 3.14.7, same library versions, so this is like-for-like.
integrations=[]Every conclusion survived, which is the useful result: the ordering and the ratios are the durable
part, as §8 has always claimed.
Scope note. §8's documented commands do not reproduce the whole file. They miss §3's structlog
table, five of §4b's nine rows, §5's
verify.pytable, §2's concurrency sweep, thetraces_sample_rate=1.0figure and §1's Sentry-alone cost. Refreshing only the documented subsetwould have left the file mixing two sessions, which is exactly what its own closing caveat says
makes the columns incomparable. So the rest was run too.
The harness no longer monkeypatches
benchmarks/stack_scenarios.pyreached "tuned" by patching around features that now exist:Both docstrings were false, and
_sentry_tunedhand-built what are now config fields. The scenariosare now plain
FastAPIConfigparts;_patch_*,PATCHESandsetup()are gone. Verified theresult is identical to what the patches produced:
ParentBasedat 0.01,exclude_spansset,auto_session_trackingFalse, breadcrumb and logs handlers bothNonewith the event handler stillat
ERROR.This matters for the page: it tells readers to set five things, so the benchmark should measure
those five things and not an approximation of them.
Corrections review caught
traces_sample_rate=1.0line never named its baseline. The figure is measured, but againstthe Sentry suite's own 6241 RPS baseline, not §3's bare 7698. Both review axes read it as an
arithmetic error, which settles it: the sentence now says which baseline it uses and that the two
tables are not directly comparable.
http_methods_to_capture=(). The listed settings summed to ~132 µsagainst ~136 µs measured; the missing one is worth ~35 µs, the second-largest saving and the one
with the sharpest trade-off. Added, and the column that merely restated its own link target is now
"what you give up".
sentry_auto_session_trackingwas documented at ~7 µs and re-measured at 7.7, and apre-existing
~33 µsforexclude_spansre-measured at 34.0. Both reconciled.about a third. Wrong before this branch; fixed here.
One finding I did not take: "most of it is Sentry's two log handlers" was called an overstatement at
9.6/33.7 = 28%, but that divides by §4c's whole per-request delta, charging the ASGI cost to every
log record. §4c minus §4b is 12.3 µs per record of handler cost against 9.6 recovered, so it is 78%.
The page now shows that subtraction instead of asserting "mostly".
Verification
ruff format,ruff check,ty, 307 tests,mkdocs build --strict, and the same lychee invocationCI runs (95 links, 0 errors).
Known, not addressed here
configuration.mdkeeps three inline µs figures outside the snippet mechanism. All three arereconciled now, but they will drift on the next re-measure. Snippeting single scalars seemed worse
than the drift.
verify.py's docstring advertisespython verify.py a b c; its argparse takes one scenario.lowest-directCI job runs onlyscripts/floor_smoke.py, nottyorpytest.