Repository navigation
perf(observability): scan client spans once for external service-map peers - #329
Merged
Merged
Conversation
…peers The service map's external-peer query reused a clients CTE in two anti-joins. DuckDB inlined it, so each run scanned spans eight times (366 MB on a one-hour OpenTelemetry Demo snapshot). On an 8-vCPU Demo host the panel took 6.5 s at the median and 9.9 s at p95 during dashboard refreshes, at the edge of the 10-second deadline. Materialize the client projection once. On the snapshot the plan has one Parquet scan reading 80 MB, and the warm panel time falls from about 450 to 140 ms. All 45 frame rows match in order. Claude-Session: https://claude.ai/code/session_01XA52nWyQ3R5f5kK8kjpAQW
vishr
force-pushed
the
perf/service-map
branch
from
October 17, 2026 04:27
b59a8ad to
6b66224
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The service map's external-peer query reused a
clientsCTE in two anti-joins. DuckDB inlined it, so each run scanned spans eight times. On an 8-vCPU host running the OpenTelemetry Demo (v2026.10.10), the panel took 6.5 s p50 and 9.9 s p95 across 20 dashboard refreshes, at the edge of the 10-second deadline.WITH clients AS MATERIALIZEDcomputes the client projection once. External peers still come from raw client spans:edge_rolluphas no entries for these external pairs and stores no peer latency quantiles, so it cannot replace them.Evidence
One-hour Demo snapshot, frozen window:
All 45 frame rows match in order; average latency differs only by floating-point roundoff (2e-14 ms). A new regression test covers peers, rates, latencies, scopes and ordering.
Verification
go test ./internal/observability/... ./internal/panel/...: passed.just check(pre-push): passed.