Repository navigation
perf(query): answer resource lookups from a per-batch inventory - #327
Merged
Merged
Conversation
The default dashboard's versions and check_states panels decoded every span's (and for versions every log record's and metric point's) resource in the window to find a handful of distinct resources. On an 8-vCPU host running the OpenTelemetry Demo each took 5 to 6.5 seconds alone and missed the 10-second deadline when four panels ran together. Completed-batch caching now also records one row per batch, signal, namespace, service and distinct resource, with its earliest and latest event time (read cache version 6). Rows are built in the same transaction as the batch marker, move with compaction, and leave with retention. Panel SQL can read them through a new relation, resource_inventory: one row per distinct resource observed in the panel's logical window, with first_seen and last_seen inside that window. Batches wholly inside the window use cached rows; boundary and uncached files are grouped exactly from Parquet under the same pinned snapshot. The relation is available to dashboard panels only, is described in the telemetry schema and MCP guidance, and is rejected by unscoped SQL. versions and check_states read it, with identical output. On a one-hour Demo snapshot the warm panel time fell from about 2.4 to 0.9 seconds (versions) and 1.7 to 0.6 seconds (check_states). Claude-Session: https://claude.ai/code/session_01XA52nWyQ3R5f5kK8kjpAQW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The default dashboard's
versionsandcheck_statespanels decoded every span's resource in the window to find a handful of distinct resources. Forversionsthat also meant every log record's and metric point's. On an 8-vCPU host running the OpenTelemetry Demo, each took 5–6.5 s alone and missed the 10-second deadline when four panels ran together.cacheBatches,transferBatchCacheand stale-deletion paths.resource_inventoryis a new panel-SQL relation: one row per distinct resource observed in the panel's logical window, withfirst_seenandlast_seeninside that window.sql_relationsin the telemetry schema, the MCP and AI guidance, and the generated docs. It is dashboard-panel only; unscoped SQL rejects it.versionsandcheck_states(service_hosts) read it, with identical output.Evidence
One-hour Demo snapshot, frozen window, frames compared row by row: 16
versionsrows and 61check_statesrows, all identical.Warm panel time through the real executor:
Cache catch-up built 6,937 inventory rows in 8.6 s across 13 passes.
Proof tests cover:
Codex adversarial review: approve, no material findings.
Verification
go testandgo test -raceon./internal/query/... ./internal/panel/... ./internal/dashboard/...: passed, including after rebase onto main.just check(pre-push): passed.Qualification on the 8-vCPU Demo host follows the release.