Repository navigation
feat(telemetry): per-controller metric views and span attributes, so controllers sharing a process stay attributable - #91
Merged
Conversation
…controllers sharing a process stay attributable core-provider docs/design/cdc-pooling.md, step 3 (Diego, 2026-11-15): per-controller telemetry. A process's identity reaches its signals process-wide (resource attributes, service.instance.id), so in a process hosting several controllers every controller's numbers would merge into one stream. Nothing errors; per-Kind numbers just become wrong. - telemetry.Metrics is now a VIEW over shared instruments. Metrics.With(attrs...) returns a view that adds attrs to every point it records (counters, histograms, the queue depth, requeue reasons, rate-limit deferrals), sharing the instruments, so it adds no metric streams beyond the new attribute values. Each view keeps its own reconciles-in-flight count, and the gauge reports one point per view. The root view from Setup adds nothing, so a process hosting one controller exports exactly what it did. An idle view adds no in-flight series. Saturation (local capacity and its input) stays process-level: it measures the process every controller shares. - controller.Options.SpanAttributes / builder.WithSpanAttributes set attributes on every reconcile root span the controller starts. The root already names the object (k8s.object.*); these name the controller. The attribute itself is the caller's choice. core-provider will set krateo.composition.gvr on every cdc, dedicated or pooled (agreed with observability, whose obs-reconcile-perf and audit-to-reconcile join key on process-wide attributes today). Tests: two views from one Metrics record distinct streams (reconcile success by attribute 2/1, the root's 1 without it), each view's own in-flight count, and a point with its own attribute keeping the view's; an idle view adds no series and With on nil is nil; a root span carries SpanAttributes, and without the option (control) does not. Mutants behind a passing baseline, each killed (4/4): With dropping its attributes; point() dropping the view's; in-flight reporting only the root; span attributes not set. go test -race ./... exit 0 (18 packages). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N8HNbBzRpE6rUjMRzosAjG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
core-provider docs/design/cdc-pooling.md, step 3 (Diego, 2026-11-15):
per-controller telemetry. A process's identity reaches its signals
process-wide (resource attributes, service.instance.id), so in a process
hosting several controllers every controller's numbers would merge into
one stream. Nothing errors; per-Kind numbers just become wrong.
Metrics.With(attrs...) returns a view that adds attrs to every point it
records (counters, histograms, the queue depth, requeue reasons,
rate-limit deferrals), sharing the instruments, so it adds no metric
streams beyond the new attribute values. Each view keeps its own
reconciles-in-flight count, and the gauge reports one point per view.
The root view from Setup adds nothing, so a process hosting one
controller exports exactly what it did. An idle view adds no in-flight
series. Saturation (local capacity and its input) stays process-level:
it measures the process every controller shares.
attributes on every reconcile root span the controller starts. The root
already names the object (k8s.object.*); these name the controller.
The attribute itself is the caller's choice. core-provider will set
krateo.composition.gvr on every cdc, dedicated or pooled (agreed with
observability, whose obs-reconcile-perf and audit-to-reconcile join key on
process-wide attributes today).
Tests: two views from one Metrics record distinct streams (reconcile
success by attribute 2/1, the root's 1 without it), each view's own
in-flight count, and a point with its own attribute keeping the view's; an
idle view adds no series and With on nil is nil; a root span carries
SpanAttributes, and without the option (control) does not. Mutants behind
a passing baseline, each killed (4/4): With dropping its attributes;
point() dropping the view's; in-flight reporting only the root; span
attributes not set. go test -race ./... exit 0 (18 packages).
🤖 Generated with Claude Code
https://claude.ai/code/session_01N8HNbBzRpE6rUjMRzosAjG