feat(metrics): identity, version, and environment resource attributes - #7266
Conversation
First Principles Review (Fable 5) — ✅ PASSPremise-level review of All claims verified against the repo. Writing the review now. First-Principles-Verdict: PASS Client-stamped resource labels are the only way these fleet questions can ever be answered, and every attribute ships with a derived bound. What this change shipsIntent: make every exported metric series identify the build, install, process, and machine class that produced it — an ADDITION (item 1 of #7232).
Every value reuses the beacon's existing clamps ( WatchThe description says the local Telemetry page "inherits these labels": the bytes land in the JSONL, but nothing in-repo reads them — the aggregator reads exactly one resource attribute, the start-time token ( Subtractions
[FIRST-PRINCIPLES-REVIEWED] ccd1829 |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Diff reviewed: Design-Verdict: CONCERNS The Watch
Suggestions
[DESIGN-REVIEWED] ccd1829 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsNo candidates were found by the discovery pass. I verified the production changes independently: No findings. [OPUS-REVIEWED] ccd1829 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/metrics/provider.py:533 -- False positive or not applicable? A repository writer can comment: |
5a50f02 to
f0ca152
Compare
f0ca152 to
2f93601
Compare
2f93601 to
3449292
Compare
First Principles CONCERNS on
|
| head | diff vs main |
|---|---|
c60481cf0 |
520 insertions, 21 deletions |
a988810bf |
379 insertions, 10 deletions |
Net between the two heads: 71 insertions against 201 deletions, so -130 lines,
in the range the review predicted. grep -rn '_id_backfill_pending\|_id_backfill_attempted\|_id_backfill_due\|needs_id_backfill' src/ test/ is empty.
Verification. One new test replaces the two deleted ones: on a fresh install
with consent enabled, building the recorder mints the id and the exported resource
attributes carry service.instance.id. Mutation-verified rather than assumed --
commenting out the create=True call turns it RED with AssertionError: build did not mint the install id, and restoring turns it green.
pytest test/metrics/test_resource_attrs.py-> 9 passedpytest test/test_beacon.py-> 152 passedpytest test/metrics/test_provider.py-> 34 passed (the consent-change and
rebuild paths, since_consent_workerandget_recorderwere both edited)- flake8, isort clean.
black --checkflagsprovider.py, which is pre-existing:
the file is in.github/black-baseline.txt(line 338) and origin/main's own copy
fails identically.
On why this was taken now rather than deferred. The usual reason to answer an
advisory concern in-thread instead of pushing is that a push re-rolls five
non-deterministic review lanes on a converged PR. That argument does not apply here:
this PR is already blocked on a main-owned red (the log-site census slack, fixed by
#7492) and has to be rebased once that lands, so the re-roll was already going to be
paid. Deferring would have meant shipping 120 lines and deleting them in a follow-up
-- paying the review cost twice for a net-negative diff.
The one thing I would flag for a reviewer of the new shape: the first synchronous
recorder build can now perform a write (three syscalls, once per install
lifetime) where before it only read. That is deliberate, it is inside install_id's
own try/except, and its failure degrades to the attribute being omitted rather than
to a wrong value or an exception on the build path.
a988810 to
10085e3
Compare
Correction: the subtraction I accepted is REVERTED in
|
10085e3 to
8ac2186
Compare
Round 7 dispositions -- rebased onto
|
8ac2186 to
783b337
Compare
Round 8 -- took the subtraction; the blocking finding is gone structurallyRebased onto GPT 5.6 BLOCKING -- hot backfill invalidates active recorder references. Accepted, and removed rather than fixed.The mechanism was real: GPT's suggested fix was "defer identity backfill until a normal rebuild or restart". First Principles, independently, asked for the same thing more completely -- move Deleted: First Principles -- subtraction taken. This reverses my round-7 rebuttal, and it was right to.Last round I argued the mint is a write while the path's existing ~14ms + ~57ms are reads, so the read cost did not license adding a write. That distinction does not survive contact with the numbers it cited: the ~57ms is a ~120-module SDK import, i.e. heavy file I/O on the same filesystem, so a cold or networked data dir makes the import the unbounded term and And the shipped behaviour was worse than either review knew. Omitting So the old design's documented residual ("until one of those runs, the attribute is simply absent") was false. Every pre-enabled fresh install exported series labelled with a per-restart uuid4 until the backfill landed -- exactly the churn this attribute exists to prevent -- and the backfill then swapped the resource mid-life, which a backend reads as a second, unrelated series set. Minting before the first build removes the window entirely: there is no export the substitute can reach. The subtraction is therefore strictly better, not merely smaller. Docstring, commit message and spec now state the fallback accurately instead of claiming absence. Consent is preserved by placement: the live branch is not reached when telemetry is off, so a disabled install still creates nothing. That is now a test. Design Review -- spec ask done in this PRAdded a On your second point -- the consent worker becoming a hand-rolled state machine -- this round removes the third branch rather than adding a fourth flag, so the worker is back to its two-branch shape. Your round-7 ask for a composite One defect I found in my own new testThe regression test for the build-path mint passed under xdist while failing in isolation -- a sibling test mints the install id into the same conftest-shared home, so the assertion was decided by execution order rather than by the code. An order-dependent test that happens to pass is worse than no test. Fixed with a Gates on
|
The MeterProvider resource carried only service.name, so nothing in an exported payload identified the build, the install, or the machine class that produced it. Set service.instance.id to the persisted install id (stable across restarts, never derived from hostname or username), service.version release-clamped via beacon.release so dev/nightly build stamps cannot mint a series set per build, the OTel semantic-convention environment attrs (os.type, host.arch and process.runtime.name as CLOSED sets folding unknown readings to 'other', the runtime version clamped to major.minor by beacon.python_minor, the single owner of that clamp), and host.cpu.logical_count so cpu.seconds can be normalized into a machine percentage downstream. Every probe fails soft: a failed read omits its attribute rather than losing telemetry. The install id is minted in _build_recorder's live branch, immediately before the resource is assembled, so the first export already carries it. That placement matters twice. Omitting the key does NOT yield an unlabelled resource -- the SDK substitutes its own per-process uuid4 -- so any window before the id lands exports exactly the per-restart series churn the attribute exists to prevent, and a mid-life swap to fix it would read downstream as a brand-new series set. And because the branch runs only under consent True, an install with telemetry off still creates nothing. process.pid carries the process identity separately from the install id: one install runs several telemetry-enabled processes at once, and a single install-scoped resource would interleave their gauges and cumulative counters into one corrupted series at a backend. The distribution channel and an install type are deliberately absent; the spec and the function docstring record why, and a regression test guards it. The metrics spec documented the SDK Resource as the payload contract for both sinks but described only what must NOT egress on it. It now carries the attribute set, each value's bound, why the install id may egress while the host-local process start-time token may not, and the fact that enabling metrics is what creates the install-id file. Refs #7232 #7257
783b337 to
ccd1829
Compare
Round 9 -- phantom description fixed; the mint placement is now a genuine deadlock between two lanesHead Design Review + First Principles: phantom description. Correct, and entirely my fault.Round 8 deleted the backfill machine and I updated the commit message and the docstrings but not the PR body, so the description still narrated a consent-worker mint, a one-shot backfill and a hot swap that no longer exist anywhere in the diff. Both lanes were right to flag it, and the rule I broke is the plain one: when the code shrinks, the description shrinks in the same push. The body is rewritten to match the shipped code, in the repo's five-section shape rather than as a round-by-round narrative. Also fixed, both from the same cause:
GPT 5.6 BLOCKING: synchronous install-id mint blocks the event loopThe mechanism is real and I am not disputing it: But the remedy -- "remove the synchronous mint and keep this build path read-only" -- is the exact shape this same lane blocked one round ago, and the two lanes are now asking for opposite things. Three properties are in play and only two can hold at once:
The third column is not cosmetic. Omitting Weighing what each corner costs: the current shape's cost is ~1ms of There is a fourth corner that satisfies all three: mint the id at process startup, off the loop, leaving I am not resolving this by pushing a fourth variant of the same 10 lines, and I am not reaching for an override on my own judgement. Flagging for the maintainer's ruling. Nothing else on this head is blocking; the remaining lanes are advisory CONCERNS that this round addressed. Gates on
|
What is the problem?
The MeterProvider resource carries only
service.name. Nothing in an exported payload identifies the build that produced it, the install it came from, the process within that install, or the machine class it ran on:service.version, so release-over-release comparison ("did the fix land") can only be done by comparing time ranges, which a gradual rollout muddies -- both versions report into the same buckets;uuid4, so every restart starts a brand-new series set and "this install over time" cannot be asked at all;host.cpu.logical_count, sokirocrew.process.cpu.secondscannot be normalized into "percent of this machine" anywhere downstream -- the core count exists only client-side;This is item 1 of #7232 and the first milestone slice of #7257.
Why this issue matters to the user
Any consumer of the OTLP egress seam (an edition-supplied collector, CloudWatch, Datadog, or the local Telemetry page reading the JSONL sink) inherits these labels on every series. Without them a fleet dashboard cannot group by version, count devices, separate concurrent processes, or normalize CPU -- and no downstream work can add them later, because resource attributes only exist if the client stamps them at export time.
How our fix solves it
_resource_attributes()is the single place the resource is built, andResource.create()now consumes it. Every value is a closed set or an explicit clamp, because a resource attribute is a label on EVERY series this process exports -- one unbounded value here multiplies every instrument. Every probe fails soft: a failed read omits its attribute rather than losing telemetry or inventing a value.service.instance.idbeacon.install_id)process.pidos.getpid()service.versionbeacon.releasemajor.minor.patch, no build stampos.typelinux/darwin/windowsotherhost.archamd64/arm64/x86(aliases folded)otherprocess.runtime.namecpython/pypy/jython/ironpythonotherprocess.runtime.versionbeacon.python_minor()major.minor, never the patchhost.cpu.logical_countos.cpu_count()Two identities, deliberately separate.
service.instance.idcounts DEVICES: a random UUID persisted on disk, never derived from hostname or username (which routinely embed an employee alias on a corporate desktop), stable across restarts.process.pidcounts PROCESSES. Collapsing them would report one machine's 6-8 concurrent processes as 6-8 machines; omitting the pid would interleave those processes into one corrupted series. PID reuse across restarts reads as an ordinary counter reset downstream.Where the id is minted, and why there. The probe is read-only (
create=False: one stat plus a 32-byte read). The single write --beacon.install_id(create=True), i.e. mkdir + mkstemp + link, race-safe per its own docstring, once per install ever -- sits in_build_recorder()'s live branch immediately before the resource is assembled. That placement is load-bearing in three ways:uuid4rather than an unlabelled resource, any gap before the id lands exports exactly the per-restart churn this attribute exists to prevent. Minting before the first build means no export can carry the substitute.The cost is one bounded write on a path that already runs synchronously on whatever thread touched telemetry first: the same first build pays
KiroCrewConfig.load()(~14ms on a changed file) plus ~57ms of SDK import, both documented indocs/system-specs/modules/metrics.md. The mint is noise against that, and a failed mint costs only the label -- never the recorder.Spec. The metrics spec documented the SDK
Resourceas the payload contract for both sinks but described only what must NOT egress on it. It now carries the attribute table, the two-identities rationale, why the install id may egress while the host-localkirocrew.process.start_timetoken may not, and the disclosed consequence that enabling metrics is what creates the install-id file even where the beacon itself is disabled.Deliberately absent, documented in the spec and guarded by a regression test: the distribution channel (the beacon's data-minimization pass removed its channel field because channel sharply narrows the anonymity crowd a stable id hides in; re-adding it is a consent-inventory question, not a code convenience) and an install type (no reliable detection exists today; a guessed label would be confidently wrong).
Imports (
beacon,__version__,platform) live at module scope per the top-level-imports rule -- stdlib-or-already-loaded, and no cycle, since the config loader itself importskiro_crew.__version__at module scope.What tests we did
test/metrics/test_resource_attrs.py, 11 tests:major.minorruntime asserted againstbeacon.python_minor()so the two surfaces cannot drift, 32-hex install id, integer core count,process.pid == os.getpid());x86_64/AMD64->amd64,aarch64->arm64);other(the closed-set guarantee, includingprocess.runtime.name);uuid4, asserted by shape so it can never be mistaken for the persisted 32-hex id;service.version; id-read failure omits onlyservice.instance.id;Each behaviour is mutation-verified -- reverting the runtime closed set, or the build-path mint, turns the suite red rather than leaving a decorative assertion. Tests that assert on the install-id file are given their own
KIROCREW_HOME, so a sibling test that mints cannot decide the outcome by execution order.Gates:
flake8/isort/mypyclean on the changed files; the new test file is black-clean andprovider.pystays on the black baseline (never whole-file reformatted);docs-lint.shclean.Any other suggestions on the work
service.instance.idsemantics are worth a second opinion before this merges. OTel semconv wants that key unique per instance, and Prometheus OTLP ingestion maps it toinstancewhile demoting other resource attributes totarget_info-- so at a Prometheus-family backend the gateway and its spawned agents still land on one(job, instance)pair despiteprocess.pid. A composite ({install_id}-{pid}) plus a dedicatedkirocrew.install.idfor fleet GROUP BYs would satisfy both readings at no extra cardinality, sinceprocess.pidalready turns the series set over per restart. It is a one-way door once dashboards key on the shape, which is why it is raised here rather than deferred.Refs #7232, #7257 (milestone M0).