Complete the metric surface described in the telemetry design #476

Activity

1. github-project-automation

   added this to Modelplane Roadmapon Sep 30, 2026
2. github-project-automation

   moved this to Backlog in Modelplane Roadmapon Sep 30, 2026
3. dennis-upbound commented on Oct 2, 2026

   

   dennis-upbound

   on Oct 2, 2026

   CollaboratorAuthor

   More actions

   One thing to fold in from the #470 review, raised by @lsviben: the per-object-state group above assumes resource-state-metrics runs on the control plane, but #470 composes a collector onto each inference cluster and none onto the control plane. So there is nothing there to scrape it, and that needs deciding before those 11 metrics can be built — either a collector on the control plane, or RSM exporting directly, or those metrics come from somewhere else entirely.

   Also worth recording against the engine group: SGLang publishes no per-request queue time and no preemption counters at v0.4.6, v0.4.9 or v0.5.0, so modelplane_request_queue_seconds and modelplane_requests_preempted_total are vLLM-only until it does. Checked against a running engine.
4. dennis-upbound

   added a commit that references this issue on Oct 5, 2026

   Say what model delivery should measure

   ...

   1e44687
5. dennis-upbound commented on Oct 5, 2026

   

   dennis-upbound

   on Oct 5, 2026

   Last edited by dennis-upbound

   CollaboratorAuthor

   More actions

   Research pass on this, prompted by the model delivery design (design/modelcache-sources.md, on dennis/oci-source-poc) wanting four cache metrics that land in the per-object-state group. Three things have changed since this was written, and the approach is now a decision rather than an open question.

   The folding group is unblocked

   Needs MetricMapping to grow the two fields left out of #470

   Both fields shipped in v0.5. labels and part are in apis/metricmappings/definition.yaml, and both are exercised on a live cluster: a mapping folding SGLang's two token counters gave modelplane_tokens_total{direction="input"} 398 and {direction="output"} 607, and part: Count over its latency histogram gave modelplane_engine_requests_total 12.

   The first step this issue names is also done. The gateway and the picker have been read off a running GKE cluster:

   • Gateway: gen_ai_server_request_duration_seconds, gen_ai_server_time_to_first_token_seconds, gen_ai_server_time_per_output_token_seconds. These are on the ext-proc sidecar's port, not the Envoy admin port — 55 gen_ai_* lines on :1064, zero on :19001.
   • Picker: llm_d_epp_scheduler_e2e_duration_seconds, llm_d_epp_scheduler_attempts_total. The inference_extension_* names are deprecated twins of the same series.

   So for this group what remains is DCGM's two (still gated on fields that are off by default) and writing the mappings.

   The per-object-state group has a blocker this issue does not name

   Every metric here reads an XR's status, and those XRs live on the control plane. Modelplane composes no collector onto the control plane. compose_collector is called only from the ServingStack path, and composes onto each inference cluster.

   So the group needs two things, not one: something to turn XR status into metrics, and somewhere for it to be scraped and exported from. design/telemetry.md:166-177 already specifies both, and neither exists.

   There is no control-plane-scoped XR to compose it from. ServingStack is namespaced and per-cluster, and every function composes through a provider that targets a workload cluster. So a control-plane collector is either a new XR and function, or something the install guide tells a user to run.

   One piece is already reusable: TelemetryDestination is cluster-scoped on the control plane, its own function composes nothing, and its schema is plain exporter configuration. A control-plane collector could read the same list that compose_collector reads. The only friction is collector.py:540, which stamps a per-cluster cluster attribute that a control-plane collector would set differently or omit.

   Two ways to get XR status into metrics

   kube-state-metrics CustomResourceStateMetrics (CRS). The decisive property: metricNamePrefix replaces the whole default prefix, so metricNamePrefix: modelplane with name: replicas_ready emits modelplane_replicas_ready directly. That passes filter/modelplane with no mapping at all. It reads nested paths, emits one series per array element or map key, selects conditions by [type=Ready], coerces RFC3339 timestamps to epoch seconds and quantities like 512Gi, and has a StateSet type that fans an enum out to one 0/1 series per value — which fits ModelCache.status.clusters[].phase exactly.

   - groupVersionKind: {group: modelplane.ai, version: v1alpha1, kind: ModelDeployment}
     metricNamePrefix: modelplane
     labelsFromPath: {deployment: [metadata, name], namespace: [metadata, namespace]}
     metrics:
       - name: replicas_ready
         each: {type: Gauge, gauge: {path: [status, replicas, ready], nilIsZero: true}}

   Limits: Gauge, StateSet and Info only, no expressions, and no count-items-matching-a-predicate. You emit per-item 0/1 gauges and sum in PromQL. It needs explicit RBAC per custom resource, and a second ksm on the control plane, since the one in the serving stack runs on workload clusters.

   resource-state-metrics (RSM), which is what design/telemetry.md already assumes. crossplane-contrib/resource-state-metrics packages it as a Crossplane Configuration (v0.7.0, pre-GA, 13 stars). Namespaced ResourceMetricsMonitor resources with CEL or Starlark resolvers, so predicate counts, timestamp arithmetic and gang completeness are expressible where CRS cannot reach them.

   Its catch is the mirror image of CRS's advantage: the prefix kube_customresource_ is hardcoded in internal/family.go:44 with no override. So RSM output needs a rename before filter/modelplane. That is cheap here, because transform/modelplane runs before filter/modelplane and a MetricMapping per metric is exactly the mechanism #470 built.

   Crossplane core gives nothing usable. Its metrics are function-run and controller counters; crossplane_managed_resource_{exists,ready,synced} covers managed resources, not composites.

   Suggested shape

   1. Decide the control-plane collector first. It blocks all 11 metrics and is the larger question, since it is the first thing Modelplane would compose or install on the control plane itself.
   2. Prefer CRS for the metrics it can express, because it emits modelplane_* natively and is upstream-stable. Keep RSM for the ones needing an expression, and pay one MetricMapping rename for those.
   3. modelplane_engine_restarts_total is plain kube-state-metrics (kube_pod_container_status_restarts_total) on the workload cluster, not CRS, and it is the metric #498 wants for a cause that outlives the pod.

   Cannot be settled without a cluster: whether LWS and Grove expose enough status for gang_incomplete without CEL, and whether the DRA driver publishes a parent GPU UUID for modelplane_replica_gpu.

   Cache metrics to add to this issue

   That design, in section 6.2, adds four, all in this group and blocked by it:

   • modelplane_cache_cluster_ready — a gauge for each cache and cluster, with the phase as a label
   • modelplane_cache_ready_seconds — created until readable
   • modelplane_cache_staged_bytes and modelplane_cache_size_bytes — progress

   #494 spent 40 minutes in one phase with no series to show it, and #495 had to read staging progress from a file listing. The StateSet type covers the first of these directly.