Skip to content

refactor(api): prune unused/redundant/reconstructable RPCs - #27

Merged
connorcarpenter15 merged 3 commits into
mainfrom
refactor/remove-unused-rpcs
Jul 23, 2026
Merged

refactor(api): prune unused/redundant/reconstructable RPCs#27
connorcarpenter15 merged 3 commits into
mainfrom
refactor/remove-unused-rpcs

Conversation

@connorcarpenter15

@connorcarpenter15 connorcarpenter15 commented Jul 23, 2026

Copy link
Copy Markdown
Member

Summary

Trims the OpenEngine v1 contract to the RPCs and message schemas a client actually needs. The Dynamo sidecar is the single OpenEngine client for the vLLM, SGLang, and TensorRT-LLM servers; scope is defined by what it calls.

Removed — 4 genuinely-unused RPCs + their message trees

  • Inference.Embed / Classify / Score — the sidecar only calls Generate. Deleted embedding.proto, classification.proto, scoring.proto, and the shared tasks.proto; dropped ModelInfo.tasks (field 27) and TaskCapabilities from model.proto.
  • Control.SubscribeRuntimeEvents — no client or server implementation. Removed SubscribeRuntimeEvents{Request,Response}, RuntimeEvent, RuntimeEventType from observability.proto (GetLoad's LoadInfo/RankLoadInfo stay).

Removed — 1 redundant RPC (data preserved elsewhere)

  • Control.GetKvConnectorInfo + GetKvConnectorInfoRequest — returned the exact KvConnectorInfo already on ServerInfo.kv_connector (GetServerInfo). All three engines populate both surfaces from the same value; the sidecar fetched ServerInfo, then re-fetched the identical connector info and discarded the first. Follows the contract's static-vs-dynamic split: static discovery data (parallelism, capacity, connector) rides on ServerInfo; only dynamic state (LoadInfo via GetLoad) keeps a poll RPC. KvConnectorInfo is kept — now ServerInfo.kv_connector's sole source.

Removed — 1 reconstructable RPC

  • Control.Drain + DrainRequest/DrainResponse/DrainState — graceful shutdown doesn't need an engine-facing drain RPC. The client stops admitting (discovery unregister), lets in-flight Generate streams complete, polls GetLoad for quiescence, and cancels stragglers via gRPC stream cancellation into the engine's existing Abort path. Dropped lifecycle.proto's now-unused error.proto import. Kept the drain-adjacent state signals that stand on their own — HEALTH_STATE_DRAINING, KvConnectorInfo.supports_drain (validated by the sidecar for disagg connectors), ERROR_CODE_DRAINING — since an engine still drains on its own SIGTERM.

Retained — SubscribeKvEvents

Not unused: the sidecar consumes it on the grpc/protobuf KV-event transport advertised by GetKvEventSources (kv.rs subscribe_loopControl::subscribe_kv_events). Only TensorRT-LLM serves it, because its KV events are available solely through an in-process API — vLLM/SGLang advertise a native ZMQ/msgpack publisher instead.

Final Control surface

GetServerInfo · GetModelInfo · GetLoad · Health · Abort · LoadLora · UnloadLora · ListLoras · GetKvEventSources · SubscribeKvEvents (+ Inference.Generate).

Validation

  • protoc -I proto --experimental_allow_proto3_optional builds the full descriptor set clean.
  • markdownlint-cli2 clean on all Markdown.

Consumer follow-up (separate repos, not this PR)

  • Embed/Classify/Score/SubscribeRuntimeEvents: engine servers implement these as trait stubs; regenerated stubs drop the methods, so the stubs should be removed.
  • GetKvConnectorInfo: the sidecar reads engine.kv_connector from GetServerInfo and deletes its second discovery call (client.rs:144–152); the three servers drop the GetKvConnectorInfo method but keep the kv_connector_info() helper that feeds GetServerInfo.
  • Drain: the sidecar's drain() override (engine.rs:736) and the three servers' Drain handlers are removed; the sidecar's LLMEngine::drain reverts to the default no-op and graceful shutdown relies on the is_quiescent/GetLoad + grace-period + stream-cancellation path already in Worker.

Compatibility

Wire-breaking (removes RPCs and message types). No known consumer depends on the removed surface; GetKvConnectorInfo's data is preserved on ServerInfo.kv_connector.

Remove five RPCs that no OpenEngine client exercises today. The Dynamo
sidecar -- the single client for the vLLM, SGLang, and TensorRT-LLM
OpenEngine servers -- never issues any of them:

- Inference.Embed / Classify / Score: the sidecar only calls Generate;
  the non-generative inference plane has no consumer.
- Control.SubscribeKvEvents: KV routing is driven entirely by
  GetKvEventSources plus the advertised out-of-band (ZMQ) event stream,
  never the in-band subscription.
- Control.SubscribeRuntimeEvents: no client or server implementation.

Drop the now-unused classification/embedding/scoring imports from the
top-level service file and sync the docs service overview. The backing
message schemas (embedding/scoring/classification, KV/runtime event
envelopes) remain for now; pruning those and their model.proto
capability references is a follow-up.

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Correct the previous commit and remove the message schemas behind the
genuinely-unused RPCs.

SubscribeKvEvents is NOT unused: the Dynamo sidecar consumes it on the
grpc/protobuf KV-event transport advertised by GetKvEventSources
(kv.rs subscribe_loop -> Control::subscribe_kv_events), decoding the
KvEventBatch/KvEvent/BlockStored payloads. The prior commit removed it
based on a grep that only matched the bare method name and missed the
snake_case call site. Restore the RPC; its messages in kv.proto stay.

Remove the schemas for the four RPCs that are actually dead (no client
call site anywhere in the sidecar):

- Delete embedding.proto, classification.proto, scoring.proto and their
  shared tasks.proto vocabulary (Embed/Classify/Score).
- Drop ModelInfo.tasks (field 27) and the TaskCapabilities message from
  model.proto.
- Remove SubscribeRuntimeEvents{Request,Response}, RuntimeEvent, and
  RuntimeEventType from observability.proto (GetLoad's LoadInfo stays);
  drop its now-unused error.proto import.
- Sync docs/api.md (service overview, Model identity block, delete the
  Non-generative task API and Runtime observability sections) and the
  proto/root README tables.

Validated with protoc (descriptors build) and markdownlint-cli2 (clean).

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@connorcarpenter15 connorcarpenter15 changed the title refactor(api): remove unused RPCs refactor(api): remove unused non-generative + runtime-event RPCs and schemas Jul 23, 2026
@connorcarpenter15 connorcarpenter15 changed the title refactor(api): remove unused non-generative + runtime-event RPCs and schemas refactor(api): remove unused RPCs and fold GetKvConnectorInfo into ServerInfo Jul 23, 2026
Remove the GetKvConnectorInfo RPC and its GetKvConnectorInfoRequest
message. The connector capabilities it returned are static per
deployment and are already carried on ServerInfo.kv_connector
(GetServerInfo) -- all engines populate both surfaces from the same
value, and the sidecar fetched GetServerInfo and then re-fetched the
identical KvConnectorInfo, discarding the first.

Follow the contract's static-vs-dynamic split: static discovery data
(parallelism, capacity, connector) rides on ServerInfo; only dynamic
state (LoadInfo via GetLoad) keeps a dedicated poll RPC.

Keep the KvConnectorInfo message -- it remains ServerInfo.kv_connector's
type, now its sole source. Sync docs/api.md (service overview + KV
connector section).

Validated with protoc and markdownlint-cli2.

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@connorcarpenter15
connorcarpenter15 force-pushed the refactor/remove-unused-rpcs branch from 66e09dd to 4b98f85 Compare July 23, 2026 05:23
@connorcarpenter15
connorcarpenter15 marked this pull request as ready for review July 23, 2026 05:25
@connorcarpenter15
connorcarpenter15 merged commit 7d69fc2 into main Jul 23, 2026
5 checks passed
@connorcarpenter15
connorcarpenter15 deleted the refactor/remove-unused-rpcs branch July 23, 2026 05:25
@connorcarpenter15 connorcarpenter15 changed the title refactor(api): remove unused RPCs and fold GetKvConnectorInfo into ServerInfo refactor(api): prune unused/redundant/reconstructable RPCs Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant