refactor(api): prune unused/redundant/reconstructable RPCs - #27
Merged
Conversation
Remove five RPCs that no OpenEngine client exercises today. The Dynamo sidecar -- the single client for the vLLM, SGLang, and TensorRT-LLM OpenEngine servers -- never issues any of them: - Inference.Embed / Classify / Score: the sidecar only calls Generate; the non-generative inference plane has no consumer. - Control.SubscribeKvEvents: KV routing is driven entirely by GetKvEventSources plus the advertised out-of-band (ZMQ) event stream, never the in-band subscription. - Control.SubscribeRuntimeEvents: no client or server implementation. Drop the now-unused classification/embedding/scoring imports from the top-level service file and sync the docs service overview. The backing message schemas (embedding/scoring/classification, KV/runtime event envelopes) remain for now; pruning those and their model.proto capability references is a follow-up. Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Correct the previous commit and remove the message schemas behind the
genuinely-unused RPCs.
SubscribeKvEvents is NOT unused: the Dynamo sidecar consumes it on the
grpc/protobuf KV-event transport advertised by GetKvEventSources
(kv.rs subscribe_loop -> Control::subscribe_kv_events), decoding the
KvEventBatch/KvEvent/BlockStored payloads. The prior commit removed it
based on a grep that only matched the bare method name and missed the
snake_case call site. Restore the RPC; its messages in kv.proto stay.
Remove the schemas for the four RPCs that are actually dead (no client
call site anywhere in the sidecar):
- Delete embedding.proto, classification.proto, scoring.proto and their
shared tasks.proto vocabulary (Embed/Classify/Score).
- Drop ModelInfo.tasks (field 27) and the TaskCapabilities message from
model.proto.
- Remove SubscribeRuntimeEvents{Request,Response}, RuntimeEvent, and
RuntimeEventType from observability.proto (GetLoad's LoadInfo stays);
drop its now-unused error.proto import.
- Sync docs/api.md (service overview, Model identity block, delete the
Non-generative task API and Runtime observability sections) and the
proto/root README tables.
Validated with protoc (descriptors build) and markdownlint-cli2 (clean).
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Remove the GetKvConnectorInfo RPC and its GetKvConnectorInfoRequest message. The connector capabilities it returned are static per deployment and are already carried on ServerInfo.kv_connector (GetServerInfo) -- all engines populate both surfaces from the same value, and the sidecar fetched GetServerInfo and then re-fetched the identical KvConnectorInfo, discarding the first. Follow the contract's static-vs-dynamic split: static discovery data (parallelism, capacity, connector) rides on ServerInfo; only dynamic state (LoadInfo via GetLoad) keeps a dedicated poll RPC. Keep the KvConnectorInfo message -- it remains ServerInfo.kv_connector's type, now its sole source. Sync docs/api.md (service overview + KV connector section). Validated with protoc and markdownlint-cli2. Signed-off-by: Connor Carpenter <connorc@nvidia.com>
connorcarpenter15
force-pushed
the
refactor/remove-unused-rpcs
branch
from
July 23, 2026 05:23
66e09dd to
4b98f85
Compare
connorcarpenter15
marked this pull request as ready for review
July 23, 2026 05:25
connorcarpenter15
requested review from
alec-flowers,
dagil-nvidia,
nv-anants,
nvda-mesharma and
saturley-hall
as code owners
July 23, 2026 05:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Trims the OpenEngine v1 contract to the RPCs and message schemas a client actually needs. The Dynamo sidecar is the single OpenEngine client for the vLLM, SGLang, and TensorRT-LLM servers; scope is defined by what it calls.
Removed — 4 genuinely-unused RPCs + their message trees
Inference.Embed/Classify/Score— the sidecar only callsGenerate. Deletedembedding.proto,classification.proto,scoring.proto, and the sharedtasks.proto; droppedModelInfo.tasks(field 27) andTaskCapabilitiesfrommodel.proto.Control.SubscribeRuntimeEvents— no client or server implementation. RemovedSubscribeRuntimeEvents{Request,Response},RuntimeEvent,RuntimeEventTypefromobservability.proto(GetLoad'sLoadInfo/RankLoadInfostay).Removed — 1 redundant RPC (data preserved elsewhere)
Control.GetKvConnectorInfo+GetKvConnectorInfoRequest— returned the exactKvConnectorInfoalready onServerInfo.kv_connector(GetServerInfo). All three engines populate both surfaces from the same value; the sidecar fetchedServerInfo, then re-fetched the identical connector info and discarded the first. Follows the contract's static-vs-dynamic split: static discovery data (parallelism, capacity, connector) rides onServerInfo; only dynamic state (LoadInfoviaGetLoad) keeps a poll RPC.KvConnectorInfois kept — nowServerInfo.kv_connector's sole source.Removed — 1 reconstructable RPC
Control.Drain+DrainRequest/DrainResponse/DrainState— graceful shutdown doesn't need an engine-facing drain RPC. The client stops admitting (discovery unregister), lets in-flightGeneratestreams complete, pollsGetLoadfor quiescence, and cancels stragglers via gRPC stream cancellation into the engine's existingAbortpath. Droppedlifecycle.proto's now-unusederror.protoimport. Kept the drain-adjacent state signals that stand on their own —HEALTH_STATE_DRAINING,KvConnectorInfo.supports_drain(validated by the sidecar for disagg connectors),ERROR_CODE_DRAINING— since an engine still drains on its own SIGTERM.Retained —
SubscribeKvEventsNot unused: the sidecar consumes it on the
grpc/protobufKV-event transport advertised byGetKvEventSources(kv.rssubscribe_loop→Control::subscribe_kv_events). Only TensorRT-LLM serves it, because its KV events are available solely through an in-process API — vLLM/SGLang advertise a native ZMQ/msgpack publisher instead.Final Control surface
GetServerInfo·GetModelInfo·GetLoad·Health·Abort·LoadLora·UnloadLora·ListLoras·GetKvEventSources·SubscribeKvEvents(+Inference.Generate).Validation
protoc -I proto --experimental_allow_proto3_optionalbuilds the full descriptor set clean.markdownlint-cli2clean on all Markdown.Consumer follow-up (separate repos, not this PR)
engine.kv_connectorfromGetServerInfoand deletes its second discovery call (client.rs:144–152); the three servers drop theGetKvConnectorInfomethod but keep thekv_connector_info()helper that feedsGetServerInfo.drain()override (engine.rs:736) and the three servers'Drainhandlers are removed; the sidecar'sLLMEngine::drainreverts to the default no-op and graceful shutdown relies on theis_quiescent/GetLoad+ grace-period + stream-cancellation path already inWorker.Compatibility
Wire-breaking (removes RPCs and message types). No known consumer depends on the removed surface;
GetKvConnectorInfo's data is preserved onServerInfo.kv_connector.