Skip to content

Add llm-d coordinator integration: hydrate and persist endpoints #215

Description

@Mohammad-nassar10

Problem statement / motivation

Let llm-d use agentic-api for Responses state without routing model traffic through the gateway. A stateful turn is exposed as two separately callable steps: hydrate before inference, persist after, so the caller makes the model
call itself.

This approach represents a separation of concerns between agentic-api which is responsible for all state management, hydration, etc., and llm-d coordinator, which is responsible for the AI request and response pipelines.

Proposed solution

  • hydrate: resolve previous_response_id and return the upstream request with history inlined and continuation and storage fields removed, plus an opaque context for the turn.
  • persist: take that context and the model's response, store the turn, and return the response envelope.

Both compose the functions the in-process executor already uses, so rehydration and persistence have one
implementation rather than two.

The endpoints are served by a separate crate and binary agentic-llm-d that depends on agentic-server-core and not on the gateway. It serves /internal/hydrate, /internal/persist, /health and /ready.

Alternatives considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions