Skip to content

fix: reconcile complete streamed reasoning items - #219

Open
StevenWang-CY wants to merge 5 commits into
vllm-project:mainfrom
StevenWang-CY:fix-streamed-reasoning-reconciliation
Open

fix: reconcile complete streamed reasoning items#219
StevenWang-CY wants to merge 5 commits into
vllm-project:mainfrom
StevenWang-CY:fix-streamed-reasoning-reconciliation

Conversation

@StevenWang-CY

@StevenWang-CY StevenWang-CY commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #152.

The streaming accumulator previously merged reasoning text and summary events into one unindexed buffer and did not reconcile reasoning response.output_item.done events. A streamed response could consequently omit completed content, summaries, encrypted_content, status, or a reasoning item whose added event was absent.

This change:

  • gives reasoning text and reasoning summary events distinct typed payloads with their output and part indexes;
  • constructs reasoning items through ReasoningOutput::try_from for both added and done-only lifecycle events;
  • routes actual reasoning text and summary done events directly through ReasoningOutput::apply_done;
  • makes the complete reasoning item authoritative for content, summary, encrypted_content, and status;
  • treats present empty arrays and null values as authoritative while preserving fields established by prior done events when the completed item omits them;
  • retains valid done-event state when a completed item is malformed;
  • verifies complete typed reasoning items survive the existing JSON storage round trip without leaking the private storage marker.

ResponseAccumulator now owns only event lifecycle routing. It does not retain a second reasoning content model or synthesize terminal events from deltas. No database schema or migration changes are required.

Test Plan

  • cargo test -p agentic-server-core reasoning_output_ — 3 passed
  • cargo test -p agentic-server-core executor::accumulator — 54 passed
  • cargo test -p agentic-server-core --test event_normalizer_test — 40 passed
  • cargo test -p agentic-server-core storage — 69 passed; 3 PostgreSQL environment tests ignored
  • cargo test -p agentic-server-core — passed
  • cargo test --workspace -- --test-threads=8 — passed
  • cargo clippy --workspace --all-targets -- -D warnings — passed
  • cargo fmt --all -- --check — passed
  • uvx pre-commit==4.4.0 run --all-files — passed

Signed-off-by: StevenWang-CY <203932027+StevenWang-CY@users.noreply.github.com>
Comment thread crates/agentic-server-core/src/executor/accumulator.rs Outdated
Signed-off-by: StevenWang-CY <203932027+StevenWang-CY@users.noreply.github.com>
use crate::utils::uuid7_str;

#[derive(Default)]
struct ReasoningBuffers {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReasoningFallback has effectively been renamed to ReasoningBuffers; the fallback behavior still exists. apply_reasoning_buffers also creates synthetic ReasoningTextDone and ReasoningSummaryTextDone events during finalization, leaving reasoning reconstruction duplicated between the accumulator and ReasoningOutput::apply_done.

The accumulator should only route real lifecycle events. OutputItemAdded and done-only OutputItemDone should use ReasoningOutput::try_from, while actual reasoning done events should locate the in-flight item and call apply_done. Because the done payload contains the authoritative final text, deltas do not need to be retained for final response construction.

For example, the flow could remain close to the original main implementation, adapted for the new event variants:

enum InFlight {
    // ...
    Reasoning {
        item: ReasoningOutput,
    },
}
SSEItemType::Reasoning => ReasoningOutput::try_from(payload)
    .ok()
    .map(|item| InFlight::Reasoning { item }),
(
    SSEEventType::ReasoningTextDone,
    payload @ EventPayload::ReasoningTextDone {
        item_id,
        output_index,
        ..
    },
) => {
    let key = self.in_flight_reasoning_key(item_id, *output_index);

    if let Some(InFlight::Reasoning { item }) = key
        .as_deref()
        .and_then(|key| self.in_flight.get_mut(key))
        .map(|entry| &mut entry.item)
    {
        item.apply_done(payload, &mut String::new());
    }
}

(
    SSEEventType::ReasoningSummaryTextDone,
    payload @ EventPayload::ReasoningSummaryTextDone {
        item_id,
        output_index,
        ..
    },
) => {
    let key = self.in_flight_reasoning_key(item_id, *output_index);

    if let Some(InFlight::Reasoning { item }) = key
        .as_deref()
        .and_then(|key| self.in_flight.get_mut(key))
        .map(|entry| &mut entry.item)
    {
        item.apply_done(payload, &mut String::new());
    }
}

The existing complete_reasoning_item can likewise call item.apply_done(payload, ...) for an in-flight item or ReasoningOutput::try_from(payload) for a done-only item. With that structure, ReasoningBuffers, apply_reasoning_buffers, and the synthetic done-event construction can be removed. Reasoning delta events can still be relayed to the client without becoming a second source of truth for the accumulated response.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 9535d3b.

ReasoningBuffers and apply_reasoning_buffers are removed. The in-flight reasoning variant now owns only ReasoningOutput. Actual ReasoningTextDone and ReasoningSummaryTextDone events locate that item and call apply_done directly; reasoning deltas are not retained as a second response model. OutputItemDone calls apply_done for an in-flight item or ReasoningOutput::try_from for a done-only item.

The accumulator tests now exercise real added, reasoning-done, and output-item-done lifecycles. Direct TryFrom and ApplyDone tests remain in the typed output module. Full core and workspace tests, workspace clippy with warnings denied, rustfmt, and all pre-commit hooks pass on the updated head.

Signed-off-by: StevenWang-CY <203932027+StevenWang-CY@users.noreply.github.com>
Comment thread crates/agentic-server-core/src/executor/accumulator.rs Outdated
Comment thread crates/agentic-server-core/src/executor/accumulator.rs Outdated
Signed-off-by: StevenWang-CY <203932027+StevenWang-CY@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Reasoning retention 1/2] Reconcile complete streamed reasoning items

2 participants