Skip to content

fix: replay persisted reasoning safely - #222

Merged
franciscojavierarceo merged 1 commit into
vllm-project:mainfrom
StevenWang-CY:fix-reasoning-continuation-replay
Aug 29, 2026
Merged

fix: replay persisted reasoning safely#222
franciscojavierarceo merged 1 commit into
vllm-project:mainfrom
StevenWang-CY:fix-reasoning-continuation-replay

Conversation

@StevenWang-CY

@StevenWang-CY StevenWang-CY commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #153.

Current vLLM has two materially different reasoning-input conversions: the generic Responses converter rejects non-null encrypted state, consumes only content[0].text, and falls back to summary[0].text; the Harmony converter joins every content part and ignores empty content. Forwarding the complete persisted item unchanged can therefore reject compatible plaintext, drop later plaintext parts, or silently promote a summary.

This change:

  • validates the complete non-compaction Responses input before the blocking/streaming split, returning ExecutorError::InvalidRequest for non-null opaque state without non-empty plaintext before normal vLLM serialization or network I/O;
  • prepares only the request-owned ctx.enriched_request.input for the initial normal inference round, and only when automatic compaction has not replaced that input;
  • canonicalizes all ordered content[].text parts into one newline-delimited reasoning_text part so generic and Harmony conversion receive identical plaintext;
  • clears summary and encrypted_content only from the vLLM-bound copy, and removes summary-only items with missing or JSON-null encrypted state so summaries cannot become continuation reasoning;
  • preserves item ID, status, surrounding history order, ctx.original_request, ctx.new_input_items, and the complete persisted reasoning item unchanged.

No schema, migration, dependency, public wire type, or logging change is introduced. Standalone/in-band compaction input and reasoning newly produced during later gateway rounds remain unchanged and out of scope; the latter remains tracked by #120.

Adversarial audit

Preserved invariants are lossless storage, stable item ordering, no private storage marker on the wire, no opaque-state or summary fallback at the normal vLLM boundary, and identical behavior for blocking and streaming continuations.

The regression coverage exercises:

  • multipart plaintext whose first part is empty and second part is usable; the outbound copy becomes exactly "\nplaintext continuation", matching Harmony joining while remaining usable by the generic converter;
  • a malformed provider content-part discriminator, canonicalized to reasoning_text;
  • missing and JSON-null encrypted state;
  • non-null string and object-shaped opaque state;
  • empty vectors, one empty part, and multiple empty parts;
  • summary-only history, which is omitted from the inference copy but retained exactly in storage;
  • a valid reasoning item followed by an invalid encrypted-only item, proving full-input validation is atomic and does not partially mutate earlier items;
  • exact ordering across historical user input, reasoning, assistant message, function call, new function output, and new user input;
  • previous-response and conversation persistence, blocking and streaming first/second turns, ID/status preservation, recursive _agentic_item_kind absence, error redaction, and zero recorded upstream calls on rejection.

The preparation is synchronous and request-owned. It introduces no shared state, cancellation cleanup, or concurrency surface, and it runs only for the initial normal inference round so active-loop reasoning semantics do not drift.

Verification

  • Unmodified upstream main d8720a7083afb40030385eb874929d72a0fe1f3d, with only the final integration regression file applied: cargo test -p agentic-server-core --test stateful_responses_integration reasoning -- --nocapture produced 0 passed / 6 failed. Four replay variants retained incompatible multipart/opaque state, summary-only reasoning remained present, and encrypted-only history reached the mock backend and returned 502 instead of a local 400.
  • Patched commit 068c4549258137873904058045bf796d74aae0bd: the same integration command passed 6/6.
  • cargo test -p agentic-server-core executor::rehydrate -- --nocapture passed 10/10 focused tests.
  • cargo test --workspace --quiet -- --test-threads=4 passed every runnable workspace test; repository-declared ignored tests remained ignored.
  • cargo clippy --workspace --all-targets -- -D warnings passed.
  • cargo fmt --all -- --check passed.
  • uvx pre-commit==4.4.0 run --all-files passed every applicable hook.

An earlier eight-thread workspace run encountered an OS WouldBlock while an unchanged MCP proxy test spawned a thread. That test passed in isolation on both this branch and an exact detached upstream-main control, and the complete four-thread workspace rerun passed.

Signed-off-by: Chuyue Wang <stevenwang0805@outlook.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes persisted reasoning replay for previous_response_id and conversation continuations by validating and normalizing reasoning history before it can reach vLLM, while keeping the stored/persisted reasoning items lossless and unchanged.

Changes:

  • Add executor-side validation to reject reasoning items that contain non-null opaque/encrypted state but no non-empty plaintext content, returning a 400-class ExecutorError::InvalidRequest before any upstream I/O.
  • Normalize vLLM-bound reasoning history by joining all content[].text parts into a single newline-delimited reasoning_text, clearing summary, and stripping encrypted_content from the request copy (and dropping summary-only reasoning items from the vLLM-bound copy).
  • Extend integration and unit coverage to verify correct ordering, preservation of persisted items, streaming/non-streaming equivalence, and “no upstream call on invalid continuation” behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
crates/agentic-server-core/tests/stateful_responses_integration.rs Adds regression/integration tests for plaintext reasoning replay, summary-only suppression, and encrypted-only early rejection (blocking + streaming, conversation + previous_response_id).
crates/agentic-server-core/src/executor/rehydrate.rs Introduces validate_reasoning_for_vllm and prepare_reasoning_for_vllm plus focused unit tests to enforce and verify the normalization/rejection rules.
crates/agentic-server-core/src/executor/engine.rs Wires validation ahead of the streaming/blocking split and applies reasoning preparation only on the initial normal inference round when automatic compaction has not replaced input.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@franciscojavierarceo
franciscojavierarceo merged commit 81d464c into vllm-project:main Aug 29, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Reasoning retention 2/2] Replay persisted reasoning safely to vLLM

3 participants