fix: replay persisted reasoning safely - #222
Merged
franciscojavierarceo merged 1 commit intoAug 29, 2026
Merged
Conversation
StevenWang-CY
requested review from
bbrowning,
franciscojavierarceo,
jiahuei,
leseb,
maralbahari,
noobHappylife,
qandrew and
tjtanaa
as code owners
August 29, 2026 14:48
StevenWang-CY
force-pushed
the
fix-reasoning-continuation-replay
branch
from
August 29, 2026 16:03
7bb45ce to
f7d9ffa
Compare
Signed-off-by: Chuyue Wang <stevenwang0805@outlook.com>
StevenWang-CY
force-pushed
the
fix-reasoning-continuation-replay
branch
from
August 29, 2026 16:11
f7d9ffa to
068c454
Compare
There was a problem hiding this comment.
Pull request overview
This PR fixes persisted reasoning replay for previous_response_id and conversation continuations by validating and normalizing reasoning history before it can reach vLLM, while keeping the stored/persisted reasoning items lossless and unchanged.
Changes:
- Add executor-side validation to reject reasoning items that contain non-null opaque/encrypted state but no non-empty plaintext content, returning a 400-class
ExecutorError::InvalidRequestbefore any upstream I/O. - Normalize vLLM-bound reasoning history by joining all
content[].textparts into a single newline-delimitedreasoning_text, clearingsummary, and strippingencrypted_contentfrom the request copy (and dropping summary-only reasoning items from the vLLM-bound copy). - Extend integration and unit coverage to verify correct ordering, preservation of persisted items, streaming/non-streaming equivalence, and “no upstream call on invalid continuation” behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| crates/agentic-server-core/tests/stateful_responses_integration.rs | Adds regression/integration tests for plaintext reasoning replay, summary-only suppression, and encrypted-only early rejection (blocking + streaming, conversation + previous_response_id). |
| crates/agentic-server-core/src/executor/rehydrate.rs | Introduces validate_reasoning_for_vllm and prepare_reasoning_for_vllm plus focused unit tests to enforce and verify the normalization/rejection rules. |
| crates/agentic-server-core/src/executor/engine.rs | Wires validation ahead of the streaming/blocking split and applies reasoning preparation only on the initial normal inference round when automatic compaction has not replaced input. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
franciscojavierarceo
approved these changes
Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #153.
Current vLLM has two materially different reasoning-input conversions: the generic Responses converter rejects non-null encrypted state, consumes only
content[0].text, and falls back tosummary[0].text; the Harmony converter joins every content part and ignores empty content. Forwarding the complete persisted item unchanged can therefore reject compatible plaintext, drop later plaintext parts, or silently promote a summary.This change:
ExecutorError::InvalidRequestfor non-null opaque state without non-empty plaintext before normal vLLM serialization or network I/O;ctx.enriched_request.inputfor the initial normal inference round, and only when automatic compaction has not replaced that input;content[].textparts into one newline-delimitedreasoning_textpart so generic and Harmony conversion receive identical plaintext;summaryandencrypted_contentonly from the vLLM-bound copy, and removes summary-only items with missing or JSON-null encrypted state so summaries cannot become continuation reasoning;ctx.original_request,ctx.new_input_items, and the complete persisted reasoning item unchanged.No schema, migration, dependency, public wire type, or logging change is introduced. Standalone/in-band compaction input and reasoning newly produced during later gateway rounds remain unchanged and out of scope; the latter remains tracked by #120.
Adversarial audit
Preserved invariants are lossless storage, stable item ordering, no private storage marker on the wire, no opaque-state or summary fallback at the normal vLLM boundary, and identical behavior for blocking and streaming continuations.
The regression coverage exercises:
"\nplaintext continuation", matching Harmony joining while remaining usable by the generic converter;reasoning_text;_agentic_item_kindabsence, error redaction, and zero recorded upstream calls on rejection.The preparation is synchronous and request-owned. It introduces no shared state, cancellation cleanup, or concurrency surface, and it runs only for the initial normal inference round so active-loop reasoning semantics do not drift.
Verification
d8720a7083afb40030385eb874929d72a0fe1f3d, with only the final integration regression file applied:cargo test -p agentic-server-core --test stateful_responses_integration reasoning -- --nocaptureproduced 0 passed / 6 failed. Four replay variants retained incompatible multipart/opaque state, summary-only reasoning remained present, and encrypted-only history reached the mock backend and returned 502 instead of a local 400.068c4549258137873904058045bf796d74aae0bd: the same integration command passed 6/6.cargo test -p agentic-server-core executor::rehydrate -- --nocapturepassed 10/10 focused tests.cargo test --workspace --quiet -- --test-threads=4passed every runnable workspace test; repository-declared ignored tests remained ignored.cargo clippy --workspace --all-targets -- -D warningspassed.cargo fmt --all -- --checkpassed.uvx pre-commit==4.4.0 run --all-filespassed every applicable hook.An earlier eight-thread workspace run encountered an OS
WouldBlockwhile an unchanged MCP proxy test spawned a thread. That test passed in isolation on both this branch and an exact detached upstream-main control, and the complete four-thread workspace rerun passed.