Skip to content

fix(memory): send Qwen thinking controls through SGLang and vLLM chat templates - #348

Open
LeslieWylie wants to merge 4 commits into
MemTensor:mainfrom
LeslieWylie:codex/sglang-vllm-thinking-providers-v2
Open

LeslieWylie wants to merge 4 commits into
MemTensor:mainfrom
LeslieWylie:codex/sglang-vllm-thinking-providers-v2

Conversation

@LeslieWylie

@LeslieWylie LeslieWylie commented Sep 3, 2026 •

Copy link
Copy Markdown

Qwen endpoints served by SGLang and vLLM expect enable_thinking inside chat_template_kwargs. Add explicit sglang and vllm memory providers so those endpoints receive the control in the right place. The generic OpenAI-compatible provider keeps its existing top-level payload and caller-supplied body options.

This branch now includes main's Responses API support. The configuration accepts openai_responses, sglang, and vllm together; Responses requests continue through their own client and receive no automatically added chat-template or thinking fields.

The earlier focused CPU validation passed the thinking and protocol-adapter tests (37 passed). It exercised the merged Memory source and AgentSourceCore TypeScript source with the tests' fetch stubs, covering provider configuration, SGLang/vLLM request bodies, the generic provider and Responses isolation. Those behavioral tests were not rerun for this build follow-up.

On head 736cef1e, both npm -w @memmy/memory run typecheck and npm -w @memmy/memory run build pass through the normal workspace scripts. Each rebuilds AgentSourceCore from this head; the Memory build also completes the viewer, TypeScript output, workspace bridge and CLI assets. The private workspace links resolve to the current source and its newly generated dist, rather than the earlier checkout's compiled output.

This uses the existing third-party dependencies with Node 26.7, TypeScript 6.0.3 and Vite 8.0.14, without installing packages or changing shared dependencies. It validates the affected Memory and AgentSourceCore workspaces, not a clean-install reproduction, the entire monorepo, release packaging or new model-server behavior. The existing protocol-test results retain their original scope.

The normal workspace commands were:

npm -w @memmy/memory run typecheck
npm -w @memmy/memory run build

The earlier live SGLang replay remains historical evidence. No new model request was made. The external-contributor Linux and Windows workflows currently require maintainer approval; these local checks do not establish their outcome.

Replaces #312. AI assistance: Codex helped resolve the merge and run the focused CPU checks.

Route Qwen enable_thinking through chat_template_kwargs only for explicit sglang and vllm providers. Keep the generic OpenAI-compatible request body unchanged so cloud-provider calls retain their existing contract.
Signed-off-by: LeslieWylie <112309453+LeslieWylie@users.noreply.github.com>
@LeslieWylie

LeslieWylie commented Oct 9, 2026 •

Copy link
Copy Markdown
Author

Updated this branch with current main (ee0ed02af8277210e749faad715584ebb526501b) in merge commit 3705dac0, preserving the PR history. The conflict in llm.ts now retains both the dedicated SGLang/vLLM thinking payload helper and upstream's summary input-budget/truncation helpers.

Validation on this head: the existing thinking, agent-region, JSON retry, token-usage, and configuration tests passed; Memory's TypeScript check passed. The dedicated transports still place Qwen's thinking switch in chat_template_kwargs, while the generic OpenAI-compatible transport keeps its existing top-level contract and preserves caller-supplied body fields. Linux installer and runtime CI results for the new head should be assessed separately from the older successful runs.

The new-head Linux CLI installer run and Windows Memory validation run both currently report action_required. No Linux job has started. A maintainer's approval is needed to start these external-contributor workflows; this is separate from a build or test failure.

…orts

Co-authored-by: Codex
Signed-off-by: LeslieWylie <112309453+LeslieWylie@users.noreply.github.com>
@LeslieWylie
LeslieWylie force-pushed the codex/sglang-vllm-thinking-providers-v2 branch from daabc68 to 7e7726d Compare October 14, 2026 19:48
Signed-off-by: LeslieWylie <112309453+LeslieWylie@users.noreply.github.com>
@LeslieWylie LeslieWylie changed the title fix(memory): add dedicated SGLang and vLLM thinking transports fix(memory): send Qwen thinking controls through SGLang and vLLM chat templates Oct 24, 2026
@LeslieWylie

LeslieWylie commented Oct 24, 2026 •

Copy link
Copy Markdown
Author

@codex review

I've kept openai_responses, sglang, and vllm available together, with Qwen's chat_template_kwargs.enable_thinking handling confined to the two dedicated providers. Please review head 736cef1efaf1231fd9a1fb92e4300ce304ac45eb for request-field isolation: generic OpenAI-compatible and Responses calls should retain their existing payloads and caller options. Focused CPU configuration and HTTP-payload checks passed against the current Memory and AgentSourceCore source with fetch stubs; live model calls, the full workspace build and typecheck, and installer validation are still pending.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant