From 16345ab8b0a9e5d6fdc8a237786339b674fc374c Mon Sep 17 00:00:00 2001 From: bitkyc08-arch Date: Thu, 20 Aug 2026 04:31:16 +0900 Subject: [PATCH 1/2] fix(responses): bridge routed tool_search through the Responses passthrough Passthrough forwarded Codex's private {type:"tool_search"} unchanged to third-party /v1/responses gateways, which only understand public function tools. The model never saw a callable tool_search and emitted zero tool_search_call items, so deferred tool discovery silently did nothing on every routed provider. Lower the private tool to a public function on noncanonical forward targets, rewrite tool_choice and replayed history to match, then restore the private tool_search_call lifecycle on the way back so Codex sees what it can execute. Canonical ChatGPT forward is untouched. Carries @Ingwannu's #2040 implementation and tests, with two corrections for findings that were still open on it: History-only replay. A turn may replay tool_search history without re-declaring the tool. The history was lowered either way, but restoration was armed only when a declaration was present, so the client got back a public function_call for what it had issued as a private search call. SSE overflow was not atomic. Both overflow branches cleared routedItemIds along with the pending buffer, so an item already restored to tool_search_call started emitting function_call_arguments frames again and the client saw a mixed private/public lifecycle for one call. Overflow now stops buffering unknown frames without forgetting what was already classified. Closes #1950 --- .../src/content/docs/ja/reference/adapters.md | 9 +- .../src/content/docs/ko/reference/adapters.md | 13 +- .../src/content/docs/reference/adapters.md | 13 +- .../content/docs/zh-cn/reference/adapters.md | 11 +- src/adapters/base.ts | 2 + src/adapters/openai-responses.ts | 10 + src/responses/parser.ts | 12 +- src/responses/tool-search-compat.ts | 301 +++++++++ src/server/responses-tool-search-repair.ts | 211 +++++++ src/server/responses/core.ts | 20 +- structure/04_transports-and-sidecars.md | 8 + tests/openai-responses-passthrough.test.ts | 30 + tests/responses-parser.test.ts | 22 + tests/responses-tool-search-repair.test.ts | 578 ++++++++++++++++++ 14 files changed, 1218 insertions(+), 22 deletions(-) create mode 100644 src/responses/tool-search-compat.ts create mode 100644 src/server/responses-tool-search-repair.ts create mode 100644 tests/responses-tool-search-repair.test.ts diff --git a/docs-site/src/content/docs/ja/reference/adapters.md b/docs-site/src/content/docs/ja/reference/adapters.md index e637c4cc26..54bd07eef8 100644 --- a/docs-site/src/content/docs/ja/reference/adapters.md +++ b/docs-site/src/content/docs/ja/reference/adapters.md @@ -43,8 +43,13 @@ interface ProviderAdapter { ## `openai-responses` -**対象:** OpenAI **Responses API**。**`passthrough: true`** — 元のリクエスト本文をそのまま渡し、レスポンスを **変換せずに** ストリーミングします。 -**認証:** `forward`（呼び出し元ヘッダー中継）または `key`。 +**対象:** OpenAI **Responses API**。**`passthrough: true`** — 通常は元のリクエストとレスポンスをそのまま渡し、ルーティング先ゲートウェイに必要な限定的な互換変換だけを適用します。 +**認証:** canonical OpenAI `forward` は安全な呼び出し元ヘッダー許可リストだけを中継します。非 canonical な `forward` は呼び出し元の authorization を中継せず、設定済みの静的ヘッダーだけを使用します。`key` は設定済み provider key を使用します。 + +非 canonical な Responses ゲートウェイには、Codex のクライアント実行型 `tool_search` +宣言を既存の公開 function tool と衝突しない名前で送り、対応するリクエスト履歴と JSON/SSE +function call をクライアント向けの非公開 `tool_search` ライフサイクルに復元します。 +canonical OpenAI forward はネイティブな非公開型を維持します。 `key` 認証では、[`retryOn429`](/ja/reference/configuration/) もここに適用されます: プリストリームの 429 は、翻訳された `openai-chat` / Anthropic リクエスト経路と同様に、他の処理やフェイルオーバーに diff --git a/docs-site/src/content/docs/ko/reference/adapters.md b/docs-site/src/content/docs/ko/reference/adapters.md index 58c61cd7e4..06eedf2773 100644 --- a/docs-site/src/content/docs/ko/reference/adapters.md +++ b/docs-site/src/content/docs/ko/reference/adapters.md @@ -49,9 +49,16 @@ interface ProviderAdapter { ## `openai-responses` -**대상:** OpenAI **Responses API**. **`passthrough: true`** — 원본 요청 본문을 전달하고 응답을 -**변환하지 않은 채** 스트리밍합니다. -**인증:** `forward`(호출자 헤더 중계) 또는 `key`. +**대상:** OpenAI **Responses API**. **`passthrough: true`** — 일반적으로 원본 요청과 응답을 +그대로 전달하되, 라우팅된 게이트웨이에 필요한 좁은 호환성 변환만 적용합니다. +**인증:** 정규 OpenAI `forward`는 안전한 호출자 헤더 허용 목록만 중계합니다. 비정규 +`forward`는 호출자 authorization을 중계하지 않고 설정된 정적 헤더만 사용하며, `key`는 +설정된 provider 키를 사용합니다. + +비정규 Responses 게이트웨이에는 Codex의 클라이언트 실행형 `tool_search` 선언을 공개 function +도구와 충돌하지 않는 이름으로 전달합니다. 일치하는 요청 기록과 JSON/SSE function call은 +클라이언트용 비공개 `tool_search` 수명 주기로 복원합니다. 정규 OpenAI forward 경로는 +네이티브 비공개 타입을 그대로 유지합니다. `key` 인증에서는 [`retryOn429`](/ko/reference/configuration/)도 여기에 적용됩니다: 사전 스트림 429는 번역된 `openai-chat`/Anthropic 요청 경로와 동일하게 다른 처리나 페일오버보다 먼저 diff --git a/docs-site/src/content/docs/reference/adapters.md b/docs-site/src/content/docs/reference/adapters.md index 8a9bd05c45..0895ad07e8 100644 --- a/docs-site/src/content/docs/reference/adapters.md +++ b/docs-site/src/content/docs/reference/adapters.md @@ -49,9 +49,16 @@ provider — xAI, Kimi, DeepSeek, GLM, Groq, OpenRouter, Ollama (local & cloud), ## `openai-responses` -**Targets:** the OpenAI **Responses API**. **`passthrough: true`** — forwards the raw request body and -streams the response back **untranslated**. -**Auth:** `forward` (relay the caller's headers) or `key`. +**Targets:** the OpenAI **Responses API**. **`passthrough: true`** — normally forwards the raw request +body and response, with narrow compatibility rewrites for routed gateways. +**Auth:** canonical OpenAI `forward` relays only the safe caller-header allowlist; noncanonical +`forward` uses configured static headers without relaying caller authorization; `key` uses the +configured provider key. + +Noncanonical Responses gateways receive Codex's client-executed `tool_search` declaration as a +collision-safe public function tool. Matching request history and JSON/SSE function calls are +translated back to the private `tool_search` lifecycle for the client. Canonical OpenAI forward +keeps the native private type unchanged. For `key` auth, [`retryOn429`](/reference/configuration/) applies here too: a pre-stream 429 waits and replays the identical request on the same key before any other handling, exactly like diff --git a/docs-site/src/content/docs/zh-cn/reference/adapters.md b/docs-site/src/content/docs/zh-cn/reference/adapters.md index 07dbfcf441..5786952810 100644 --- a/docs-site/src/content/docs/zh-cn/reference/adapters.md +++ b/docs-site/src/content/docs/zh-cn/reference/adapters.md @@ -45,9 +45,14 @@ interface ProviderAdapter { ## `openai-responses` -**目标：** OpenAI **Responses API**。**`passthrough: true`** —— 转发原始请求 body，并把响应 -**不经转换**地流式传回。 -**认证：** `forward`（转发调用方 header）或 `key`。 +**目标：** OpenAI **Responses API**。**`passthrough: true`** —— 通常原样转发请求与响应，仅对 +路由网关应用范围有限的兼容性转换。 +**认证：** 规范 OpenAI `forward` 只转发安全的调用方 header allowlist；非规范 `forward` 不会 +转发调用方 authorization，只使用已配置的静态 header；`key` 使用已配置的 provider key。 + +对于非规范 Responses 网关，Codex 的客户端执行型 `tool_search` 声明会作为公共 function tool +以不与现有 function 名称冲突的方式发送；匹配的请求历史和 JSON/SSE function call 会恢复为 +客户端私有的 `tool_search` 生命周期。规范 OpenAI forward 路径仍保持原生私有类型不变。 使用 `key` 认证时，[`retryOn429`](/zh-cn/reference/configuration/) 同样适用：流开始前的 429 会等待并先于其他处理或故障转移，在相同 key 上重放完全相同请求，与翻译后的 diff --git a/src/adapters/base.ts b/src/adapters/base.ts index 7d3a8be07b..faeea0e959 100644 --- a/src/adapters/base.ts +++ b/src/adapters/base.ts @@ -70,6 +70,8 @@ export interface AdapterRequest { body: string; /** Custom-tool names actually lowered to upstream function calls while building this request. */ convertedRoutedCustomToolNames?: ReadonlySet; + /** Client tool-search names actually lowered to upstream function calls for this request. */ + convertedRoutedToolSearchNames?: ReadonlySet; /** Releases observation of a serialized request body after its final fetch attempt settles. */ releaseBodyObservation?: () => void; /** Exact reasoning parameter emitted by the adapter, for request-log diagnostics only. */ diff --git a/src/adapters/openai-responses.ts b/src/adapters/openai-responses.ts index a8bbab8bfd..7bc0bf9cec 100644 --- a/src/adapters/openai-responses.ts +++ b/src/adapters/openai-responses.ts @@ -11,6 +11,7 @@ import { OCX_REASONING_PREFIX } from "../responses/reasoning-envelope"; import { modelRecordValue } from "../reasoning-effort"; import type { TranslatorBudget } from "../lib/translator-budget"; import { rewriteRoutedCustomToolsForUpstream } from "../responses/custom-tool-compat"; +import { rewriteRoutedToolSearchForUpstream } from "../responses/tool-search-compat"; import { openaiResponsesUrl } from "./openai-responses-url"; import { createAdapterTierMetadata, @@ -1468,6 +1469,7 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig): const forward = provider.authMode === "forward"; let convertedRoutedCustomToolNames: Set | undefined; + let convertedRoutedToolSearchNames: Set | undefined; const unexpandedMiss = !!parsed.previousResponseId && parsed._previousResponseInputExpanded !== true; let outBody = stripPreviousResponseId( parsed._rawBody, @@ -1522,6 +1524,13 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig): outBody = rewritten.body; convertedRoutedCustomToolNames = rewritten.names; } + if (!isCanonicalOpenAiForwardProvider(provider)) { + // Run after custom-tool lowering so the search compatibility layer can choose a + // collision-free public function name against the final routed function catalog. + const rewritten = rewriteRoutedToolSearchForUpstream(outBody); + outBody = rewritten.body; + convertedRoutedToolSearchNames = rewritten.names; + } const sanitizedBody = normalizeToolSchemas(stripSparkCompatibility(stripUnsupportedReasoningParams(stripItemIdsWhenUnstored(stripInvalidItemIds(stripUnsupportedHostedTools(sanitizeReasoningInputContent(scrubOcxCompactionItems(outBody), { preserveRawReasoningContent: provider.preserveResponsesReasoningContent === true }))))))); const finalBody = stripDisabledReasoningSummaries( normalizeConfiguredReasoningSummaryDelivery(sanitizedBody, provider, parsed.modelId), @@ -1549,6 +1558,7 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig): body, releaseBodyObservation, ...(convertedRoutedCustomToolNames ? { convertedRoutedCustomToolNames } : {}), + ...(convertedRoutedToolSearchNames ? { convertedRoutedToolSearchNames } : {}), ...(tierLog ? { tierLog } : {}), }; }, diff --git a/src/responses/parser.ts b/src/responses/parser.ts index ca9f425f29..ed7e83588c 100644 --- a/src/responses/parser.ts +++ b/src/responses/parser.ts @@ -20,6 +20,7 @@ import { previousResponseReplayPrefixLength } from "./state"; import { decodeReasoningEnvelope } from "./reasoning-envelope"; import { extractHostedWebSearch, WEB_SEARCH_TOOL_NAME } from "../web-search/synthetic-tool"; import { extractHostedImageGeneration, IMAGE_GEN_TOOL_NAME } from "../images/synthetic-tool"; +import { toolSearchDescription, toolSearchParameters } from "./tool-search-compat"; function isObj(v: unknown): v is Record { return typeof v === "object" && v !== null && !Array.isArray(v); @@ -214,15 +215,8 @@ function buildTools(tools: unknown[] | undefined): OcxTool[] | undefined { // Expose as a function so chat models can call it; the bridge relays it as a tool_search_call. out.push({ name: "tool_search", - description: (t.description as string) ?? "Search for additional tools to load for the next turn.", - parameters: (isObj(t.parameters) ? t.parameters : { - type: "object", - properties: { - query: { type: "string", description: "Search query for tools to load." }, - limit: { type: "number", description: "Maximum number of tools to return." }, - }, - required: ["query"], - }) as Record, + description: toolSearchDescription(t), + parameters: normalizeParameters(toolSearchParameters(t)), toolSearch: true, }); } diff --git a/src/responses/tool-search-compat.ts b/src/responses/tool-search-compat.ts new file mode 100644 index 0000000000..38eb6deb8b --- /dev/null +++ b/src/responses/tool-search-compat.ts @@ -0,0 +1,301 @@ +export const TOOL_SEARCH_FUNCTION_NAME = "tool_search"; +export const TOOL_SEARCH_DEFAULT_DESCRIPTION = "Search for additional tools to load for the next turn."; +const TOOL_SEARCH_WIRE_ALIAS_PREFIX = "opencodex_tool_search"; + +function isPlainObject(value: unknown): value is Record { + return !!value && typeof value === "object" && !Array.isArray(value); +} + +export function toolSearchDescription(tool: unknown): string { + return isPlainObject(tool) && typeof tool.description === "string" + ? tool.description + : TOOL_SEARCH_DEFAULT_DESCRIPTION; +} + +export function toolSearchParameters(tool: unknown): Record { + if (isPlainObject(tool) && isPlainObject(tool.parameters)) return tool.parameters; + // Fresh per parse/build: downstream normalizers are allowed to clone or extend schemas, and a + // shared mutable default would couple otherwise unrelated requests. + return { + type: "object", + properties: { + query: { type: "string", description: "Search query for tools to load." }, + limit: { type: "number", description: "Maximum number of tools to return." }, + }, + required: ["query"], + }; +} + +function collectToolDeclarationNames(tools: unknown[], names: Set, namespace?: string): void { + for (const tool of tools) { + if (!isPlainObject(tool) || tool.type === "tool_search") continue; + if (tool.type === "function" && isPlainObject(tool.function) && typeof tool.function.name === "string") { + names.add(tool.function.name); + } + if (typeof tool.name === "string") { + names.add(tool.name); + if (namespace) names.add(`${namespace}__${tool.name}`); + } + if (tool.type === "namespace" && Array.isArray(tool.tools)) { + collectToolDeclarationNames(tool.tools, names, typeof tool.name === "string" ? tool.name : undefined); + } + } +} + +function declaredToolNames(body: Record): Set { + const names = new Set(); + if (Array.isArray(body.tools)) collectToolDeclarationNames(body.tools, names); + if (Array.isArray(body.input)) { + for (const item of body.input) { + if (isPlainObject(item) && item.type === "additional_tools" && Array.isArray(item.tools)) { + collectToolDeclarationNames(item.tools, names); + } + } + } + return names; +} + +function hasToolSearchDeclaration(tools: unknown[]): boolean { + return tools.some(tool => isPlainObject(tool) && tool.type === "tool_search"); +} + +function chooseToolSearchWireName(usedNames: ReadonlySet): string { + if (!usedNames.has(TOOL_SEARCH_FUNCTION_NAME)) return TOOL_SEARCH_FUNCTION_NAME; + if (!usedNames.has(TOOL_SEARCH_WIRE_ALIAS_PREFIX)) return TOOL_SEARCH_WIRE_ALIAS_PREFIX; + for (let suffix = 2; ; suffix++) { + const candidate = `${TOOL_SEARCH_WIRE_ALIAS_PREFIX}_${suffix}`; + if (!usedNames.has(candidate)) return candidate; + } +} + +function rewriteToolList(tools: unknown[], wireName: string): { tools: unknown[]; changed: boolean } { + let changed = false; + const rewritten = tools.map(tool => { + if (!isPlainObject(tool) || tool.type !== "tool_search") return tool; + const { + execution: _execution, + defer_loading: _deferLoading, + ...rest + } = tool; + changed = true; + return { + ...rest, + type: "function", + name: wireName, + description: toolSearchDescription(tool), + parameters: toolSearchParameters(tool), + }; + }); + return changed ? { tools: rewritten, changed: true } : { tools, changed: false }; +} + +function rewriteToolChoice(choice: unknown, wireName: string): unknown { + if (!isPlainObject(choice)) return choice; + if (choice.type === "tool_search") { + return { type: "function", name: wireName }; + } + if (choice.type !== "allowed_tools" || !Array.isArray(choice.tools)) return choice; + let changed = false; + const tools = choice.tools.map(tool => { + if (!isPlainObject(tool) || tool.type !== "tool_search") return tool; + changed = true; + return { type: "function", name: wireName }; + }); + return changed ? { ...choice, tools } : choice; +} + +function toolChoiceAllowsPrivateSearch(choice: unknown): boolean { + if (choice === undefined || choice === null || choice === "auto" || choice === "required") return true; + if (choice === "none") return false; + if (!isPlainObject(choice)) return true; + if (choice.type === "tool_search") return true; + if (choice.type === "allowed_tools" && Array.isArray(choice.tools)) { + return choice.tools.some(tool => isPlainObject(tool) && tool.type === "tool_search"); + } + // A forced ordinary function named `tool_search` is distinct from the private tool kind. + return false; +} + +function upstreamToolSearchItemId(id: unknown): unknown { + if (typeof id !== "string") return id; + return id.startsWith("tsc_") ? `fc_${id.slice(4)}` : id; +} + +function toolSearchArgumentsText(value: unknown): string { + if (typeof value === "string") return value; + return JSON.stringify(isPlainObject(value) ? value : {}); +} + +function toolSearchOutputText(value: Record): string { + const payload: Record = { + tools: Array.isArray(value.tools) ? value.tools : [], + }; + if (typeof value.status === "string") payload.status = value.status; + return JSON.stringify(payload); +} + +function rewriteHistoryItem(item: Record, wireName: string): Record { + if (item.type === "tool_search_call") { + const { + execution: _execution, + arguments: argumentsValue, + id, + ...rest + } = item; + return { + ...rest, + type: "function_call", + ...(id === undefined ? {} : { id: upstreamToolSearchItemId(id) }), + name: wireName, + arguments: toolSearchArgumentsText(argumentsValue), + }; + } + if (item.type === "tool_search_output") { + const { + execution: _execution, + id: _id, + tools: _tools, + status: _status, + ...rest + } = item; + return { + ...rest, + type: "function_call_output", + output: toolSearchOutputText(item), + }; + } + return item; +} + +/** + * Third-party Responses gateways generally implement the public function-tool schema, not Codex's + * private client-executed `tool_search` declaration. Lower only request catalogs sent to a + * noncanonical upstream; the caller-facing response is restored separately. + */ +export function rewriteRoutedToolSearchForUpstream(body: unknown): { + body: unknown; + names: Set; +} { + const names = new Set(); + if (!isPlainObject(body)) return { body, names }; + + const topLevelSearch = Array.isArray(body.tools) && hasToolSearchDeclaration(body.tools); + const inputItems = Array.isArray(body.input) ? body.input : []; + const additionalSearch = inputItems.some(item => + isPlainObject(item) + && item.type === "additional_tools" + && Array.isArray(item.tools) + && hasToolSearchDeclaration(item.tools)); + const historySearch = inputItems.some(item => + isPlainObject(item) + && (item.type === "tool_search_call" || item.type === "tool_search_output")); + if (!topLevelSearch && !additionalSearch && !historySearch) return { body, names }; + + const wireName = chooseToolSearchWireName(declaredToolNames(body)); + const declarationChanged = topLevelSearch || additionalSearch; + + let tools = body.tools; + if (Array.isArray(tools)) { + const result = rewriteToolList(tools, wireName); + tools = result.tools; + } + + let input = body.input; + let historyLowered = false; + if (Array.isArray(input)) { + input = input.map(item => { + if (!isPlainObject(item)) return item; + if (item.type === "additional_tools" && Array.isArray(item.tools)) { + const result = rewriteToolList(item.tools, wireName); + return result.changed ? { ...item, tools: result.tools } : item; + } + const rewritten = rewriteHistoryItem(item, wireName); + if (rewritten !== item) historyLowered = true; + return rewritten; + }); + } + + // A turn can replay `tool_search_call` history WITHOUT re-declaring the tool — Codex normally + // sends the declaration, but a history-only body is legal. The history is lowered to + // `function_call` either way, so restoration has to be armed on that too: leaving `names` + // empty there would hand the client a public `function_call` for what it issued as a private + // search call, and the round trip would silently stop matching. + if ((declarationChanged || historyLowered) && toolChoiceAllowsPrivateSearch(body.tool_choice)) { + names.add(wireName); + } + const toolChoice = declarationChanged ? rewriteToolChoice(body.tool_choice, wireName) : body.tool_choice; + return { + body: { + ...body, + ...(tools !== body.tools ? { tools } : {}), + ...(input !== body.input ? { input } : {}), + ...(toolChoice !== body.tool_choice ? { tool_choice: toolChoice } : {}), + }, + names, + }; +} + +export function toolSearchItemId(id: unknown): unknown { + if (typeof id !== "string") return id; + return id.startsWith("fc_") ? `tsc_${id.slice(3)}` : id; +} + +function toolSearchArguments(value: unknown): Record { + if (isPlainObject(value)) return value; + if (typeof value !== "string" || value.length === 0) return {}; + try { + const parsed: unknown = JSON.parse(value); + return isPlainObject(parsed) ? parsed : {}; + } catch { + return {}; + } +} + +export function restoreRoutedToolSearchCalls( + value: unknown, + names: ReadonlySet, +): { value: unknown; changed: boolean } { + if (Array.isArray(value)) { + let changed = false; + const restored = value.map(entry => { + const result = restoreRoutedToolSearchCalls(entry, names); + changed ||= result.changed; + return result.value; + }); + return changed ? { value: restored, changed: true } : { value, changed: false }; + } + if (!isPlainObject(value)) return { value, changed: false }; + + let changed = false; + const restored: Record = {}; + for (const [key, entry] of Object.entries(value)) { + const result = restoreRoutedToolSearchCalls(entry, names); + restored[key] = result.value; + changed ||= result.changed; + } + + if (value.type === "function_call" && typeof value.name === "string" && names.has(value.name)) { + restored.type = "tool_search_call"; + restored.id = toolSearchItemId(value.id); + restored.execution = "client"; + restored.arguments = toolSearchArguments(value.arguments); + delete restored.name; + changed = true; + } + return changed ? { value: restored, changed: true } : { value, changed: false }; +} + +export function restoreRoutedToolSearchCallsInJson( + text: string, + names: ReadonlySet, +): string { + if (names.size === 0) return text; + let payload: unknown; + try { + payload = JSON.parse(text); + } catch { + return text; + } + const restored = restoreRoutedToolSearchCalls(payload, names); + return restored.changed ? JSON.stringify(restored.value) : text; +} diff --git a/src/server/responses-tool-search-repair.ts b/src/server/responses-tool-search-repair.ts new file mode 100644 index 0000000000..8aa5c48130 --- /dev/null +++ b/src/server/responses-tool-search-repair.ts @@ -0,0 +1,211 @@ +import { + isTranslatorBudgetExceededError, + type TranslatorBudget, +} from "../lib/translator-budget"; +import { + restoreRoutedToolSearchCalls, +} from "../responses/tool-search-compat"; +import { + replaceSseDataPayload, + sseDataPayload, + type SseBlockRewrite, +} from "./sse-payload-rewrite"; + +function isPlainObject(value: unknown): value is Record { + return !!value && typeof value === "object" && !Array.isArray(value); +} + +type PendingArgumentBlock = { + block: string; + itemId?: string; + outputIndex?: number; + retainedBytes: number; +}; + +const MAX_PENDING_ARGUMENT_FRAMES = 256; +const MAX_PENDING_ARGUMENT_BYTES = 1024 * 1024; + +/** + * Public Responses gateways stream a lowered search as a normal function lifecycle. Codex expects + * only `tool_search_call` items, so classify each item before dropping its function-argument + * frames. Unknown early argument frames stay bounded until their item arrives. + */ +export function createRoutedToolSearchRestoreBlockRewrite( + names: ReadonlySet, + budget?: TranslatorBudget, +): SseBlockRewrite { + const routedItemIds = new Set(); + const ordinaryItemIds = new Set(); + let pendingArguments: PendingArgumentBlock[] = []; + let pendingArgumentBytes = 0; + let passthrough = false; + let disposed = false; + + const releaseAll = (): void => { + if (disposed) return; + disposed = true; + if (pendingArgumentBytes > 0) { + budget?.releaseRetained(pendingArgumentBytes, { kind: "retained_collectors" }); + } + pendingArguments = []; + pendingArgumentBytes = 0; + routedItemIds.clear(); + ordinaryItemIds.clear(); + }; + + const retainPending = ( + block: string, + itemId: string | undefined, + outputIndex: number | undefined, + ): readonly string[] | null => { + const retainedBytes = Buffer.byteLength(block, "utf8"); + const overflow = pendingArguments.length >= MAX_PENDING_ARGUMENT_FRAMES + || pendingArgumentBytes + retainedBytes > MAX_PENDING_ARGUMENT_BYTES; + if (overflow) { + const flushed = [...pendingArguments.map(pending => pending.block), block]; + if (pendingArgumentBytes > 0) { + budget?.releaseRetained(pendingArgumentBytes, { kind: "retained_collectors" }); + } + pendingArguments = []; + pendingArgumentBytes = 0; + passthrough = true; + // Deliberately KEEP routedItemIds. Overflow means we stop buffering UNKNOWN frames, not + // that we forget what we already classified: an item restored to `tool_search_call` + // upstream of here would otherwise start emitting `function_call_arguments.*` again and + // the client would see a mixed private/public lifecycle for one call. + ordinaryItemIds.clear(); + return flushed; + } + if (retainedBytes > 0) { + try { + budget?.chargeRetained(retainedBytes, { kind: "retained_collectors" }); + } catch (error) { + if (!isTranslatorBudgetExceededError(error)) throw error; + const flushed = [...pendingArguments.map(pending => pending.block), block]; + if (pendingArgumentBytes > 0) { + budget?.releaseRetained(pendingArgumentBytes, { kind: "retained_collectors" }); + } + pendingArguments = []; + pendingArgumentBytes = 0; + passthrough = true; + // Same reasoning as the frame/byte overflow above: an already-restored routed item + // must keep its frames suppressed even once buffering stops. + ordinaryItemIds.clear(); + return flushed; + } + } + pendingArguments.push({ block, itemId, outputIndex, retainedBytes }); + pendingArgumentBytes += retainedBytes; + return null; + }; + + const takePending = ( + itemId: string | undefined, + outputIndex: number | undefined, + ): string[] => { + const matched: PendingArgumentBlock[] = []; + const remaining: PendingArgumentBlock[] = []; + for (const pending of pendingArguments) { + const matches = pending.itemId !== undefined + ? itemId !== undefined && pending.itemId === itemId + : outputIndex !== undefined && pending.outputIndex === outputIndex; + (matches ? matched : remaining).push(pending); + } + pendingArguments = remaining; + const retainedBytes = matched.reduce((total, pending) => total + pending.retainedBytes, 0); + if (retainedBytes > 0) { + budget?.releaseRetained(retainedBytes, { kind: "retained_collectors" }); + pendingArgumentBytes = Math.max(0, pendingArgumentBytes - retainedBytes); + } + return matched.map(pending => { + if (pending.itemId !== undefined || itemId === undefined) return pending.block; + const payload = sseDataPayload(pending.block); + if (payload === null) return pending.block; + try { + const parsed: unknown = JSON.parse(payload); + return isPlainObject(parsed) + ? replaceSseDataPayload(pending.block, JSON.stringify({ ...parsed, item_id: itemId })) + : pending.block; + } catch { + return pending.block; + } + }); + }; + + const rewrite: SseBlockRewrite = (block: string): readonly string[] => { + if (disposed) return [block]; + const payload = sseDataPayload(block); + if (payload === null || payload === "[DONE]") return [block]; + let parsed: unknown; + try { + parsed = JSON.parse(payload); + } catch { + return [block]; + } + if (!isPlainObject(parsed)) return [block]; + + const type = typeof parsed.type === "string" ? parsed.type : ""; + // After overflow we stop BUFFERING unknown frames, but an item already restored to + // `tool_search_call` must keep its public argument frames suppressed — otherwise the client + // receives a private item followed by `function_call_arguments.*` for the same id, which is + // exactly the mixed lifecycle this rewrite exists to prevent. Everything else passes through. + if (passthrough) { + const passthroughItemId = typeof parsed.item_id === "string" ? parsed.item_id : undefined; + const isArgumentEvent = type === "response.function_call_arguments.delta" + || type === "response.function_call_arguments.done"; + if (isArgumentEvent && passthroughItemId && routedItemIds.has(passthroughItemId)) return []; + return [block]; + } + const outputIndex = typeof parsed.output_index === "number" + && Number.isInteger(parsed.output_index) + && parsed.output_index >= 0 + ? parsed.output_index + : undefined; + if ( + (type === "response.output_item.added" || type === "response.output_item.done") + && isPlainObject(parsed.item) + && parsed.item.type === "function_call" + && typeof parsed.item.name === "string" + ) { + const itemId = typeof parsed.item.id === "string" ? parsed.item.id : undefined; + const routed = names.has(parsed.item.name); + if (itemId) { + if (routed) { + routedItemIds.add(itemId); + ordinaryItemIds.delete(itemId); + } else { + ordinaryItemIds.add(itemId); + routedItemIds.delete(itemId); + } + } + const pending = takePending(itemId, outputIndex); + const restored = routed ? restoreRoutedToolSearchCalls(parsed, names) : { value: parsed, changed: false }; + const restoredBlock = restored.changed + ? replaceSseDataPayload(block, JSON.stringify(restored.value)) + : block; + if (type === "response.output_item.done" && itemId) { + routedItemIds.delete(itemId); + ordinaryItemIds.delete(itemId); + } + return routed ? [restoredBlock] : [...pending, restoredBlock]; + } + + const itemId = typeof parsed.item_id === "string" ? parsed.item_id : undefined; + const argumentEvent = type === "response.function_call_arguments.delta" + || type === "response.function_call_arguments.done"; + if (argumentEvent && (!itemId || (!routedItemIds.has(itemId) && !ordinaryItemIds.has(itemId)))) { + return retainPending(block, itemId, outputIndex) ?? []; + } + if (argumentEvent && itemId && routedItemIds.has(itemId)) return []; + + const terminal = type === "response.completed" || type === "response.failed" || type === "response.incomplete"; + if (!terminal) return [block]; + const restored = restoreRoutedToolSearchCalls(parsed, names); + releaseAll(); + return restored.changed + ? [replaceSseDataPayload(block, JSON.stringify(restored.value))] + : [block]; + }; + rewrite.dispose = releaseAll; + return rewrite; +} diff --git a/src/server/responses/core.ts b/src/server/responses/core.ts index 9e2813d0b5..67f6c07ceb 100644 --- a/src/server/responses/core.ts +++ b/src/server/responses/core.ts @@ -267,6 +267,8 @@ import { } from "../sse-payload-rewrite"; import { restoreRoutedCustomCallsInJson } from "../../responses/custom-tool-compat"; import { createRoutedCustomToolRestoreBlockRewrite } from "../responses-custom-tool-repair"; +import { restoreRoutedToolSearchCallsInJson } from "../../responses/tool-search-compat"; +import { createRoutedToolSearchRestoreBlockRewrite } from "../responses-tool-search-repair"; import { collectDeclaredWireToolNames, createUndeclaredToolCallGuardBlockRewrite, @@ -2399,6 +2401,7 @@ async function handleResponsesInner( ? new Map() : imageGenToolCallAliases(toolBridgeMaps.toolNsMap, parsed._rawBody, translatorBudget); const routedCustomToolNames = new Set(); + const routedToolSearchNames = new Set(); // Local continuation cache for the ChatGPT passthrough. Codex WS turns chain with // previous_response_id, ocx converts them to internal HTTP requests, and the ChatGPT Codex // REST backend rejects the parameter — the adapter strips it in forward mode, so the ONLY @@ -2432,6 +2435,12 @@ async function handleResponsesInner( if (toolBridgeMaps.freeformToolNames.has(name)) routedCustomToolNames.add(name); } } + for (const name of request.convertedRoutedToolSearchNames ?? []) { + // The adapter already keeps this set empty when tool_choice forbids the private search. + // Its wire name may be collision-aliased, so comparing it to the caller-facing name here + // would incorrectly disable restoration for the exact ambiguous-name case the alias fixes. + routedToolSearchNames.add(name); + } // #1700: the bridged paths refuse a call to a tool the request never declared // (`declaredToolNames`, src/bridge.ts). The passthrough had no equivalent, so a routed // provider's top-level `apply_patch` — which under Codex code mode exists only as a nested @@ -2894,6 +2903,9 @@ async function handleResponsesInner( routedCustomToolNames.size > 0 ? createRoutedCustomToolRestoreBlockRewrite(routedCustomToolNames, translatorBudget) : undefined, + routedToolSearchNames.size > 0 + ? createRoutedToolSearchRestoreBlockRewrite(routedToolSearchNames, translatorBudget) + : undefined, githubCopilotRepairEnabled ? createGithubCopilotResponsesBlockRewrite(translatorBudget) : undefined, @@ -3091,9 +3103,13 @@ async function handleResponsesInner( restoreImageGenCallsInJson(text, imageGenCallAliases), routedCustomToolNames, ); + const restoredToolSearch = restoreRoutedToolSearchCallsInJson( + restored, + routedToolSearchNames, + ); const repaired = hasResponsesSnapshotRepair(route.provider.responsesSnapshotRepair) - ? repairResponsesSnapshotJson(restored, outboundRequestBody) - : restored; + ? repairResponsesSnapshotJson(restoredToolSearch, outboundRequestBody) + : restoredToolSearch; const modelRewritten = parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId ? rewriteResponsesModelJson(backfillResponsesFieldsJson(repaired), parsed._responseModelId) : backfillResponsesFieldsJson(repaired); diff --git a/structure/04_transports-and-sidecars.md b/structure/04_transports-and-sidecars.md index bdb0d15e0c..708140e828 100644 --- a/structure/04_transports-and-sidecars.md +++ b/structure/04_transports-and-sidecars.md @@ -31,6 +31,14 @@ Responses-compatible streaming output. - 다른 대안 대신 이 방식을 선택한 이유: Model guidance is not an enforcement boundary, while automatic translation would invent executable caller intent and arguments after generation. - 장점, 단점 및 영향: Streaming and non-streaming routed responses now fail closed with an actionable provider-contract error; providers that emit aliases they never advertised must correct their adapter mapping instead of relying on client abort behavior. +[Decision Log] +- 목적과 의도: Keep Codex client-side deferred tool discovery usable through third-party Responses-compatible gateways that implement public function tools but reject the private `tool_search` declaration. +- 기존 구현 및 제약 조건: The chat translation path already exposed search as a function and bridged its call back to `tool_search_call`; passthrough only promoted definitions returned by an earlier search, so it could not initiate discovery on a strict third-party Responses endpoint. +- 검토한 주요 대안: Require every gateway to implement Codex-private tool types; route affected models through `openai-chat`; lower the declaration only; lower the noncanonical request and restore both JSON and SSE response lifecycles. +- 선택한 방식: On noncanonical Responses passthrough only, lower an actually declared `tool_search` to a collision-free public function name, translate its replayed call/output history to public function pairs, record only caller-authorized request-local conversions, and restore matching JSON/SSE calls to client `tool_search_call` items. Canonical OpenAI forward remains byte-shape native. +- 다른 대안 대신 이 방식을 선택한 이유: Provider-specific workarounds fragment the contract, while unconditional restoration could turn an untrusted ordinary function call into a privileged client discovery action. +- 장점, 단점 및 영향: Strict third-party Responses gateways can start and continue deferred discovery without changing native ChatGPT behavior; ordinary same-named functions remain distinct, and the proxy performs a capped SSE lifecycle rewrite only when the request actually required compatibility translation. + The option-aware `openai` provider uses `openai-responses` with `authMode: "forward"`. Pool mode resolves main plus added accounts through affinity/quota/cooldown ownership; Direct forwards only the allowed Codex/OpenAI auth/session headers from the current request and short-circuits pool diff --git a/tests/openai-responses-passthrough.test.ts b/tests/openai-responses-passthrough.test.ts index c2e9d38a71..ed5c484d1a 100644 --- a/tests/openai-responses-passthrough.test.ts +++ b/tests/openai-responses-passthrough.test.ts @@ -346,6 +346,34 @@ describe("OpenAI Responses passthrough sanitization", () => { .not.toHaveProperty("defer_loading"); expect(namespace?.tools?.find(tool => tool.name === "deferred_read")) .not.toHaveProperty("defer_loading"); + expect(body.tools.find(tool => tool.name === "tool_search")).toMatchObject({ + type: "function", + name: "tool_search", + description: "Search deferred tools", + }); + }); + + test("noncanonical forward passthrough also lowers tool_search without caller credential relay", () => { + const adapter = createResponsesPassthroughAdapter({ + adapter: "openai-responses", + baseUrl: "https://provider.example/v1", + authMode: "forward", + headers: { authorization: "Bearer provider-static" }, + }); + const request = adapter.buildRequest({ + modelId: deferredToolBody.model, + context: { messages: [] }, + stream: true, + options: {}, + _rawBody: deferredToolBody, + }, { headers: new Headers({ authorization: "Bearer caller-secret" }) }); + const body = JSON.parse(request.body) as { tools: Array> }; + + expect(body.tools.find(tool => tool.name === "tool_search")).toMatchObject({ + type: "function", + name: "tool_search", + }); + expect(request.headers.authorization).toBe("Bearer provider-static"); }); test("canonical forward passthrough leaves tool-search loading to the native backend", () => { @@ -397,6 +425,8 @@ describe("OpenAI Responses passthrough sanitization", () => { "declared_deferred_read", "deferred_read", ]); + expect(additionalTools?.find(tool => tool.name === "tool_search")) + .toMatchObject({ type: "function", name: "tool_search" }); }); test("normalizes top-level function schemas in the serialized raw body (#745)", () => { diff --git a/tests/responses-parser.test.ts b/tests/responses-parser.test.ts index 1f5f991e14..f236a6f2e5 100644 --- a/tests/responses-parser.test.ts +++ b/tests/responses-parser.test.ts @@ -32,6 +32,28 @@ describe("Responses parser", () => { ]); }); + test("normalizes tool_search parameter schemas to the same object-root contract", () => { + const parsed = parseRequest({ + model: "test-model", + input: "find a tool", + tools: [{ + type: "tool_search", + parameters: { properties: { query: { type: "string" } }, required: ["query"] }, + }], + }); + + expect(parsed.context.tools).toEqual([{ + name: "tool_search", + description: "Search for additional tools to load for the next turn.", + parameters: { + type: "object", + properties: { query: { type: "string" } }, + required: ["query"], + }, + toolSearch: true, + }]); + }); + test("unwraps Chat-shaped function tools while retaining flat function tools", () => { const parameters = { type: "object", diff --git a/tests/responses-tool-search-repair.test.ts b/tests/responses-tool-search-repair.test.ts new file mode 100644 index 0000000000..c208d3c1d0 --- /dev/null +++ b/tests/responses-tool-search-repair.test.ts @@ -0,0 +1,578 @@ +import { describe, expect, test } from "bun:test"; +import { + restoreRoutedToolSearchCallsInJson, + rewriteRoutedToolSearchForUpstream, +} from "../src/responses/tool-search-compat"; +import { createRoutedToolSearchRestoreBlockRewrite } from "../src/server/responses-tool-search-repair"; +import { handleResponses } from "../src/server/responses"; +import type { OcxConfig } from "../src/types"; +import { createTestTranslatorBudget } from "./helpers/translator-budget"; + +function frame(event: string, payload: Record): string { + return `event: ${event}\ndata: ${JSON.stringify({ type: event, ...payload })}`; +} + +function dataPayload(block: string): Record { + const line = block.split(/\r?\n/).find(entry => entry.startsWith("data:")); + if (!line) throw new Error("missing SSE data line"); + return JSON.parse(line.slice(5).trim()) as Record; +} + +const searchTool = { + type: "tool_search", + execution: "client", + description: "Search deferred tools", + parameters: { + type: "object", + properties: { query: { type: "string" } }, + required: ["query"], + additionalProperties: false, + }, +}; + +function fixtureConfig(): OcxConfig { + return { + port: 0, + defaultProvider: "fixture", + providers: { + fixture: { + adapter: "openai-responses", + baseUrl: "https://fixture.test/v1", + authMode: "key", + apiKey: "fixture-key", + }, + }, + } as OcxConfig; +} + +describe("routed Responses tool-search compatibility", () => { + test("lowers top-level and Responses Lite declarations plus tool choice without mutating input", () => { + const raw = { + model: "deepseek-v4-flash", + input: [{ type: "additional_tools", tools: [searchTool] }], + tools: [searchTool, { type: "function", name: "ordinary", parameters: { type: "object" } }], + tool_choice: { + type: "allowed_tools", + mode: "required", + tools: [{ type: "tool_search" }, { type: "function", name: "ordinary" }], + }, + }; + + const rewritten = rewriteRoutedToolSearchForUpstream(raw); + const body = rewritten.body as { + tools: Array>; + input: Array<{ tools: Array> }>; + tool_choice: { mode: string; tools: Array> }; + }; + + expect(rewritten.names).toEqual(new Set(["tool_search"])); + expect(rewritten.body).not.toBe(raw); + expect(raw.tools[0]).toHaveProperty("type", "tool_search"); + expect(body.tools[0]).toEqual({ + type: "function", + name: "tool_search", + description: "Search deferred tools", + parameters: searchTool.parameters, + }); + expect(body.input[0]?.tools[0]).toMatchObject({ type: "function", name: "tool_search" }); + expect(body.tool_choice.mode).toBe("required"); + expect(body.tool_choice.tools[0]).toEqual({ type: "function", name: "tool_search" }); + expect(body.tools[1]).toEqual(raw.tools[1]); + }); + + test("does not mark an ordinary same-named function for privileged restoration", () => { + const raw = { + tools: [{ type: "function", name: "tool_search", parameters: { type: "object" } }], + }; + const rewritten = rewriteRoutedToolSearchForUpstream(raw); + + expect(rewritten.body).toBe(raw); + expect(rewritten.names).toEqual(new Set()); + }); + + test("aliases a private search away from an ordinary same-named function", () => { + const raw = { + tools: [ + searchTool, + { type: "function", name: "tool_search", parameters: { type: "object" } }, + ], + tool_choice: { + type: "allowed_tools", + mode: "required", + tools: [ + { type: "tool_search" }, + { type: "function", name: "tool_search" }, + ], + }, + }; + const rewritten = rewriteRoutedToolSearchForUpstream(raw); + const body = rewritten.body as { + tools: Array>; + tool_choice: { tools: Array> }; + }; + + expect(rewritten.names).toEqual(new Set(["opencodex_tool_search"])); + expect(body.tools[0]).toMatchObject({ type: "function", name: "opencodex_tool_search" }); + expect(body.tools[1]).toEqual(raw.tools[1]); + expect(body.tool_choice.tools).toEqual([ + { type: "function", name: "opencodex_tool_search" }, + { type: "function", name: "tool_search" }, + ]); + + const restored = JSON.parse(restoreRoutedToolSearchCallsInJson(JSON.stringify({ + output: [ + { type: "function_call", name: "opencodex_tool_search", arguments: "{}" }, + { type: "function_call", name: "tool_search", arguments: "{}" }, + ], + }), rewritten.names)) as { output: Array> }; + expect(restored.output[0]?.type).toBe("tool_search_call"); + expect(restored.output[1]).toMatchObject({ type: "function_call", name: "tool_search" }); + }); + + test("a forced ordinary same-named function does not authorize private-search restoration", () => { + const rewritten = rewriteRoutedToolSearchForUpstream({ + tools: [ + searchTool, + { type: "function", name: "tool_search", parameters: { type: "object" } }, + ], + tool_choice: { type: "function", name: "tool_search" }, + }); + const body = rewritten.body as { + tools: Array>; + tool_choice: Record; + }; + + expect(body.tools[0]).toMatchObject({ type: "function", name: "opencodex_tool_search" }); + expect(body.tool_choice).toEqual({ type: "function", name: "tool_search" }); + expect(rewritten.names).toEqual(new Set()); + }); + + test("rewrites prior private search history to public function call pairs", () => { + const previousTools = [{ type: "function", name: "deferred_read", parameters: { type: "object" } }]; + const rewritten = rewriteRoutedToolSearchForUpstream({ + tools: [searchTool], + input: [ + { + type: "tool_search_call", + id: "tsc_previous", + call_id: "call_previous", + execution: "client", + arguments: { query: "database" }, + status: "completed", + }, + { + type: "tool_search_output", + call_id: "call_previous", + execution: "client", + status: "completed", + tools: previousTools, + }, + ], + }); + const input = (rewritten.body as { input: Array> }).input; + + expect(input[0]).toEqual({ + type: "function_call", + id: "fc_previous", + call_id: "call_previous", + name: "tool_search", + arguments: '{"query":"database"}', + status: "completed", + }); + expect(input[1]).toMatchObject({ + type: "function_call_output", + call_id: "call_previous", + }); + expect(JSON.parse(String(input[1]?.output))).toEqual({ tools: previousTools, status: "completed" }); + }); + + test("restores only the converted function name in bounded JSON", () => { + const restored = JSON.parse(restoreRoutedToolSearchCallsInJson(JSON.stringify({ + id: "resp_1", + output: [ + { type: "function_call", id: "fc_search", call_id: "call_search", name: "tool_search", arguments: '{"query":"database"}', status: "completed" }, + { type: "function_call", id: "fc_other", call_id: "call_other", name: "ordinary", arguments: "{}", status: "completed" }, + ], + }), new Set(["tool_search"]))) as { output: Array> }; + + expect(restored.output[0]).toEqual({ + type: "tool_search_call", + id: "tsc_search", + call_id: "call_search", + execution: "client", + arguments: { query: "database" }, + status: "completed", + }); + expect(restored.output[1]).toMatchObject({ type: "function_call", name: "ordinary" }); + }); + + test("restores the SSE item lifecycle and suppresses function argument frames", () => { + const budget = createTestTranslatorBudget(); + const rewrite = createRoutedToolSearchRestoreBlockRewrite(new Set(["tool_search"]), budget); + + expect(rewrite(frame("response.function_call_arguments.delta", { + output_index: 0, + item_id: "fc_search", + delta: '{"query":', + }))).toEqual([]); + expect(budget.snapshot().currentBytes).toBeGreaterThan(0); + + const added = rewrite(frame("response.output_item.added", { + output_index: 0, + item: { + type: "function_call", + id: "fc_search", + call_id: "call_search", + name: "tool_search", + arguments: "", + status: "in_progress", + }, + })); + expect(added).toHaveLength(1); + expect(dataPayload(added[0]!).item).toEqual({ + type: "tool_search_call", + id: "tsc_search", + call_id: "call_search", + execution: "client", + arguments: {}, + status: "in_progress", + }); + expect(budget.snapshot().currentBytes).toBe(0); + + expect(rewrite(frame("response.function_call_arguments.done", { + output_index: 0, + item_id: "fc_search", + arguments: '{"query":"database"}', + }))).toEqual([]); + + const done = rewrite(frame("response.output_item.done", { + output_index: 0, + item: { + type: "function_call", + id: "fc_search", + call_id: "call_search", + name: "tool_search", + arguments: '{"query":"database"}', + status: "completed", + }, + })); + expect(dataPayload(done[0]!).item).toMatchObject({ + type: "tool_search_call", + id: "tsc_search", + arguments: { query: "database" }, + }); + rewrite.dispose?.(); + expect(budget.snapshot().currentBytes).toBe(0); + }); + + test("replays an ordinary pending argument frame and releases unmatched terminal state", () => { + const budget = createTestTranslatorBudget(); + const rewrite = createRoutedToolSearchRestoreBlockRewrite(new Set(["tool_search"]), budget); + + expect(rewrite(frame("response.function_call_arguments.delta", { + output_index: 1, + delta: '{"path":', + }))).toEqual([]); + const ordinary = rewrite(frame("response.output_item.added", { + output_index: 1, + item: { + type: "function_call", + id: "fc_regular", + call_id: "call_regular", + name: "ordinary", + arguments: "", + status: "in_progress", + }, + })); + expect(ordinary).toHaveLength(2); + expect(dataPayload(ordinary[0]!).item_id).toBe("fc_regular"); + expect(dataPayload(ordinary[1]!).item).toMatchObject({ type: "function_call", name: "ordinary" }); + + expect(rewrite(frame("response.function_call_arguments.delta", { + output_index: 9, + delta: "unmatched", + }))).toEqual([]); + expect(budget.snapshot().currentBytes).toBeGreaterThan(0); + expect(rewrite(frame("response.completed", { + response: { id: "resp_done", status: "completed", output: [] }, + }))).toHaveLength(1); + expect(budget.snapshot().currentBytes).toBe(0); + }); + + test("falls back to raw pass-through when the pending collector budget is exhausted", () => { + const budget = createTestTranslatorBudget({ maxTurnBytes: 64 }); + const rewrite = createRoutedToolSearchRestoreBlockRewrite(new Set(["tool_search"]), budget); + const pending = frame("response.function_call_arguments.delta", { + output_index: 0, + delta: "x".repeat(256), + }); + + expect(rewrite(pending)).toEqual([pending]); + expect(budget.snapshot().currentBytes).toBe(0); + const added = frame("response.output_item.added", { + output_index: 0, + item: { + type: "function_call", + id: "fc_search", + call_id: "call_search", + name: "tool_search", + arguments: "", + }, + }); + expect(rewrite(added)).toEqual([added]); + }); + + test("handleResponses lowers and restores a streaming search call", async () => { + const savedFetch = globalThis.fetch; + let outboundBody: Record | undefined; + const upstreamItem = { + type: "function_call", + id: "fc_search", + call_id: "call_search", + name: "tool_search", + arguments: '{"query":"database"}', + status: "completed", + }; + const upstream = [ + frame("response.output_item.added", { output_index: 0, item: { ...upstreamItem, arguments: "", status: "in_progress" } }), + frame("response.function_call_arguments.delta", { output_index: 0, item_id: "fc_search", delta: upstreamItem.arguments }), + frame("response.function_call_arguments.done", { output_index: 0, item_id: "fc_search", arguments: upstreamItem.arguments }), + frame("response.output_item.done", { output_index: 0, item: upstreamItem }), + frame("response.completed", { response: { id: "resp_1", status: "completed", output: [upstreamItem] } }), + "data: [DONE]", + ].join("\n\n") + "\n\n"; + globalThis.fetch = (async (_input, init) => { + outboundBody = JSON.parse(String(init?.body)) as Record; + return new Response(upstream, { headers: { "content-type": "text/event-stream" } }); + }) as typeof fetch; + + try { + const response = await handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + model: "fixture/deepseek-v4-flash", + stream: true, + input: [{ role: "user", content: [{ type: "input_text", text: "find a database tool" }] }], + tools: [ + searchTool, + { type: "custom", name: "exec", description: "Run JavaScript", format: { type: "grammar", syntax: "lark" } }, + ], + }), + }), fixtureConfig(), { model: "", provider: "" }); + const clientSse = await response.text(); + const outboundTools = outboundBody?.tools as Array> | undefined; + + expect(outboundTools?.[0]).toMatchObject({ type: "function", name: "tool_search" }); + expect(outboundTools?.[1]).toMatchObject({ type: "function", name: "exec" }); + expect(clientSse).toContain('"type":"tool_search_call"'); + expect(clientSse).toContain('"id":"tsc_search"'); + expect(clientSse).not.toContain("response.function_call_arguments"); + } finally { + globalThis.fetch = savedFetch; + } + }); + + test("handleResponses restores a non-streaming search call", async () => { + const savedFetch = globalThis.fetch; + let outboundBody: Record | undefined; + globalThis.fetch = (async (_input, init) => { + outboundBody = JSON.parse(String(init?.body)) as Record; + return new Response(JSON.stringify({ + id: "resp_json", + status: "completed", + output: [{ + type: "function_call", + id: "fc_search", + call_id: "call_search", + name: "tool_search", + arguments: '{"query":"database"}', + status: "completed", + }], + }), { headers: { "content-type": "application/json" } }); + }) as typeof fetch; + + try { + const response = await handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + model: "fixture/deepseek-v4-flash", + stream: false, + input: [{ role: "user", content: [{ type: "input_text", text: "find a database tool" }] }], + tools: [searchTool], + }), + }), fixtureConfig(), { model: "", provider: "" }); + const body = await response.json() as { output: Array> }; + const outboundTools = outboundBody?.tools as Array> | undefined; + + expect(outboundTools?.[0]).toMatchObject({ type: "function", name: "tool_search" }); + expect(body.output[0]).toEqual({ + type: "tool_search_call", + id: "tsc_search", + call_id: "call_search", + execution: "client", + arguments: { query: "database" }, + status: "completed", + }); + } finally { + globalThis.fetch = savedFetch; + } + }); + + test("handleResponses forwards second-turn search history in public function form", async () => { + const savedFetch = globalThis.fetch; + let outboundBody: Record | undefined; + globalThis.fetch = (async (_input, init) => { + outboundBody = JSON.parse(String(init?.body)) as Record; + return new Response(JSON.stringify({ id: "resp_next", status: "completed", output: [] }), { + headers: { "content-type": "application/json" }, + }); + }) as typeof fetch; + + try { + await handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + model: "fixture/deepseek-v4-flash", + stream: false, + tools: [searchTool], + input: [ + { + type: "tool_search_call", + id: "tsc_previous", + call_id: "call_previous", + execution: "client", + arguments: { query: "database" }, + status: "completed", + }, + { + type: "tool_search_output", + call_id: "call_previous", + execution: "client", + status: "completed", + tools: [{ type: "function", name: "deferred_read", parameters: { type: "object" } }], + }, + { role: "user", content: [{ type: "input_text", text: "use the loaded tool" }] }, + ], + }), + }), fixtureConfig(), { model: "", provider: "" }); + const input = outboundBody?.input as Array> | undefined; + + expect(input?.[0]).toMatchObject({ + type: "function_call", + id: "fc_previous", + call_id: "call_previous", + name: "tool_search", + }); + expect(typeof input?.[0]?.arguments).toBe("string"); + expect(input?.[1]).toMatchObject({ + type: "function_call_output", + call_id: "call_previous", + }); + expect(input?.some(item => item.type === "tool_search_call" || item.type === "tool_search_output")).toBe(false); + } finally { + globalThis.fetch = savedFetch; + } + }); + + test("tool_choice none prevents restoring an ignored upstream search call", async () => { + const savedFetch = globalThis.fetch; + globalThis.fetch = (async () => new Response(JSON.stringify({ + id: "resp_json", + status: "completed", + output: [{ + type: "function_call", + id: "fc_search", + call_id: "call_search", + name: "tool_search", + arguments: '{"query":"database"}', + status: "completed", + }], + }), { headers: { "content-type": "application/json" } })) as typeof fetch; + + try { + const response = await handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + model: "fixture/deepseek-v4-flash", + stream: false, + input: [{ role: "user", content: [{ type: "input_text", text: "do not call tools" }] }], + tools: [searchTool], + tool_choice: "none", + }), + }), fixtureConfig(), { model: "", provider: "" }); + const body = await response.json() as { output: Array> }; + + expect(body.output[0]).toMatchObject({ + type: "function_call", + name: "tool_search", + }); + expect(body.output[0]).not.toHaveProperty("execution"); + } finally { + globalThis.fetch = savedFetch; + } + }); + + test("history-only replay still arms restoration for the lowered call", () => { + // A turn may replay tool_search history WITHOUT re-declaring the tool. The history is + // lowered to function_call either way, so leaving \`names\` empty would hand the client a + // public function_call for what it issued as a private search call. + const { body, names } = rewriteRoutedToolSearchForUpstream({ + model: "fixture/deepseek-v4-flash", + tools: [{ type: "function", name: "read_file", parameters: { type: "object" } }], + input: [ + { + type: "tool_search_call", + id: "tsc_old", + call_id: "call_old", + execution: "client", + arguments: { query: "x" }, + status: "completed", + }, + { type: "tool_search_output", call_id: "call_old", execution: "client", status: "completed", tools: [] }, + ], + }); + + expect(names.has("tool_search")).toBe(true); + const input = body.input as Array>; + expect(input[0]).toMatchObject({ type: "function_call", id: "fc_old", name: "tool_search" }); + expect(input[1]).toMatchObject({ type: "function_call_output", call_id: "call_old" }); + }); + + test("overflow keeps suppressing frames for an already-restored routed item", () => { + const rewrite = createRoutedToolSearchRestoreBlockRewrite(new Set(["tool_search"])); + + // Open and restore a routed item. + const opened = rewrite(frame("response.output_item.added", { + output_index: 0, + item: { type: "function_call", id: "fc_search", name: "tool_search", arguments: "" }, + })); + expect(dataPayload(opened[0]!).item).toMatchObject({ type: "tool_search_call", id: "tsc_search" }); + + // Overflow the pending buffer on a DIFFERENT, unclassified item id. + for (let i = 0; i <= 256; i += 1) { + rewrite(frame("response.function_call_arguments.delta", { + output_index: 1, + item_id: "fc_unknown", + delta: "x", + })); + } + + // The restored item's public argument frames must still be suppressed. Clearing the routed + // id on overflow would leak them and give the client a mixed private/public lifecycle for + // one call — the exact defect this rewrite exists to prevent. + const leaked = rewrite(frame("response.function_call_arguments.delta", { + output_index: 0, + item_id: "fc_search", + delta: "{", + })); + expect(leaked).toEqual([]); + + // Unrelated traffic still passes through untouched after overflow. + const other = rewrite(frame("response.output_text.delta", { output_index: 2, delta: "hi" })); + expect(other).toHaveLength(1); + }); +}); From 8b7b65cb8946c763ea9ad44d8e9f2cc89ba78abf Mon Sep 17 00:00:00 2001 From: bitkyc08-arch Date: Thu, 20 Aug 2026 11:27:04 +0900 Subject: [PATCH 2/2] fix(responses): keep tool-search item classification past output_item.done The routed and ordinary id sets were cleared at output_item.done, but done ends the item, not the id's relevance. For a ROUTED id that was a leak: an upstream that emits a trailing function_call_arguments.done after the item closes found the classification gone, fell through to the unknown-id branch, and the frame reached the client as a public argument event for an item the client was told is a private tool_search_call. That mixed lifecycle is the defect this rewrite exists to prevent. For an ORDINARY id it was not a leak but was not free either: the same fall-through buffered its trailing frames as unknown and held them for an item that would never arrive. Both are now retained until the terminal event releases everything. Found by CodeRabbit on #2145. --- src/server/responses-tool-search-repair.ts | 14 ++++-- tests/responses-tool-search-repair.test.ts | 53 ++++++++++++++++++++++ 2 files changed, 63 insertions(+), 4 deletions(-) diff --git a/src/server/responses-tool-search-repair.ts b/src/server/responses-tool-search-repair.ts index 8aa5c48130..6c18708406 100644 --- a/src/server/responses-tool-search-repair.ts +++ b/src/server/responses-tool-search-repair.ts @@ -183,10 +183,16 @@ export function createRoutedToolSearchRestoreBlockRewrite( const restoredBlock = restored.changed ? replaceSseDataPayload(block, JSON.stringify(restored.value)) : block; - if (type === "response.output_item.done" && itemId) { - routedItemIds.delete(itemId); - ordinaryItemIds.delete(itemId); - } + // Classification is retained past `output_item.done` for BOTH kinds, until the terminal + // event releases everything. + // + // `done` ends the item, not the id's relevance. Forgetting a ROUTED id let a trailing + // `function_call_arguments.*` — which some upstreams emit after done — fall through to + // the unknown-id branch and reach the client as a public frame for an item the client + // was told is a private `tool_search_call`: the mixed lifecycle this rewrite exists to + // prevent. Forgetting an ORDINARY id is not leak-shaped but is not free either, because + // the same fall-through buffers its trailing frames as unknown and delays them until an + // item that will never arrive. Neither id is dropped early. return routed ? [restoredBlock] : [...pending, restoredBlock]; } diff --git a/tests/responses-tool-search-repair.test.ts b/tests/responses-tool-search-repair.test.ts index c208d3c1d0..a080574a3d 100644 --- a/tests/responses-tool-search-repair.test.ts +++ b/tests/responses-tool-search-repair.test.ts @@ -575,4 +575,57 @@ describe("routed Responses tool-search compatibility", () => { const other = rewrite(frame("response.output_text.delta", { output_index: 2, delta: "hi" })); expect(other).toHaveLength(1); }); + + // `output_item.done` ends the ITEM, not the id's relevance. Forgetting the routed id there + // meant a trailing argument frame — which some upstreams emit after done — fell through to + // the unknown-id branch and reached the client as a public `function_call_arguments.*` for + // an item the client was told is a private `tool_search_call`. + test("a routed id stays suppressed after output_item.done", () => { + const rewrite = createRoutedToolSearchRestoreBlockRewrite(new Set(["tool_search"])); + + rewrite(frame("response.output_item.added", { + output_index: 0, + item: { type: "function_call", id: "fc_search", name: "tool_search", arguments: "" }, + })); + const done = rewrite(frame("response.output_item.done", { + output_index: 0, + item: { type: "function_call", id: "fc_search", name: "tool_search", arguments: "{}" }, + })); + expect(dataPayload(done[0]!).item).toMatchObject({ type: "tool_search_call" }); + + const trailing = rewrite(frame("response.function_call_arguments.done", { + output_index: 0, + item_id: "fc_search", + arguments: "{}", + })); + expect(trailing).toEqual([]); + + const trailingDelta = rewrite(frame("response.function_call_arguments.delta", { + output_index: 0, + item_id: "fc_search", + delta: "x", + })); + expect(trailingDelta).toEqual([]); + }); + + test("an ordinary item's frames still pass through after its done", () => { + const rewrite = createRoutedToolSearchRestoreBlockRewrite(new Set(["tool_search"])); + + rewrite(frame("response.output_item.added", { + output_index: 0, + item: { type: "function_call", id: "fc_plain", name: "not_routed", arguments: "" }, + })); + rewrite(frame("response.output_item.done", { + output_index: 0, + item: { type: "function_call", id: "fc_plain", name: "not_routed", arguments: "{}" }, + })); + + // Retaining routed ids must not accidentally start swallowing ordinary ones. + const trailing = rewrite(frame("response.function_call_arguments.delta", { + output_index: 0, + item_id: "fc_plain", + delta: "x", + })); + expect(trailing).toHaveLength(1); + }); });