diff --git a/docs-site/src/content/docs/ja/reference/configuration/providers.md b/docs-site/src/content/docs/ja/reference/configuration/providers.md index f1210892a5..57a576434c 100644 --- a/docs-site/src/content/docs/ja/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ja/reference/configuration/providers.md @@ -355,3 +355,15 @@ OpenRouter は、複数の推論プロバイダーを通じて 1 つのモデル "visionSidecar": { "enabled": true } } ``` + +### Responses ターミナル イベント修復ポリシー + +これらの3つのキーは、ネイティブ Responses ストリームがターミナル イベントを配信しない場合に、制限付きの修復を必要とするカスタム プロバイダー向けの制御機能です。要求されたモデルごとに、有効なアダプター（プロバイダーのアダプターまたはその modelAdapters オーバーライド）が openai-responses である必要があります。Chat Completions やその他の配線はオプトインしません。モデルの一致は大文字と小文字を区別しません。 + +解決の優先順位： + +1. 一致する modelResponsesCompatibility エントリは、モデルをターミナル修復にオプトインします。デフォルトの猶予時間は 500 ms です（一致する modelResponsesTerminalRepair が明示的な猶予時間を指定していない限り）。 +2. それ以外の場合、一致する modelResponsesTerminalRepair エントリがモデルごとの猶予時間を指定します。 +3. それ以外の場合、responsesTerminalRepair がプロバイダー レベルのフォールバックを提供します。 + +猶予時間の値は正の有限ミリ秒であり、ランタイムは切り捨てて最大 60 秒に制限します。設定検証では無効な値を拒否し、正規の ChatGPT forward プロバイダーではこれら3つのキーを拒否します（プロバイダー名ではなく adapter、authMode、正規化 baseUrl で判定）。 diff --git a/docs-site/src/content/docs/ko/reference/configuration/providers.md b/docs-site/src/content/docs/ko/reference/configuration/providers.md index ccacb0a94f..2e18270805 100644 --- a/docs-site/src/content/docs/ko/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ko/reference/configuration/providers.md @@ -356,3 +356,15 @@ OpenRouter는 하나의 모델을 여러 추론 공급자로 제공할 수 있 "visionSidecar": { "enabled": true } } ``` + +### Responses 터미널 이벤트 복구 정책 + +이 세 가지 키는 네이티브 Responses 스트림이 터미널 이벤트를 전달하지 않을 때 제한된 복구가 필요한 커스텀 공급자를 위한 제어 항목입니다. 요청된 각 모델에 대해 유효한 어댑터(공급자 어댑터 또는 해당 modelAdapters 재정의)가 openai-responses여야 합니다. Chat Completions 및 기타 와이어는 활성화되지 않습니다. 모델 매칭은 대소문자를 구분하지 않습니다. + +해석 우선순위: + +1. 일치하는 modelResponsesCompatibility 항목은 모델을 터미널 복구에 포함시킵니다. 기본 유예 시간은 500ms입니다(modelResponsesTerminalRepair에 명시적인 유예 시간이 제공되지 않는 한). +2. 그렇지 않으면 일치하는 modelResponsesTerminalRepair 항목이 모델별 유예 시간을 제공합니다. +3. 그렇지 않으면 responsesTerminalRepair가 공급자 수준의 대체값을 제공합니다. + +유예 시간 값은 양의 유한 밀리초여야 하며, 런타임은 이를 버림 처리하고 최대 60초로 제한합니다. 설정 검증은 잘못된 값을 거부하고 정식 ChatGPT forward 공급자에서 이 세 가지 키를 거부합니다(공급자 이름이 아닌 adapter, authMode, 정규화된 baseUrl 기준). diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 85cef14e1e..982fdc6b89 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -100,6 +100,9 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids. | `modelSupportsReasoningSummaries?` | `Record` | Set a model to `false` to stop advertising summaries and strip summary-delivery fields. | | `modelReasoningSummaryDelivery?` | `Record` | Per-model Responses delivery enum; rewrites an existing delivery field. | | `modelAdapters?` | `Record` | Per-model `openai-chat` or `openai-responses` wire override for mixed-wire gateways. Explicit entries beat registry defaults. The OpenCode Go preset selects Responses for `gpt-5.6-luna` while leaving sibling models on their documented wires; DeepSeek can select native Responses for `deepseek-v4-flash`; and GitHub Copilot declares Responses-only defaults for its GPT-5 family (`gpt-5.3-codex`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.5`, `gpt-5.6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`) because those models reject `/chat/completions` for agent traffic. Models without a built-in default (for example `gpt-5.4-nano`) can be opted in here. Single-wire upstream pins and canonical ChatGPT forward reject overrides. | +| `modelResponsesCompatibility?` | `Record` | Case-insensitive per-model opt-in for the Responses terminal-repair policy on custom providers. A matching model gets the 500 ms default grace unless `modelResponsesTerminalRepair` supplies an explicit grace. The effective model wire must be `openai-responses`; canonical ChatGPT forward rejects this key. | +| `modelResponsesTerminalRepair?` | `Record` | Case-insensitive per-model terminal-repair grace in milliseconds. It overrides the compatibility default and takes precedence over `responsesTerminalRepair`; positive values are floored and capped at 60 seconds. Invalid or ambiguous case-folded entries fail closed at resolution. | +| `responsesTerminalRepair?` | `"terminal-repair" \| number \| { graceMs: number }` | Provider-level terminal-repair fallback for models using the `openai-responses` wire. The string selects the 500 ms default; numeric/object values set the grace and are capped at 60 seconds. It is considered only after compatibility and explicit per-model settings, and is rejected on the canonical ChatGPT forward provider. | | xAI Responses opt-in (dashboard) | switch | For `xai` only, atomically sets or clears the `grok-4.5` and `grok-4.6` `modelAdapters` entries. A hand-edited single entry appears as mixed until the next switch write normalizes both. Other overrides and tier behavior are unchanged. | | `modelPreferHostedTools?` | `Record` | Exact-model opt-in for non-forward Responses gateways that reserve a hosted-tool namespace. Currently accepts only `["image_generation"]`; a matching model must use the `openai-responses` wire and support that hosted tool. It removes colliding client `image_gen` declarations and rewrites their selectors to preserve caller tool choice. For OpenAI API virtual `-pro` models, the selected public ID is matched first and the resolved base wire-model ID is a fallback. `modelAdapters` resolves the public ID first, then the base ID; the second resolution determines the final wire. Other models retain normal alias behavior. | | `reasoningEffortMap?` | `Record` | Provider-wide wire aliases for reasoning labels. | @@ -132,6 +135,36 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids. | `unsafeAllowNativeLocalExec?` | `boolean` | Cursor legacy boolean, equivalent to `nativeLocalExec: "on"` only when the newer field is unset. | | `nativeLocalExec?` | `"off" \| "codex-sandbox" \| "on"` | Cursor local-exec policy. `off` is default; `codex-sandbox` currently fails closed like `off`. | +### Responses terminal-repair policy + +These three keys are overlapping controls for custom providers that need a bounded repair when a +native Responses stream does not deliver its terminal event. For each requested model, the +effective adapter (the provider adapter or its `modelAdapters` override) must be +`openai-responses`; Chat Completions and other wires never opt in. Model matching is +case-insensitive. + +Resolution uses this precedence: + +1. A matching `modelResponsesCompatibility` entry opts the model into terminal repair. Its + default grace is 500 ms, unless a matching `modelResponsesTerminalRepair` entry supplies an + explicit grace. +2. Otherwise, a matching `modelResponsesTerminalRepair` entry supplies the per-model grace. +3. Otherwise, `responsesTerminalRepair` supplies the provider-level fallback. + +Grace values are positive finite milliseconds, and the runtime floors them and caps any result at +60 seconds. Config validation rejects malformed values and rejects all three keys on the canonical +ChatGPT forward provider (matching by adapter, authMode, and normalized baseUrl configuration, not provider name). The runtime resolver is defense in depth: an invalid or ambiguous +case-folded per-model entry is not selected, so resolution fails closed instead of choosing an +arbitrary entry. + +#### Decision Log: why three overlapping knobs? + +`modelResponsesCompatibility` provides a readable opt-in with a safe default, while +`modelResponsesTerminalRepair` handles models that need a different grace period. The +provider-level `responsesTerminalRepair` covers a gateway whose Responses models share one policy. +Keeping all three preserves simple compatibility migration without giving a broad default priority +over an explicit per-model choice. + ### FastWire B1 capability migration Fast capability and arbitrary Chat caller-tier forwarding are independent after FastWire B1. The diff --git a/docs-site/src/content/docs/ru/reference/configuration/providers.md b/docs-site/src/content/docs/ru/reference/configuration/providers.md index c415517074..4b1f6410be 100644 --- a/docs-site/src/content/docs/ru/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ru/reference/configuration/providers.md @@ -441,3 +441,15 @@ Pool/Direct рекламирует `922000`; синхронизированны "visionSidecar": { "enabled": true } } ``` + +### Политика восстановления терминальных событий Responses + +Эти три ключа представляют собой перекрывающиеся элементы управления для кастомных провайдеров, которым требуется ограниченное восстановление, когда нативный поток Responses не доставляет свое терминальное событие. Для каждой запрошенной модели эффективный адаптер (адаптер провайдера или его переопределение modelAdapters) должен быть openai-responses. Сопоставление моделей нечувствительно к регистру. + +Приоритет разрешения: + +1. Соответствующая запись modelResponsesCompatibility включает модель в восстановление (default 500 ms, если modelResponsesTerminalRepair не задает явный grace). +2. Иначе соответствующая запись modelResponsesTerminalRepair задает grace для конкретной модели. +3. Иначе responsesTerminalRepair задает значение fallback на уровне провайдера. + +Значения grace — положительные конечные миллисекунды (максимум 60 секунд). Валидация конфигурации отклоняет некорректные значения и запрещает эти ключи для canonical ChatGPT forward (по adapter, authMode и baseUrl). diff --git a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md index 3630a9ba6c..eec75c4d72 100644 --- a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md @@ -358,3 +358,15 @@ OpenRouter 可以通过多个推理提供者来提供同一个模型。`openRout "visionSidecar": { "enabled": true } } ``` + +### Responses 终端事件修复策略 (Terminal-Repair Policy) + +这三个配置键是针对自定义 Provider 提供的重叠控制项，用于在原生 Responses 流未送达终端事件时提供有界的修复机制。对于每个请求的模型，有效适配器（Provider 的 adapter 或其 modelAdapters 覆盖）必须为 openai-responses；Chat Completions 与其他线协议绝不启用。模型匹配不区分大小写。 + +解析优先级如下： + +1. 匹配的 modelResponsesCompatibility 条目为模型启用终端修复，默认宽限期为 500 毫秒（除非匹配的 modelResponsesTerminalRepair 提供显式宽限期）。 +2. 否则，匹配的 modelResponsesTerminalRepair 条目提供 per-model 宽限期。 +3. 否则，由 responsesTerminalRepair 提供 Provider 级别的回退值。 + +宽限期数值必须为正有限毫秒数，Runtime 会对其取整并将上限约束在 60 秒。配置验证会拒绝格式错误的值，并在规范的 ChatGPT forward Provider 上拒绝这三个键（通过 adapter、authMode 与规范化 baseUrl 判定，而非 Provider 名称启发式）。Runtime 解析器实施纵深防御：无效或大小写冲突的 per-model 条目不会被选取，从而 fail-closed 拒绝，而非任意挑选条目。 diff --git a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md index b0a46f49ec..54a0feae71 100644 --- a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md @@ -326,3 +326,15 @@ OpenRouter 可透過多個推論供應商提供一個模型。`openRouterRouting "visionSidecar": { "enabled": true } } ``` + +### Responses 終端事件修復策略 (Terminal-Repair Policy) + +這三個設定鍵是針對自定義 Provider 所提供的重疊控制項，用於在原生 Responses 串流未送達終端事件時提供有界的修復機制。對於每個請求的模型，有效適配器（Provider 的 adapter 或其 modelAdapters 覆寫）必須為 openai-responses；Chat Completions 與其他 wire 格式絕不啟用。模型匹配不區分大小寫。 + +解析優先級如下： + +1. 匹配的 modelResponsesCompatibility 項目為模型啟用終端修復，預設寬限期為 500 毫秒（除非匹配的 modelResponsesTerminalRepair 提供明確寬限期）。 +2. 否則，匹配的 modelResponsesTerminalRepair 項目提供 per-model 寬限期。 +3. 否則，由 responsesTerminalRepair 提供 Provider 級別的回退值。 + +寬限期數值必須為正有限毫秒數，Runtime 會對其取整並將上限約束在 60 秒。設定驗證會拒絕格式錯誤的值，並在規範的 ChatGPT forward Provider 上拒絕這三個鍵（透過 adapter、authMode 與正規化 baseUrl 判定，而非 Provider 名稱啟發式）。Runtime 解析器實施深度防禦：無效或大小寫衝突的 per-model 項目不會被選取，從而 fail-closed 拒絕，而非任意挑選項目。 diff --git a/src/config.ts b/src/config.ts index 9420ede9b3..e5e418385a 100644 --- a/src/config.ts +++ b/src/config.ts @@ -72,7 +72,7 @@ import { type FastWire, type ProviderCostOverlay, } from "./types"; -import { OPENAI_CODEX_PROVIDER_ID } from "./providers/openai-tiers"; +import { isCanonicalOpenAiForwardProvider, OPENAI_CODEX_PROVIDER_ID } from "./providers/openai-tiers"; import { modelAutoCompactTokenLimitsConfigError } from "./providers/auto-compact-budget"; import { fastWireDeclarationError, hasFastWireCapabilityConflict } from "./providers/fastwire"; import { @@ -709,6 +709,92 @@ export function modelPreferHostedToolsConfigError( return null; } +/** + * Validate a provider's per-model wire override map (#404). + * + * Rejects, rather than silently ignoring, configurations the resolver would refuse: + * a value outside the allowed wires, a model the upstream pins to one wire, and any + * override on a canonical forward provider (where switching wires would drop the + * caller's forwarded credential). Silently dropping them would leave the user + * believing an override is in effect. + */ +export function modelResponsesCompatibilityConfigError( + value: unknown, + field = "modelResponsesCompatibility", + providerName?: string, + provider?: { adapter?: unknown; authMode?: unknown; baseUrl?: unknown }, +): string | null { + if (value === undefined) return null; + if (!value || typeof value !== "object" || Array.isArray(value)) return `${field} must be a plain object`; + const prototype = Object.getPrototypeOf(value); + if (prototype !== Object.prototype && prototype !== null) return `${field} must be a plain object with own properties`; + const entries = Object.entries(value); + if (entries.length > 0 && provider && isCanonicalOpenAiForwardProvider(provider as OcxProviderConfig)) { + return `${field} is not supported on the canonical ChatGPT forward provider`; + } + const seen = new Set(); + for (const [key, entry] of entries) { + const lower = key.toLowerCase(); + if (seen.has(lower)) { + return `${field} contains duplicate case-insensitive model id "${key}"`; + } + seen.add(lower); + if (!key.trim() || key !== key.trim()) return `${field} keys must be nonblank trimmed model ids`; + if (entry !== "terminal-repair") { + return `${field}.${key} must be "terminal-repair"`; + } + } + return null; +} + +export function modelResponsesTerminalRepairConfigError( + value: unknown, + field = "modelResponsesTerminalRepair", + providerName?: string, + provider?: { adapter?: unknown; authMode?: unknown; baseUrl?: unknown }, +): string | null { + if (value === undefined) return null; + if (!value || typeof value !== "object" || Array.isArray(value)) return `${field} must be a plain object`; + const prototype = Object.getPrototypeOf(value); + if (prototype !== Object.prototype && prototype !== null) return `${field} must be a plain object with own properties`; + const entries = Object.entries(value); + if (entries.length > 0 && provider && isCanonicalOpenAiForwardProvider(provider as OcxProviderConfig)) { + return `${field} is not supported on the canonical ChatGPT forward provider`; + } + const seen = new Set(); + for (const [key, entry] of entries) { + const lower = key.toLowerCase(); + if (seen.has(lower)) { + return `${field} contains duplicate case-insensitive model id "${key}"`; + } + seen.add(lower); + if (!key.trim() || key !== key.trim()) return `${field} keys must be nonblank trimmed model ids`; + const grace = typeof entry === "number" ? entry : (typeof entry === "object" && entry ? (entry as { graceMs?: unknown }).graceMs : null); + if (typeof grace !== "number" || !Number.isFinite(grace) || Math.floor(grace) <= 0) { + return `${field}.${key} must be a positive number of milliseconds or { graceMs: number }`; + } + } + return null; +} + +export function responsesTerminalRepairConfigError( + value: unknown, + field = "responsesTerminalRepair", + providerName?: string, + provider?: { adapter?: unknown; authMode?: unknown; baseUrl?: unknown }, +): string | null { + if (value === undefined) return null; + if (provider && isCanonicalOpenAiForwardProvider(provider as OcxProviderConfig)) { + return `${field} is not supported on the canonical ChatGPT forward provider`; + } + if (value === "terminal-repair") return null; + const grace = typeof value === "number" ? value : (typeof value === "object" && value ? (value as { graceMs?: unknown }).graceMs : null); + if (typeof grace !== "number" || !Number.isFinite(grace) || Math.floor(grace) <= 0) { + return `${field} must be "terminal-repair", a positive number of milliseconds, or { graceMs: number }`; + } + return null; +} + const CODEX_ACCOUNT_NAMESPACES_RECORD_ERROR = "codexAccountNamespaces must be a plain object mapping account selectors to Codex account ids"; const CODEX_ACCOUNT_NAMESPACE_KEY_ERROR = @@ -1081,6 +1167,45 @@ const configSchema = z.object({ message: modelAdaptersError, }); } + const compatError = modelResponsesCompatibilityConfigError( + (provider as { modelResponsesCompatibility?: unknown }).modelResponsesCompatibility, + "modelResponsesCompatibility", + name, + provider, + ); + if (compatError) { + ctx.addIssue({ + code: "custom", + path: ["providers", redactSecretString(name), "modelResponsesCompatibility"], + message: compatError, + }); + } + const modelRepairError = modelResponsesTerminalRepairConfigError( + (provider as { modelResponsesTerminalRepair?: unknown }).modelResponsesTerminalRepair, + "modelResponsesTerminalRepair", + name, + provider, + ); + if (modelRepairError) { + ctx.addIssue({ + code: "custom", + path: ["providers", redactSecretString(name), "modelResponsesTerminalRepair"], + message: modelRepairError, + }); + } + const repairError = responsesTerminalRepairConfigError( + (provider as { responsesTerminalRepair?: unknown }).responsesTerminalRepair, + "responsesTerminalRepair", + name, + provider, + ); + if (repairError) { + ctx.addIssue({ + code: "custom", + path: ["providers", redactSecretString(name), "responsesTerminalRepair"], + message: repairError, + }); + } const preferHostedToolsError = modelPreferHostedToolsConfigError( (provider as { modelPreferHostedTools?: unknown }).modelPreferHostedTools, "modelPreferHostedTools", diff --git a/src/providers/registry.ts b/src/providers/registry.ts index dae7310ea2..60973d1519 100644 --- a/src/providers/registry.ts +++ b/src/providers/registry.ts @@ -19,6 +19,7 @@ import { } from "../adapters/cursor/discovery"; import { COMMAND_CODE_MODEL_REASONING_EFFORTS } from "./command-code-efforts"; import { isCanonicalOpenRouterTarget } from "./openrouter-routing"; +import { isCanonicalOpenAiForwardProvider } from "./openai-tiers"; export type ProviderAuthKind = "forward" | "oauth" | "key" | "local"; export type MetadataModelIdNormalize = "case-insensitive"; @@ -2910,18 +2911,104 @@ export function providerModelResponsesUpstreamStreaming( return entry.modelResponsesUpstreamStreaming[modelId.trim().toLowerCase()]; } -/** Resolve a registry-only terminal-repair policy for native Responses streams. */ +const DEFAULT_TERMINAL_REPAIR_GRACE_MS = 500; +const MAX_TERMINAL_REPAIR_GRACE_MS = 60_000; + +type CaseInsensitiveLookup = + | { kind: "missing" } + | { kind: "ambiguous" } + | { kind: "value"; value: T }; + +function lookupCaseInsensitive(map: Record | undefined, key: string): CaseInsensitiveLookup { + if (!map) return { kind: "missing" }; + const target = key.trim().toLowerCase(); + if (!target) return { kind: "missing" }; + let matchedValue: T | undefined = undefined; + let matchCount = 0; + for (const [k, v] of Object.entries(map)) { + if (k.trim().toLowerCase() === target) { + matchedValue = v; + matchCount++; + } + } + // If multiple keys case-fold to the same target (e.g. "My-Model" and "my-model"), reject as ambiguous + if (matchCount > 1) return { kind: "ambiguous" }; + return matchCount === 1 && matchedValue !== undefined + ? { kind: "value", value: matchedValue } + : { kind: "missing" }; +} + +/** + * Resolve terminal-repair policy for native Responses streams (supports registry presets + * and custom-provider configuration overrides, issue #1809). + */ export function providerModelResponsesTerminalRepair( id: string, - provider: Pick & Partial>, + provider: Pick & Partial>, modelId: string, ): ResponsesTerminalRepairPolicy | undefined { + // Canonical ChatGPT forward traffic must never undergo synthetic terminal repair + if (isCanonicalOpenAiForwardProvider(provider as OcxProviderConfig)) { + return undefined; + } + + const modelKey = modelId.trim().toLowerCase(); + // Match resolveWireProtocolOverride: explicit modelAdapters entries are exact-keyed. The + // compatibility/repair maps are intentionally case-insensitive, but folding this map here + // would make the policy disagree with the adapter selected for the actual request. + const effectiveAdapter = provider.modelAdapters?.[modelId] ?? provider.adapter; + + // Custom provider opt-in: effective wire must be openai-responses + if (effectiveAdapter === "openai-responses") { + // 1. Check explicit modelResponsesCompatibility + const compat = lookupCaseInsensitive(provider.modelResponsesCompatibility, modelId); + if (compat.kind === "ambiguous") return undefined; + if (compat.kind === "value" && compat.value === "terminal-repair") { + const raw = lookupCaseInsensitive(provider.modelResponsesTerminalRepair, modelId); + if (raw.kind === "ambiguous") return undefined; + if (raw.kind === "value") { + const grace = typeof raw.value === "number" ? raw.value : (typeof raw.value === "object" && raw.value && "graceMs" in raw.value ? (raw.value as { graceMs?: unknown }).graceMs : undefined); + const graceMs = Math.floor(typeof grace === "number" ? grace : 0); + if (!Number.isFinite(graceMs) || graceMs <= 0) return undefined; + return { graceMs: Math.min(graceMs, MAX_TERMINAL_REPAIR_GRACE_MS) }; + } + return { graceMs: DEFAULT_TERMINAL_REPAIR_GRACE_MS }; + } + + // 2. Check explicit modelResponsesTerminalRepair + const rawModel = lookupCaseInsensitive(provider.modelResponsesTerminalRepair, modelId); + if (rawModel.kind === "ambiguous") return undefined; + if (rawModel.kind === "value") { + const grace = typeof rawModel.value === "number" ? rawModel.value : (typeof rawModel.value === "object" && rawModel.value && "graceMs" in rawModel.value ? (rawModel.value as { graceMs?: unknown }).graceMs : undefined); + const graceMs = Math.floor(typeof grace === "number" ? grace : 0); + if (Number.isFinite(graceMs) && graceMs > 0) { + return { graceMs: Math.min(graceMs, MAX_TERMINAL_REPAIR_GRACE_MS) }; + } + // Explicit model-level setting exists but is non-positive/invalid: fail closed, do not fall back to provider default + return undefined; + } + + // 3. Check provider-level responsesTerminalRepair + if (provider.responsesTerminalRepair !== undefined) { + if (provider.responsesTerminalRepair === "terminal-repair") return { graceMs: DEFAULT_TERMINAL_REPAIR_GRACE_MS }; + const grace = typeof provider.responsesTerminalRepair === "number" + ? provider.responsesTerminalRepair + : (typeof provider.responsesTerminalRepair === "object" && provider.responsesTerminalRepair && "graceMs" in provider.responsesTerminalRepair ? (provider.responsesTerminalRepair as { graceMs?: unknown }).graceMs : undefined); + const graceMs = Math.floor(typeof grace === "number" ? grace : 0); + if (Number.isFinite(graceMs) && graceMs > 0) { + return { graceMs: Math.min(graceMs, MAX_TERMINAL_REPAIR_GRACE_MS) }; + } + return undefined; + } + } + + // Fall back to registry-defined policy const entry = getProviderRegistryEntry(id); if (!entry?.modelResponsesTerminalRepair || !providerMatchesRegistryTransport(id, provider)) return undefined; - const policy = entry.modelResponsesTerminalRepair[modelId.trim().toLowerCase()]; + const policy = entry.modelResponsesTerminalRepair[modelKey]; const graceMs = Math.floor(policy?.graceMs ?? 0); if (!Number.isFinite(graceMs) || graceMs <= 0) return undefined; - return { graceMs }; + return { graceMs: Math.min(graceMs, MAX_TERMINAL_REPAIR_GRACE_MS) }; } /** diff --git a/src/server/auth-cors.ts b/src/server/auth-cors.ts index d10cc09287..74d4c45ae7 100644 --- a/src/server/auth-cors.ts +++ b/src/server/auth-cors.ts @@ -7,6 +7,9 @@ import { requestPacingConfigError, retryOn429PolicyConfigError, sanitizeModelCostsForDisplay, + modelResponsesCompatibilityConfigError, + modelResponsesTerminalRepairConfigError, + responsesTerminalRepairConfigError, } from "../config"; import { apiKeyTransportConfigError, @@ -626,6 +629,27 @@ export function providerManagementConfigError(name: unknown, provider: unknown): if (reasoningSummaryDeliveryError) return `provider ${name} ${reasoningSummaryDeliveryError}`; const modelAdaptersError = modelAdapterRecordConfigError(raw.modelAdapters, "modelAdapters", name, typed); if (modelAdaptersError) return `provider ${name} ${modelAdaptersError}`; + const compatError = modelResponsesCompatibilityConfigError( + raw.modelResponsesCompatibility, + "modelResponsesCompatibility", + name, + typed, + ); + if (compatError) return `provider ${name} ${compatError}`; + const modelRepairError = modelResponsesTerminalRepairConfigError( + raw.modelResponsesTerminalRepair, + "modelResponsesTerminalRepair", + name, + typed, + ); + if (modelRepairError) return `provider ${name} ${modelRepairError}`; + const repairError = responsesTerminalRepairConfigError( + raw.responsesTerminalRepair, + "responsesTerminalRepair", + name, + typed, + ); + if (repairError) return `provider ${name} ${repairError}`; const preferHostedToolsError = modelPreferHostedToolsConfigError( raw.modelPreferHostedTools, "modelPreferHostedTools", @@ -722,6 +746,9 @@ export function safeConfigDTO(config: OcxConfig): unknown { "modelMaxOutputTokens", "openRouterRouting", "modelOpenRouterRouting", + "modelResponsesCompatibility", + "modelResponsesTerminalRepair", + "responsesTerminalRepair", "reasoningEfforts", "modelReasoningEfforts", "reasoningWireFormat", diff --git a/src/types/provider.ts b/src/types/provider.ts index e6a66d577f..adc01938a2 100644 --- a/src/types/provider.ts +++ b/src/types/provider.ts @@ -204,6 +204,20 @@ export interface OcxProviderConfig { * `ocxr1` envelopes are still stripped because no upstream can decrypt them. */ preserveResponsesReasoningContent?: boolean; + /** + * Optional per-model Responses compatibility escape hatch for custom providers (issue #1809). + * Keyed by the upstream model id (case-insensitive). + * "terminal-repair" opts into the bounded terminal repair state machine (default 500ms grace). + */ + modelResponsesCompatibility?: Record; + /** + * Explicit per-model terminal-repair grace period for native Responses streams (in ms). + */ + modelResponsesTerminalRepair?: Record; + /** + * Provider-level default terminal-repair grace period for native Responses streams. + */ + responsesTerminalRepair?: { graceMs: number } | number | "terminal-repair"; /** * Explicit opt-in for a relay that genuinely fronts OpenAI and can decode native * compaction blobs. Absent or false degrades foreign blobs to an opaque note. diff --git a/tests/deepseek-inbound-wire.test.ts b/tests/deepseek-inbound-wire.test.ts index 1f298bacce..b8a75d46c9 100644 --- a/tests/deepseek-inbound-wire.test.ts +++ b/tests/deepseek-inbound-wire.test.ts @@ -12,6 +12,15 @@ * assert the captured upstream URL, which is externally observable. */ import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { safeConfigDTO, providerManagementConfigError } from "../src/server/auth-cors"; +import { + getDefaultConfig, + modelResponsesCompatibilityConfigError, + modelResponsesTerminalRepairConfigError, + responsesTerminalRepairConfigError, + validateConfigCandidate, +} from "../src/config"; +import type { OcxConfig } from "../src/types"; import { enrichProviderFromRegistry, providerConfigSeed } from "../src/providers/derive"; import { getProviderRegistryEntry, @@ -1014,4 +1023,350 @@ describe("stateless Responses upstreams get no stateful parameters", () => { expect(input.some(item => item.call_id === "call_orphan")).toBe(false); expect(input.some(item => item.type === "message")).toBe(true); }); + + describe("Custom provider Responses terminal repair escape hatch (#1809)", () => { + test("custom provider opts into default 500ms terminal repair via modelResponsesCompatibility", () => { + const customProv = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesCompatibility: { + "My-Model": "terminal-repair" as const, + }, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", customProv, "my-model")).toEqual({ graceMs: 500 }); + expect(providerModelResponsesTerminalRepair("custom-gateway", customProv, "MY-MODEL")).toEqual({ graceMs: 500 }); + expect(providerModelResponsesTerminalRepair("custom-gateway", customProv, "My-Model")).toEqual({ graceMs: 500 }); + expect(providerModelResponsesTerminalRepair("custom-gateway", customProv, "other-model")).toBeUndefined(); + }); + + test("custom provider specifies explicit graceMs via modelResponsesTerminalRepair", () => { + const customProv = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesTerminalRepair: { + "Model-Num": 1500, + "Model-Obj": { graceMs: 2000 }, + }, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", customProv, "model-num")).toEqual({ graceMs: 1500 }); + expect(providerModelResponsesTerminalRepair("custom-gateway", customProv, "MODEL-NUM")).toEqual({ graceMs: 1500 }); + expect(providerModelResponsesTerminalRepair("custom-gateway", customProv, "model-obj")).toEqual({ graceMs: 2000 }); + expect(providerModelResponsesTerminalRepair("custom-gateway", customProv, "MODEL-OBJ")).toEqual({ graceMs: 2000 }); + expect(providerModelResponsesTerminalRepair("custom-gateway", customProv, "unconfigured")).toBeUndefined(); + }); + + test("custom provider specifies provider-level responsesTerminalRepair", () => { + const customProvString = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + responsesTerminalRepair: "terminal-repair" as const, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", customProvString, "any-model")).toEqual({ graceMs: 500 }); + + const customProvNumber = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + responsesTerminalRepair: 750, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", customProvNumber, "any-model")).toEqual({ graceMs: 750 }); + }); + + test("rejects repair for non-responses wires even when compatibility is set", () => { + const chatProv = { + adapter: "openai-chat", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesCompatibility: { + "my-model": "terminal-repair" as const, + }, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", chatProv, "my-model")).toBeUndefined(); + }); + + test("respects per-model modelAdapters overrides", () => { + const hybridProv = { + adapter: "openai-chat", + baseUrl: "https://custom-gateway.test/v1", + modelAdapters: { + "responses-model": "openai-responses", + }, + modelResponsesCompatibility: { + "responses-model": "terminal-repair" as const, + "chat-model": "terminal-repair" as const, + }, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", hybridProv, "responses-model")).toEqual({ graceMs: 500 }); + expect(providerModelResponsesTerminalRepair("custom-gateway", hybridProv, "chat-model")).toBeUndefined(); + }); + + test("fails closed on non-positive or invalid grace values", () => { + const invalidProv = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + responsesTerminalRepair: 750, + modelResponsesTerminalRepair: { + "zero-grace": 0, + "neg-grace": -500, + "nan-grace": NaN, + }, + modelResponsesCompatibility: { + "compat-zero": "terminal-repair" as const, + "compat-neg": "terminal-repair" as const, + "compat-nan": "terminal-repair" as const, + }, + }; + const invalidCompatProv = { + ...invalidProv, + modelResponsesTerminalRepair: { + "compat-zero": 0, + "compat-neg": -500, + "compat-nan": NaN, + }, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", invalidProv, "zero-grace")).toBeUndefined(); + expect(providerModelResponsesTerminalRepair("custom-gateway", invalidProv, "neg-grace")).toBeUndefined(); + expect(providerModelResponsesTerminalRepair("custom-gateway", invalidProv, "nan-grace")).toBeUndefined(); + expect(providerModelResponsesTerminalRepair("custom-gateway", invalidCompatProv, "compat-zero")).toBeUndefined(); + expect(providerModelResponsesTerminalRepair("custom-gateway", invalidCompatProv, "compat-neg")).toBeUndefined(); + expect(providerModelResponsesTerminalRepair("custom-gateway", invalidCompatProv, "compat-nan")).toBeUndefined(); + }); + + test("canonical ChatGPT forward provider never undergoes terminal repair", () => { + const canonicalOpenAi = { + adapter: "openai-responses", + authMode: "forward" as const, + baseUrl: "https://chatgpt.com/backend-api/codex", + responsesTerminalRepair: "terminal-repair" as const, + modelResponsesTerminalRepair: { "gpt-5": 1000 }, + modelResponsesCompatibility: { "gpt-5": "terminal-repair" as const }, + }; + expect(providerModelResponsesTerminalRepair("openai", canonicalOpenAi, "gpt-5")).toBeUndefined(); + }); + + test("validateConfigCandidate rejects every terminal-repair key on the canonical forward provider", () => { + const base = getDefaultConfig(); + const entries = [ + ["modelResponsesCompatibility", { "gpt-5": "terminal-repair" }], + ["modelResponsesTerminalRepair", { "gpt-5": 500 }], + ["responsesTerminalRepair", "terminal-repair"], + ] as const; + + for (const [field, value] of entries) { + const result = validateConfigCandidate({ + ...base, + providers: { + ...base.providers, + openai: { ...base.providers.openai!, [field]: value }, + }, + }); + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.error).toContain(`${field} is not supported on the canonical ChatGPT forward provider`); + } + } + }); + + test("duplicate case-folded keys fail closed on ambiguity", () => { + const conflictProv = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesTerminalRepair: { + "My-Model": 500, + "my-model": 1500, + }, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", conflictProv, "My-Model")).toBeUndefined(); + expect(providerModelResponsesTerminalRepair("custom-gateway", conflictProv, "my-model")).toBeUndefined(); + expect(providerModelResponsesTerminalRepair("custom-gateway", conflictProv, "MY-MODEL")).toBeUndefined(); + }); + + test("ambiguous explicit values do not fall back to a provider-level grace", () => { + const conflictProv = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + responsesTerminalRepair: 750, + modelResponsesTerminalRepair: { + "My-Model": 500, + "my-model": 1500, + }, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", conflictProv, "MY-MODEL")).toBeUndefined(); + + const compatibilityConflict = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + responsesTerminalRepair: 750, + modelResponsesCompatibility: { + "My-Model": "terminal-repair" as const, + "my-model": "terminal-repair" as const, + }, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", compatibilityConflict, "MY-MODEL")).toBeUndefined(); + }); + + test("matches modelAdapters with the exact wire resolver key semantics", () => { + const provider = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelAdapters: { "My-Model": "openai-chat" }, + modelResponsesTerminalRepair: { "my-model": 1500 }, + }; + // resolveWireProtocolOverride does not match the differently-cased key, so the + // effective wire remains openai-responses and terminal repair is applicable. + expect(providerModelResponsesTerminalRepair("custom-gateway", provider, "my-model")).toEqual({ graceMs: 1500 }); + }); + + test("clamps grace period to maximum 60,000 ms", () => { + const hugeProv = { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesTerminalRepair: { + "huge-model": 120_000, + "max-safe": Number.MAX_SAFE_INTEGER, + }, + }; + expect(providerModelResponsesTerminalRepair("custom-gateway", hugeProv, "huge-model")).toEqual({ graceMs: 60_000 }); + expect(providerModelResponsesTerminalRepair("custom-gateway", hugeProv, "max-safe")).toEqual({ graceMs: 60_000 }); + }); + + test("safeConfigDTO preserves terminal-repair configuration keys", () => { + const config: OcxConfig = { + providers: { + "custom-gw": { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesCompatibility: { "my-model": "terminal-repair" }, + modelResponsesTerminalRepair: { "my-model": 1500 }, + responsesTerminalRepair: { graceMs: 800 }, + }, + }, + } as unknown as OcxConfig; + const dto = safeConfigDTO(config) as { providers: Record> }; + expect(dto.providers["custom-gw"].modelResponsesCompatibility).toEqual({ "my-model": "terminal-repair" }); + expect(dto.providers["custom-gw"].modelResponsesTerminalRepair).toEqual({ "my-model": 1500 }); + expect(dto.providers["custom-gw"].responsesTerminalRepair).toEqual({ graceMs: 800 }); + }); + + test("providerManagementConfigError validates terminal-repair configuration", () => { + expect(providerManagementConfigError("custom-gw", { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesCompatibility: { "my-model": "terminal-repair" }, + modelResponsesTerminalRepair: { "my-model": 1500 }, + responsesTerminalRepair: 800, + })).toBeNull(); + + expect(providerManagementConfigError("custom-gw", { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesCompatibility: { "my-model": "invalid" }, + })).toContain('modelResponsesCompatibility.my-model must be "terminal-repair"'); + + expect(providerManagementConfigError("custom-gw", { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + responsesTerminalRepair: -500, + })).toContain('responsesTerminalRepair must be "terminal-repair", a positive number'); + + const canonicalOpenAi = { + adapter: "openai-responses", + authMode: "forward", + baseUrl: "https://chatgpt.com/backend-api/codex", + }; + expect(responsesTerminalRepairConfigError("terminal-repair", "responsesTerminalRepair", "openai", canonicalOpenAi)) + .toContain("responsesTerminalRepair is not supported on the canonical ChatGPT forward provider"); + expect(modelResponsesCompatibilityConfigError({ "gpt-5": "terminal-repair" }, "modelResponsesCompatibility", "openai", canonicalOpenAi)) + .toContain("modelResponsesCompatibility is not supported on the canonical ChatGPT forward provider"); + expect(modelResponsesTerminalRepairConfigError({ "gpt-5": 500 }, "modelResponsesTerminalRepair", "openai", canonicalOpenAi)) + .toContain("modelResponsesTerminalRepair is not supported on the canonical ChatGPT forward provider"); + + // Reject duplicate case-folded keys + expect(providerManagementConfigError("custom-gw", { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesCompatibility: { "My-Model": "terminal-repair", "my-model": "terminal-repair" }, + })).toContain('modelResponsesCompatibility contains duplicate case-insensitive model id "my-model"'); + + expect(providerManagementConfigError("custom-gw", { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesTerminalRepair: { "My-Model": 1500, "my-model": 2000 }, + })).toContain('modelResponsesTerminalRepair contains duplicate case-insensitive model id "my-model"'); + + // Reject fractional grace flooring to zero + expect(providerManagementConfigError("custom-gw", { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + responsesTerminalRepair: 0.5, + })).toContain('responsesTerminalRepair must be "terminal-repair", a positive number'); + + expect(providerManagementConfigError("custom-gw", { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + modelResponsesTerminalRepair: { "my-model": 0.5 }, + })).toContain('modelResponsesTerminalRepair.my-model must be a positive number'); + }); + + test("handleResponses executes terminal repair on custom openai-responses provider", async () => { + const source = controlledSse(); + const scheduler = new ManualTerminalScheduler(); + const testAbort = new AbortController(); + + globalThis.fetch = (async () => { + return new Response(source.stream, { + status: 200, + headers: { "content-type": "text/event-stream" }, + }); + }) as typeof fetch; + + const config = { + providers: { + "custom-gw": { + adapter: "openai-responses", + baseUrl: "https://custom-gateway.test/v1", + responsesTerminalRepair: 500, + }, + }, + } as unknown as OcxConfig; + + const response = await handleResponses( + new Request("http://localhost/v1/responses", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "custom-gw/custom-model", input: "hello", stream: true }), + }), + config, + { model: "", provider: "" }, + { + abortSignal: testAbort.signal, + responsesTerminalRepairScheduler: scheduler, + } as Parameters[3], + ); + + const reader = response.body!.getReader(); + try { + source.push([ + sse({ type: "response.created", response: { id: "resp_custom", status: "in_progress", output: [] }, sequence_number: 0 }), + sse({ type: "response.output_item.added", item: { type: "message", id: "msg_custom", role: "assistant", status: "in_progress", content: [] }, output_index: 0, sequence_number: 1 }), + sse({ type: "response.output_item.done", item: { type: "message", id: "msg_custom", role: "assistant", status: "completed", content: [{ type: "output_text", text: "hi" }] }, output_index: 0, sequence_number: 2 }), + ].join("")); + + const first = await readUntil(reader, "response.output_item.done"); + expect(first).toContain("response.output_item.done"); + + for (let attempts = 0; attempts < 20 && scheduler.pending() === 0; attempts += 1) { + await Bun.sleep(0); + } + expect(scheduler.pending()).toBe(1); + scheduler.advance(500); + + const tail = await readUntil(reader, "response.completed"); + expect(tail).toContain("response.completed"); + expect(tail).toContain("resp_custom"); + } finally { + testAbort.abort(); + source.cancel(); + await reader.cancel().catch(() => {}); + } + }); + }); });