diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 70cd94af7..553aff98d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2417,8 +2417,8 @@ replaced; do not carry obsolete compatibility code forward to satisfy this secti creation with `model_provider_required`; legacy rows without a snapshot are rejected at message admission and dispatch, never sent to a harness that would fall back to a built-in endpoint. There is no operator options file: its - retirement fails startup, and historical snapshots keep their frozen native - options. Keep runtime dispatch on the common adapter path and fail closed for + retirement fails startup. Sessions retain only the common frozen provider + bundle; historical native-option snapshots are unsupported. Keep runtime dispatch on the common adapter path and fail closed for missing/decryption-failed snapshots. Agent edits/deletion, default changes, restart and idle suspend/resume never resolve defaults again. Record caller intent for every new hosted Session before resolving defaults; other inline @@ -2437,6 +2437,25 @@ replaced; do not carry obsolete compatibility code forward to satisfy this secti identity; retries cannot replace it. Keep this administrator query separate from runtime observations and do not touch activity or wake sandboxes. The versioned contract is `contracts/agents-api/execution-configuration.md`. +- Model communication uses `internal/modeltransport` in the Runtime. Core sends one + frozen confidential `model_provider` bundle, independent of engine and placement; + it must not manufacture native provider options or retain a historical-options + dispatch path. The adapter declares native protocols. Native matches connect + directly; mismatches use one private loopback endpoint owned by the existing + Session execution resource. Its credential is distinct from the upstream key. + Keep endpoint cleanup on final native teardown, including preparation failure, + rather than on individual Turn completion. No second Session manager is added. + CLIProxyAPI's pinned translator is an embedded conversion dependency, not a + gateway service. Requests, JSON responses and incremental SSE events share the + same conversion implementation for hosted and self-hosted execution. The + dependency owns protocol fields, tools, reasoning and usage conversion. Do not + add local field maps, parameter restoration or parallel compatibility rules. + Keep SDK format selection, HTTP/SSE framing, limits, cancellation and resource + cleanup here. Pin dependency versions and qualify upgrades with contract and + native engine tests; document upstream limitations instead of silently promising + lossless conversion. Preserve model identity, reject incomplete streams and + never redirect upstream credentials. See `contracts/agents-api/model-protocol-conversion.md` for coverage, + limits and dependency qualification. Go 1.26.8 is the pinned build toolchain. - Provider input validation uses the adapter-owned rules in `internal/harnessconfig`. Keep one internal registry for protocol and token-limit validation; Core owns credential environment and endpoint admission policy. These rules are not a @@ -3473,7 +3492,7 @@ Do not bypass them by dropping fields, changing model identity or fabricating us Operators may configure the daemon provider environment or the deployment default model provider for `claude_sdk` (HTTPS `base_url` and a write-only key), which Core freezes in the Session's encrypted snapshot and delivers as the adapter-owned -`claude_provider`; it never enters public Session configuration. The adapter exclusively selects the +`model_provider`; it never enters public Session configuration. The adapter exclusively selects the provider environment and removes credentials from native tool environments. Product `claude_code` and product execution are unchanged. The `none` public profile accepts only text, explicit model/system instructions, managed state, exact native resume and diff --git a/apps/docs/content/docs/api-reference/core/deployment-model-providers.mdx b/apps/docs/content/docs/api-reference/core/deployment-model-providers.mdx index 5f8f685ff..86fe0cb03 100644 --- a/apps/docs/content/docs/api-reference/core/deployment-model-providers.mdx +++ b/apps/docs/content/docs/api-reference/core/deployment-model-providers.mdx @@ -28,10 +28,11 @@ _openapi: Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this - harness: an HTTPS base_url without credentials, query or fragment, the - harness's protocol (responses for codex, anthropic for claude_sdk and - mcode) and, for mcode, positive context_window and max_output_tokens. - New openai_hosted and none Sessions that resolve no Session or Agent + harness: an HTTPS base_url without credentials, query or fragment, an + upstream protocol (responses, anthropic or chat_completions), + automatically adapted by Runtime to the selected harness and, for + mcode, positive context_window and max_output_tokens. New + openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator diff --git a/apps/docs/openapi/core-api.yaml b/apps/docs/openapi/core-api.yaml index 472e1f59e..5731f3f83 100644 --- a/apps/docs/openapi/core-api.yaml +++ b/apps/docs/openapi/core-api.yaml @@ -195,7 +195,7 @@ paths: tags: - Deployment Model Providers put: - description: 'Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, the harness''s protocol (responses for codex, anthropic for claude_sdk and mcode) and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key.' + description: 'Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, an upstream protocol (responses, anthropic or chat_completions), automatically adapted by Runtime to the selected harness and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key.' parameters: - schema: type: string @@ -5677,6 +5677,7 @@ components: enum: - anthropic - responses + - chat_completions type: string required: - api_key @@ -5697,6 +5698,7 @@ components: enum: - anthropic - responses + - chat_completions type: string required: - api_key_configured diff --git a/apps/docs/openapi/public-api.yaml b/apps/docs/openapi/public-api.yaml index 29898c58f..c7460d188 100644 --- a/apps/docs/openapi/public-api.yaml +++ b/apps/docs/openapi/public-api.yaml @@ -4397,6 +4397,7 @@ components: enum: - anthropic - responses + - chat_completions type: string required: - api_key @@ -4417,6 +4418,7 @@ components: enum: - anthropic - responses + - chat_completions type: string required: - api_key_configured diff --git a/apps/docs/openapi/sources.json b/apps/docs/openapi/sources.json index ddcaef8ac..37a1a38fa 100644 --- a/apps/docs/openapi/sources.json +++ b/apps/docs/openapi/sources.json @@ -1,7 +1,7 @@ { "sources": { - "contracts/agents-api/openapi.yaml": "a8896e8b67d78ed0608c7cc5636077ec33f4b4895a04bd6cfb8b038afe941e01", - "contracts/agents-api/core.openapi.yaml": "488a50fa308194224ee5ee879b7f6e38166d1b08dedc2bbc5bb39f38a844c87f", + "contracts/agents-api/openapi.yaml": "433a86baa0117791900f4d4c46490400b2fd00b0ca09eda1186a7b134aef24d7", + "contracts/agents-api/core.openapi.yaml": "5d6d13fdafa55c8459a61004aca6317c1d45de244df75dbfd83a589e813f2a6d", "contracts/agents-api/runtime.openapi.yaml": "b3fb025b20a0bcf4635d60339479b2b07f6dbed80f51e72f40a63b31e42b5f4a" }, "outputs": { @@ -18,11 +18,11 @@ "content/docs/api-reference/skills.mdx": "774ea05a7a8250fe5af5164388b98dafca99d56dc2eae686e8f628acb4e7c10b", "content/docs/api-reference/vaults.mdx": "d80eade1c2f8100030866abbb0fb6b7a09c94250a859c97cdf1abce32fdfdf1e", "content/docs/api-reference/credentials.mdx": "f129a51f74a3c7021c016600518e2350023f8d29571666ac5d64b7584cd7c7d0", - "openapi/public-api.yaml": "84ae8a33717c817950d757990018edd10a4d31d5a343c8172baa099d101c5a1f", + "openapi/public-api.yaml": "54759eb63257a9b4ca791f6cabcae4b9bf3cff791071f599f525cd76bb41f385", "content/docs/api-reference/meta.json": "56bf1dc145ccc8adde38c5f956f1a06f18ef18b646122250d01a294c516f8f83", "content/docs/api-reference/index.mdx": "9fd6b1c54149874d2999235067c101326f9e8a2e707a70b6672c81eb3ebfe63f", "content/docs/api-reference/core/core-administration.mdx": "a1261c7998017de3761ca90ca073a0b89a6c528ff37a634f3d8b986ee9540fc5", - "content/docs/api-reference/core/deployment-model-providers.mdx": "b2795fadcc05a1e7195b6d34c3f234cb29cc5e3f580d1521e31965453e654347", + "content/docs/api-reference/core/deployment-model-providers.mdx": "be4582d72ad10b225cb80c27cc8df2396e28fb2b5e857f7f0e20353a4425cd13", "content/docs/api-reference/core/administrator-projects.mdx": "8fd687bb2a7119fa16e938d7c71710c2816afb9cf959eb6849ef5e6300c5419b", "content/docs/api-reference/core/agents.mdx": "b6c089fbea4b07bc4249e7c7ca5699d47f1512a4fc2d8ee01cb26910ae541a84", "content/docs/api-reference/core/environment-templates.mdx": "05907f35d7f02d66af9da8b1efaa3765badb4cdf7b94d0f6c3ab0413a2cfa1a9", @@ -40,7 +40,7 @@ "content/docs/api-reference/core/vaults.mdx": "12d0aac96cd4a3472ef4eaa287f7945f78862017956e79bfc3ed0ef8e772e78a", "content/docs/api-reference/core/credentials.mdx": "27d586b9a8fd738b9947802f32c80458195370252886bc26abedccf2125ee981", "content/docs/api-reference/core/sandbox-manager.mdx": "36f2895b9969d27be843d5cebf935bf14ca9400ad076e56074a25905e3df5d1b", - "openapi/core-api.yaml": "9872f80cec2661c8539b6652a294c16c668f581399afd20f76b198279faae9ed", + "openapi/core-api.yaml": "fd969e19b1f8f310abafa577ed3228a67ba5878790fe80de6d5035593684192c", "content/docs/api-reference/core/meta.json": "588b163ed2de2100b0f8376688274f80698a6bf6ac037eef56d17c93fa8a5062", "content/docs/api-reference/core/index.mdx": "ddf0a8d87a6c21761beac703a32f2a40486a1c74abb341bd726f40c44a866405", "content/docs/api-reference/machine/sandbox-node.mdx": "5b367a08d43cea82686345d2ba1c75cc90627a7b2970fe42a0df9d77fb3e1e80", diff --git a/apps/parsar-daemon/internal/agent/claudesdk/executor.go b/apps/parsar-daemon/internal/agent/claudesdk/executor.go index 0abc0ca47..74e2e6131 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/executor.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/executor.go @@ -12,17 +12,19 @@ import ( "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/internal/modeltransport" ) type executor struct { - mu sync.Mutex - base *session - start startRequest - active *session - ready chan error - done chan struct{} - invalid bool - nativeID string + modelEndpoint *modeltransport.Endpoint + mu sync.Mutex + base *session + start startRequest + active *session + ready chan error + done chan struct{} + invalid bool + nativeID string } func NewExecutorFactory(config Config) agent.ExecutorFactory { @@ -40,6 +42,17 @@ func NewExecutorFactory(config Config) agent.ExecutorFactory { if req.RunID != "" || len(req.Input) != 0 || req.ConversationID != "" { return nil, errors.New("claudesdk: Executor preparation cannot submit input") } + options, endpoint, err := modeltransport.PrepareOptions(req.AgentOptions, modeltransport.Anthropic) + if err != nil { + return nil, err + } + transferred := false + defer func() { + if !transferred && endpoint != nil { + _ = endpoint.Close() + } + }() + req.AgentOptions = options start, env, err := prepareConfiguration(config, req) if err != nil { return nil, err @@ -58,7 +71,8 @@ func NewExecutorFactory(config Config) agent.ExecutorFactory { } base.reads.supported = slices.Contains(info.Features, "workspace_read") base.directories.supported = slices.Contains(info.Features, "workspace_directory") - e := &executor{base: base, start: start, ready: make(chan error, 1), done: make(chan struct{}), nativeID: start.Resume} + e := &executor{modelEndpoint: endpoint, base: base, start: start, ready: make(chan error, 1), done: make(chan struct{}), nativeID: start.Resume} + transferred = true go e.read() if err = e.write(start); err == nil { select { @@ -177,6 +191,9 @@ func (e *executor) read() { if !ready { e.ready <- readyFailure } + if e.modelEndpoint != nil { + _ = e.modelEndpoint.Close() + } close(e.done) } diff --git a/apps/parsar-daemon/internal/agent/claudesdk/executor_test.go b/apps/parsar-daemon/internal/agent/claudesdk/executor_test.go index 39d18f03a..b3fef4abc 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/executor_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/executor_test.go @@ -6,6 +6,7 @@ import ( "bufio" "context" "encoding/json" + "net/http" "os" "path/filepath" "strconv" @@ -144,14 +145,35 @@ func awaitExecutorTurn(t *testing.T, turn agent.Turn, out <-chan proto.Envelope, } func TestExecutorRetainsProcessAcrossTurnsAndCancellation(t *testing.T) { config, req := persistentConfig(t, "") + req.AgentOptions["model_provider"] = map[string]any{ + "protocol": "chat_completions", "base_url": "https://provider.example/v1", "api_key": "fixture-key", + } owner, err := NewExecutorFactory(config)(t.Context(), req) if err != nil { t.Fatal(err) } defer owner.Close(context.Background()) pid := owner.(*executor).base.process.Cmd.Process.Pid + endpoint := owner.(*executor).modelEndpoint + if endpoint == nil { + t.Fatal("missing model conversion endpoint") + } + client := &http.Client{Timeout: time.Second} + checkEndpoint := func() { + t.Helper() + response, err := client.Get(endpoint.BaseURL + "/v1/messages") + if err != nil { + t.Fatal("model endpoint closed before Executor", err) + } + response.Body.Close() + if response.StatusCode != http.StatusUnauthorized { + t.Fatal("model endpoint lost its credential boundary") + } + } + checkEndpoint() first, out := consumeExecutorTurn(t, owner, "first", "hello") awaitExecutorTurn(t, first, out, true) + checkEndpoint() next, out := consumeExecutorTurn(t, owner, "next", "wait") if event := <-out; event.Type != proto.TypeDelta { t.Fatal(event.Type) @@ -163,6 +185,7 @@ func TestExecutorRetainsProcessAcrossTurnsAndCancellation(t *testing.T) { t.Fatal(err) } awaitExecutorTurn(t, next, out, true) + checkEndpoint() third, out := consumeExecutorTurn(t, owner, "third", "hello") awaitExecutorTurn(t, third, out, true) if owner.(*executor).base.process.Cmd.Process.Pid != pid { @@ -173,6 +196,14 @@ func TestExecutorRetainsProcessAcrossTurnsAndCancellation(t *testing.T) { t.Fatal("native process exited between Turns") default: } + checkEndpoint() + if err := owner.Close(t.Context()); err != nil { + t.Fatal(err) + } + if response, err := client.Get(endpoint.BaseURL + "/v1/messages"); err == nil { + response.Body.Close() + t.Fatal("model endpoint survived Executor.Close") + } } func TestExecutorLateTurnEventInvalidatesWithoutRetargeting(t *testing.T) { config, req := persistentConfig(t, "late") diff --git a/apps/parsar-daemon/internal/agent/claudesdk/local_test.go b/apps/parsar-daemon/internal/agent/claudesdk/local_test.go index 51c7ca7b0..32c2b796f 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/local_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/local_test.go @@ -63,7 +63,7 @@ func TestWorkspaceProviderCredentialsReplaceAmbientSelection(t *testing.T) { config := workspaceFixture(t) original := slices.Clone(config.Env) req := workspaceRequest() - req.AgentOptions["claude_provider"] = map[string]any{"base_url": "https://provider.example/anthropic", "bearer_token": "selected-secret"} + req.AgentOptions["model_provider"] = map[string]any{"protocol": "anthropic", "base_url": "https://provider.example/anthropic", "api_key": "selected-secret"} start, env, err := prepare(config, req) if err != nil { t.Fatal(err) @@ -75,8 +75,8 @@ func TestWorkspaceProviderCredentialsReplaceAmbientSelection(t *testing.T) { if !slices.Contains(env, "ANTHROPIC_AUTH_TOKEN=selected-secret") || slices.Contains(env, "ANTHROPIC_AUTH_TOKEN=selected-provider-fixture") { t.Fatal("provider selection was not exclusive") } - for _, value := range []any{nil, "secret", map[string]any{"base_url": "http://provider.example", "bearer_token": "secret"}, map[string]any{"base_url": "https://user:pass@provider.example", "bearer_token": "secret"}} { - req.AgentOptions["claude_provider"] = value + for _, value := range []any{nil, "secret", map[string]any{"protocol": "anthropic", "base_url": "http://provider.example", "api_key": "secret"}, map[string]any{"protocol": "anthropic", "base_url": "https://user:pass@provider.example", "api_key": "secret"}} { + req.AgentOptions["model_provider"] = value if _, _, err := prepare(config, req); err == nil || strings.Contains(err.Error(), "secret") { t.Fatal("unsafe provider accepted or disclosed") } diff --git a/apps/parsar-daemon/internal/agent/claudesdk/options.go b/apps/parsar-daemon/internal/agent/claudesdk/options.go index a9ecb8fee..36822c5f7 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/options.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/options.go @@ -105,7 +105,7 @@ func prepareConfiguration(config Config, req proto.PromptRequestPayload) (startR } var provider []string for name, raw := range req.AgentOptions { - if name == "claude_provider" { + if name == "model_provider" { var err error provider, err = providerEnvironment(raw) if err != nil { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/provider.go b/apps/parsar-daemon/internal/agent/claudesdk/provider.go index 45f6b3621..61d64b125 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/provider.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/provider.go @@ -1,32 +1,19 @@ package claudesdk import ( - "fmt" - "net/url" "strings" + + "github.com/MiniMax-AI-Dev/parsar/internal/modeltransport" ) -// Provider options are transient operator input, never public Agent fields or -// native tool environment. The workspace sandbox removes these variables. +// The factory prepares the native endpoint before producing launch variables. +// Upstream credentials remain in Runtime when translation is required. func providerEnvironment(value any) ([]string, error) { - fail := func() ([]string, error) { return nil, fmt.Errorf("claudesdk: invalid provider configuration") } - options, ok := value.(map[string]any) - if !ok || len(options) != 2 { - return fail() - } - base, ok := options["base_url"].(string) - if !ok { - return fail() - } - token, ok := options["bearer_token"].(string) - if !ok || strings.TrimSpace(token) == "" || strings.ContainsAny(token, "\x00\r\n") { - return fail() - } - u, err := url.Parse(base) - if err != nil || u.Scheme != "https" || u.Host == "" || u.User != nil || u.RawQuery != "" || u.Fragment != "" { - return fail() + provider, err := modeltransport.ParseProvider(value) + if err != nil || provider.Protocol != modeltransport.Anthropic { + return nil, modeltransport.ErrConfiguration } - return []string{"ANTHROPIC_BASE_URL=" + base, "ANTHROPIC_AUTH_TOKEN=" + token}, nil + return []string{"ANTHROPIC_BASE_URL=" + provider.BaseURL, "ANTHROPIC_AUTH_TOKEN=" + provider.APIKey}, nil } func withProvider(env, provider []string) []string { diff --git a/apps/parsar-daemon/internal/agent/codex/executor_native_test.go b/apps/parsar-daemon/internal/agent/codex/executor_native_test.go index 5d4ecbc97..22d381544 100644 --- a/apps/parsar-daemon/internal/agent/codex/executor_native_test.go +++ b/apps/parsar-daemon/internal/agent/codex/executor_native_test.go @@ -52,7 +52,7 @@ func TestExecutorNativeReuse(t *testing.T) { req := proto.PromptRequestPayload{ AgentKind: "codex", AgentStateKey: "executor-native", WorkDir: filepath.Join(isolated, "workspace"), StrictResume: true, DisableExecutionEnvironment: true, DisableSubagents: true, ObserveMessages: true, - AgentOptions: map[string]any{"model": model, "codex_provider": map[string]any{"base_url": endpoint, "wire_api": "responses", "bearer_token": strings.TrimSpace(string(key))}}, + AgentOptions: map[string]any{"model": model, "model_provider": map[string]any{"base_url": endpoint, "protocol": "responses", "api_key": strings.TrimSpace(string(key))}}, FunctionTools: []proto.FunctionTool{{Name: "hold", Description: "Wait until the host supplies a result.", Parameters: json.RawMessage("{\"type\":\"object\",\"properties\":{},\"additionalProperties\":false}")}}, } ctx, cancel := context.WithTimeout(t.Context(), 4*time.Minute) diff --git a/apps/parsar-daemon/internal/agent/codex/options.go b/apps/parsar-daemon/internal/agent/codex/options.go index 001a9c837..a08c7e8f9 100644 --- a/apps/parsar-daemon/internal/agent/codex/options.go +++ b/apps/parsar-daemon/internal/agent/codex/options.go @@ -10,6 +10,7 @@ import ( "strings" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/paths" + "github.com/MiniMax-AI-Dev/parsar/internal/modeltransport" ) // SessionPlan holds the resolved per-prompt launch plan derived from @@ -79,23 +80,7 @@ type SessionPlan struct { // env map[string]any extra env vars (KEY=string-value) // mcp_servers map[string]any rendered MCP server config — written // to /config.toml [mcp_servers] -// codex_provider map[string]any full provider config — written to -// /config.toml -// [model_providers.]. -// Required for any real prompt; when -// missing, codex falls back to its -// builtin "openai" provider which only -// speaks api.openai.com. -// Recognised keys: name (string), -// base_url (string, required when -// codex_provider is present), -// bearer_token (string, required), -// wire_api (string; defaults to -// "responses"), -// http_headers (map[string]string), -// query_params (map[string]string), -// request_max_retries (number), -// stream_max_retries (number). +// model_provider object frozen upstream protocol, endpoint, credential and token limits // reasoning_summary string one of auto/concise/detailed/none — // routed via -c model_reasoning_summary // mode string Codex collaboration mode: default/plan @@ -181,17 +166,23 @@ func BuildSessionPlan(runID, agentStateKey, workDir string, opts map[string]any) } } - // codex_provider carries the full ModelProviderInfo the server-side - // injectCodexManagedModel resolved. When set, we materialise it into - // the [model_providers.] block and pin - // thread/start.model_provider to that slug, so codex skips its - // builtin "openai" provider entirely. - provider, hasProvider, err := normaliseProviderConfig(opts["codex_provider"]) + // The native adapter selects Responses; Runtime converts other upstream protocols. + preparedOptions, endpoint, err := modeltransport.PrepareOptions(opts, modeltransport.Responses) if err != nil { return plan, err } + if endpoint != nil { + cleanup = func() { _ = endpoint.Close() } + plan.Cleanup = cleanup + } + provider, hasProvider, err := normaliseProviderConfig(preparedOptions["model_provider"]) + if err != nil { + cleanup() + return plan, err + } if hasProvider { if err := writeCodexProviderConfig(codexHome, provider); err != nil { + cleanup() return plan, err } plan.ModelProvider = oacProviderSlug @@ -383,55 +374,16 @@ func normaliseMCPServers(raw any) (map[string]mcpServerConfig, error) { return out, nil } -// normaliseProviderConfig flattens agent_options["codex_provider"] (a -// string-keyed map produced by injectCodexManagedModel) into a typed -// providerConfig. Returns hasProvider=false when the key is absent, so -// BuildSessionPlan can skip the TOML write entirely; that path is only -// exercised by tests that don't care about model auth. -// -// base_url + bearer_token are validated by writeCodexProviderConfig -// itself (single source of truth) so this function only normalises -// shapes. +// normaliseProviderConfig receives the adapter's prepared native endpoint. func normaliseProviderConfig(raw any) (providerConfig, bool, error) { if raw == nil { return providerConfig{}, false, nil } - m, ok := raw.(map[string]any) - if !ok { - return providerConfig{}, false, fmt.Errorf("codex: codex_provider must be object, got %T", raw) - } - cfg := providerConfig{} - if v, ok := m["name"].(string); ok { - cfg.Name = v - } - if v, ok := m["base_url"].(string); ok { - cfg.BaseURL = v - } - if v, ok := m["bearer_token"].(string); ok { - cfg.BearerToken = v - } - if v, ok := m["wire_api"].(string); ok { - cfg.WireAPI = v - } - if hdrs, ok := m["http_headers"].(map[string]any); ok { - cfg.HTTPHeaders = make(map[string]string, len(hdrs)) - for k, v := range hdrs { - if s, ok := v.(string); ok { - cfg.HTTPHeaders[k] = s - } - } - } - if params, ok := m["query_params"].(map[string]any); ok { - cfg.QueryParams = make(map[string]string, len(params)) - for k, v := range params { - if s, ok := v.(string); ok { - cfg.QueryParams[k] = s - } - } + provider, err := modeltransport.ParseProvider(raw) + if err != nil || provider.Protocol != modeltransport.Responses { + return providerConfig{}, false, modeltransport.ErrConfiguration } - cfg.RequestMaxRetries = intOpt(m, "request_max_retries") - cfg.StreamMaxRetries = intOpt(m, "stream_max_retries") - return cfg, true, nil + return providerConfig{BaseURL: provider.BaseURL, BearerToken: provider.APIKey, WireAPI: "responses"}, true, nil } // intOpt extracts an integer-shaped value from a map. JSON-decoded diff --git a/apps/parsar-daemon/internal/agent/codex/provider_config.go b/apps/parsar-daemon/internal/agent/codex/provider_config.go index 1693943d4..b3c571e0d 100644 --- a/apps/parsar-daemon/internal/agent/codex/provider_config.go +++ b/apps/parsar-daemon/internal/agent/codex/provider_config.go @@ -18,8 +18,8 @@ import ( const oacProviderSlug = "oac" // providerConfig is the daemon-internal view of the model provider the -// server-side injector resolved for this prompt. Flattened from -// agent_options["codex_provider"] (a string-keyed map) into a typed +// Runtime prepared for this Session. Flattened from the common +// agent_options["model_provider"] bundle into a typed // struct before TOML emission. Field names mirror the codex-rs // ModelProviderInfo enum (model-provider-info/src/lib.rs) so the // rendered TOML deserialises 1:1 against upstream. diff --git a/apps/parsar-daemon/internal/agent/codex/provider_config_test.go b/apps/parsar-daemon/internal/agent/codex/provider_config_test.go index ec36e2718..133716715 100644 --- a/apps/parsar-daemon/internal/agent/codex/provider_config_test.go +++ b/apps/parsar-daemon/internal/agent/codex/provider_config_test.go @@ -132,38 +132,15 @@ func TestWriteCodexProviderConfig_AppendsAlongsideMCP(t *testing.T) { } } -func TestNormaliseProviderConfig_FullRoundTrip(t *testing.T) { - raw := map[string]any{ - "name": "mygw", - "base_url": "https://x/v1", - "bearer_token": "sk-x", - "wire_api": "responses", - "http_headers": map[string]any{ - "X-Sub-Module": "codex-internal", - }, - "query_params": map[string]any{ - "api-version": "2025-04-01-preview", - }, - "request_max_retries": float64(4), // JSON numbers arrive as float64 - } - cfg, hasProvider, err := normaliseProviderConfig(raw) - if err != nil { - t.Fatalf("normalise: %v", err) - } - if !hasProvider { - t.Fatal("hasProvider must be true for non-nil raw") - } - if cfg.Name != "mygw" || cfg.BaseURL != "https://x/v1" || cfg.BearerToken != "sk-x" { - t.Fatalf("scalar fields wrong: %+v", cfg) - } - if cfg.HTTPHeaders["X-Sub-Module"] != "codex-internal" { - t.Fatalf("headers lost: %+v", cfg.HTTPHeaders) - } - if cfg.QueryParams["api-version"] != "2025-04-01-preview" { - t.Fatalf("query_params lost: %+v", cfg.QueryParams) +func TestNormaliseProviderConfig_FixedBundle(t *testing.T) { + raw := map[string]any{"protocol": "responses", "base_url": "https://x/v1", "api_key": "sk-x"} + cfg, present, err := normaliseProviderConfig(raw) + if err != nil || !present || cfg.BaseURL != "https://x/v1" || cfg.BearerToken != "sk-x" || cfg.WireAPI != "responses" { + t.Fatal("provider not preserved", err) } - if cfg.RequestMaxRetries != 4 { - t.Fatalf("request_max_retries = %d, want 4", cfg.RequestMaxRetries) + raw["http_headers"] = map[string]any{"Authorization": "other"} + if _, _, err := normaliseProviderConfig(raw); err == nil { + t.Fatal("undeclared configuration accepted") } } @@ -180,9 +157,10 @@ func TestNormaliseProviderConfig_Nil(t *testing.T) { func TestBuildSessionPlan_PinsModelProviderWhenProviderSet(t *testing.T) { plan, err := BuildSessionPlan("run-x", "conv-1/agent-1/codex", "", map[string]any{ - "codex_provider": map[string]any{ - "base_url": "https://x/v1", - "bearer_token": "sk-x", + "model": "fixture-model", + "model_provider": map[string]any{"protocol": "responses", + "base_url": "https://x/v1", + "api_key": "sk-x", }, }) if err != nil { diff --git a/apps/parsar-daemon/internal/agent/mcode/model_provider.go b/apps/parsar-daemon/internal/agent/mcode/model_provider.go new file mode 100644 index 000000000..c5057ca33 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/mcode/model_provider.go @@ -0,0 +1,34 @@ +package mcode + +import ( + "fmt" + + "github.com/MiniMax-AI-Dev/parsar/internal/modeltransport" +) + +func modelProviderConfig(raw any, model string) (map[string]any, error) { + provider, err := modeltransport.ParseProvider(raw) + if err != nil { + return nil, fmt.Errorf("mcode: %w", err) + } + if provider.ContextWindow <= 0 || provider.MaxOutputTokens <= 0 { + return nil, fmt.Errorf("mcode: positive model context_window and max_output_tokens are required") + } + var api string + switch provider.Protocol { + case modeltransport.Anthropic: + api = "anthropic-messages" + case modeltransport.ChatCompletions: + api = "openai-completions" + case modeltransport.Responses: + api = "openai-responses" + } + return map[string]any{ + "name": "Configured provider", "kind": "custom", "enabled": true, "api": api, + "options": map[string]any{"baseURL": provider.BaseURL, "apiKey": provider.APIKey}, + "models": map[string]any{model: map[string]any{ + "name": model, "tool_call": true, + "limit": map[string]any{"context": provider.ContextWindow, "output": provider.MaxOutputTokens}, + }}, + }, nil +} diff --git a/apps/parsar-daemon/internal/agent/mcode/model_provider_test.go b/apps/parsar-daemon/internal/agent/mcode/model_provider_test.go new file mode 100644 index 000000000..715bf181c --- /dev/null +++ b/apps/parsar-daemon/internal/agent/mcode/model_provider_test.go @@ -0,0 +1,84 @@ +package mcode + +import ( + "encoding/json" + "os" + "path/filepath" + "reflect" + "testing" +) + +func TestOptionsModelProviderProtocols(t *testing.T) { + for _, tc := range []struct{ protocol, api string }{ + {"anthropic", "anthropic-messages"}, {"responses", "openai-responses"}, {"chat_completions", "openai-completions"}, + } { + t.Run(tc.protocol, func(t *testing.T) { + req := testRequest(t) + req.AgentOptions["model_provider"].(map[string]any)["protocol"] = tc.protocol + req.AgentOptions["model"] = "chosen-model" + opts, err := prepareOptions(t.Context(), req) + if err != nil { + t.Fatal(err) + } + body, err := os.ReadFile(filepath.Join(opts.DataDir, "config.yaml")) + if err != nil { + t.Fatal(err) + } + var config map[string]any + if err := json.Unmarshal(body, &config); err != nil { + t.Fatal(err) + } + want := map[string]any{"oac": map[string]any{ + "name": "Configured provider", "kind": "custom", "enabled": true, "api": tc.api, + "options": map[string]any{"baseURL": "https://provider.example/v1", "apiKey": "fixture-key"}, + "models": map[string]any{"chosen-model": map[string]any{ + "name": "chosen-model", "tool_call": true, + "limit": map[string]any{"context": float64(64000), "output": float64(4096)}, + }}, + }} + if opts.Model != "chosen-model" || !reflect.DeepEqual(config["custom_provider"], want) { + t.Fatal("native provider configuration did not preserve the upstream bundle and selected model") + } + }) + } +} + +func TestOptionsRejectInvalidProvider(t *testing.T) { + for _, tc := range []struct { + name, field string + value any + }{ + {"unknown protocol", "protocol", "unknown"}, + {"protocol alias", "protocol", "chat-completions"}, + {"remote HTTP", "base_url", "http://provider.example/v1"}, + {"empty key", "api_key", ""}, + {"missing context", "context_window", 0}, + {"missing output", "max_output_tokens", 0}, + {"excess output", "max_output_tokens", 64001}, + {"native options", "options", map[string]any{"apiKey": "fixture-key"}}, + } { + t.Run(tc.name, func(t *testing.T) { + req := testRequest(t) + req.AgentOptions["model_provider"].(map[string]any)[tc.field] = tc.value + if _, err := prepareOptions(t.Context(), req); err == nil { + t.Fatal("invalid model provider accepted") + } + }) + } + t.Run("retired native option", func(t *testing.T) { + req := testRequest(t) + req.AgentOptions["mcode_provider"] = req.AgentOptions["model_provider"] + delete(req.AgentOptions, "model_provider") + if _, err := prepareOptions(t.Context(), req); err == nil { + t.Fatal("retired native provider option accepted") + } + }) +} + +func TestOptionsAllowLoopbackProviderFixture(t *testing.T) { + req := testRequest(t) + req.AgentOptions["model_provider"].(map[string]any)["base_url"] = "http://127.0.0.1:4321/v1" + if _, err := prepareOptions(t.Context(), req); err != nil { + t.Fatal(err) + } +} diff --git a/apps/parsar-daemon/internal/agent/mcode/native_test.go b/apps/parsar-daemon/internal/agent/mcode/native_test.go index 902017574..53e8f9251 100644 --- a/apps/parsar-daemon/internal/agent/mcode/native_test.go +++ b/apps/parsar-daemon/internal/agent/mcode/native_test.go @@ -97,7 +97,7 @@ func TestNativeMCodeACP(t *testing.T) { digest := sha256.Sum256(archive.Bytes()) skill := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write(archive.Bytes()) })) defer skill.Close() - req.AgentOptions["mcode_provider"] = map[string]any{"name": "OpenAgentCore", "kind": "custom", "enabled": true, "npm": "@ai-sdk/anthropic", "options": map[string]any{"apiKey": "fixture-only", "baseURL": model.URL}, "models": map[string]any{"fixture": map[string]any{"name": "Fixture", "tool_call": true, "limit": map[string]int{"context": 64000, "output": 4096}}}} + req.AgentOptions["model_provider"] = map[string]any{"protocol": "anthropic", "base_url": model.URL, "api_key": "fixture-only", "context_window": 64000, "max_output_tokens": 4096} req.AgentOptions["skills"] = []any{map[string]any{"name": "qa-mcode-skill", "version": "1", "download_url": skill.URL, "sha256": hex.EncodeToString(digest[:])}} req.AgentOptions["mcp_servers"] = map[string]any{"qa": map[string]any{"type": "http", "url": mcp.URL}} req.AgentOptions["system_prompt"] = "SP-MCODE-672: use the available tools when requested." @@ -151,10 +151,6 @@ func TestNativeMCodeACP(t *testing.T) { req.AgentSessionID = done.Metadata[proto.DoneMetaAgentSessionID].(string) req.AgentOptions["system_prompt"] = "SP-MCODE-NEW: reply concisely." req.AgentOptions["skills"] = []any{} - provider := req.AgentOptions["mcode_provider"].(map[string]any) - models := provider["models"].(map[string]any) - models["fixture-new"] = models["fixture"] - delete(models, "fixture") req.AgentOptions["model"] = "fixture-new" req.Input = proto.TextInput("Now reply OAC-MCODE-OK.") run() diff --git a/apps/parsar-daemon/internal/agent/mcode/options.go b/apps/parsar-daemon/internal/agent/mcode/options.go index 79b2cfd9f..507b560b6 100644 --- a/apps/parsar-daemon/internal/agent/mcode/options.go +++ b/apps/parsar-daemon/internal/agent/mcode/options.go @@ -74,15 +74,15 @@ func prepareOptionsWithSkills(ctx context.Context, req proto.PromptRequestPayloa return result, err } config := map[string]any{"logLevel": "error", "skills": map[string]any{"external": map[string]any{"enabled": false}}} - if provider, ok := opts["mcode_provider"].(map[string]any); ok { - config["custom_provider"] = map[string]any{"oac": provider} - } else { - return result, fmt.Errorf("mcode: a OpenAgentCore-managed model is required") - } result.Model = optionString(opts, "model") if result.Model == "" { return result, fmt.Errorf("mcode: model is required") } + provider, err := modelProviderConfig(opts["model_provider"], result.Model) + if err != nil { + return result, err + } + config["custom_provider"] = map[string]any{"oac": provider} if req.StrictResume { configureTextExecution(config) if !req.DisableSubagents { diff --git a/apps/parsar-daemon/internal/agent/mcode/options_test.go b/apps/parsar-daemon/internal/agent/mcode/options_test.go index e6e64afa6..a19117546 100644 --- a/apps/parsar-daemon/internal/agent/mcode/options_test.go +++ b/apps/parsar-daemon/internal/agent/mcode/options_test.go @@ -69,7 +69,7 @@ func TestOptionsRejectDroppedContext(t *testing.T) { r.Input = proto.MessageInput{{Content: []proto.InputContent{{Type: "input_image"}}}} }}, {"missing model", func(r *proto.PromptRequestPayload) { delete(r.AgentOptions, "model") }}, - {"missing provider", func(r *proto.PromptRequestPayload) { delete(r.AgentOptions, "mcode_provider") }}, + {"missing provider", func(r *proto.PromptRequestPayload) { delete(r.AgentOptions, "model_provider") }}, {"invalid permission mode", func(r *proto.PromptRequestPayload) { r.AgentOptions["mode"] = "plan" }}, } for _, tt := range tests { diff --git a/apps/parsar-daemon/internal/agent/mcode/session_test.go b/apps/parsar-daemon/internal/agent/mcode/session_test.go index 8d0fb9537..9fa42641f 100644 --- a/apps/parsar-daemon/internal/agent/mcode/session_test.go +++ b/apps/parsar-daemon/internal/agent/mcode/session_test.go @@ -19,7 +19,7 @@ func testRequest(t *testing.T) proto.PromptRequestPayload { t.Helper() t.Setenv("OAC_RUNTIME_HOME", t.TempDir()) return proto.PromptRequestPayload{RunID: "run-1", ConversationID: "conversation-1", AgentStateKey: "conversation-1/agent-1/mcode", Input: proto.TextInput("Hello"), AgentOptions: map[string]any{ - "model": "fixture", "mcode_provider": map[string]any{"kind": "custom", "enabled": true}, "system_prompt": "Current instructions", + "model": "fixture", "model_provider": map[string]any{"protocol": "anthropic", "base_url": "https://provider.example/v1", "api_key": "fixture-key", "context_window": 64000, "max_output_tokens": 4096}, "system_prompt": "Current instructions", }} } diff --git a/apps/parsar-daemon/internal/dispatch/functions_native_test.go b/apps/parsar-daemon/internal/dispatch/functions_native_test.go index b47d0aec9..ceb51c485 100644 --- a/apps/parsar-daemon/internal/dispatch/functions_native_test.go +++ b/apps/parsar-daemon/internal/dispatch/functions_native_test.go @@ -134,7 +134,7 @@ func TestNativeFunctionBridge(t *testing.T) { run := fmt.Sprintf("run-%d", index) request := proto.PromptRequestPayload{AgentKind: "codex", Input: proto.TextInput("Look up ticket 42."), RunID: run, AgentStateKey: "native-functions", AgentSessionID: nativeID, StrictResume: true, ReleaseOnCompletion: true, DisableExecutionEnvironment: true, ObserveTools: true, FunctionTools: []proto.FunctionTool{{Name: "lookup_ticket", Description: "Read a synthetic ticket", Parameters: json.RawMessage(`{"type":"object","properties":{"ticket":{"type":"string"}},"required":["ticket"],"additionalProperties":false}`)}}, - AgentOptions: map[string]any{"model": "gpt-5.5", "codex_provider": map[string]any{"base_url": model.URL + "/v1", "bearer_token": "synthetic-local-token"}}} + AgentOptions: map[string]any{"model": "gpt-5.5", "model_provider": map[string]any{"protocol": "responses", "base_url": model.URL + "/v1", "api_key": "synthetic-local-token"}}} env, _ := proto.NewEnvelope(proto.TypePromptRequest, run, request) if err := router.Handle(ctx, env); err != nil { t.Fatal(err) diff --git a/apps/web/e2e/fixture-console.mjs b/apps/web/e2e/fixture-console.mjs index e2b25125b..1a9b6f385 100644 --- a/apps/web/e2e/fixture-console.mjs +++ b/apps/web/e2e/fixture-console.mjs @@ -422,8 +422,6 @@ function registeredNode(nodeId) { const HARNESS_PROVIDER = /^\/harnesses\/([^/]+)\/model-provider$/; const PROVIDER_FIELDS = new Set(["protocol", "base_url", "api_key", "context_window", "max_output_tokens"]); -/** Each harness's protocol, as Core's registry declares it; only mcode requires token limits. */ -const HARNESS_PROTOCOL = { claude_sdk: "anthropic", codex: "responses", mcode: "anthropic" }; /** Core's one message for a body that is not a complete provider; it never echoes a value. */ const PROVIDER_SHAPE = "The body must be a complete model provider: protocol, base_url, api_key and optional nonnegative context_window and max_output_tokens."; const httpsBase = (value) => { try { const url = new URL(value); return url.protocol === "https:" && Boolean(url.hostname) && !url.username && !url.password && !url.search && !url.hash; } catch { return false; } }; @@ -436,10 +434,9 @@ function providerProblem(harness, input) { if (Object.keys(input).some((key) => !PROVIDER_FIELDS.has(key)) || ["protocol", "base_url", "api_key"].some((key) => typeof input[key] !== "string") || !tokenLimit(input.context_window) || !tokenLimit(input.max_output_tokens)) return PROVIDER_SHAPE; if (!httpsBase(input.base_url)) return "model provider requires an HTTPS base_url without credentials, query or fragment"; - if (input.protocol !== "anthropic" && input.protocol !== "responses") return "unsupported model provider protocol"; + if (!["anthropic", "responses", "chat_completions"].includes(input.protocol)) return "unsupported model provider protocol"; if (!input.api_key.trim() || Buffer.byteLength(input.api_key) > 16384 || /[\0\r\n]/.test(input.api_key)) return "invalid model provider API key"; if ((input.max_output_tokens ?? 0) > (input.context_window ?? 0)) return "invalid model token limits"; - if (input.protocol !== HARNESS_PROTOCOL[harness]) return "selected harness does not support this model provider protocol"; if (harness === "mcode" && !(input.context_window > 0 && input.max_output_tokens > 0)) return "selected harness requires positive model context_window and max_output_tokens"; return null; } diff --git a/apps/web/e2e/system.spec.ts b/apps/web/e2e/system.spec.ts index a61336473..52e485da6 100644 --- a/apps/web/e2e/system.spec.ts +++ b/apps/web/e2e/system.spec.ts @@ -20,6 +20,11 @@ test("sets, replaces and clears a harness's default model provider, and keeps it await codex.getByRole("button", { name: "Set the default model provider for Codex" }).click(); const set = page.getByRole("dialog", { name: "Set default model provider for Codex" }); + await expect(set.getByRole("combobox", { name: "Protocol", exact: true })).toHaveText("OpenAI Responses"); + await set.getByRole("combobox", { name: "Protocol", exact: true }).click(); + await expect(page.getByRole("option")).toHaveText(["Anthropic Messages", "OpenAI Responses", "OpenAI Chat Completions"]); + await page.screenshot({ path: test.info().outputPath("provider-protocol-options.png"), animations: "disabled" }); + await page.getByRole("option", { name: "OpenAI Chat Completions", exact: true }).click(); await set.getByLabel("Base URL").fill("https://model.example/v1"); await set.getByLabel("API key").fill(KEY); await expect(set).toContainText("Your application specifies the model in the Agent’s model field."); @@ -46,20 +51,26 @@ test("sets, replaces and clears a harness's default model provider, and keeps it await set.getByLabel("Context window").fill("200000"); await set.getByRole("button", { name: "Save" }).click(); await expect(set).toBeHidden(); - for (const text of ["OpenAI Responses", "https://model.example/v1", "Configured", "200,000", "32,000"]) await expect(codex).toContainText(text); + for (const text of ["OpenAI Chat Completions", "https://model.example/v1", "Configured", "200,000", "32,000"]) await expect(codex).toContainText(text); expect(await page.content()).not.toContain(KEY); // Replacing starts from the saved fields but never from the key; closing the form forgets a typed key. await codex.getByRole("button", { name: "Replace the default model provider for Codex" }).click(); let replace = page.getByRole("dialog", { name: "Replace default model provider for Codex" }); + await expect(replace.getByRole("combobox", { name: "Protocol", exact: true })).toHaveText("OpenAI Chat Completions"); await expect(replace.getByLabel("Base URL")).toHaveValue("https://model.example/v1"); await expect(replace.getByLabel("API key")).toHaveValue(""); await expect(replace.getByRole("button", { name: "Save" })).toBeDisabled(); await replace.getByLabel("API key").fill(KEY); + await replace.getByRole("combobox", { name: "Protocol", exact: true }).click(); + await page.getByRole("option", { name: "Anthropic Messages", exact: true }).click(); await replace.getByRole("button", { name: "Cancel" }).click(); expect(await page.content()).not.toContain(KEY); await codex.getByRole("button", { name: "Replace the default model provider for Codex" }).click(); replace = page.getByRole("dialog", { name: "Replace default model provider for Codex" }); + await expect(replace.getByRole("combobox", { name: "Protocol", exact: true })).toHaveText("OpenAI Chat Completions"); + await replace.getByRole("combobox", { name: "Protocol", exact: true }).click(); + await page.getByRole("option", { name: "Anthropic Messages", exact: true }).click(); await expect(replace.getByLabel("API key")).toHaveValue(""); await replace.getByLabel("Base URL").fill("https://model.example/v2"); await replace.getByLabel("API key").fill(KEY); @@ -67,6 +78,7 @@ test("sets, replaces and clears a harness's default model provider, and keeps it await replace.getByLabel("API key").press("Enter"); await expect(replace).toBeHidden(); await expect(codex).toContainText("https://model.example/v2"); + await expect(codex).toContainText("Anthropic Messages"); await codex.getByRole("button", { name: "Clear the default model provider for Codex" }).click(); const confirm = page.getByRole("dialog", { name: "Clear default model provider" }); @@ -77,6 +89,9 @@ test("sets, replaces and clears a harness's default model provider, and keeps it // MiniMax Code needs both limits; the form shows Core's reason. A disabled harness may still be configured. await mcode.getByRole("button", { name: "Set the default model provider for MiniMax Code" }).click(); const limits = page.getByRole("dialog", { name: "Set default model provider for MiniMax Code" }); + await expect(limits.getByRole("combobox", { name: "Protocol", exact: true })).toHaveText("Anthropic Messages"); + await limits.getByRole("combobox", { name: "Protocol", exact: true }).click(); + await page.getByRole("option", { name: "OpenAI Responses", exact: true }).click(); await limits.getByLabel("Base URL").fill("https://model.example/anthropic"); await limits.getByLabel("API key").fill(KEY); await limits.getByRole("button", { name: "Save" }).click(); diff --git a/apps/web/src/features/system/DefaultModelsSection.tsx b/apps/web/src/features/system/DefaultModelsSection.tsx index fa22b6cc7..62bb813d9 100644 --- a/apps/web/src/features/system/DefaultModelsSection.tsx +++ b/apps/web/src/features/system/DefaultModelsSection.tsx @@ -3,6 +3,7 @@ import { useQuery, useQueryClient } from "@tanstack/react-query"; import { useCallback, useId, useRef, useState, type ReactNode } from "react"; import { useTranslation } from "react-i18next"; +import { ConsoleSelect } from "../../components/console-select"; import { ConfirmDialog } from "../../components/ConfirmDialog"; import { EmptyState, HelpTip, revealInPageBody, Section, StatusDot } from "../../components/console-ui"; import { ErrorState } from "../../components/ErrorState"; @@ -21,8 +22,9 @@ import { harnessesQuery } from "./harness-queries"; const WRITE_TIMEOUT_MS = 30_000; type Protocol = ModelProviderInput["protocol"]; -/** The one protocol Core accepts for each harness; the form sends it and cannot change it. */ -const harnessProtocol: Record = { codex: "responses", claude_sdk: "anthropic", mcode: "anthropic" }; +/** New providers start with the harness native protocol; saved providers keep their upstream protocol. */ +const defaultHarnessProtocol: Record = { codex: "responses", claude_sdk: "anthropic", mcode: "anthropic" }; +const protocolOptions = (["anthropic", "responses", "chat_completions"] as const).map((value) => ({ value, label: protocolNames[value] })); /** Core stores both token limits as 32-bit integers. */ const INT32_MAX = 2_147_483_647; @@ -189,8 +191,8 @@ function HarnessCard({ harness, busy, onEdit, onClear }: { harness: CoreHarness; /** * Sets or replaces one harness's provider. Non-secret fields start from the - * current provider; the API key never does. The protocol is the one Core - * accepts for the harness. The form checks the HTTPS provider URL and whole-number + * current provider; the API key never does. The protocol describes the upstream + * model provider. The form checks the HTTPS provider URL and whole-number * limits within Core's range, with max output no larger than the context window. * Core's other rules come back as its 400 message, shown * beside the form. Enter saves; a save in flight blocks another. @@ -206,6 +208,7 @@ function ModelProviderDialog({ harness, onClose, onSaved, onReread }: { const id = useId(); const formId = `${id}-form`; const current = harness?.model_provider ?? null; + const [protocol, setProtocol] = useState(current?.protocol ?? (harness ? defaultHarnessProtocol[harness.id] : "responses")); const [baseUrl, setBaseUrl] = useState(current?.base_url ?? ""); const [apiKey, setApiKey] = useState(""); const [contextWindow, setContextWindow] = useState(current?.context_window === undefined ? "" : String(current.context_window)); @@ -215,7 +218,6 @@ function ModelProviderDialog({ harness, onClose, onSaved, onReread }: { const [error, setError] = useState(null); const name = harness ? harnessNames[harness.id] : ""; - const protocol = harness ? harnessProtocol[harness.id] : "responses"; const limitsRequired = harness?.id === "mcode"; const url = baseUrl.trim(); const urlProblem = url && !isProviderUrl(url) ? t("models.form.baseUrlInvalid") : null; @@ -298,7 +300,10 @@ function ModelProviderDialog({ harness, onClose, onSaved, onReread }: { {t("models.protocol")} {t("models.form.protocolHelp")} -

{protocolNames[protocol]}

+ { + const option = protocolOptions.find((option) => option.value === value); + if (option) setProtocol(option.value); + }} />
diff --git a/apps/web/src/i18n/locales/en/system.ts b/apps/web/src/i18n/locales/en/system.ts index 7d03f61b8..9fa3add9c 100644 --- a/apps/web/src/i18n/locales/en/system.ts +++ b/apps/web/src/i18n/locales/en/system.ts @@ -74,7 +74,7 @@ export const system = { contextHelpRequired: "The model's context window, in tokens. MiniMax Code needs it.", outputHelp: "The most tokens the model writes in one response. Optional; it needs a context window at least as large.", outputHelpRequired: "The most tokens the model writes in one response. MiniMax Code needs it, with a context window at least as large.", - protocolHelp: "Each harness speaks one protocol; Core accepts no other.", + protocolHelp: "Choose the protocol supported by your model provider. Runtime handles protocol conversion automatically.", wholeNumber: "Enter a whole number.", tooLarge: "Enter a whole number up to 2147483647.", needsContext: "Set a context window at least this large.", diff --git a/apps/web/src/i18n/locales/zh-CN/system.ts b/apps/web/src/i18n/locales/zh-CN/system.ts index f4d044f81..4dd4a2745 100644 --- a/apps/web/src/i18n/locales/zh-CN/system.ts +++ b/apps/web/src/i18n/locales/zh-CN/system.ts @@ -76,7 +76,7 @@ export const system: TranslationShape = { contextHelpRequired: "模型的上下文窗口,单位为 token。MiniMax Code 必须填写。", outputHelp: "模型单次回复最多输出的 token 数。可选;填写时,上下文窗口必须不小于它。", outputHelpRequired: "模型单次回复最多输出的 token 数。MiniMax Code 必须填写,且上下文窗口不能小于它。", - protocolHelp: "每个执行框架只使用一种协议,Core 不接受其他协议。", + protocolHelp: "选择模型服务支持的协议,Runtime 会自动完成协议转换。", wholeNumber: "请输入整数。", tooLarge: "请输入不超过 2147483647 的整数。", needsContext: "请设置不小于该值的上下文窗口。", diff --git a/apps/web/src/lib/harness-labels.ts b/apps/web/src/lib/harness-labels.ts index 4fb0fc074..a81dd8c9e 100644 --- a/apps/web/src/lib/harness-labels.ts +++ b/apps/web/src/lib/harness-labels.ts @@ -4,4 +4,4 @@ import type { CoreHarnessKind, ModelProviderView } from "@agents-core-web/agents export const harnessNames: Record = { claude_sdk: "Claude Code", codex: "Codex", mcode: "MiniMax Code" }; /** Provider protocols by their product names, the same in every language. */ -export const protocolNames: Record = { anthropic: "Anthropic Messages", responses: "OpenAI Responses" }; +export const protocolNames: Record = { anthropic: "Anthropic Messages", responses: "OpenAI Responses", chat_completions: "OpenAI Chat Completions" }; diff --git a/contracts/agents-api/core.openapi.yaml b/contracts/agents-api/core.openapi.yaml index d857e9a2d..765c8a0a3 100644 --- a/contracts/agents-api/core.openapi.yaml +++ b/contracts/agents-api/core.openapi.yaml @@ -1667,6 +1667,7 @@ definitions: enum: - anthropic - responses + - chat_completions type: string required: - api_key @@ -1687,6 +1688,7 @@ definitions: enum: - anthropic - responses + - chat_completions type: string required: - api_key_configured @@ -3178,7 +3180,7 @@ paths: put: consumes: - application/json - description: 'Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, the harness''s protocol (responses for codex, anthropic for claude_sdk and mcode) and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key.' + description: 'Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, an upstream protocol (responses, anthropic or chat_completions), automatically adapted by Runtime to the selected harness and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key.' parameters: - description: Harness enum: diff --git a/contracts/agents-api/model-execution.md b/contracts/agents-api/model-execution.md index 90c5d325d..7dbc4653b 100644 --- a/contracts/agents-api/model-execution.md +++ b/contracts/agents-api/model-execution.md @@ -24,8 +24,9 @@ requires a model for an inline Agent and when creating a saved Agent. There is n model-name inference. Provider precedence is: complete Session bundle, complete saved bundle, then the deployment default for the resolved harness. Never merge a replacement endpoint with an inherited key. A model-only override reuses the entire -inherited bundle. Codex requires `responses`; Claude SDK and MiniMax Code require -`anthropic`, with positive context/output limits for MiniMax Code. Validate the +inherited bundle. All three engines accept `responses`, `anthropic` and `chat_completions` as +upstream protocols. Runtime automatically uses native support or converts to the +engine protocol. MiniMax Code requires positive context/output limits. Validate the resolved combination before writing a Session. Where each source applies depends on who owns the compute that receives the key: @@ -102,7 +103,9 @@ See the [TypeScript client example](../../packages/agents-client/saved-agent-def } ``` -`protocol` is `anthropic` for Claude Code/MiniMax Code or `responses` for Codex. +`protocol` names the upstream API: `anthropic`, `responses` or `chat_completions`. +It does not select an engine. [Protocol conversion](model-protocol-conversion.md) +is automatic when the selected engine cannot use that upstream protocol natively. The endpoint must use HTTPS without embedded credentials, a query or a fragment. Keys must be nonempty, at most 16 KiB, and contain no NUL/CR/LF. Unknown fields and unsupported protocol/Harness/environment combinations are rejected before creating @@ -122,8 +125,8 @@ Session before mutable Agent/template resolution. No public Session, Agent, Environment, event or ordinary configuration contains the key. The top-level extension is write-only and has no update endpoint. -At dispatch, Core resolves its encrypted snapshot into the existing native adapter -options. It does not fall back to other credentials when a snapshot is missing or +At dispatch, Core delivers its encrypted snapshot as one common confidential +provider bundle. Runtime adapters own native options and protocol conversion. It does not fall back to other credentials when a snapshot is missing or cannot decrypt. The same snapshot path serves every environment: Core sends the options only over the daemon connection bound to the Session. For `self_hosted`, that is the executor enrolled for the Session's own Environment with a current @@ -158,25 +161,10 @@ default never reaches existing Sessions, so a Session's first and later Turns al use the same provider. The execution-configuration read shows the frozen safe view with source `deployment`. -The former `AGENTS_API_EXECUTION_OPTIONS_FILE` is retired: setting it stops Core at -startup with the replacement named; remove it and set the deployment defaults -in Web (System) or with `PUT /core/v1/harnesses/{harness}/model-provider`. Only its provider identity (endpoint, key, -protocol and MiniMax Code limits) has a home in the deployment default; its other -native options (headers, query parameters, environment, MCP servers, feature and -permission settings) are dropped. Sessions frozen from that file keep their -complete private snapshot. Historical `openai_hosted` and `self_hosted` Sessions -created without any snapshot cannot start new work: message input returns 400 -`model_provider_required`, while cancellation and history reads keep working. -Input they reserved before the upgrade settles as failed with that reason, and the -Session reports it, instead of waiting for its deadline. A retry of a Session that -carried its own provider key, first sent before this release, conflicts once after -the upgrade, because its retry hash now holds a keyed fingerprint instead of the -key. Key-bearing retries likewise conflict after a credential key change, which is -not supported anyway. -Recreate them with a bundle. Run `deploy/install/model_provider_sessions.py` -against an installation before upgrading to count them; it only reads. Historical -`none` Sessions that relied on the retired options file now run with the device's -own environment, and the check does not count them. +Configure deployment defaults through `PUT /core/v1/harnesses/{harness}/model-provider`. +The operator options file and historical native-option snapshots are unsupported. +A missing or invalid provider snapshot fails closed; no upgrade reader, automatic +migration or fallback to another model/provider is provided. Parsar manages its own workspace catalog and encrypted keys, sends this extension only on the first Core Session request, and retains a private encrypted snapshot diff --git a/contracts/agents-api/model-protocol-benchmarks.md b/contracts/agents-api/model-protocol-benchmarks.md new file mode 100644 index 000000000..f041dea01 --- /dev/null +++ b/contracts/agents-api/model-protocol-benchmarks.md @@ -0,0 +1,99 @@ +# Model transport benchmarks + +Measured on 2026-09-28 on Linux amd64, Intel Xeon Platinum 8358P @ 2.60 GHz, +Go 1.26.8, GOMAXPROCS=8, CLIProxyAPI v8.0.3. This measures the thin Exchange +integration, including the composed Responses-to-Messages route. The shared host +was not isolated from other workloads; figures are microbenchmarks, not latency SLOs. +No model or network is used by these benchmark workloads. + +Short input is one user message and a function declaration. Large input contains +128 alternating messages with 2 KiB each. Tool loop includes a streamed function +call, its result in a second request, and a text stream. Concurrent SSE uses 32 +workers, 64 deltas and 8 KiB of text per stream. Outputs are consumed and discarded. +HTTP framing, backpressure and native engine processing are excluded. + +The tables report the median of three testing.Benchmark allocation runs with a +200 ms target. Sequential latency uses 20 warmups and 500 samples; concurrent +latency uses 800 streams. p95/p99 include scheduling and GC. Concurrent mean is +aggregate throughput per stream, rather than an individual stream's latency. + +## Short request + +| Client → upstream | Mean (ms) | p95 (ms) | p99 (ms) | Bytes/op | Allocs/op | +| --- | ---: | ---: | ---: | ---: | ---: | +| anthropic → responses | 0.076 | 0.046 | 0.076 | 13,605 | 137 | +| anthropic → chat_completions | 0.059 | 0.065 | 0.099 | 7,614 | 84 | +| responses → anthropic | 0.205 | 0.276 | 0.552 | 19,807 | 239 | +| responses → chat_completions | 0.074 | 0.100 | 0.215 | 10,996 | 120 | +| chat_completions → anthropic | 0.129 | 0.189 | 0.316 | 10,777 | 132 | +| chat_completions → responses | 0.090 | 0.040 | 0.065 | 11,579 | 106 | + +## 256 KiB history + +| Client → upstream | Mean (ms) | p95 (ms) | p99 (ms) | Bytes/op | Allocs/op | +| --- | ---: | ---: | ---: | ---: | ---: | +| anthropic → responses | 15.484 | 18.166 | 34.967 | 4,021,333 | 2,568 | +| anthropic → chat_completions | 15.872 | 18.228 | 19.874 | 3,588,969 | 1,553 | +| responses → anthropic | 56.984 | 63.098 | 75.256 | 8,020,006 | 5,743 | +| responses → chat_completions | 26.077 | 27.209 | 33.847 | 3,631,486 | 1,603 | +| chat_completions → anthropic | 37.060 | 41.171 | 46.010 | 5,740,777 | 4,153 | +| chat_completions → responses | 17.402 | 30.225 | 43.999 | 5,161,877 | 3,370 | + +## Two-request tool loop + +| Client → upstream | Mean (ms) | p95 (ms) | p99 (ms) | Bytes/op | Allocs/op | +| --- | ---: | ---: | ---: | ---: | ---: | +| anthropic → responses | 0.676 | 1.056 | 1.599 | 133,175 | 923 | +| anthropic → chat_completions | 0.555 | 1.006 | 1.515 | 116,540 | 794 | +| responses → anthropic | 1.927 | 3.170 | 3.520 | 315,546 | 2,410 | +| responses → chat_completions | 1.180 | 1.874 | 2.153 | 234,483 | 1,541 | +| chat_completions → anthropic | 0.782 | 0.803 | 0.984 | 95,143 | 993 | +| chat_completions → responses | 0.605 | 0.965 | 1.525 | 83,962 | 808 | + +## Concurrent SSE + +| Client → upstream | Mean (ms) | p95 (ms) | p99 (ms) | Bytes/op | Allocs/op | +| --- | ---: | ---: | ---: | ---: | ---: | +| anthropic → responses | 0.318 | 40.509 | 64.359 | 728,156 | 2,814 | +| anthropic → chat_completions | 0.285 | 22.059 | 33.589 | 593,693 | 3,028 | +| responses → anthropic | 0.625 | 38.772 | 55.790 | 1,493,381 | 6,200 | +| responses → chat_completions | 0.513 | 28.864 | 39.006 | 1,233,740 | 4,003 | +| chat_completions → anthropic | 0.167 | 19.135 | 28.550 | 323,167 | 2,796 | +| chat_completions → responses | 0.211 | 24.962 | 33.010 | 461,607 | 2,814 | + +## Sampled heap + +During the concurrent run, runtime.MemStats is sampled every 1 ms after a baseline +GC. These are observed Go heap peaks, not exact maxima, OS RSS, per-Session retained +memory or a ceiling. Sampling can miss short spikes and perturbs timings. + +| Client → upstream | Baseline heap (B) | Sampled peak (B) | Increase (B) | +| --- | ---: | ---: | ---: | +| anthropic → responses | 2,256,344 | 8,569,920 | 6,313,576 | +| anthropic → chat_completions | 2,260,440 | 7,682,336 | 5,421,896 | +| responses → anthropic | 2,283,664 | 12,052,232 | 9,768,568 | +| responses → chat_completions | 2,310,952 | 10,051,656 | 7,740,704 | +| chat_completions → anthropic | 2,316,000 | 9,032,160 | 6,716,160 | +| chat_completions → responses | 2,437,928 | 7,721,664 | 5,283,736 | + +## Binary cost + +With Go 1.26.8 and `go build -trimpath -ldflags="-s -w"`, the Runtime executable +is 10,051,849 bytes at baseline `5b5c10fc4afeb20a1f5ac52207a9810ece85dbed` and +19,157,257 bytes with this integration, an increase of 9,105,408 bytes. +This is separate from container image size and process memory. Same-protocol +production connections bypass the converter and are covered by selection tests. + +## Reproduce + +```sh +GOMAXPROCS=8 go test ./internal/modeltransport -run '^$' \ + -bench '^BenchmarkProduct' -benchmem -benchtime=200ms -count=3 +OAC_MODELTRANSPORT_PROFILE=1 GOMAXPROCS=8 go test ./internal/modeltransport \ + -run '^TestProductBenchmarkProfile$' -count=1 -v +``` + +The profile is opt-in. The source fixtures live in `internal/modeltransport`. +Raw samples are retained as `thin-bench.log` and `thin-profile.log` in the Linux +qualification workspace. The independent library comparison uses Go 1.27 and +is not a toolchain-controlled comparison with these product numbers. diff --git a/contracts/agents-api/model-protocol-conversion.md b/contracts/agents-api/model-protocol-conversion.md new file mode 100644 index 000000000..bf0015a44 --- /dev/null +++ b/contracts/agents-api/model-protocol-conversion.md @@ -0,0 +1,89 @@ +# Runtime model protocol conversion + +The model provider bundle names the upstream protocol, endpoint, credential and +limits. The Agent model and execution harness are separate selections. Runtime +uses the adapter's native protocols to choose a direct connection or a local +conversion endpoint automatically; model names never select a protocol. + +| Engine | Anthropic Messages upstream | OpenAI Responses upstream | OpenAI Chat Completions upstream | +| --- | --- | --- | --- | +| Claude SDK / Claude Code | Native | Convert Messages to Responses and back | Convert Messages to Chat Completions and back | +| Codex | Convert Responses to Messages and back | Native | Convert Responses to Chat Completions and back | +| MiniMax Code | Native | Native | Native | + +These are model communication paths, not claims that every model implements every +engine capability. MiniMax Code uses its pinned native `custom_provider.api` +selection (`anthropic-messages`, `openai-responses`, `openai-completions`). Do not +force it through another protocol merely to exercise a converter. + +## Ownership and transport + +Core freezes one confidential provider bundle and sends it through the existing +execution contract. It does not emit engine-specific provider options. The same +Runtime implementation serves Core-created and self-hosted environments. Session +scheduling, environments, native execution, tools, Skills and MCP retain their +existing owners. No gateway service, accounts, routing catalog, billing subsystem +or additional Agent loop is installed. + +A conversion endpoint binds only loopback and requires a random per-endpoint +credential. Native configuration receives that credential; the upstream key stays +inside Runtime. Only the configured model endpoint is exposed. Upstream redirects +are refused, and upstream error bodies are never exposed as native diagnostics. +Native URL query flags are accepted on the configured operation but are not +forwarded to the provider. Streaming Chat requests enable `include_usage` so the +provider can send its final usage frame. Token conversion remains in the SDK. +SSE data is converted incrementally and flushed without waiting for the complete +answer. Chat completion requires either `[DONE]` after a valid finish reason, or +a separate empty-choice usage tail after that finish reason. The latter is used by compatible providers that omit `[DONE]`. A bare EOF without a protocol end cannot complete an exchange. +Closing the execution resource cancels requests and releases the endpoint; +individual Turn completion does not establish Session resource ownership. +Same-protocol execution keeps the original native connection and behavior. + +The upstream base URL has the native protocol's usual semantics: Responses and +Chat Completions append `/responses` or `/chat/completions`; include `/v1` in that +base when the provider requires it. Messages appends `/v1/messages`, avoiding a +second `/v1` when already present. Public provider admission requires HTTPS. +There is no endpoint probing or automatic model substitution. + +## Conversion limits + +The conversion dependency is pinned to CLIProxyAPI v8.0.3 (MIT). Only its +public translator SDK is embedded; gateway, account and management services are +not started. The SDK owns field, tool, image, reasoning and usage conversion. +Runtime supplies per-request SDK state and the HTTP/SSE connection. It does not +maintain field allowlists, parameter restoration, tool-name mappings, reasoning +signature interpretation or token accounting alongside the dependency. + +Cross-protocol conversion inherits the pinned library's behavior. Provider-only +tools, grammar enforcement, opaque reasoning state, detailed reasoning usage and +other native extensions are not universally portable. The SDK may omit or +normalize those fields. Exact preservation of arbitrary provider extensions is +not a supported cross-protocol guarantee. The SDK's Responses target applies +Codex defaults to stream/store/parallel-tool settings and can omit sampling and +token-limit controls. Custom grammar tools become string-argument functions; +the target does not enforce the original grammar. For Chat providers that send +empty or provisional usage on a finish chunk before definitive usage, the pinned +SDK's Messages output can retain the earlier counts. These inherited behaviors +are not repaired with local conversion rules. Same-protocol connections retain +the native API. Gemini conversion is not enabled in this first set. + +Dependency updates are explicit version changes, validated with request/response, +stream and real engine regression tests. Prefer an upstream release for conversion +repairs; any unavoidable local patch requires a separately justified narrow scope. + +The project is unpublished. Historical engine-specific provider option snapshots +and alternate legacy wire shapes are unsupported. No automatic migration, old +configuration reader or data deletion accompanies this change. + +## Validation + +Use the same engine/protocol/capability matrix for hosted and self-hosted model +communication. Validate multiple tool rounds, text deltas, supported image and +reasoning forms, cancellation, abnormal termination, errors and usage. A synthetic +HTTP fixture or one-line native answer alone is not real model qualification. +Pure conversion benchmarks and HTTP control tests establish separate evidence; +record real native-model acceptance and any unverified cells explicitly. + +See the [pinned library evaluation](model-protocol-library-evaluation.md) and +[product benchmarks](model-protocol-benchmarks.md) for dependency and measured +conversion costs. These measurements are separate from live model acceptance. diff --git a/contracts/agents-api/model-protocol-library-evaluation.md b/contracts/agents-api/model-protocol-library-evaluation.md new file mode 100644 index 000000000..aed60e395 --- /dev/null +++ b/contracts/agents-api/model-protocol-library-evaluation.md @@ -0,0 +1,116 @@ +# Model protocol conversion library evaluation (2026-09-28) + +Scope: in-process conversion among OpenAI Chat Completions, OpenAI Responses and Anthropic Messages. No gateway processes, account pools, billing, agent loops, real model requests or credentials were used. + +## Decision + +The repository is MIT-licensed. Prefer native protocol operation when a harness already supports the configured provider protocol. For cross-protocol operation, use the maintained CLIProxyAPI public translator SDK at v8.0.3 behind the Runtime-owned Exchange interface. This keeps the upstream transformation implementation upgradeable. The dependency owns protocol conversion, including field selection, tools, reasoning and usage. The host only selects registered conversion routes, supplies per-request SDK state and handles HTTP/SSE transport. Pin the dependency and qualify upgrades; do not maintain a second converter through local field maps or parameter restoration. SDK behavior is not universally lossless; limitations need explicit documentation and engine-level regression evidence. + +RelayKit is the cleanest small protocol-only module, but is AGPL-3.0. Do not add it as the MIT repository's default dependency without resolving licensing compatibility. Bifrost is Apache-2.0, faster than CLIProxyAPI in this bounded sample, but its provider/schema surface is substantially broader than a converter, requires Go 1.27, and brings MCP and transport dependencies even when only the two provider packages are imported. + +## Fixed sources + +- RelayKit: new-api commit 789c970199ea527e6a26e071915f4a4cd2c64178; module github.com/QuantumNous/new-api/relaykit; Go 1.25.1. Published relaykit/v0.2.1 points to 0aec08fee811ec6136828fda790551b49e410301 (2026-09-21). Bench/tests used the newer pinned commit. +- CLIProxyAPI: v8.0.3, acdace936fa7df2905500c7f5e0a97d683138dea; github.com/router-for-me/CLIProxyAPI/v8; MIT; Go 1.26.0 minimum. +- Bifrost: 51172dbe37df018384d0f15f94940e01c40a7fff; github.com/maximhq/bifrost/core; Apache-2.0; Go 1.27.0. + +CLI v6.10.9 (2026-05-07), v6.9.0 and v6.8.55 already require Go 1.26.0. v6.8.9 uses Go 1.24.0, but predates the relevant recent translator repairs; no recent equivalent Go-1.25 version was established. Do not silently lower its go directive. + +## Import closure and reusable APIs + +| Candidate/import | Non-standard packages | Modules, including candidate | Relevant dependencies | +|---|---:|---:|---| +| RelayKit relayconvert | 35 | 8 | uuid, lo, gjson/sjson, match/pretty, x/text | +| CLI sdk/translator/builtin | 135 | 26 | Gin, Redis, logrus, protobuf, crypto, gjson/sjson | +| Bifrost providers/anthropic + providers/openai | 126 | 32 | sonic, fasthttp, websocket, MCP, compression, JSON-schema utilities | + +Counts are actual go list -deps results, not root go.mod requirement counts. Raw evidence remains in the Linux qualification workspace; it is not a repository artifact. + +RelayKit exposes ConvertRequest, ConvertResponse, NewResponseStreamState, ConvertStreamResponseChunk and FinalizeStreamResponse. It owns no HTTP or SSE reading/writing. Result objects include route, quality, usage and diagnostics. Images needing materialization require a host MediaResolver; the library does not download content. + +CLI exposes sdk/translator.Registry and sdk/translator/builtin.Registry(). Request translation returns bytes; response/stream translation returns bytes/chunks plus caller-owned per-exchange state through *any. Responses as an incoming client format is openai-response; the outgoing generic Responses-shaped implementation is registered as codex. The bare registry imports only 4 internal packages and logrus/gjson/sjson/yaml, but contains no registered concrete conversions. builtin imports 50 internal packages. A static closure of the six relevant conversion directions still traverses 23 internal packages and Gin/config/util dependencies. Internal conversion packages cannot be imported outside that module. A copied subset would require removing registration init files and extracting tool/schema/JSON helpers from internal/util plus common/signature/thinking helpers; this is a maintained fork, not a drop-in tiny module. + +Bifrost can be used without constructing its gateway. OpenAIChatRequest.ToBifrostChatRequest plus anthropic.ToAnthropicChatRequest is the tested conversion path. AnthropicMessageRequest.ToBifrostResponsesRequest, ToAnthropicResponsesRequest, response methods and AnthropicResponsesStreamState/ToBifrostResponsesStream/ToAnthropicResponsesStreamResponse cover the corresponding typed conversions. Chat/Responses interchange additionally uses core/schemas conversion methods. Importing those packages also compiles their provider HTTP code and shared schema/MCP definitions. + +## Evidence of maintenance and defects + +RelayKit commit 4eb3b9160566dc4c1f4f0e9340de254f7febf056 (2026-09-21) repairs tool_result image blocks previously sent as text and Responses reasoning segmentation after a mid-stream finish. It adds 292 lines of tests across request/response tests. Current tests include conversion matrix golden fixtures, tool-loss policies, terminal stream tails, usage and failed Responses events. Strict loss policy rejects only request conversion loss; response and stream loss still return successful results with diagnostics. This is a deliberate documented limitation, not a hard fail-closed library. + +CLI commit 0a45f253344089c2f47ae698c34abe8837ec6135 (2026-09-28) repairs missing blank-line SSE terminators and adds regression tests. 4ad5bba repairs false namespace-prefix matches (2026-09-28); 75b854e repairs Claude tool-name sanitation and parameterless schemas (2026-09-24). Tests cover parallel tools, cached usage, incomplete Responses, custom-tool replay, namespace collisions and tool-result adjacency. However: +- The public transforms have no error return; missing registrations return the original body. +- TestConvertOpenAIResponsesRequestToClaude_DropsApplyPatchCustomTool explicitly requires dropping the reserved apply_patch custom grammar tool. +- The codex target rewrites generic parameters, reasoning defaults, stream/store/include. +- Claude-to-Chat and Claude-to-Responses non-stream functions expect an aggregated SSE transcript, while the Codex non-stream response functions expect a response.completed/incomplete envelope. +- Some stream conversions emit client completion at a Chat finish_reason before upstream [DONE]. +- Claude-to-Responses estimates reasoning_tokens from visible text length. + +The integration uses only public registered transformers. Responses-to-Messages +can compose the library's Responses-to-Chat and Chat-to-Messages routes, retaining +separate SDK state and original/translated request bytes for each step. This avoids +the direct route's reserved custom-tool behavior without local tool-name or schema +mapping. Composition must pass the same tool-history and stream tests as direct +routes. HTTP framing handles stream termination; it does not calculate usage or +interpret reasoning signatures. Non-stream Messages conversion can request an +upstream stream and pass its bounded transcript to the library's aggregate entry. +No local synthetic Messages events or field restoration are retained. + +The dependency may normalize parameters, omit provider-specific extensions or +estimate a reasoning-token breakdown. Those behaviors belong to the pinned SDK; +this integration does not claim native fidelity across protocols. Prefer a native +connection when exact provider-specific semantics are needed. A library upgrade +must pass the stored contract and real engine checks before adoption. + +Bifrost commit a44106ad (2026-09-24) repairs Responses types and Anthropic tool-result is_error preservation with dedicated tests; e0ad6638 (2026-09-23) preserves Anthropic extra parameters through Bedrock conversion. Its current tests also deliberately drop forced tool choice on unsupported model capabilities. Typed conversion and model-aware normalization can therefore still change semantics; typed APIs do not imply universal field preservation. + +## First-batch boundary + +CLIProxyAPI registers Gemini conversions in the same public registry. Enabling +Gemini also requires a provider configuration contract, model/credential URL +handling and real engine qualification. Those are not established in this batch; +the admitted protocols remain the initial three. No parallel Gemini gateway or +new provider manager is introduced. + +The measured costs support using CLIProxyAPI despite its larger dependency +closure: the chosen license and tested tool coverage avoid a maintained fork. +The product qualification budget for this host is a short-request p99 below 1 ms, +a 256 KiB request p99 below 100 ms and an executable increase below 12 MiB. These +are review thresholds for dependency upgrades, not production SLOs. Concurrent +heap samples are retained for comparison and do not establish a memory ceiling. + +## Bounded performance sample + +All final samples ran on the same Linux host using Go 1.27.0, GOMAXPROCS=1, stripped binaries (-s -w), no HTTP or real model. Direction: Chat -> Claude, including JSON decode/encode for typed libraries. CLI's public API operates directly on bytes. Baseline: JSON decode and re-encode only. Inputs: 94-byte single text; 263,212-byte history with 32 alternating user/assistant messages (8 KiB text each); 1,496-byte function declaration/call/result/follow-up. testing.Benchmark supplies mean and allocation counts; a separate 1,000-operation sample after GC supplies percentiles. These are comparative microbenchmarks, not production SLOs. + +| Candidate | Input | Mean | p95 | p99 | Bytes/op | Allocs/op | +|---|---|---:|---:|---:|---:|---:| +| Baseline | small | 6.44 us | 6.29 us | 8.47 us | 1,176 | 32 | +| Baseline | large | 0.636 ms | 0.603 ms | 0.671 ms | 548,346 | 350 | +| Baseline | tools | 24.86 us | 34.50 us | 40.57 us | 8,273 | 132 | +| RelayKit | small | 6.77 us | 10.29 us | 15.91 us | 5,008 | 22 | +| RelayKit | large | 0.721 ms | 0.898 ms | 0.961 ms | 567,051 | 194 | +| RelayKit | tools | 31.12 us | 41.94 us | 89.09 us | 15,698 | 132 | +| CLIProxyAPI | small | 19.60 us | 21.79 us | 58.41 us | 3,536 | 59 | +| CLIProxyAPI | large | 7.788 ms | 7.705 ms | 8.402 ms | 3,252,011 | 1,040 | +| CLIProxyAPI | tools | 93.12 us | 129.67 us | 176.01 us | 27,677 | 212 | +| Bifrost | small | 9.00 us | 12.31 us | 41.72 us | 3,802 | 32 | +| Bifrost | large | 1.221 ms | 1.753 ms | 1.956 ms | 1,988,175 | 296 | +| Bifrost | tools | 46.38 us | 90.94 us | 144.14 us | 25,284 | 173 | + +Stripped binary sizes: baseline 2,834,592 B; RelayKit 6,549,767 B (+3,715,175 B); CLI 14,782,727 B (+11,948,135 B); Bifrost 9,924,871 B (+7,090,279 B). This is an independent harness delta, not a prediction of the exact final daemon delta. + +## Validation and reproducing + +RelayKit GOWORK=off go test ./... passed at the pinned commit. CLI sdk/translator and the selected Claude/OpenAI conversion test packages passed. Bifrost selected pure converter tests (TestToAnthropic, TestToBifrost, tool conversion/stream names) passed. None of these results establishes live provider compatibility. + +Each checkout contains benchprotocol/main.go; baseline/main.go is the identical fixture/measurement harness without conversion. *-bench.log, *-tests.log, and *-deps.json hold raw output. The root task's product worktree contains its separate Exchange contract tests and HTTP mock-provider tests; this report does not claim full repository make check. + +## Primary source links + +- [RelayKit API and license](https://github.com/QuantumNous/new-api/blob/789c970199ea527e6a26e071915f4a4cd2c64178/relaykit/README.md) +- [RelayKit actual dependencies](https://github.com/QuantumNous/new-api/blob/789c970199ea527e6a26e071915f4a4cd2c64178/relaykit/go.mod) +- [RelayKit media/stream repair](https://github.com/QuantumNous/new-api/commit/4eb3b9160566dc4c1f4f0e9340de254f7febf056) +- [CLI public registry](https://github.com/router-for-me/CLIProxyAPI/blob/acdace936fa7df2905500c7f5e0a97d683138dea/sdk/translator/registry.go) +- [CLI reserved custom-tool test](https://github.com/router-for-me/CLIProxyAPI/blob/acdace936fa7df2905500c7f5e0a97d683138dea/internal/translator/claude/openai/responses/claude_openai-responses_request_test.go) +- [CLI SSE repair](https://github.com/router-for-me/CLIProxyAPI/commit/0a45f253344089c2f47ae698c34abe8837ec6135) +- [Bifrost module](https://github.com/maximhq/bifrost/blob/51172dbe37df018384d0f15f94940e01c40a7fff/core/go.mod) +- [Bifrost Responses converters](https://github.com/maximhq/bifrost/blob/51172dbe37df018384d0f15f94940e01c40a7fff/core/providers/anthropic/responses.go) diff --git a/contracts/agents-api/model-protocol-qualification.md b/contracts/agents-api/model-protocol-qualification.md new file mode 100644 index 000000000..019db696d --- /dev/null +++ b/contracts/agents-api/model-protocol-qualification.md @@ -0,0 +1,47 @@ +# Model protocol qualification (2026-09-28) + +The pinned converter is CLIProxyAPI v8.0.3. Real execution used MiniMax-M2.7 +through the configured MiniMax Chat Completions, Responses and Messages APIs, +Codex 0.153.4, Claude Agent SDK 0.3.269 / Claude Code 2.1.269 and MiniMax Code +0.4.12. The Claude bridge uses Executor protocol 3. Each run creates a fresh +public Session through the pinned official Python client and freezes the provider +through normal deployment-default admission. No test Dispatcher.Options override +supplies model selection. + +| Engine | Messages | Responses | Chat Completions | +| --- | --- | --- | --- | +| Claude Code | Native: passed | Converted: passed | Converted: passed | +| Codex | Converted: passed | Native: passed | Converted: passed | +| MiniMax Code | Native: passed | Native: passed | Native: passed | + +Each Claude/Codex case covers two successful public function rounds, one failed +function result, incremental text, cancellation, continuation after cancellation +and a Runtime restart followed by exact native Session continuation. Persisted +function-result delivery is checked for loss or duplication. The engine's public +Session response is checked for provider-key exposure. Each MiniMax Code case +covers remembered context and exact continuation after a Runtime restart; public +function calls and cancellation are not claimed by those MiniMax cases. + +This is a model-communication acceptance suite using the existing environment:none +profile. Hosted and self-hosted use the same Runtime path in code; this batch does +not repeat a complete Docker/E2B/self-hosted deployment qualification. Native +model reasoning appeared in the real responses. Image payload preservation, +visible reasoning projection, usage, namespaced/custom tools and abnormal stream +termination are additionally checked by controlled converter fixtures. No real +vision-model qualification is claimed. Public token usage can remain null when an +adapter preserves only a partial native breakdown; this task does not add billing +or invent counters to fill that gap. + +The endpoint tests cover upstream credential isolation, redirect refusal, +authentication, first-text flushing before completion, cancellation, final cleanup, +non-stream SDK aggregation and interrupted streams. An Executor test verifies that +the endpoint survives completed and cancelled Turns and closes on final teardown. +The console's three affected browser scenarios passed, including protocol choice, +save/edit/cancel, token limits and write-only credentials. + +The real entrypoints are TestNativeModelProtocolPublicExecution and +services/agents-api/tests/official_model_protocol_native.py. Private model settings +are supplied by OAC_TEST_MODEL_PROTOCOL_OPTIONS; do not commit them or print their +values. The suite records only controlled checks, IDs and event counts under +OAC_TEST_NATIVE_PROOF_DIR. Library and product performance evidence are separate: +see model-protocol-library-evaluation.md and model-protocol-benchmarks.md. diff --git a/contracts/agents-api/openapi.yaml b/contracts/agents-api/openapi.yaml index 00bc1685d..c06ed0feb 100644 --- a/contracts/agents-api/openapi.yaml +++ b/contracts/agents-api/openapi.yaml @@ -913,6 +913,7 @@ definitions: enum: - anthropic - responses + - chat_completions type: string required: - api_key @@ -933,6 +934,7 @@ definitions: enum: - anthropic - responses + - chat_completions type: string required: - api_key_configured diff --git a/contracts/agents-api/v1/model_execution.go b/contracts/agents-api/v1/model_execution.go index 4edf6dd27..338b7624f 100644 --- a/contracts/agents-api/v1/model_execution.go +++ b/contracts/agents-api/v1/model_execution.go @@ -14,7 +14,7 @@ type SessionExecutionInput struct { } type ModelProviderInput struct { - Protocol string `json:"protocol" enums:"anthropic,responses" binding:"required"` + Protocol string `json:"protocol" enums:"anthropic,responses,chat_completions" binding:"required"` BaseURL string `json:"base_url" binding:"required"` APIKey string `json:"api_key" binding:"required"` ContextWindow int32 `json:"context_window,omitempty"` diff --git a/contracts/agents-api/v1/model_execution_test.go b/contracts/agents-api/v1/model_execution_test.go index 2c6dd664c..679a2f5ba 100644 --- a/contracts/agents-api/v1/model_execution_test.go +++ b/contracts/agents-api/v1/model_execution_test.go @@ -3,13 +3,27 @@ package v1 import "testing" func TestModelExecutionValidation(t *testing.T) { - for _, tc := range []struct { - protocol, harness string - valid bool - }{{"anthropic", "claude_sdk", true}, {"anthropic", "mcode", true}, {"responses", "codex", true}, {"responses", "mcode", false}, {"anthropic", "codex", false}, {"responses", "", false}} { + for _, harness := range []string{"codex", "claude_sdk", "mcode"} { + for _, protocol := range []string{"anthropic", "responses", "chat_completions"} { + t.Run(harness+"/"+protocol, func(t *testing.T) { + p := ModelProviderInput{Protocol: protocol, BaseURL: "https://example.com/v1", APIKey: "secret", ContextWindow: 200000, MaxOutputTokens: 8000} + if err := p.ValidateHarness(harness); err != nil { + t.Fatalf("Runtime-translatable provider rejected: %v", err) + } + p.ContextWindow, p.MaxOutputTokens = 0, 0 + if err := p.ValidateHarness(harness); (err == nil) != (harness != "mcode") { + t.Fatalf("incorrect optional token-limit admission: %v", err) + } + }) + } + } + for _, tc := range []struct{ protocol, harness string }{ + {"responses", ""}, {"responses", "unknown"}, {"unknown", "codex"}, + {"openai", "codex"}, {"chat", "claude_sdk"}, {"chat-completions", "mcode"}, + } { p := ModelProviderInput{Protocol: tc.protocol, BaseURL: "https://example.com/v1", APIKey: "secret", ContextWindow: 200000, MaxOutputTokens: 8000} - if (p.ValidateHarness(tc.harness) == nil) != tc.valid { - t.Fatalf("incorrect protocol validation: %s/%s", tc.protocol, tc.harness) + if p.ValidateHarness(tc.harness) == nil { + t.Fatalf("unsupported protocol or harness accepted: %s/%s", tc.protocol, tc.harness) } } for _, url := range []string{"http://example.com", "https://user:pass@example.com", "https://example.com?key=secret", "https://example.com#secret", "https://", diff --git a/contracts/agents-api/v1/saved_core_extension.go b/contracts/agents-api/v1/saved_core_extension.go index 247fe3f9b..b8eaf7087 100644 --- a/contracts/agents-api/v1/saved_core_extension.go +++ b/contracts/agents-api/v1/saved_core_extension.go @@ -21,7 +21,7 @@ type SavedAgentCore struct { } type ModelProviderView struct { - Protocol string `json:"protocol" enums:"anthropic,responses" binding:"required"` + Protocol string `json:"protocol" enums:"anthropic,responses,chat_completions" binding:"required"` BaseURL string `json:"base_url" binding:"required"` ContextWindow int32 `json:"context_window,omitempty"` MaxOutputTokens int32 `json:"max_output_tokens,omitempty"` diff --git a/contracts/agents-api/v1/saved_core_extension_test.go b/contracts/agents-api/v1/saved_core_extension_test.go index 1f393be21..156c016aa 100644 --- a/contracts/agents-api/v1/saved_core_extension_test.go +++ b/contracts/agents-api/v1/saved_core_extension_test.go @@ -28,26 +28,31 @@ func TestSavedProviderSafeView(t *testing.T) { } func TestSavedProviderExplicitHarnessCompatibility(t *testing.T) { - for _, tc := range []struct { - harness, protocol string - valid bool - }{ - {"", "responses", true}, {"", "anthropic", true}, - {"codex", "responses", true}, {"codex", "anthropic", false}, - {"claude_sdk", "anthropic", true}, {"claude_sdk", "responses", false}, - {"mcode", "anthropic", true}, {"mcode", "responses", false}, - } { - t.Run(tc.harness+"/"+tc.protocol, func(t *testing.T) { - x := &SavedAgentCoreInput{Harness: tc.harness, ModelProvider: &ModelProviderInput{ - Protocol: tc.protocol, BaseURL: "https://example.test", APIKey: "fixture", ContextWindow: 100, MaxOutputTokens: 20, + for _, harness := range []string{"", "codex", "claude_sdk", "mcode"} { + for _, protocol := range []string{"anthropic", "responses", "chat_completions"} { + t.Run(harness+"/"+protocol, func(t *testing.T) { + x := &SavedAgentCoreInput{Harness: harness, ModelProvider: &ModelProviderInput{ + Protocol: protocol, BaseURL: "https://example.test", APIKey: "fixture", ContextWindow: 100, MaxOutputTokens: 20, + }} + if err := x.Validate(); err != nil { + t.Fatalf("Runtime-translatable saved provider rejected: %v", err) + } + if harness != "" { + if err := x.SafeView().ModelProvider.ValidateHarness(harness); err != nil { + t.Fatalf("safe saved provider rejected: %v", err) + } + } + }) + } + } + for _, protocol := range []string{"anthropic", "responses", "chat_completions"} { + for _, limits := range [][2]int32{{0, 0}, {100, 0}, {0, 20}} { + x := &SavedAgentCoreInput{Harness: "mcode", ModelProvider: &ModelProviderInput{ + Protocol: protocol, BaseURL: "https://example.test", APIKey: "fixture", ContextWindow: limits[0], MaxOutputTokens: limits[1], }} - if err := x.Validate(); (err == nil) != tc.valid { - t.Fatalf("valid=%v error=%v", tc.valid, err) + if x.Validate() == nil || x.SafeView().ModelProvider.ValidateHarness("mcode") == nil { + t.Fatal("MiniMax limits must be complete for every upstream protocol") } - }) - } - x := &SavedAgentCoreInput{Harness: "mcode", ModelProvider: &ModelProviderInput{Protocol: "anthropic", BaseURL: "https://example.test", APIKey: "fixture"}} - if x.Validate() == nil { - t.Fatal("MiniMax limits must be complete") + } } } diff --git a/go.mod b/go.mod index 026a7c9c3..6337a4eb0 100644 --- a/go.mod +++ b/go.mod @@ -1,6 +1,6 @@ module github.com/MiniMax-AI-Dev/parsar -go 1.25.13 +go 1.26.8 require ( github.com/BurntSushi/toml v1.6.0 @@ -13,6 +13,7 @@ require ( github.com/moby/moby/client v0.6.0 github.com/openai/openai-go/v3 v3.61.0 github.com/pressly/goose/v3 v3.27.3 + github.com/router-for-me/CLIProxyAPI/v8 v8.0.3 go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.44.0 go.opentelemetry.io/otel/sdk v1.44.0 go.opentelemetry.io/otel/sdk/metric v1.44.0 @@ -37,10 +38,10 @@ require ( github.com/moby/docker-image-spec v1.3.1 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/image-spec v1.1.1 // indirect - github.com/tidwall/gjson v1.19.0 // indirect + github.com/tidwall/gjson v1.19.0 github.com/tidwall/match v1.1.1 // indirect github.com/tidwall/pretty v1.2.1 // indirect - github.com/tidwall/sjson v1.2.5 // indirect + github.com/tidwall/sjson v1.2.5 go.opentelemetry.io/auto/sdk v1.2.1 // indirect go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 // indirect go.opentelemetry.io/otel v1.44.0 @@ -58,15 +59,38 @@ require ( ) require ( + github.com/bytedance/sonic v1.11.6 // indirect + github.com/bytedance/sonic/loader v0.1.1 // indirect github.com/cenkalti/backoff/v5 v5.0.3 // indirect + github.com/cloudwego/base64x v0.1.4 // indirect + github.com/cloudwego/iasm v0.2.0 // indirect + github.com/gabriel-vasile/mimetype v1.4.3 // indirect + github.com/gin-contrib/sse v0.1.0 // indirect + github.com/gin-gonic/gin v1.10.1 // indirect + github.com/go-playground/locales v0.14.1 // indirect + github.com/go-playground/universal-translator v0.18.1 // indirect + github.com/go-playground/validator/v10 v10.20.0 // indirect + github.com/goccy/go-json v0.10.2 // indirect github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect + github.com/json-iterator/go v1.1.12 // indirect + github.com/klauspost/cpuid/v2 v2.3.0 // indirect + github.com/leodido/go-urn v1.4.0 // indirect + github.com/mattn/go-isatty v0.0.23 // indirect github.com/mfridman/interpolate v0.0.2 // indirect - github.com/rogpeppe/go-internal v1.15.0 // indirect + github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect + github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/pelletier/go-toml/v2 v2.2.2 // indirect + github.com/redis/go-redis/v9 v9.19.0 // indirect github.com/sethvargo/go-retry v0.4.0 // indirect + github.com/sirupsen/logrus v1.9.3 // indirect + github.com/twitchyliquid64/golang-asm v0.15.1 // indirect + github.com/ugorji/go/codec v1.2.12 // indirect + go.uber.org/atomic v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect + golang.org/x/arch v0.8.0 // indirect golang.org/x/sync v0.22.0 // indirect golang.org/x/text v0.41.0 // indirect golang.org/x/time v0.15.0 // indirect diff --git a/go.sum b/go.sum index d0481a593..a69e2c6d5 100644 --- a/go.sum +++ b/go.sum @@ -2,10 +2,22 @@ github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho= github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= +github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs= +github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c= +github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA= +github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0= +github.com/bytedance/sonic v1.11.6 h1:oUp34TzMlL+OY1OUWxHqsdkgC/Zfc85zGqw9siXjrc0= +github.com/bytedance/sonic v1.11.6/go.mod h1:LysEHSvpvDySVdC2f87zGWf6CIKJcAvqab1ZaiQtds4= +github.com/bytedance/sonic/loader v0.1.1 h1:c+e5Pt1k/cy5wMveRDyk2X4B9hF4g7an8N3zCYjJFNM= +github.com/bytedance/sonic/loader v0.1.1/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU= github.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM= github.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/0Y= +github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w= +github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg= +github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY= github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= @@ -23,6 +35,12 @@ github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkp github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/felixge/httpsnoop v1.1.0 h1:3YtUj32ZZkqZtt3sZZsClsymw/QDuVfpNhoA31zeORc= github.com/felixge/httpsnoop v1.1.0/go.mod h1:Zqxgdd+1Rkcz8euOqdr7lqgCRJztwr5hp9vDSi5UZCE= +github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0= +github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk= +github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE= +github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI= +github.com/gin-gonic/gin v1.10.1 h1:T0ujvqyCSqRopADpgPgiTT63DUQVSfojyME59Ei63pQ= +github.com/gin-gonic/gin v1.10.1/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y= github.com/go-chi/chi/v5 v5.3.2 h1:5YQkICvTCSZ25hoRsyJazN0scjzKGiu4VAUc7H1o1nY= github.com/go-chi/chi/v5 v5.3.2/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto= github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= @@ -30,12 +48,23 @@ github.com/go-logr/logr v1.4.4 h1:tG4xh9yMsRCAiodLVTxyrkzSZ9+o0L1Kg/+cPVcbP/8= github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s= +github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4= +github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA= +github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY= +github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY= +github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY= +github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8= +github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM= +github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU= +github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I= github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY= github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE= github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/jsonschema-go v0.4.3 h1:/DBOLZTfDow7pe2GmaJNhltueGTtDKICi8V8p+DQPd0= github.com/google/jsonschema-go v0.4.3/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= @@ -52,10 +81,18 @@ github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0= github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= +github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= +github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y= +github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= +github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ= +github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI= github.com/mattn/go-isatty v0.0.23 h1:cYwCQTQf3HB6xUC+BtyCLZNr7IzbOmoZbmssVNzSyiQ= github.com/mattn/go-isatty v0.0.23/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A= github.com/mfridman/interpolate v0.0.2 h1:pnuTK7MQIxxFz1Gr+rjSIx9u7qVjf5VOoM/u6BbAxPY= @@ -68,6 +105,11 @@ github.com/moby/moby/client v0.6.0 h1:AJjEB21QPbXSXjDsZorFBoDZPhMrfbpaPLgSMAW9Bg github.com/moby/moby/client v0.6.0/go.mod h1:OCo00wNRyA3m4lmJ228W3JbyCN4ZNNYjpOXiJydBdcQ= github.com/modelcontextprotocol/go-sdk v1.7.0 h1:yqjY2dsbKAC0LSuWZVBMrHgiG8ukXv6NRo0JiALay44= github.com/modelcontextprotocol/go-sdk v1.7.0/go.mod h1:dL7u98E/zjJTGzEq+j30jQ8K2k1mb6LeAH4inEcSGts= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= github.com/openai/openai-go/v3 v3.61.0 h1:nMLuGFdKBF0sB3qFVNwE8kpBpenVN1ucYRoKcx7E1u0= @@ -76,23 +118,39 @@ github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8 github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040= github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= +github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM= +github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pressly/goose/v3 v3.27.3 h1:pIglVHjw99r4e/hDHHwbl9vfOsDMqUokfkXo6+n/RxA= github.com/pressly/goose/v3 v3.27.3/go.mod h1:Dag+xpV6o20HR2LFY1j0q6MDwc3f7vPUFDA77R+0yGY= +github.com/redis/go-redis/v9 v9.19.0 h1:XPVaaPSnG6RhYf7p+rmSa9zZfeVAnWsH5h3lxthOm/k= +github.com/redis/go-redis/v9 v9.19.0/go.mod h1:v/M13XI1PVCDcm01VtPFOADfZtHf8YW3baQf57KlIkA= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc= github.com/rogpeppe/go-internal v1.15.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs= +github.com/router-for-me/CLIProxyAPI/v8 v8.0.3 h1:dlgkrtwgl+t8UXBZZ86t2WTeqMe638Yt8OULRB3KgEM= +github.com/router-for-me/CLIProxyAPI/v8 v8.0.3/go.mod h1:j1Mtcj0TrC66O1JwbSBB03lY3uYPx4J7yHNvU6nDBCo= github.com/segmentio/asm v1.2.1 h1:DTNbBqs57ioxAD4PrArqftgypG4/qNpXoJx8TVXxPR0= github.com/segmentio/asm v1.2.1/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs= github.com/segmentio/encoding v0.5.4 h1:OW1VRern8Nw6ITAtwSZ7Idrl3MXCFwXHPgqESYfvNt0= github.com/segmentio/encoding v0.5.4/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0= github.com/sethvargo/go-retry v0.4.0 h1:9qy1OoIAxBL+gBYnkTnTnWle5wlfsXQlwRzIbbpdqPw= github.com/sethvargo/go-retry v0.4.0/go.mod h1:tvsjdKG6xfiCx4LSiUZ06kcv38xvdVQwv8R6/VnnVWg= +github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ= +github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= +github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= +github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= @@ -105,8 +163,14 @@ github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4= github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU= github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY= github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28= +github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI= +github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08= +github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE= +github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg= github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4= github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4= +github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs= +github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s= go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 h1:8tvICD4vSTOOsNrsI4Ljf6C+6UKvpTEH5XY3JMoyPoo= @@ -127,8 +191,13 @@ go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/ go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE= go.opentelemetry.io/proto/otlp v1.10.0 h1:IQRWgT5srOCYfiWnpqUYz9CVmbO8bFmKcwYxpuCSL2g= go.opentelemetry.io/proto/otlp v1.10.0/go.mod h1:/CV4QoCR/S9yaPj8utp3lvQPoqMtxXdzn7ozvvozVqk= +go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= +go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= +golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8= +golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc= +golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys= golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M= golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis= golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To= @@ -137,6 +206,7 @@ golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs= golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q= golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8= @@ -171,5 +241,7 @@ modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI= modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw= modernc.org/sqlite v1.54.0 h1:JCxR4qwkJvOaqAoYcgDoO25Nc+ROg6EJ2LfBVzdrgog= modernc.org/sqlite v1.54.0/go.mod h1:4ntCLuNmnH8+GNqjka1wNg7KJd5/Hi5FYp8K+XQ7GZw= +nullprogram.com/x/optparse v1.0.0/go.mod h1:KdyPE+Igbe0jQUrVfMqDMeJQIJZEuyV7pjYmp6pbG50= pgregory.net/rapid v1.2.0 h1:keKAYRcjm+e1F0oAuU5F5+YPAWcyxNNRK2wud503Gnk= pgregory.net/rapid v1.2.0/go.mod h1:PY5XlDGj0+V1FCq0o192FdRhpKHGTRIWBgqjDBTrq04= +rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4= diff --git a/go.work b/go.work index 463cff599..5f2749f73 100644 --- a/go.work +++ b/go.work @@ -1,3 +1,3 @@ -go 1.25.13 +go 1.26.8 use . diff --git a/internal/harnessconfig/claudesdk/configuration.go b/internal/harnessconfig/claudesdk/configuration.go index ab955c681..8fb442a65 100644 --- a/internal/harnessconfig/claudesdk/configuration.go +++ b/internal/harnessconfig/claudesdk/configuration.go @@ -4,5 +4,9 @@ package claudesdk import "github.com/MiniMax-AI-Dev/parsar/internal/harnessconfig" func Configuration() harnessconfig.Configuration { - return harnessconfig.Configuration{Providers: []harnessconfig.Provider{{Protocol: "anthropic"}}} + return harnessconfig.Configuration{Providers: []harnessconfig.Provider{ + {Protocol: "anthropic"}, + {Protocol: "responses"}, + {Protocol: "chat_completions"}, + }} } diff --git a/internal/harnessconfig/codex/configuration.go b/internal/harnessconfig/codex/configuration.go index 42a267ab8..601fe3f85 100644 --- a/internal/harnessconfig/codex/configuration.go +++ b/internal/harnessconfig/codex/configuration.go @@ -4,5 +4,9 @@ package codex import "github.com/MiniMax-AI-Dev/parsar/internal/harnessconfig" func Configuration() harnessconfig.Configuration { - return harnessconfig.Configuration{Providers: []harnessconfig.Provider{{Protocol: "responses"}}} + return harnessconfig.Configuration{Providers: []harnessconfig.Provider{ + {Protocol: "anthropic"}, + {Protocol: "responses"}, + {Protocol: "chat_completions"}, + }} } diff --git a/internal/harnessconfig/mcode/configuration.go b/internal/harnessconfig/mcode/configuration.go index a98a6a644..1032ea2ff 100644 --- a/internal/harnessconfig/mcode/configuration.go +++ b/internal/harnessconfig/mcode/configuration.go @@ -4,5 +4,9 @@ package mcode import "github.com/MiniMax-AI-Dev/parsar/internal/harnessconfig" func Configuration() harnessconfig.Configuration { - return harnessconfig.Configuration{Providers: []harnessconfig.Provider{{Protocol: "anthropic", RequiresTokenLimits: true}}} + return harnessconfig.Configuration{Providers: []harnessconfig.Provider{ + {Protocol: "anthropic", RequiresTokenLimits: true}, + {Protocol: "responses", RequiresTokenLimits: true}, + {Protocol: "chat_completions", RequiresTokenLimits: true}, + }} } diff --git a/internal/modeltransport/benchmark_test.go b/internal/modeltransport/benchmark_test.go new file mode 100644 index 000000000..e02c9c0fe --- /dev/null +++ b/internal/modeltransport/benchmark_test.go @@ -0,0 +1,250 @@ +package modeltransport + +import ( + "context" + "encoding/json" + "fmt" + "os" + "runtime" + "sort" + "strings" + "sync" + "testing" + "time" +) + +type productBenchmarkCase struct { + name string + run func() error +} + +func productRequest(p Protocol, large bool) []byte { + var request map[string]any + _ = json.Unmarshal([]byte(exchangeRequests[p]), &request) + field := "messages" + if p == Responses { + field = "input" + } + messages := []any{map[string]any{"role": "user", "content": "Hello"}} + if large { + messages = nil + for i := 0; i < 128; i++ { + role := "user" + if i%2 == 1 { + role = "assistant" + } + messages = append(messages, map[string]any{"role": role, "content": strings.Repeat("history ", 256)}) + } + } + request[field] = messages + raw, _ := json.Marshal(request) + return raw +} + +func productStream(source, target Protocol, request []byte, events [][]byte) error { + e, err := NewExchange(source, target, "upstream", request, true) + if err != nil { + return err + } + _ = e.Request() + for _, event := range events { + if _, err := e.Event(context.Background(), event); err != nil { + return err + } + } + return e.Finish() +} + +func productCases() []productBenchmarkCase { + var cases []productBenchmarkCase + for _, source := range protocols() { + for _, target := range protocols() { + if source == target { + continue + } + for _, large := range []bool{false, true} { + kind := "short" + if large { + kind = "history256KiB" + } + request := productRequest(source, large) + cases = append(cases, productBenchmarkCase{kind + "/" + string(source) + "_to_" + string(target), func() error { + e, err := NewExchange(source, target, "upstream", request, false) + if err == nil { + _ = e.Request() + } + return err + }}) + } + initial, followup := productRequest(source, false), []byte(exchangeRequests[source]) + tool, text := toolEvents(target), textEvents(target) + cases = append(cases, productBenchmarkCase{"toolLoop/" + string(source) + "_to_" + string(target), func() error { + if err := productStream(source, target, initial, tool); err != nil { + return err + } + return productStream(source, target, followup, text) + }}) + } + } + return cases +} + +func BenchmarkProductExchange(b *testing.B) { + for _, tc := range productCases() { + b.Run(tc.name, func(b *testing.B) { + b.ReportAllocs() + b.ResetTimer() + for b.Loop() { + if err := tc.run(); err != nil { + b.Fatal(err) + } + } + }) + } +} + +// Emit 8 KiB in 64 deltas while retaining each protocol's normal terminal frame. +func productLongEvents(p Protocol) [][]byte { + chunk := strings.Repeat("x", 128) + var out [][]byte + for _, event := range textEvents(p) { + s := string(event) + if strings.Contains(s, `"Hel"`) { + for i := 0; i < 64; i++ { + delta := strings.ReplaceAll(s, `"Hel"`, `"`+chunk+`"`) + if i > 0 { + delta = strings.Replace(delta, `"role":"assistant",`, "", 1) + } + out = append(out, []byte(delta)) + } + continue + } + if strings.Contains(s, `"lo"`) { + continue + } + out = append(out, []byte(strings.ReplaceAll(s, `"Hello"`, `"`+strings.Repeat(chunk, 64)+`"`))) + } + return out +} + +func BenchmarkProductConcurrentSSE(b *testing.B) { + for _, source := range protocols() { + for _, target := range protocols() { + if source == target { + continue + } + request, events := productRequest(source, false), productLongEvents(target) + b.Run(string(source)+"_to_"+string(target), func(b *testing.B) { + b.ReportAllocs() + b.SetParallelism(4) + b.ResetTimer() + b.RunParallel(func(pb *testing.PB) { + for pb.Next() { + if err := productStream(source, target, request, events); err != nil { + b.Error(err) + return + } + } + }) + }) + } + } +} + +// Opt-in measurements are separate from ns/op so timing and heap sampling do +// not distort the allocation benchmark. These are process-local samples. +func TestProductBenchmarkProfile(t *testing.T) { + if os.Getenv("OAC_MODELTRANSPORT_PROFILE") != "1" { + t.Skip("explicit product benchmark profile required") + } + t.Logf("go=%s GOMAXPROCS=%d", runtime.Version(), runtime.GOMAXPROCS(0)) + for _, tc := range productCases() { + samples := make([]time.Duration, 500) + for i := 0; i < 20; i++ { + if err := tc.run(); err != nil { + t.Fatal(err) + } + } + for i := range samples { + start := time.Now() + if err := tc.run(); err != nil { + t.Fatal(err) + } + samples[i] = time.Since(start) + } + sort.Slice(samples, func(i, j int) bool { return samples[i] < samples[j] }) + t.Logf("latency %s samples=%d p95_ns=%d p99_ns=%d", tc.name, len(samples), samples[474].Nanoseconds(), samples[494].Nanoseconds()) + } + for _, source := range protocols() { + for _, target := range protocols() { + if source == target { + continue + } + t.Run(fmt.Sprint(source, "_to_", target), func(t *testing.T) { profileConcurrentStreams(t, source, target) }) + } + } +} + +func profileConcurrentStreams(t *testing.T, source, target Protocol) { + request, events := productRequest(source, false), productLongEvents(target) + runtime.GC() + var baseline runtime.MemStats + runtime.ReadMemStats(&baseline) + peakHeap, peakInuse := baseline.HeapAlloc, baseline.HeapInuse + stop, sampled := make(chan struct{}), make(chan struct{}) + go func() { + defer close(sampled) + ticker := time.NewTicker(time.Millisecond) + defer ticker.Stop() + for { + select { + case <-ticker.C: + var m runtime.MemStats + runtime.ReadMemStats(&m) + if m.HeapAlloc > peakHeap { + peakHeap = m.HeapAlloc + } + if m.HeapInuse > peakInuse { + peakInuse = m.HeapInuse + } + case <-stop: + return + } + } + }() + const workers, perWorker = 32, 25 + start := make(chan struct{}) + errors := make(chan error, workers) + latencies := make(chan time.Duration, workers*perWorker) + var wg sync.WaitGroup + wg.Add(workers) + for i := 0; i < workers; i++ { + go func() { + defer wg.Done() + <-start + for j := 0; j < perWorker; j++ { + before := time.Now() + if err := productStream(source, target, request, events); err != nil { + errors <- err + return + } + latencies <- time.Since(before) + } + }() + } + close(start) + wg.Wait() + close(stop) + <-sampled + close(errors) + close(latencies) + for err := range errors { + t.Fatal(err) + } + samples := make([]time.Duration, 0, workers*perWorker) + for d := range latencies { + samples = append(samples, d) + } + sort.Slice(samples, func(i, j int) bool { return samples[i] < samples[j] }) + t.Logf("concurrent %s_to_%s workers=%d streams=%d deltas=64 text_bytes=8192 p95_ns=%d p99_ns=%d baseline_heap_bytes=%d sampled_peak_heap_bytes=%d sampled_peak_heap_delta_bytes=%d sampled_peak_inuse_delta_bytes=%d", source, target, workers, len(samples), samples[(len(samples)*95+99)/100-1].Nanoseconds(), samples[(len(samples)*99+99)/100-1].Nanoseconds(), baseline.HeapAlloc, peakHeap, peakHeap-baseline.HeapAlloc, peakInuse-baseline.HeapInuse) +} diff --git a/internal/modeltransport/config.go b/internal/modeltransport/config.go new file mode 100644 index 000000000..a9db966c5 --- /dev/null +++ b/internal/modeltransport/config.go @@ -0,0 +1,85 @@ +// Package modeltransport adapts model API protocols inside Runtime execution. +package modeltransport + +import ( + "bytes" + "encoding/json" + "errors" + "net" + "net/url" + "strings" +) + +type Protocol string + +const ( + Anthropic Protocol = "anthropic" + Responses Protocol = "responses" + ChatCompletions Protocol = "chat_completions" +) + +// Provider is the confidential, frozen upstream bundle supplied by Core. +// Engine selection and model identity remain separate execution inputs. +type Provider struct { + Protocol Protocol `json:"protocol"` + BaseURL string `json:"base_url"` + APIKey string `json:"api_key"` + ContextWindow int32 `json:"context_window,omitempty"` + MaxOutputTokens int32 `json:"max_output_tokens,omitempty"` +} + +var ErrConfiguration = errors.New("invalid model provider configuration") +var ErrUnsupported = errors.New("model protocol capability is unsupported") + +func ParseProvider(raw any) (Provider, error) { + var provider Provider + body, err := json.Marshal(raw) + if err != nil { + return provider, ErrConfiguration + } + decoder := json.NewDecoder(bytes.NewReader(body)) + decoder.DisallowUnknownFields() + if decoder.Decode(&provider) != nil { + return Provider{}, ErrConfiguration + } + if err := provider.Validate(); err != nil { + return Provider{}, err + } + return provider, nil +} + +func (p Provider) Validate() error { + switch p.Protocol { + case Anthropic, Responses, ChatCompletions: + default: + return ErrConfiguration + } + u, err := url.Parse(p.BaseURL) + if err != nil || u.Hostname() == "" || u.User != nil || u.RawQuery != "" || u.Fragment != "" || strings.ContainsAny(p.BaseURL, "\x00\r\n") { + return ErrConfiguration + } + // Remote providers require HTTPS. Runtime-local providers may use loopback + // HTTP; Core retains stricter public provider admission. + if u.Scheme != "https" && !(u.Scheme == "http" && net.ParseIP(u.Hostname()).IsLoopback()) { + return ErrConfiguration + } + if strings.TrimSpace(p.APIKey) == "" || len(p.APIKey) > 16384 || strings.ContainsAny(p.APIKey, "\x00\r\n") { + return ErrConfiguration + } + if p.ContextWindow < 0 || p.MaxOutputTokens < 0 || p.MaxOutputTokens > p.ContextWindow { + return ErrConfiguration + } + return nil +} + +func (p Protocol) Path() string { + switch p { + case Anthropic: + return "/messages" + case Responses: + return "/responses" + case ChatCompletions: + return "/chat/completions" + } + return "" +} diff --git a/internal/modeltransport/config_test.go b/internal/modeltransport/config_test.go new file mode 100644 index 000000000..83c1c4678 --- /dev/null +++ b/internal/modeltransport/config_test.go @@ -0,0 +1,107 @@ +package modeltransport + +import ( + "errors" + "reflect" + "testing" +) + +func TestParseProviderValidatesFrozenBundle(t *testing.T) { + valid := map[string]any{"protocol": "responses", "base_url": "https://model.example/v1", "api_key": "fixture-upstream-key", "context_window": int32(64000), "max_output_tokens": int32(4096)} + for _, protocol := range []Protocol{Anthropic, Responses, ChatCompletions} { + t.Run(string(protocol), func(t *testing.T) { + raw := copyBundle(valid) + raw["protocol"] = string(protocol) + got, err := ParseProvider(raw) + want := Provider{Protocol: protocol, BaseURL: "https://model.example/v1", APIKey: "fixture-upstream-key", ContextWindow: 64000, MaxOutputTokens: 4096} + if err != nil || got != want { + t.Fatalf("bundle changed or rejected: %v", err) + } + }) + } + for _, tc := range []struct { + name, field string + value any + }{ + {"unknown field", "native_options", map[string]any{}}, {"alias", "protocol", "openai"}, + {"missing key", "api_key", ""}, {"newline key", "api_key", "fixture\nkey"}, + {"URL credentials", "base_url", "https://user:secret@model.example/v1"}, + {"remote HTTP", "base_url", "http://model.example/v1"}, + {"query", "base_url", "https://model.example/v1?key=secret"}, + {"fragment", "base_url", "https://model.example/v1#secret"}, + {"negative limit", "context_window", -1}, {"excess output", "max_output_tokens", 64001}, + {"fractional limit", "context_window", 1.5}, + } { + t.Run(tc.name, func(t *testing.T) { + raw := copyBundle(valid) + raw[tc.field] = tc.value + if _, err := ParseProvider(raw); !errors.Is(err, ErrConfiguration) { + t.Fatalf("invalid bundle accepted: %v", err) + } + }) + } + for _, raw := range []any{nil, []any{}, "provider", map[string]any{}} { + if _, err := ParseProvider(raw); !errors.Is(err, ErrConfiguration) { + t.Fatalf("invalid root accepted: %v", err) + } + } +} + +func copyBundle(raw map[string]any) map[string]any { + out := make(map[string]any, len(raw)) + for k, v := range raw { + out[k] = v + } + return out +} + +func TestPrepareSelectsNativeProtocolWithoutListener(t *testing.T) { + for _, protocol := range []Protocol{Anthropic, Responses, ChatCompletions} { + for _, native := range [][]Protocol{{protocol}, {Anthropic, Responses, ChatCompletions}} { + p := Provider{Protocol: protocol, BaseURL: "https://model.example/v1", APIKey: "fixture-upstream-key"} + endpoint, err := Prepare(p, "chosen-model", native...) + if err != nil { + t.Fatal(err) + } + if endpoint.Protocol != protocol || endpoint.BaseURL != p.BaseURL || endpoint.APIKey != p.APIKey || endpoint.server != nil || endpoint.transport != nil || endpoint.cancel != nil { + t.Fatal("native protocol unnecessarily converted or upstream bundle changed") + } + if err := endpoint.Close(); err != nil { + t.Fatal(err) + } + if err := endpoint.Close(); err != nil { + t.Fatal(err) + } + } + } +} + +func TestPrepareRejectsInvalidSelection(t *testing.T) { + p := Provider{Protocol: Responses, BaseURL: "https://model.example/v1", APIKey: "fixture-key"} + for _, tc := range []struct { + model string + native []Protocol + }{{"", []Protocol{Responses}}, {"chosen-model", nil}, {"chosen-model", []Protocol{"unknown"}}} { + if _, err := Prepare(p, tc.model, tc.native...); !errors.Is(err, ErrConfiguration) { + t.Fatalf("invalid selection accepted: %v", err) + } + } +} + +func TestPrepareOptionsPreservesCallerBundle(t *testing.T) { + raw := map[string]any{"protocol": "chat_completions", "base_url": "http://127.0.0.1:4321/v1", "api_key": "fixture-upstream-key", "context_window": 64000, "max_output_tokens": 4096} + original := copyBundle(raw) + options := map[string]any{"model": "chosen-model", "model_provider": raw, "system_prompt": "unchanged"} + mapped, endpoint, err := PrepareOptions(options, Anthropic) + if err != nil { + t.Fatal(err) + } + defer endpoint.Close() + p, err := ParseProvider(mapped["model_provider"]) + if err != nil || p.Protocol != Anthropic || p.BaseURL != endpoint.BaseURL || p.APIKey != endpoint.APIKey || p.ContextWindow != 64000 || p.MaxOutputTokens != 4096 { + t.Fatal("prepared native bundle lost its provider limits") + } + if endpoint.APIKey == original["api_key"] || !reflect.DeepEqual(options["model_provider"], original) || mapped["system_prompt"] != "unchanged" { + t.Fatal("provider preparation mutated caller configuration or reused upstream key") + } +} diff --git a/internal/modeltransport/endpoint.go b/internal/modeltransport/endpoint.go new file mode 100644 index 000000000..03157fcd1 --- /dev/null +++ b/internal/modeltransport/endpoint.go @@ -0,0 +1,246 @@ +package modeltransport + +import ( + "bytes" + "context" + "crypto/rand" + "crypto/subtle" + "encoding/base64" + "encoding/json" + "errors" + "io" + "net" + "net/http" + "strings" + "sync" + "time" +) + +const maxBody = 32 << 20 + +// Endpoint belongs to one Session Executor. Close it only after the native +// executor stops using it, including preparation failure and final teardown. +type Endpoint struct { + Protocol Protocol + BaseURL string + APIKey string + server *http.Server + transport *http.Transport + cancel context.CancelFunc + mu sync.Mutex + closing bool + requests sync.WaitGroup + once sync.Once +} + +// Prepare chooses native support first. Only a protocol mismatch creates a +// local conversion endpoint; no provider request or model execution happens here. +func Prepare(provider Provider, model string, native ...Protocol) (*Endpoint, error) { + if provider.Validate() != nil || strings.TrimSpace(model) == "" || len(native) == 0 { + return nil, ErrConfiguration + } + for _, protocol := range native { + if protocol.Path() == "" { + return nil, ErrConfiguration + } + if protocol == provider.Protocol { + return &Endpoint{Protocol: protocol, BaseURL: provider.BaseURL, APIKey: provider.APIKey}, nil + } + } + return newEndpoint(provider, model, native[0]) +} + +func newEndpoint(provider Provider, model string, native Protocol) (*Endpoint, error) { + listener, err := net.Listen("tcp4", "127.0.0.1:0") + if err != nil { + return nil, errors.New("model conversion endpoint unavailable") + } + token := make([]byte, 32) + if _, err := rand.Read(token); err != nil { + listener.Close() + return nil, errors.New("model conversion credential unavailable") + } + ctx, cancel := context.WithCancel(context.Background()) + transport := http.DefaultTransport.(*http.Transport).Clone() + transport.ResponseHeaderTimeout = time.Minute + endpoint := &Endpoint{Protocol: native, BaseURL: "http://" + listener.Addr().String(), + APIKey: base64.RawURLEncoding.EncodeToString(token), transport: transport, cancel: cancel} + if native != Anthropic { + endpoint.BaseURL += "/v1" + } + client := &http.Client{Transport: transport, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} + endpoint.server = &http.Server{ReadHeaderTimeout: 10 * time.Second, ReadTimeout: time.Minute, + BaseContext: func(net.Listener) context.Context { return ctx }, + Handler: http.HandlerFunc(func(w http.ResponseWriter, request *http.Request) { + endpoint.mu.Lock() + if endpoint.closing { + endpoint.mu.Unlock() + http.Error(w, "model endpoint closed", http.StatusServiceUnavailable) + return + } + endpoint.requests.Add(1) + endpoint.mu.Unlock() + defer endpoint.requests.Done() + endpoint.serve(w, request, client, provider, model) + })} + go func() { _ = endpoint.server.Serve(listener) }() + return endpoint, nil +} + +func (e *Endpoint) Close() error { + e.once.Do(func() { + e.mu.Lock() + e.closing = true + e.mu.Unlock() + if e.cancel != nil { + e.cancel() + } + if e.server != nil { + _ = e.server.Close() + } + e.requests.Wait() + if e.transport != nil { + e.transport.CloseIdleConnections() + } + }) + return nil +} + +func (e *Endpoint) authenticated(request *http.Request) bool { + values := request.Header.Values("Authorization") + keys := request.Header.Values("X-Api-Key") + if len(values) > 1 || len(keys) > 1 || len(values)+len(keys) == 0 { + return false + } + for _, value := range values { + if !strings.HasPrefix(value, "Bearer ") || subtle.ConstantTimeCompare([]byte(strings.TrimPrefix(value, "Bearer ")), []byte(e.APIKey)) != 1 { + return false + } + } + for _, key := range keys { + if subtle.ConstantTimeCompare([]byte(key), []byte(e.APIKey)) != 1 { + return false + } + } + return true +} + +func (e *Endpoint) serve(w http.ResponseWriter, request *http.Request, client *http.Client, provider Provider, model string) { + if !e.authenticated(request) { + modelError(w, e.Protocol, http.StatusUnauthorized, "invalid local model credential") + return + } + if request.Method != http.MethodPost || request.URL.Path != "/v1"+e.Protocol.Path() { + modelError(w, e.Protocol, http.StatusNotFound, "model protocol operation is unsupported") + return + } + request.Body = http.MaxBytesReader(w, request.Body, maxBody) + raw, err := io.ReadAll(request.Body) + if err != nil { + modelError(w, e.Protocol, http.StatusRequestEntityTooLarge, "model request exceeds limit") + return + } + var envelope struct { + Model string `json:"model"` + Stream bool `json:"stream"` + } + if !json.Valid(raw) || json.Unmarshal(raw, &envelope) != nil || envelope.Model != model { + modelError(w, e.Protocol, http.StatusBadRequest, "model request does not match the selected model") + return + } + exchange, err := NewExchange(e.Protocol, provider.Protocol, model, raw, envelope.Stream) + if err != nil { + message := "model request capability is unsupported" + if errors.Is(err, ErrUnsupported) { + message = err.Error() + } + modelError(w, e.Protocol, http.StatusBadRequest, message) + return + } + upstream, err := http.NewRequestWithContext(request.Context(), http.MethodPost, upstreamURL(provider), bytes.NewReader(exchange.Request())) + if err != nil { + modelError(w, e.Protocol, http.StatusBadGateway, "model upstream unavailable") + return + } + upstream.Header.Set("Content-Type", "application/json") + if provider.Protocol == Anthropic { + upstream.Header.Set("X-Api-Key", provider.APIKey) + upstream.Header.Set("Anthropic-Version", "2023-06-01") + } else { + upstream.Header.Set("Authorization", "Bearer "+provider.APIKey) + } + var upstreamEnvelope struct { + Stream bool `json:"stream"` + } + _ = json.Unmarshal(exchange.Request(), &upstreamEnvelope) + if upstreamEnvelope.Stream { + upstream.Header.Set("Accept", "text/event-stream") + } + response, err := client.Do(upstream) + if err != nil { + modelError(w, e.Protocol, http.StatusBadGateway, "model upstream unavailable") + return + } + defer response.Body.Close() + if response.StatusCode < 200 || response.StatusCode >= 300 { + status := response.StatusCode + if status < 400 || status > 599 { + status = http.StatusBadGateway + } + modelError(w, e.Protocol, status, "upstream model request failed") + return + } + if envelope.Stream { + e.serveStream(w, request, response, exchange) + return + } + if upstreamEnvelope.Stream && !strings.HasPrefix(response.Header.Get("Content-Type"), "text/event-stream") { + modelError(w, e.Protocol, http.StatusBadGateway, "upstream model did not return an event stream") + return + } + body, err := io.ReadAll(io.LimitReader(response.Body, maxBody+1)) + if err != nil || len(body) > maxBody { + modelError(w, e.Protocol, http.StatusBadGateway, "invalid upstream model response") + return + } + converted, err := exchange.Response(request.Context(), body) + if err != nil { + modelError(w, e.Protocol, http.StatusBadGateway, "unsupported upstream model response") + return + } + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write(converted) +} + +func upstreamURL(provider Provider) string { + base := strings.TrimRight(provider.BaseURL, "/") + if provider.Protocol == Anthropic && !strings.HasSuffix(base, "/v1") { + base += "/v1" + } + return base + provider.Protocol.Path() +} + +func modelError(w http.ResponseWriter, protocol Protocol, status int, message string) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(errorBody(protocol, status, message)) +} + +func errorBody(protocol Protocol, status int, message string) map[string]any { + kind := "api_error" + switch status { + case 400, 404, 413, 422: + kind = "invalid_request_error" + case 401: + kind = "authentication_error" + case 403: + kind = "permission_error" + case 429: + kind = "rate_limit_error" + } + body := map[string]any{"error": map[string]any{"type": kind, "message": message}} + if protocol == Anthropic { + body["type"] = "error" + } + return body +} diff --git a/internal/modeltransport/endpoint_test.go b/internal/modeltransport/endpoint_test.go new file mode 100644 index 000000000..92a05790f --- /dev/null +++ b/internal/modeltransport/endpoint_test.go @@ -0,0 +1,325 @@ +package modeltransport + +import ( + "bufio" + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "net/http/httptest" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/tidwall/gjson" +) + +const endpointRequest = `{"model":"chosen-model","max_tokens":256,"messages":[{"role":"user","content":"hello fixture"}],"tools":[{"name":"lookup","description":"Look up a city","input_schema":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}]}` +const chatToolResponse = `{"id":"chat-fixture","object":"chat.completion","created":1,"model":"chosen-model","choices":[{"index":0,"message":{"role":"assistant","content":null,"tool_calls":[{"id":"call_fixture","type":"function","function":{"name":"lookup","arguments":"{\"city\":\"Paris\"}"}}]},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":11,"completion_tokens":5,"total_tokens":16}}` +const chatFirst = `{"id":"chat-fixture","object":"chat.completion.chunk","created":1,"model":"chosen-model","choices":[{"index":0,"delta":{"role":"assistant","content":"early-token"},"finish_reason":null}]}` +const chatLast = `{"id":"chat-fixture","object":"chat.completion.chunk","created":1,"model":"chosen-model","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":11,"completion_tokens":5,"total_tokens":16}}` + +func endpointFixture(t *testing.T, handler http.HandlerFunc) (*Endpoint, *httptest.Server) { + t.Helper() + upstream := httptest.NewServer(handler) + t.Cleanup(upstream.Close) + endpoint, err := Prepare(Provider{Protocol: ChatCompletions, BaseURL: upstream.URL + "/v1", APIKey: "fixture-upstream-key"}, "chosen-model", Anthropic) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = endpoint.Close() }) + return endpoint, upstream +} + +func localRequest(t *testing.T, ctx context.Context, endpoint *Endpoint, stream bool) *http.Request { + t.Helper() + var body map[string]any + if err := json.Unmarshal([]byte(endpointRequest), &body); err != nil { + t.Fatal(err) + } + body["stream"] = stream + raw, err := json.Marshal(body) + if err != nil { + t.Fatal(err) + } + request, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint.BaseURL+"/v1/messages?beta=true", strings.NewReader(string(raw))) + if err != nil { + t.Fatal(err) + } + request.Header.Set("Content-Type", "application/json") + request.Header.Set("X-Api-Key", endpoint.APIKey) + return request +} + +func readEndpoint(t *testing.T, request *http.Request) (int, string) { + t.Helper() + response, err := (&http.Client{Timeout: 5 * time.Second}).Do(request) + if err != nil { + t.Fatal(err) + } + defer response.Body.Close() + raw, err := io.ReadAll(response.Body) + if err != nil { + t.Fatal(err) + } + return response.StatusCode, string(raw) +} + +func TestEndpointTranslatesRequestToolsUsageAndCredentials(t *testing.T) { + type observed struct{ path, auth, apiKey, body string } + seen := make(chan observed, 1) + endpoint, _ := endpointFixture(t, func(w http.ResponseWriter, r *http.Request) { + raw, _ := io.ReadAll(r.Body) + seen <- observed{r.URL.Path, r.Header.Get("Authorization"), r.Header.Get("X-Api-Key"), string(raw)} + w.Header().Set("Content-Type", "application/json") + _, _ = io.WriteString(w, chatToolResponse) + }) + status, body := readEndpoint(t, localRequest(t, t.Context(), endpoint, false)) + if status != http.StatusOK { + t.Fatalf("conversion failed: %d %s", status, body) + } + got := <-seen + if got.path != "/v1/chat/completions" || got.auth != "Bearer fixture-upstream-key" || got.apiKey != "" || strings.Contains(got.body, endpoint.APIKey) || strings.Contains(got.auth, endpoint.APIKey) { + t.Fatal("wrong upstream destination or credential boundary") + } + if gjson.Get(got.body, "model").String() != "chosen-model" || !strings.Contains(got.body, "hello fixture") || gjson.Get(got.body, "tools.0.function.name").String() != "lookup" { + t.Fatal("request model, input or declared tool was lost") + } + if gjson.Get(body, "content.0.id").String() != "call_fixture" || gjson.Get(body, "content.0.name").String() != "lookup" || gjson.Get(body, "content.0.input.city").String() != "Paris" || gjson.Get(body, "stop_reason").String() != "tool_use" { + t.Fatalf("tool call was lost: %s", body) + } + if gjson.Get(body, "usage.input_tokens").Int() != 11 || gjson.Get(body, "usage.output_tokens").Int() != 5 { + t.Fatalf("usage changed: %s", body) + } + if strings.Contains(body, "fixture-upstream-key") || strings.Contains(body, endpoint.APIKey) { + t.Fatal("credential leaked in response") + } +} + +func TestEndpointFlushesFirstTextBeforeUpstreamCompletion(t *testing.T) { + release := make(chan struct{}) + var once sync.Once + unblock := func() { once.Do(func() { close(release) }) } + defer unblock() + endpoint, _ := endpointFixture(t, func(w http.ResponseWriter, r *http.Request) { + raw, _ := io.ReadAll(r.Body) + if !gjson.GetBytes(raw, "stream_options.include_usage").Bool() || r.URL.RawQuery != "" { + t.Error("missing usage stream option or native query forwarded upstream") + http.Error(w, "invalid transport options", http.StatusBadRequest) + return + } + w.Header().Set("Content-Type", "text/event-stream") + fmt.Fprintf(w, "data: %s\n\n", chatFirst) + w.(http.Flusher).Flush() + select { + case <-release: + case <-r.Context().Done(): + return + } + fmt.Fprintf(w, "data: %s\n\ndata: [DONE]\n\n", chatLast) + }) + response, err := (&http.Client{Timeout: 5 * time.Second}).Do(localRequest(t, t.Context(), endpoint, true)) + if err != nil { + t.Fatal(err) + } + defer response.Body.Close() + first := make(chan struct{}) + complete := make(chan string, 1) + go func() { + scanner := bufio.NewScanner(response.Body) + var body strings.Builder + announced := false + for scanner.Scan() { + line := scanner.Text() + body.WriteString(line) + body.WriteByte('\n') + if !announced && strings.Contains(line, "early-token") { + close(first) + announced = true + } + } + complete <- body.String() + }() + select { + case <-first: + case <-time.After(3 * time.Second): + t.Fatal("first text was buffered until upstream completion") + } + select { + case <-complete: + t.Fatal("stream finished before upstream completed") + default: + } + unblock() + select { + case body := <-complete: + if !strings.Contains(body, "message_stop") || strings.Contains(body, "event: error") { + t.Fatalf("stream did not complete: %s", body) + } + case <-time.After(3 * time.Second): + t.Fatal("completed upstream did not drain") + } +} + +func TestEndpointCancellationAndCloseStopUpstream(t *testing.T) { + for _, mode := range []string{"cancel request", "close endpoint"} { + t.Run(mode, func(t *testing.T) { + entered, stopped := make(chan struct{}), make(chan struct{}) + endpoint, _ := endpointFixture(t, func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "text/event-stream") + fmt.Fprintf(w, "data: %s\n\n", chatFirst) + w.(http.Flusher).Flush() + close(entered) + <-r.Context().Done() + close(stopped) + }) + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + response, err := (&http.Client{Timeout: 5 * time.Second}).Do(localRequest(t, ctx, endpoint, true)) + if err != nil { + t.Fatal(err) + } + defer response.Body.Close() + select { + case <-entered: + case <-time.After(3 * time.Second): + t.Fatal("upstream did not start") + } + if mode == "cancel request" { + cancel() + } else { + done := make(chan struct{}) + go func() { _ = endpoint.Close(); close(done) }() + select { + case <-done: + case <-time.After(3 * time.Second): + t.Fatal("Close did not drain requests") + } + } + select { + case <-stopped: + case <-time.After(3 * time.Second): + t.Fatal("upstream outlived its local request") + } + _ = endpoint.Close() + }) + } +} + +func TestEndpointRejectsLocalAuthenticationBeforeUpstream(t *testing.T) { + var requests atomic.Int32 + endpoint, _ := endpointFixture(t, func(w http.ResponseWriter, r *http.Request) { requests.Add(1); http.Error(w, "unexpected", 500) }) + for _, mode := range []string{"missing", "wrong", "duplicate", "conflicting"} { + t.Run(mode, func(t *testing.T) { + request := localRequest(t, t.Context(), endpoint, false) + switch mode { + case "missing": + request.Header.Del("X-Api-Key") + case "wrong": + request.Header.Set("X-Api-Key", "wrong") + case "duplicate": + request.Header.Add("X-Api-Key", endpoint.APIKey) + case "conflicting": + request.Header.Set("Authorization", "Bearer wrong") + } + status, _ := readEndpoint(t, request) + if status != http.StatusUnauthorized { + t.Fatalf("invalid credential returned %d", status) + } + }) + } + if requests.Load() != 0 { + t.Fatal("unauthorized request reached upstream") + } +} + +func TestEndpointDoesNotFollowCredentialRedirect(t *testing.T) { + var redirected atomic.Int32 + destination := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + redirected.Add(1) + _, _ = io.WriteString(w, chatToolResponse) + })) + defer destination.Close() + endpoint, _ := endpointFixture(t, func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, strings.Replace(destination.URL, "127.0.0.1", "localhost", 1)+"/stolen", http.StatusTemporaryRedirect) + }) + status, body := readEndpoint(t, localRequest(t, t.Context(), endpoint, false)) + if status != http.StatusBadGateway || redirected.Load() != 0 || strings.Contains(body, "fixture-upstream-key") { + t.Fatalf("redirect was followed or not rejected: status=%d requests=%d", status, redirected.Load()) + } +} + +func TestEndpointRedactsUpstreamFailureBody(t *testing.T) { + for _, status := range []int{http.StatusUnauthorized, http.StatusTooManyRequests, http.StatusInternalServerError} { + t.Run(fmt.Sprint(status), func(t *testing.T) { + endpoint, _ := endpointFixture(t, func(w http.ResponseWriter, r *http.Request) { + http.Error(w, "fixture-upstream-key secret-diagnostic-canary", status) + }) + got, body := readEndpoint(t, localRequest(t, t.Context(), endpoint, false)) + if got != status || strings.Contains(body, "secret-diagnostic-canary") || strings.Contains(body, "fixture-upstream-key") { + t.Fatalf("unsafe upstream failure: status=%d body=%s", got, body) + } + }) + } +} + +func TestEndpointTruncatedStreamCannotReportSuccess(t *testing.T) { + for _, tail := range []string{"", "data: " + chatLast + "\n\n"} { + t.Run(fmt.Sprint(len(tail)), func(t *testing.T) { + endpoint, _ := endpointFixture(t, func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "text/event-stream") + fmt.Fprintf(w, "data: %s\n\n%s", chatFirst, tail) + }) + status, body := readEndpoint(t, localRequest(t, t.Context(), endpoint, true)) + if status != http.StatusOK || !strings.Contains(body, "event: error") || strings.Contains(body, "event: message_stop") { + t.Fatalf("truncated stream reported completion: %d %s", status, body) + } + }) + } +} + +func TestEndpointNonStreamingUsesSDKUpstreamStream(t *testing.T) { + for _, target := range []Protocol{Anthropic, Responses} { + for _, complete := range []bool{true, false} { + t.Run(fmt.Sprintf("%s/complete=%t", target, complete), func(t *testing.T) { + observed := make(chan bool, 1) + upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + request, _ := io.ReadAll(r.Body) + observed <- gjson.GetBytes(request, "stream").Bool() && r.Header.Get("Accept") == "text/event-stream" + chunks := textEvents(target) + if !complete { + chunks = chunks[:len(chunks)-1] + } + w.Header().Set("Content-Type", "text/event-stream") + _, _ = w.Write(eventBody(chunks)) + })) + defer upstream.Close() + endpoint, err := Prepare(Provider{Protocol: target, BaseURL: upstream.URL + "/v1", APIKey: "fixture-key"}, "chosen-model", ChatCompletions) + if err != nil { + t.Fatal(err) + } + defer endpoint.Close() + request, err := http.NewRequestWithContext(t.Context(), http.MethodPost, endpoint.BaseURL+"/chat/completions", strings.NewReader(`{"model":"chosen-model","stream":false,"messages":[{"role":"user","content":"Hello"}]}`)) + if err != nil { + t.Fatal(err) + } + request.Header.Set("Authorization", "Bearer "+endpoint.APIKey) + status, body := readEndpoint(t, request) + if !<-observed { + t.Fatal("SDK aggregation did not request upstream SSE") + } + if complete { + if status != http.StatusOK || gjson.Get(body, "choices.0.message.content").String() != "Hello" || + gjson.Get(body, "usage.prompt_tokens").Int() != 5 || gjson.Get(body, "usage.completion_tokens").Int() != 7 { + t.Fatalf("SDK aggregation failed: %d %s", status, body) + } + } else if status != http.StatusBadGateway { + t.Fatal("incomplete upstream SSE accepted") + } + }) + } + } +} diff --git a/internal/modeltransport/exchange.go b/internal/modeltransport/exchange.go new file mode 100644 index 000000000..10e0c3d9f --- /dev/null +++ b/internal/modeltransport/exchange.go @@ -0,0 +1,155 @@ +package modeltransport + +import ( + "bytes" + "context" + "encoding/json" + "errors" + + sdk "github.com/router-for-me/CLIProxyAPI/v8/sdk/translator" + "github.com/router-for-me/CLIProxyAPI/v8/sdk/translator/builtin" + "github.com/tidwall/gjson" + "github.com/tidwall/sjson" +) + +// Exchange owns the upstream invocation and the SDK state for one request. +type Exchange interface { + Request() []byte + Response(context.Context, []byte) ([]byte, error) + Event(context.Context, []byte) ([][]byte, error) + Finish() error +} + +var ErrExchange = errors.New("invalid or incomplete model exchange") + +type translationStep struct { + from, to sdk.Format + original, request []byte + state any +} + +type exchange struct { + source, target Protocol + model string + request []byte + steps []translationStep + upstreamStream bool + terminal, stopped, started bool + failure error + pending [][]byte + streamBytes int64 +} + +func NewExchange(source, target Protocol, model string, request []byte, stream bool) (Exchange, error) { + if source.Path() == "" || target.Path() == "" || model == "" || !jsonObject(request) { + return nil, ErrUnsupported + } + e := &exchange{source: source, target: target, model: model} + current, err := sjson.SetBytes(request, "model", model) + if err != nil { + return nil, ErrExchange + } + current, _ = sjson.SetBytes(current, "stream", stream) + route := []Protocol{source, target} + // The SDK's direct Responses->Claude route drops custom apply_patch. + // Its Chat route owns the freeform bridge and reverse namespace/custom mapping. + if source == Responses && target == Anthropic { + route = []Protocol{source, ChatCompletions, target} + } + if source != target { + registry := builtin.Registry() + for i := 1; i < len(route); i++ { + step := translationStep{from: clientFormat(route[i-1]), to: providerFormat(route[i]), original: bytes.Clone(current)} + if !registry.HasRequestTransformer(step.from, step.to) || !registry.HasStreamResponseTransformer(step.from, step.to) || !registry.HasNonStreamResponseTransformer(step.from, step.to) { + return nil, ErrUnsupported + } + upstreamStream := stream + // Claude's non-stream SDK entry consumes a complete SSE transcript; + // Codex's consumes a terminal Responses event. Request those upstream + // representations rather than synthesizing protocol events ourselves. + if i == len(route)-1 && (target == Anthropic || target == Responses) { + upstreamStream = true + } + current = registry.TranslateRequest(step.from, step.to, model, current, upstreamStream) + if !jsonObject(current) { + return nil, ErrExchange + } + step.request = bytes.Clone(current) + e.steps = append(e.steps, step) + } + } + // Request the usage tail together with streaming. Some Chat APIs close + // after finish_reason without [DONE] unless this transport option is set. + if target == ChatCompletions && gjson.GetBytes(current, "stream").Bool() { + current, _ = sjson.SetBytes(current, "stream_options.include_usage", true) + if len(e.steps) > 0 { + e.steps[len(e.steps)-1].request = bytes.Clone(current) + } + } + e.request = bytes.Clone(current) + e.upstreamStream = gjson.GetBytes(current, "stream").Bool() + return e, nil +} + +func clientFormat(p Protocol) sdk.Format { + switch p { + case Anthropic: + return "claude" + case Responses: + return "openai-response" + default: + return "openai" + } +} +func providerFormat(p Protocol) sdk.Format { + if p == Responses { + return "codex" + } + return clientFormat(p) +} +func (e *exchange) Request() []byte { return bytes.Clone(e.request) } +func jsonObject(raw []byte) bool { return json.Valid(raw) && gjson.ParseBytes(raw).IsObject() } +func (e *exchange) fail() error { e.failure = ErrExchange; return e.failure } + +func (e *exchange) Response(ctx context.Context, raw []byte) ([]byte, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + if e.failure != nil { + return nil, e.failure + } + if len(raw) > maxBody { + return nil, e.fail() + } + input := raw + if e.upstreamStream { + payloads, err := ssePayloads(raw) + if err != nil { + return nil, e.fail() + } + for _, payload := range payloads { + if e.terminal && bytes.Equal(payload, []byte("[DONE]")) && e.target != Anthropic { + continue + } + if err := e.observe(payload); err != nil { + return nil, err + } + if e.target == Responses && e.terminal && jsonObject(payload) { + input = payload + } + } + if e.Finish() != nil { + return nil, e.fail() + } + } else if !jsonObject(raw) || upstreamError(gjson.ParseBytes(raw)) { + return nil, e.fail() + } + for i := len(e.steps) - 1; i >= 0; i-- { + step := &e.steps[i] + input = builtin.Registry().TranslateNonStream(ctx, step.to, step.from, e.model, step.original, step.request, input, &step.state) + if !jsonObject(input) { + return nil, e.fail() + } + } + return bytes.Clone(input), nil +} diff --git a/internal/modeltransport/exchange_stream.go b/internal/modeltransport/exchange_stream.go new file mode 100644 index 000000000..2289695f5 --- /dev/null +++ b/internal/modeltransport/exchange_stream.go @@ -0,0 +1,249 @@ +package modeltransport + +import ( + "bufio" + "bytes" + "context" + "strings" + + "github.com/router-for-me/CLIProxyAPI/v8/sdk/translator/builtin" + "github.com/tidwall/gjson" +) + +func (e *exchange) Event(ctx context.Context, payload []byte) ([][]byte, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + if e.failure != nil { + return nil, e.failure + } + payload = bytes.TrimSpace(payload) + if e.terminal && bytes.Equal(payload, []byte("[DONE]")) && e.target != Anthropic { + return nil, nil + } + if err := e.observe(payload); err != nil { + return nil, err + } + outputs := [][]byte{payload} + // A usage-only tail following finish_reason is an explicit upstream end + // used by compatible APIs that omit [DONE]. Pass that end to the SDK. + if e.target == ChatCompletions && e.terminal && !bytes.Equal(payload, []byte("[DONE]")) { + outputs = append(outputs, []byte("[DONE]")) + } + for i := len(e.steps) - 1; i >= 0; i-- { + step := &e.steps[i] + var next [][]byte + for _, data := range outputs { + chunks := builtin.Registry().TranslateStream(ctx, step.to, step.from, e.model, step.original, step.request, append([]byte("data: "), data...), &step.state) + for _, chunk := range chunks { + if len(bytes.TrimSpace(chunk)) == 0 { + continue + } + payloads, err := sdkPayloads(chunk) + if err != nil { + return nil, e.fail() + } + next = append(next, payloads...) + } + } + // Claude->Chat's SDK emits the usage chunk at message_stop; the Chat + // stream transport sentinel is supplied at that same confirmed boundary. + if e.terminal && step.from == clientFormat(ChatCompletions) { + hasDone := false + for _, data := range next { + hasDone = hasDone || bytes.Equal(data, []byte("[DONE]")) + } + if !hasDone { + next = append(next, []byte("[DONE]")) + } + } + outputs = next + } + var frames [][]byte + for _, data := range outputs { + frames = append(frames, frame(e.source, data)) + } + return e.releaseFrames(frames), nil +} + +func upstreamError(r gjson.Result) bool { + return r.Get("error").Type != gjson.Null || r.Get("response.error").Type != gjson.Null || + r.Get("type").String() == "error" || r.Get("type").String() == "response.failed" +} + +// Observe only transport lifecycle, never content, tool schemas, or token math. +func (e *exchange) observe(payload []byte) error { + e.streamBytes += int64(len(payload)) + if e.failure != nil { + return e.failure + } + if e.streamBytes > 64<<20 || e.terminal { + return e.fail() + } + if bytes.Equal(payload, []byte("[DONE]")) { + if e.target != ChatCompletions || !e.stopped { + return e.fail() + } + e.terminal = true + return nil + } + if !jsonObject(payload) { + return e.fail() + } + r := gjson.ParseBytes(payload) + if upstreamError(r) { + return e.fail() + } + switch e.target { + case ChatCompletions: + if !r.Get("choices").IsArray() { + return e.fail() + } + for _, choice := range r.Get("choices").Array() { + e.started = true + if choice.Get("finish_reason").String() != "" { + e.stopped = true + } + } + if e.stopped && len(r.Get("choices").Array()) == 0 && r.Get("usage").IsObject() { + e.terminal = true + } + case Anthropic: + switch r.Get("type").String() { + case "message_start": + e.started = true + case "message_delta": + if r.Get("delta.stop_reason").String() != "" { + e.stopped = true + } + case "message_stop": + if !e.started || !e.stopped { + return e.fail() + } + e.terminal = true + } + case Responses: + switch r.Get("type").String() { + case "response.completed", "response.incomplete": + if !r.Get("response").IsObject() { + return e.fail() + } + e.started, e.stopped, e.terminal = true, true, true + } + } + return nil +} + +func (e *exchange) Finish() error { + if e.failure != nil { + return e.failure + } + if !e.terminal || !e.stopped { + return e.fail() + } + return nil +} + +func frame(p Protocol, payload []byte) []byte { + var out []byte + if p != ChatCompletions && jsonObject(payload) { + if typ := gjson.GetBytes(payload, "type").String(); typ != "" { + out = append(out, []byte("event: "+typ+"\n")...) + } + } + out = append(out, []byte("data: ")...) + out = append(out, payload...) + return append(out, '\n', '\n') +} + +func sdkPayloads(raw []byte) ([][]byte, error) { + raw = bytes.TrimSpace(raw) + if jsonObject(raw) || bytes.Equal(raw, []byte("[DONE]")) { + return [][]byte{raw}, nil + } + return ssePayloads(append(bytes.Clone(raw), '\n', '\n')) +} + +func ssePayloads(raw []byte) ([][]byte, error) { + scanner := bufio.NewScanner(bytes.NewReader(raw)) + scanner.Buffer(make([]byte, 4096), 8<<20) + var payloads [][]byte + var data []string + size := 0 + for scanner.Scan() { + line := scanner.Text() + if line == "" { + if len(data) > 0 { + payloads = append(payloads, []byte(strings.Join(data, "\n"))) + data = nil + size = 0 + } + } else if strings.HasPrefix(line, "data:") { + size += len(line) + if size > 8<<20 { + return nil, ErrExchange + } + data = append(data, strings.TrimPrefix(strings.TrimPrefix(line, "data:"), " ")) + } + } + if scanner.Err() != nil || len(data) > 0 { + return nil, ErrExchange + } + return payloads, nil +} + +// Completion may precede the upstream terminator in the SDK. Delay only that +// boundary so an interrupted transport cannot publish a successful completion. +func (e *exchange) releaseFrames(outputs [][]byte) [][]byte { + var ready [][]byte + for _, f := range outputs { + if len(e.pending) > 0 || e.completionFrame(f) { + e.pending = append(e.pending, f) + } else { + ready = append(ready, f) + } + } + if e.terminal { + ready = append(ready, e.pending...) + e.pending = nil + if e.source == ChatCompletions { + hasDone := false + for _, f := range ready { + hasDone = hasDone || bytes.Contains(f, []byte("data: [DONE]")) + } + if !hasDone { + ready = append(ready, frame(ChatCompletions, []byte("[DONE]"))) + } + } + } + return ready +} +func (e *exchange) completionFrame(f []byte) bool { + for _, line := range bytes.Split(f, []byte("\n")) { + if !bytes.HasPrefix(line, []byte("data:")) { + continue + } + data := bytes.TrimSpace(line[5:]) + if bytes.Equal(data, []byte("[DONE]")) { + return true + } + r := gjson.ParseBytes(data) + switch e.source { + case Anthropic: + if r.Get("type").String() == "message_stop" || r.Get("delta.stop_reason").String() != "" { + return true + } + case Responses: + if r.Get("type").String() == "response.completed" || r.Get("type").String() == "response.incomplete" { + return true + } + case ChatCompletions: + for _, choice := range r.Get("choices").Array() { + if choice.Get("finish_reason").String() != "" { + return true + } + } + } + } + return false +} diff --git a/internal/modeltransport/exchange_test.go b/internal/modeltransport/exchange_test.go new file mode 100644 index 000000000..e6fa83936 --- /dev/null +++ b/internal/modeltransport/exchange_test.go @@ -0,0 +1,232 @@ +package modeltransport + +import ( + "context" + "encoding/json" + "errors" + "strings" + "testing" + + "github.com/tidwall/gjson" +) + +var exchangeRequests = map[Protocol]string{ + ChatCompletions: `{"model":"client","max_tokens":123,"messages":[{"role":"user","content":"Read"},{"role":"assistant","tool_calls":[{"id":"call_a","type":"function","function":{"name":"read","arguments":"{\"path\":\"a\"}"}}]},{"role":"tool","tool_call_id":"call_a","content":"file"},{"role":"user","content":"Continue"}],"tools":[{"type":"function","function":{"name":"read","parameters":{"type":"object","properties":{"path":{"type":"string"}}}}}]}`, + Anthropic: `{"model":"client","max_tokens":123,"messages":[{"role":"user","content":"Read"},{"role":"assistant","content":[{"type":"tool_use","id":"call_a","name":"read","input":{"path":"a"}}]},{"role":"user","content":[{"type":"tool_result","tool_use_id":"call_a","content":"file"},{"type":"text","text":"Continue"}]}],"tools":[{"name":"read","input_schema":{"type":"object","properties":{"path":{"type":"string"}}}}]}`, + Responses: `{"model":"client","max_output_tokens":123,"input":[{"role":"user","content":"Read"},{"type":"function_call","call_id":"call_a","name":"read","arguments":"{\"path\":\"a\"}"},{"type":"function_call_output","call_id":"call_a","output":"file"},{"role":"user","content":"Continue"}],"tools":[{"type":"function","name":"read","parameters":{"type":"object","properties":{"path":{"type":"string"}}}}]}`, +} +var exchangeResponses = map[Protocol]string{ + ChatCompletions: `{"id":"chat_1","object":"chat.completion","model":"upstream","choices":[{"index":0,"message":{"role":"assistant","content":"Hello"},"finish_reason":"stop"}],"usage":{"prompt_tokens":5,"completion_tokens":7,"total_tokens":12}}`, + Anthropic: `{"id":"msg_1","type":"message","role":"assistant","model":"upstream","content":[{"type":"text","text":"Hello"}],"stop_reason":"end_turn","usage":{"input_tokens":5,"output_tokens":7}}`, + Responses: `{"id":"resp_1","object":"response","model":"upstream","status":"completed","error":null,"output":[{"id":"m1","type":"message","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Hello","annotations":[]}]}],"usage":{"input_tokens":5,"output_tokens":7,"total_tokens":12}}`, +} + +func protocols() []Protocol { return []Protocol{Anthropic, Responses, ChatCompletions} } +func TestExchangeRequestAndResponseMatrix(t *testing.T) { + for _, source := range protocols() { + for _, target := range protocols() { + t.Run(string(source)+"_"+string(target), func(t *testing.T) { + e, err := NewExchange(source, target, "exact/upstream-model", []byte(exchangeRequests[source]), false) + if err != nil { + t.Fatal(err) + } + r := gjson.ParseBytes(e.Request()) + if r.Get("model").String() != "exact/upstream-model" { + t.Fatal("request identity or streaming changed") + } + if !strings.Contains(string(e.Request()), "call_a") || !strings.Contains(string(e.Request()), "read") || !strings.Contains(string(e.Request()), "file") { + t.Fatal("tool history lost") + } + out, err := e.Response(context.Background(), responseBody(source, target)) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(out), "Hello") { + t.Fatalf("text lost: %s", out) + } + inputPath := "usage.input_tokens" + outputPath := "usage.output_tokens" + if source == ChatCompletions { + inputPath = "usage.prompt_tokens" + outputPath = "usage.completion_tokens" + } + if gjson.GetBytes(out, inputPath).Int() != 5 || gjson.GetBytes(out, outputPath).Int() != 7 { + t.Fatalf("usage lost: %s", out) + } + }) + } + } +} +func textEvents(p Protocol) [][]byte { + var s []string + switch p { + case Anthropic: + s = []string{ + `{"type":"message_start","message":{"id":"m1","type":"message","role":"assistant","model":"upstream","content":[],"usage":{"input_tokens":5,"output_tokens":0}}}`, + `{"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}`, + `{"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hel"}}`, + `{"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"lo"}}`, + `{"type":"content_block_stop","index":0}`, + `{"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":7}}`, + `{"type":"message_stop"}`} + case ChatCompletions: + s = []string{ + `{"id":"c1","model":"upstream","choices":[{"index":0,"delta":{"role":"assistant","content":"Hel"}}]}`, + `{"id":"c1","model":"upstream","choices":[{"index":0,"delta":{"content":"lo"}}]}`, + `{"id":"c1","model":"upstream","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":5,"completion_tokens":7,"total_tokens":12}}`, + "[DONE]"} + case Responses: + s = []string{ + `{"type":"response.created","response":{"id":"r1","model":"upstream","status":"in_progress","output":[]}}`, + `{"type":"response.output_item.added","output_index":0,"item":{"id":"m1","type":"message","role":"assistant","content":[]}}`, + `{"type":"response.content_part.added","output_index":0,"item_id":"m1","content_index":0,"part":{"type":"output_text","text":""}}`, + `{"type":"response.output_text.delta","output_index":0,"item_id":"m1","content_index":0,"delta":"Hel"}`, + `{"type":"response.output_text.delta","output_index":0,"item_id":"m1","content_index":0,"delta":"lo"}`, + `{"type":"response.output_text.done","output_index":0,"item_id":"m1","content_index":0,"text":"Hello"}`, + `{"type":"response.output_item.done","output_index":0,"item":{"id":"m1","type":"message","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Hello"}]}}`, + `{"type":"response.completed","response":` + exchangeResponses[Responses] + `}`} + } + var out [][]byte + for _, s := range s { + out = append(out, []byte(s)) + } + return out +} +func TestExchangeIncrementalStreamMatrix(t *testing.T) { + for _, source := range protocols() { + for _, target := range protocols() { + t.Run(string(source)+"_"+string(target), func(t *testing.T) { + e, err := NewExchange(source, target, "upstream", []byte(exchangeRequests[source]), true) + if err != nil { + t.Fatal(err) + } + chunks := textEvents(target) + seenBeforeTerminal := false + var result strings.Builder + for i, c := range chunks { + out, err := e.Event(context.Background(), c) + if err != nil { + t.Fatalf("event %d: %v", i, err) + } + for _, b := range out { + if !strings.HasSuffix(string(b), "\n\n") { + t.Fatal("unframed output") + } + result.Write(b) + } + if i < len(chunks)-1 && strings.Contains(result.String(), "Hel") { + seenBeforeTerminal = true + } + } + if !seenBeforeTerminal { + t.Fatal("text buffered until terminal") + } + if err := e.Finish(); err != nil { + t.Fatal(err) + } + }) + } + } +} +func TestExchangeRejectsIncompleteAndError(t *testing.T) { + for _, target := range protocols() { + e, err := NewExchange(Responses, target, "upstream", []byte(exchangeRequests[Responses]), true) + if err != nil { + t.Fatal(err) + } + if _, err = e.Event(context.Background(), textEvents(target)[0]); err != nil { + t.Fatal(err) + } + if !errors.Is(e.Finish(), ErrExchange) { + t.Fatal("truncated stream accepted") + } + } + e, _ := NewExchange(Responses, Anthropic, "upstream", []byte(exchangeRequests[Responses]), true) + if _, err := e.Event(context.Background(), []byte(`{"type":"error","error":{"type":"overloaded_error","message":"private"}}`)); !errors.Is(err, ErrExchange) { + t.Fatal("error accepted") + } + if _, err := e.Response(context.Background(), []byte(`{"error":{"message":"private"}}`)); !errors.Is(err, ErrExchange) { + t.Fatal("HTTP error accepted") + } +} + +// These are upstream wire fixtures, not generated protocol content in production. +func responseBody(source, target Protocol) []byte { + if source == target || target == ChatCompletions { + return []byte(exchangeResponses[target]) + } + return eventBody(textEvents(target)) +} +func eventBody(events [][]byte) []byte { + var out []byte + for _, event := range events { + out = append(out, frame(ChatCompletions, event)...) + } + return out +} +func TestExchangeCustomPatchTool(t *testing.T) { + req := `{"model":"client","input":[{"role":"user","content":"Patch"},{"type":"custom_tool_call","call_id":"call_old","name":"apply_patch","input":"old patch"},{"type":"custom_tool_call_output","call_id":"call_old","output":"applied"}],"tools":[{"type":"custom","name":"apply_patch","format":{"type":"grammar","syntax":"lark","definition":"start: patch"}}]}` + for _, target := range []Protocol{Anthropic, ChatCompletions} { + for _, stream := range []bool{false, true} { + e, err := NewExchange(Responses, target, "upstream", []byte(req), stream) + if err != nil { + t.Fatal(err) + } + request := gjson.ParseBytes(e.Request()) + schema := "tools.0.function.parameters.properties.input.type" + name := "tools.0.function.name" + if target == Anthropic { + schema = "tools.0.input_schema.properties.input.type" + name = "tools.0.name" + } + if request.Get(name).String() != "apply_patch" || request.Get(schema).String() != "string" || + !strings.Contains(string(e.Request()), "call_old") || !strings.Contains(string(e.Request()), "old patch") { + t.Fatal("SDK custom declaration or history bridge lost") + } + chunks := toolEvents(target) + for i, chunk := range chunks { + value := strings.ReplaceAll(string(chunk), `"name":"read"`, `"name":"apply_patch"`) + value = strings.ReplaceAll(value, "path", "input") + chunks[i] = []byte(value) + } + var out []byte + if stream { + for _, chunk := range chunks { + frames, err := e.Event(context.Background(), chunk) + if err != nil { + t.Fatal(err) + } + for _, f := range frames { + out = append(out, f...) + } + } + if e.Finish() != nil { + t.Fatal("stream incomplete") + } + } else { + body := eventBody(chunks) + if target == ChatCompletions { + body = []byte(`{"id":"c1","choices":[{"index":0,"message":{"role":"assistant","tool_calls":[{"id":"call_a","type":"function","function":{"name":"apply_patch","arguments":"{\"input\":\"a\"}"}}]},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":5,"completion_tokens":7,"total_tokens":12}}`) + } + out, err = e.Response(context.Background(), body) + if err != nil { + t.Fatal(err) + } + } + if !strings.Contains(string(out), `"name":"apply_patch"`) || !strings.Contains(string(out), "custom_tool_call") || !strings.Contains(string(out), `"input":"a"`) { + t.Fatalf("SDK custom round trip lost: %s", out) + } + } + } +} + +func TestExchangeImageAndDelegatedParameters(t *testing.T) { + req := `{"model":"client","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"data:image/png;base64,AA=="}}]}],"reasoning_effort":"medium","vendor_future_field":true}` + e, err := NewExchange(ChatCompletions, Anthropic, "claude-sonnet-4-5", []byte(req), true) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(e.Request()), "image") || !json.Valid(e.Request()) { + t.Fatal("image lost") + } +} diff --git a/internal/modeltransport/exchange_tool_test.go b/internal/modeltransport/exchange_tool_test.go new file mode 100644 index 000000000..69c0c4366 --- /dev/null +++ b/internal/modeltransport/exchange_tool_test.go @@ -0,0 +1,247 @@ +package modeltransport + +import ( + "context" + "encoding/json" + "strings" + "testing" + + "github.com/tidwall/gjson" +) + +func toolEvents(p Protocol) [][]byte { + var values []string + switch p { + case Anthropic: + values = []string{ + `{"type":"message_start","message":{"id":"m1","type":"message","role":"assistant","model":"upstream","content":[],"usage":{"input_tokens":5,"output_tokens":0}}}`, + `{"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"call_a","name":"read","input":{}}}`, + `{"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"path\":"}}`, + `{"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"\"a\"}"}}`, + `{"type":"content_block_stop","index":0}`, + `{"type":"message_delta","delta":{"stop_reason":"tool_use"},"usage":{"output_tokens":7}}`, + `{"type":"message_stop"}`} + case ChatCompletions: + values = []string{ + `{"id":"c1","model":"upstream","choices":[{"index":0,"delta":{"role":"assistant","tool_calls":[{"index":0,"id":"call_a","type":"function","function":{"name":"read","arguments":"{\"path\":"}}]}}]}`, + `{"id":"c1","model":"upstream","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"a\"}"}}]}}]}`, + `{"id":"c1","model":"upstream","choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":5,"completion_tokens":7,"total_tokens":12}}`, + "[DONE]"} + case Responses: + values = []string{ + `{"type":"response.created","response":{"id":"r1","model":"upstream","status":"in_progress","output":[]}}`, + `{"type":"response.output_item.added","output_index":0,"item":{"id":"fc1","type":"function_call","call_id":"call_a","name":"read","arguments":""}}`, + `{"type":"response.function_call_arguments.delta","output_index":0,"item_id":"fc1","delta":"{\"path\":"}`, + `{"type":"response.function_call_arguments.delta","output_index":0,"item_id":"fc1","delta":"\"a\"}"}`, + `{"type":"response.function_call_arguments.done","output_index":0,"item_id":"fc1","arguments":"{\"path\":\"a\"}"}`, + `{"type":"response.output_item.done","output_index":0,"item":{"id":"fc1","type":"function_call","call_id":"call_a","name":"read","arguments":"{\"path\":\"a\"}"}}`, + `{"type":"response.completed","response":{"id":"r1","object":"response","model":"upstream","status":"completed","output":[{"id":"fc1","type":"function_call","call_id":"call_a","name":"read","arguments":"{\"path\":\"a\"}"}],"usage":{"input_tokens":5,"output_tokens":7,"total_tokens":12}}}`} + } + out := make([][]byte, len(values)) + for i, v := range values { + out[i] = []byte(v) + } + return out +} +func TestExchangeToolStreamMatrix(t *testing.T) { + for _, source := range protocols() { + for _, target := range protocols() { + t.Run(string(source)+"_"+string(target), func(t *testing.T) { + e, err := NewExchange(source, target, "upstream", []byte(exchangeRequests[source]), true) + if err != nil { + t.Fatal(err) + } + var argument strings.Builder + var terminal, identity, usage bool + for _, event := range toolEvents(target) { + out, err := e.Event(context.Background(), event) + if err != nil { + t.Fatal(err) + } + for _, frame := range out { + for _, line := range strings.Split(string(frame), "\n") { + if !strings.HasPrefix(line, "data:") { + continue + } + data := strings.TrimSpace(strings.TrimPrefix(line, "data:")) + if data == "[DONE]" { + terminal = source == ChatCompletions + continue + } + r := gjson.Parse(data) + if strings.Contains(data, "call_a") { + identity = true + } + switch source { + case Anthropic: + if r.Get("type").String() == "content_block_delta" { + argument.WriteString(r.Get("delta.partial_json").String()) + } + if r.Get("type").String() == "message_stop" { + terminal = true + } + if r.Get("usage.output_tokens").Int() == 7 { + usage = true + } + case ChatCompletions: + for _, c := range r.Get("choices.0.delta.tool_calls").Array() { + argument.WriteString(c.Get("function.arguments").String()) + } + if r.Get("usage.completion_tokens").Int() == 7 { + usage = true + } + case Responses: + if r.Get("type").String() == "response.function_call_arguments.delta" { + argument.WriteString(r.Get("delta").String()) + } + if r.Get("type").String() == "response.completed" { + terminal = true + } + if r.Get("response.usage.output_tokens").Int() == 7 { + usage = true + } + } + } + } + } + if err := e.Finish(); err != nil { + t.Fatal(err) + } + if !terminal || !identity || !usage { + t.Fatalf("missing terminal=%v identity=%v usage=%v", terminal, identity, usage) + } + if !json.Valid([]byte(argument.String())) || gjson.Get(argument.String(), "path").String() != "a" { + t.Fatalf("tool arguments lost: %q", argument.String()) + } + }) + } + } +} + +// This deterministic fixture keeps the shape of Codex 0.153.4's request without +// retaining captured instructions, prompts, client IDs, or local paths. +const codexNamespaceRequest = `{ + "model":"client","input":[ + {"type":"message","role":"user","content":[{"type":"input_text","text":"Read a skill."}]}, + {"type":"function_call","call_id":"call_old","namespace":"skills","name":"list","arguments":"{}"}, + {"type":"function_call_output","call_id":"call_old","output":"fixture"}, + {"type":"message","role":"user","content":[{"type":"input_text","text":"Continue."}]} + ], + "instructions":"Fixture only.","stream":true,"store":false, + "client_metadata":{"fixture_id":"not-a-real-client"}, + "prompt_cache_key":"fixture-cache","include":["reasoning.encrypted_content"], + "reasoning":{"summary":"auto"},"parallel_tool_calls":true,"tool_choice":"auto", + "tools":[ + {"type":"function","name":"lookup","parameters":{"type":"object","properties":{}}}, + {"type":"namespace","name":"skills","description":"Fixture skills","tools":[ + {"type":"function","name":"list","description":"List fixture skills","parameters":{"type":"object","properties":{}}}, + {"type":"function","name":"read","description":"Read fixture skill","parameters":{"type":"object","properties":{"path":{"type":"string"}},"required":["path"]}} + ]} + ]}` + +func TestExchangeCodexNamespaceRoundTrip(t *testing.T) { + for _, target := range []Protocol{ChatCompletions, Anthropic} { + t.Run(string(target), func(t *testing.T) { + e, err := NewExchange(Responses, target, "upstream", []byte(codexNamespaceRequest), true) + if err != nil { + t.Fatal(err) + } + request := gjson.ParseBytes(e.Request()) + if len(request.Get("tools").Array()) != 3 { + t.Fatal("namespace leaves lost") + } + for _, name := range []string{"lookup", "skills__list", "skills__read", "call_old"} { + if !strings.Contains(string(e.Request()), name) { + t.Fatal("SDK namespace or history lost") + } + } + var completed gjson.Result + for _, event := range toolEvents(target) { + event = []byte(strings.ReplaceAll(string(event), `"name":"read"`, `"name":"skills__read"`)) + frames, err := e.Event(context.Background(), event) + if err != nil { + t.Fatal(err) + } + for _, frame := range frames { + for _, line := range strings.Split(string(frame), "\n") { + if !strings.HasPrefix(line, "data:") { + continue + } + data := gjson.Parse(strings.TrimSpace(strings.TrimPrefix(line, "data:"))) + if data.Get("type").String() == "response.completed" { + completed = data.Get("response") + } + if item := data.Get("item"); item.Get("type").String() == "function_call" { + if item.Get("namespace").String() != "skills" || item.Get("name").String() != "read" { + t.Fatal("streamed namespace identity lost") + } + } + } + } + } + if err := e.Finish(); err != nil { + t.Fatal(err) + } + call := completed.Get("output.0") + if call.Get("namespace").String() != "skills" || call.Get("name").String() != "read" || call.Get("call_id").String() != "call_a" { + t.Fatal("completed namespace identity lost") + } + + e, err = NewExchange(Responses, target, "upstream", []byte(codexNamespaceRequest), false) + if err != nil { + t.Fatal(err) + } + response := []byte(`{"id":"fixture","model":"upstream","choices":[{"message":{"role":"assistant","content":null,"tool_calls":[{"id":"call_a","type":"function","function":{"name":"skills__read","arguments":"{\"path\":\"a\"}"}}]},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":5,"completion_tokens":7,"total_tokens":12}}`) + if target == Anthropic { + chunks := toolEvents(target) + for i, chunk := range chunks { + chunks[i] = []byte(strings.ReplaceAll(string(chunk), `"name":"read"`, `"name":"skills__read"`)) + } + response = eventBody(chunks) + } + out, err := e.Response(context.Background(), []byte(response)) + if err != nil { + t.Fatal(err) + } + call = gjson.GetBytes(out, "output.0") + if call.Get("namespace").String() != "skills" || call.Get("name").String() != "read" { + t.Fatal("nonstream namespace identity lost") + } + }) + } +} + +func TestExchangeSDKVisibleReasoning(t *testing.T) { + for _, target := range []Protocol{ChatCompletions, Anthropic} { + e, err := NewExchange(Responses, target, "upstream", []byte(`{"input":"Hello","reasoning":{"summary":"auto"}}`), false) + if err != nil { + t.Fatal(err) + } + response := []byte(`{"id":"fixture","model":"upstream","choices":[{"message":{"role":"assistant","content":"Answer","reasoning_content":"Visible fixture reasoning"},"finish_reason":"stop"}],"usage":{"prompt_tokens":5,"completion_tokens":7,"total_tokens":12}}`) + if target == Anthropic { + response = eventBody([][]byte{ + []byte(`{"type":"message_start","message":{"id":"fixture","type":"message","role":"assistant","model":"upstream","content":[],"usage":{"input_tokens":5,"output_tokens":0}}}`), + []byte(`{"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":""}}`), + []byte(`{"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"Visible fixture reasoning"}}`), + []byte(`{"type":"content_block_stop","index":0}`), + []byte(`{"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":7}}`), + []byte(`{"type":"message_stop"}`), + }) + } + out, err := e.Response(context.Background(), response) + if err != nil { + t.Fatal(err) + } + found := false + for _, item := range gjson.GetBytes(out, "output").Array() { + if item.Get("type").String() == "reasoning" { + for _, part := range item.Get("summary").Array() { + found = found || part.Get("text").String() == "Visible fixture reasoning" + } + } + } + if !found { + t.Fatal("SDK lost visible reasoning") + } + } +} diff --git a/internal/modeltransport/options.go b/internal/modeltransport/options.go new file mode 100644 index 000000000..3df4f3113 --- /dev/null +++ b/internal/modeltransport/options.go @@ -0,0 +1,25 @@ +package modeltransport + +import "maps" + +// PrepareOptions replaces only the confidential provider bundle in a copied +// execution option set. The caller retains and closes the returned endpoint. +func PrepareOptions(options map[string]any, native ...Protocol) (map[string]any, *Endpoint, error) { + raw, present := options["model_provider"] + if !present { + return options, nil, nil + } + provider, err := ParseProvider(raw) + if err != nil { + return nil, nil, err + } + model, _ := options["model"].(string) + endpoint, err := Prepare(provider, model, native...) + if err != nil { + return nil, nil, err + } + provider.Protocol, provider.BaseURL, provider.APIKey = endpoint.Protocol, endpoint.BaseURL, endpoint.APIKey + copied := maps.Clone(options) + copied["model_provider"] = provider + return copied, endpoint, nil +} diff --git a/internal/modeltransport/stream.go b/internal/modeltransport/stream.go new file mode 100644 index 000000000..10b13edde --- /dev/null +++ b/internal/modeltransport/stream.go @@ -0,0 +1,76 @@ +package modeltransport + +import ( + "bufio" + "encoding/json" + "fmt" + "net/http" + "strings" +) + +// Read SSE events incrementally, preserving multi-line data without buffering +// the complete answer. The exchange validates protocol termination separately. +func (e *Endpoint) serveStream(w http.ResponseWriter, request *http.Request, response *http.Response, exchange Exchange) { + if !strings.HasPrefix(response.Header.Get("Content-Type"), "text/event-stream") { + modelError(w, e.Protocol, http.StatusBadGateway, "upstream model did not return an event stream") + return + } + w.Header().Set("Content-Type", "text/event-stream") + w.Header().Set("Cache-Control", "no-cache") + controller := http.NewResponseController(w) + scanner := bufio.NewScanner(response.Body) + scanner.Buffer(make([]byte, 4096), 8<<20) + var data []string + eventBytes := 0 + emit := func() bool { + if len(data) == 0 { + return true + } + frames, err := exchange.Event(request.Context(), []byte(strings.Join(data, "\n"))) + data = nil + eventBytes = 0 + if err != nil { + e.streamError(w) + _ = controller.Flush() + return false + } + for _, frame := range frames { + if _, err := w.Write(frame); err != nil { + return false + } + } + return controller.Flush() == nil + } + for scanner.Scan() { + line := scanner.Text() + if line == "" { + if !emit() { + return + } + continue + } + if strings.HasPrefix(line, "data:") { + eventBytes += len(line) + if eventBytes > 8<<20 { + e.streamError(w) + _ = controller.Flush() + return + } + data = append(data, strings.TrimPrefix(strings.TrimPrefix(line, "data:"), " ")) + } + } + if request.Context().Err() != nil { + return + } + if scanner.Err() != nil || len(data) != 0 || exchange.Finish() != nil { + e.streamError(w) + _ = controller.Flush() + } +} + +func (e *Endpoint) streamError(w http.ResponseWriter) { + body := errorBody(e.Protocol, http.StatusBadGateway, "upstream model stream failed or ended incompletely") + body["type"] = "error" + raw, _ := json.Marshal(body) + _, _ = fmt.Fprintf(w, "event: error\ndata: %s\n\n", raw) +} diff --git a/internal/modeltransport/stream_tail_test.go b/internal/modeltransport/stream_tail_test.go new file mode 100644 index 000000000..a1178c141 --- /dev/null +++ b/internal/modeltransport/stream_tail_test.go @@ -0,0 +1,60 @@ +package modeltransport + +import ( + "bytes" + "github.com/tidwall/gjson" + "testing" +) + +func TestChatUsageTailCompletesWithoutSentinel(t *testing.T) { + for _, source := range []Protocol{Anthropic, Responses} { + t.Run(string(source), func(t *testing.T) { + e, err := NewExchange(source, ChatCompletions, "upstream", []byte(exchangeRequests[source]), true) + if err != nil { + t.Fatal(err) + } + chunks := [][]byte{ + []byte(`{"id":"c1","choices":[{"index":0,"delta":{"role":"assistant","content":"answer"},"finish_reason":null}]}`), + []byte(`{"id":"c1","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":null}`), + []byte(`{"id":"c1","choices":[],"usage":{"prompt_tokens":5,"completion_tokens":7,"total_tokens":12}}`), + } + var frames [][]byte + for _, chunk := range chunks { + out, err := e.Event(t.Context(), chunk) + if err != nil { + t.Fatal(err) + } + frames = append(frames, out...) + } + if e.Finish() != nil { + t.Fatal("explicit usage tail did not complete stream") + } + out, err := e.Event(t.Context(), []byte("[DONE]")) + if err != nil || len(out) != 0 { + t.Fatal("optional sentinel duplicated completion") + } + completed, usage := 0, false + for _, frame := range frames { + for _, line := range bytes.Split(frame, []byte("\n")) { + if !bytes.HasPrefix(line, []byte("data:")) { + continue + } + r := gjson.ParseBytes(bytes.TrimSpace(line[5:])) + if r.Get("type").String() == "response.completed" { + completed++ + usage = r.Get("response.usage.input_tokens").Int() == 5 && r.Get("response.usage.output_tokens").Int() == 7 + } + if r.Get("type").String() == "message_stop" { + completed++ + } + if r.Get("type").String() == "message_delta" { + usage = r.Get("usage.output_tokens").Int() == 7 + } + } + } + if completed != 1 || !usage { + t.Fatalf("terminal count=%d actual usage=%v", completed, usage) + } + }) + } +} diff --git a/packages/agents-client/src/execution-configuration-projection.ts b/packages/agents-client/src/execution-configuration-projection.ts index f794df086..6ae24dc72 100644 --- a/packages/agents-client/src/execution-configuration-projection.ts +++ b/packages/agents-client/src/execution-configuration-projection.ts @@ -32,7 +32,7 @@ function safeBaseURL(value: string): boolean { /** The safe provider view shared by frozen Session configuration and saved Agent reads. */ export function safeProvider(value: unknown, invalid: Invalid): ModelProviderView { if (!isRecord(value) || !onlyFields(value, providerFields) || - (value.protocol !== "responses" && value.protocol !== "anthropic") || + (value.protocol !== "responses" && value.protocol !== "anthropic" && value.protocol !== "chat_completions") || typeof value.base_url !== "string" || typeof value.api_key_configured !== "boolean" || (value.context_window !== undefined && !isNonnegativeInteger(value.context_window)) || (value.max_output_tokens !== undefined && !isNonnegativeInteger(value.max_output_tokens)) || diff --git a/packages/agents-client/src/execution-configuration.test.ts b/packages/agents-client/src/execution-configuration.test.ts index 549544c30..4e4e81972 100644 --- a/packages/agents-client/src/execution-configuration.test.ts +++ b/packages/agents-client/src/execution-configuration.test.ts @@ -26,6 +26,16 @@ describe("frozen execution configuration", () => { }); expect(await client.retrieveSessionExecutionConfiguration(projectId, id, { signal: abort.signal })).toEqual(snapshot); }); + it.each(["anthropic", "responses", "chat_completions"] as const)("preserves the %s upstream protocol for every harness", async (protocol) => { + for (const harness of ["codex", "claude_sdk", "mcode"]) { + const value = structuredClone(snapshot); + value.harness.value = harness; + value.model_provider.configuration!.protocol = protocol; + value.model_provider.configuration!.context_window = 200000; + value.model_provider.configuration!.max_output_tokens = 8000; + expect(await clientReturning(value).retrieveSessionExecutionConfiguration(projectId, id)).toEqual(value); + } + }); it.each([ { status: "redacted", source: "deployment", configuration: null }, { status: "available", source: "deployment", configuration: { protocol: "responses", base_url: "https://deployment.example/v1", api_key_configured: true } }, @@ -42,6 +52,7 @@ describe("frozen execution configuration", () => { (value: any) => { value.model_provider.configuration.base_url += "?key=secret-canary"; }, (value: any) => { value.session_id = "wrong-session"; }, (value: any) => { value.model.source = "guessed"; }, + (value: any) => { value.model_provider.configuration.protocol = "chat-completions"; }, ])("rejects unsafe or inconsistent responses without echoing them", async (mutate) => { const value = structuredClone(snapshot); mutate(value); await expect(clientReturning(value).retrieveSessionExecutionConfiguration(projectId, id)).rejects.toMatchObject({ code: "invalid_admin_response" }); diff --git a/packages/agents-client/src/saved-agent-defaults.test.ts b/packages/agents-client/src/saved-agent-defaults.test.ts index 13e09f946..a50089a9b 100644 --- a/packages/agents-client/src/saved-agent-defaults.test.ts +++ b/packages/agents-client/src/saved-agent-defaults.test.ts @@ -4,12 +4,12 @@ import { OpenAIAgentsClient } from "./client"; import type { ModelProviderInput, SavedAgentCore, WebSearchToolInput } from "./types"; describe("saved Agent execution defaults", () => { - it("sends complete replacement inputs and preserves omission versus null", async () => { + it.each(["anthropic", "responses", "chat_completions"] as const)("sends %s replacement inputs and preserves omission versus null", async (protocol) => { const calls: unknown[] = []; const safe: SavedAgentCore = { harness: "codex", model_provider: { - protocol: "responses", base_url: "https://model.example/v1", api_key_configured: true, + protocol, base_url: "https://model.example/v1", api_key_configured: true, }, }; const client = new OpenAIAgentsClient({ @@ -22,7 +22,7 @@ describe("saved Agent execution defaults", () => { }) as typeof fetch, }); const provider: ModelProviderInput = { - protocol: "responses", base_url: "https://model.example/v1", api_key: "write-only-fixture", + protocol, base_url: "https://model.example/v1", api_key: "write-only-fixture", }; const agent = await client.createAgent({ model: "example-model", x_agents_core: { harness: "codex", model_provider: provider } }); expect(agent.x_agents_core).toEqual(safe); diff --git a/packages/agents-client/src/types.ts b/packages/agents-client/src/types.ts index a074e4ef4..44fe169b5 100644 --- a/packages/agents-client/src/types.ts +++ b/packages/agents-client/src/types.ts @@ -1186,7 +1186,7 @@ export type CoreHarnessKind = "claude_sdk" | "codex" | "mcode"; /** A complete replacement bundle. API keys are write-only. */ export interface ModelProviderInput { - protocol: "anthropic" | "responses"; + protocol: "anthropic" | "responses" | "chat_completions"; base_url: string; api_key: string; context_window?: number; @@ -1195,7 +1195,7 @@ export interface ModelProviderInput { } export interface ModelProviderView { - protocol: "anthropic" | "responses"; + protocol: "anthropic" | "responses" | "chat_completions"; base_url: string; context_window?: number; max_output_tokens?: number; diff --git a/services/agents-api/deploy/codex/README.md b/services/agents-api/deploy/codex/README.md index e2b67f96b..49aa41f5e 100644 --- a/services/agents-api/deploy/codex/README.md +++ b/services/agents-api/deploy/codex/README.md @@ -171,11 +171,12 @@ Node providers use explicit local Unix Docker sockets, ignoring ambient socket is mounted in a Runtime. Caller-managed `self_hosted` enrollment retains its separate public lifecycle. -Hosted Codex needs a Responses-compatible model provider: either the Session's +Hosted Codex needs a model provider: either the Session's `x_agents_core.model_provider` or the deployment default for `codex`, set with the Core key in Web or through `PUT /core/v1/harnesses/codex/model-provider`. Core freezes the bundle in the Session's encrypted snapshot and delivers it as the -adapter's `codex_provider`; a hosted Session without one is rejected with 400 +common `model_provider` bundle. Runtime automatically adapts Chat Completions or +Anthropic to Codex Responses; a hosted Session without a provider is rejected with 400 `model_provider_required`. Do not place credentials in images. With the qualified Codex image and Docker provider configured, create an idle or diff --git a/services/agents-api/internal/api/harness_model_providers.go b/services/agents-api/internal/api/harness_model_providers.go index e5ccf9bb4..f8a24c502 100644 --- a/services/agents-api/internal/api/harness_model_providers.go +++ b/services/agents-api/internal/api/harness_model_providers.go @@ -141,7 +141,7 @@ var harnessModelProviderShape = shape{kind: objectValue, members: []member{ }} // @Summary Replace a harness's deployment default model provider -// @Description Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, the harness's protocol (responses for codex, anthropic for claude_sdk and mcode) and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key. +// @Description Core key only. The body is the complete x_agents_core.model_provider bundle, including the write-only api_key; there is no partial update and bundles are never merged. The provider is validated for this harness: an HTTPS base_url without credentials, query or fragment, an upstream protocol (responses, anthropic or chat_completions), automatically adapted by Runtime to the selected harness and, for mcode, positive context_window and max_output_tokens. New openai_hosted and none Sessions that resolve no Session or Agent bundle freeze this default into their encrypted snapshot; existing Sessions never change. self_hosted Sessions never use it. The key is encrypted and never returned. Each write records an administrator audit entry without the key. // @Tags Deployment Model Providers // @Accept json // @Produce json diff --git a/services/agents-api/internal/api/saved_provider_test.go b/services/agents-api/internal/api/saved_provider_test.go index fcd8960a3..11cec687a 100644 --- a/services/agents-api/internal/api/saved_provider_test.go +++ b/services/agents-api/internal/api/saved_provider_test.go @@ -131,7 +131,7 @@ func TestSavedProviderInvalidInput(t *testing.T) { `{"model_provider":{"protocol":"responses","base_url":"https://user:saved-provider-secret@example.test","api_key":"saved-provider-secret"}}`, `{"model_provider":{"protocol":"responses","base_url":"https://example.test","api_key":"saved-provider-secret","API_KEY":"secret"}}`, `{"model_provider":{"protocol":"responses","base_url":"https://example.test","api_key":"saved-provider-secret","api_key":"other"}}`, - `{"harness":"claude_sdk","model_provider":` + savedProviderFixture + `}`, + `{"harness":"claude_sdk","model_provider":` + strings.Replace(savedProviderFixture, `"responses"`, `"unknown"`, 1) + `}`, `{"model_provider":` + strings.Replace(savedProviderFixture, `"max_output_tokens":8000`, `"max_output_tokens":100001`, 1) + `}`, } { for _, path := range []string{"/v1/agents", "/v1/agents/" + uuid.NewString()} { @@ -143,3 +143,34 @@ func TestSavedProviderInvalidInput(t *testing.T) { } } } + +func TestSavedProviderProtocolHarnessMatrix(t *testing.T) { + for _, harness := range []string{"codex", "claude_sdk", "mcode"} { + for _, protocol := range []string{"anthropic", "responses", "chat_completions"} { + t.Run(harness+"/"+protocol, func(t *testing.T) { + h, recording, _ := testHandler(t) + s := &savedProviderStore{} + recording.ResourceStore = s + provider := strings.Replace(savedProviderFixture, `"responses"`, `"`+protocol+`"`, 1) + body := `{"model":"fixture","x_agents_core":{"harness":"` + harness + `","model_provider":` + provider + `}}` + for _, path := range []string{"/v1/agents", "/v1/agents/" + uuid.NewString()} { + response := credentialRequest(h, http.MethodPost, path, body) + want := http.StatusOK + if path == "/v1/agents" { + want = http.StatusCreated + } + if response.Code != want || s.provider == nil || s.provider.Protocol != protocol || s.provider.APIKey != "saved-provider-secret" { + t.Fatalf("provider bundle rejected or changed: status=%d", response.Code) + } + assertSavedProviderRedacted(t, response.Body.String()) + var result struct { + Core v1.SavedAgentCore `json:"x_agents_core"` + } + if json.Unmarshal(response.Body.Bytes(), &result) != nil || result.Core.Harness != harness || result.Core.ModelProvider == nil || result.Core.ModelProvider.Protocol != protocol { + t.Fatal("safe view changed the selected harness or upstream protocol") + } + } + }) + } + } +} diff --git a/services/agents-api/internal/execution/model_execution.go b/services/agents-api/internal/execution/model_execution.go index 2b68cc29b..462b8a1f1 100644 --- a/services/agents-api/internal/execution/model_execution.go +++ b/services/agents-api/internal/execution/model_execution.go @@ -8,32 +8,23 @@ import ( "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" ) -func (d *Dispatcher) sessionModelOptions(ctx context.Context, session store.Session, model string) (map[string]any, error) { +func (d *Dispatcher) sessionModelOptions(ctx context.Context, session store.Session) (map[string]any, error) { if d.Store == nil { return nil, errors.New("session model configuration is unavailable") } - provider, frozenOptions, err := d.Store.SessionModelExecutionWithOptions(ctx, session.TenantID, session.ID) + provider, err := d.Store.SessionModelExecution(ctx, session.TenantID, session.ID) if err != nil { return nil, err } - return resolvedSessionModelOptions(provider, frozenOptions, session.Engine, model) + return resolvedSessionModelOptions(provider, session.Engine) } -func resolvedSessionModelOptions(provider *v1.ModelProviderInput, frozenOptions map[string]any, engine, model string) (map[string]any, error) { +func resolvedSessionModelOptions(provider *v1.ModelProviderInput, engine string) (map[string]any, error) { if err := provider.ValidateHarness(engine); err != nil { return nil, err } - if frozenOptions != nil { - return frozenOptions, nil - } - switch engine { - case "codex": - return map[string]any{"codex_provider": map[string]any{"base_url": provider.BaseURL, "bearer_token": provider.APIKey, "wire_api": "responses"}}, nil - case "claude_sdk": - return map[string]any{"claude_provider": map[string]any{"base_url": provider.BaseURL, "bearer_token": provider.APIKey}}, nil - case "mcode": - return map[string]any{"mcode_provider": map[string]any{"name": "Configured provider", "kind": "custom", "enabled": true, "npm": "@ai-sdk/anthropic", "options": map[string]any{"baseURL": provider.BaseURL, "apiKey": provider.APIKey}, "models": map[string]any{model: map[string]any{"name": model, "tool_call": true, "limit": map[string]any{"context": provider.ContextWindow, "output": provider.MaxOutputTokens}}}}}, nil - default: - return nil, errors.New("unsupported model provider harness") - } + return map[string]any{"model_provider": map[string]any{ + "protocol": provider.Protocol, "base_url": provider.BaseURL, "api_key": provider.APIKey, + "context_window": provider.ContextWindow, "max_output_tokens": provider.MaxOutputTokens, + }}, nil } diff --git a/services/agents-api/internal/execution/model_execution_test.go b/services/agents-api/internal/execution/model_execution_test.go index 62af0db2c..694d9618f 100644 --- a/services/agents-api/internal/execution/model_execution_test.go +++ b/services/agents-api/internal/execution/model_execution_test.go @@ -3,18 +3,19 @@ package execution import ( "context" "errors" + "reflect" + "testing" + v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/device" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" - "reflect" - "testing" ) func TestSessionModelExecutionNeverFallsBack(t *testing.T) { called := false d := Dispatcher{Options: func(context.Context, store.Session) (map[string]any, error) { called = true - return map[string]any{"codex_provider": map[string]any{"base_url": "http://127.0.0.1:1/v1"}}, nil + return map[string]any{"model_provider": map[string]any{"base_url": "http://127.0.0.1:1/v1"}}, nil }} if _, err := d.executionRequest(t.Context(), store.Session{Engine: "codex"}, Snapshot{ModelProviderConfigured: true}, device.KindCapabilities{}, store.SessionExecutionBinding{}); err == nil || called { t.Fatal("missing Session credentials fell back") @@ -28,31 +29,41 @@ func TestSessionModelExecutionNeverFallsBack(t *testing.T) { } // A none device may supply its own provider environment. request, err := d.executionRequest(t.Context(), store.Session{Engine: "codex"}, Snapshot{Environment: &v1.Environment{Type: "none"}}, device.KindCapabilities{}, store.SessionExecutionBinding{}) - if err != nil || !called || request.AgentOptions["codex_provider"] == nil { + if err != nil || !called || request.AgentOptions["model_provider"] == nil { t.Fatal("none Session lost its adapter options", err) } } -func TestFrozenDeploymentOptionsRetainNativeProviderSettings(t *testing.T) { - provider := &v1.ModelProviderInput{Protocol: "responses", BaseURL: "https://example.com/v1", APIKey: "private-key"} - frozen := map[string]any{"codex_provider": map[string]any{ - "base_url": provider.BaseURL, "bearer_token": provider.APIKey, "wire_api": "responses", - "http_headers": map[string]any{"x-custom-header": "private-header"}, - "query_params": map[string]any{"api-version": "2026-01-01"}, - }, "mode": "default"} - got, err := resolvedSessionModelOptions(provider, frozen, "codex", "actual-model") - if err != nil || !reflect.DeepEqual(got, frozen) { - t.Fatal("native deployment settings were lost", err) +func TestSessionModelOptionsPreserveUpstreamBundleForEveryHarness(t *testing.T) { + for _, engine := range []string{"codex", "claude_sdk", "mcode"} { + for _, protocol := range []string{"anthropic", "responses", "chat_completions"} { + t.Run(engine+"/"+protocol, func(t *testing.T) { + provider := &v1.ModelProviderInput{Protocol: protocol, BaseURL: "https://example.com/v1", APIKey: "private-key", ContextWindow: 200000, MaxOutputTokens: 8000} + got, err := resolvedSessionModelOptions(provider, engine) + want := map[string]any{"model_provider": map[string]any{ + "protocol": protocol, "base_url": provider.BaseURL, "api_key": provider.APIKey, + "context_window": provider.ContextWindow, "max_output_tokens": provider.MaxOutputTokens, + }} + if err != nil || !reflect.DeepEqual(got, want) { + t.Fatal("upstream provider bundle was changed", err) + } + }) + } } - if _, err := resolvedSessionModelOptions(provider, frozen, "claude_sdk", "actual-model"); err == nil { - t.Fatal("frozen options bypassed provider/harness validation") +} + +func TestSessionModelOptionsValidateAdmission(t *testing.T) { + provider := &v1.ModelProviderInput{Protocol: "responses", BaseURL: "https://example.com/v1", APIKey: "private-key"} + for _, engine := range []string{"mcode", "unknown", ""} { + if _, err := resolvedSessionModelOptions(provider, engine); err == nil { + t.Fatalf("invalid provider/harness configuration accepted for %q", engine) + } } - explicit, err := resolvedSessionModelOptions(provider, nil, "codex", "actual-model") - if err != nil { - t.Fatal(err) + if _, err := resolvedSessionModelOptions(nil, "codex"); err == nil { + t.Fatal("missing provider accepted") } - codex, ok := explicit["codex_provider"].(map[string]any) - if !ok || codex["base_url"] != provider.BaseURL || codex["bearer_token"] != provider.APIKey || len(codex) != 3 { - t.Fatal("explicit provider mapping changed") + provider.Protocol = "chat-completions" + if _, err := resolvedSessionModelOptions(provider, "codex"); err == nil { + t.Fatal("protocol alias accepted") } } diff --git a/services/agents-api/internal/execution/request.go b/services/agents-api/internal/execution/request.go index 000435f17..658f169f9 100644 --- a/services/agents-api/internal/execution/request.go +++ b/services/agents-api/internal/execution/request.go @@ -22,13 +22,13 @@ func (d *Dispatcher) executionRequest(ctx context.Context, session store.Session } options := map[string]any{} if snapshot.ModelProviderConfigured { - options, err = d.sessionModelOptions(ctx, session, snapshot.Agent.Model) + options, err = d.sessionModelOptions(ctx, session) if err != nil { return proto.PromptRequestPayload{}, err } } else if snapshot.Environment != nil && v1.ModelProviderRequired(snapshot.Environment.Type) { - // A Session from before deployment defaults moved into Core. Without a - // bundle its harness would fall back to a built-in endpoint and hang. + // Require the frozen bundle before dispatch so the harness cannot + // select an implicit provider endpoint. return proto.PromptRequestPayload{}, store.ErrModelProviderRequired } else if d.Options != nil { options, err = d.Options(ctx, session) diff --git a/services/agents-api/internal/store/agent_execution_defaults_http_test.go b/services/agents-api/internal/store/agent_execution_defaults_http_test.go index df7a70eb4..c6e27d2b9 100644 --- a/services/agents-api/internal/store/agent_execution_defaults_http_test.go +++ b/services/agents-api/internal/store/agent_execution_defaults_http_test.go @@ -96,10 +96,27 @@ func TestAgentExecutionDefaultsPublicSnapshotAndPrecedence(t *testing.T) { replacement := `{"agent_id":"` + agentID + `","environment":{"type":"openai_hosted"},"x_agents_core":{"model_provider":{"protocol":"responses","base_url":"https://override.example/v1","api_key":"override-canary"}}}` sid = id(call("POST", "/v1/agents/sessions", replacement, uuid.NewString(), 201)) assertSnapshot(sid, "model-original", "https://override.example/v1", "override-canary") + for _, protocol := range []string{"anthropic", "chat_completions"} { + crossProtocol := strings.Replace(replacement, `"protocol":"responses"`, `"protocol":"`+protocol+`"`, 1) + created := id(call("POST", "/v1/agents/sessions", crossProtocol, uuid.NewString(), 201)) + assertSnapshot(created, "model-original", "https://override.example/v1", "override-canary") + provider, err := st.SessionModelExecution(t.Context(), tenant, created) + if err != nil || provider == nil || provider.Protocol != protocol { + t.Fatal("Session did not freeze the chosen upstream protocol", err) + } + } + crossHarness := strings.Replace(modelOnly, `"model":"model-override"`, `"model":"model-override","x_agents_core":{"harness":"claude_sdk"}`, 1) + created := id(call("POST", "/v1/agents/sessions", crossHarness, uuid.NewString(), 201)) + assertSnapshot(created, "model-override", "https://saved.example/v1", "saved-canary") + resolved, err := st.GetSession(t.Context(), tenant, created) + frozen, providerErr := st.SessionModelExecution(t.Context(), tenant, created) + if err != nil || providerErr != nil || resolved.Engine != "claude_sdk" || frozen == nil || frozen.Protocol != "responses" { + t.Fatal("harness override changed the inherited upstream protocol", err, providerErr) + } for _, raw := range []string{ strings.Replace(replacement, `,"api_key":"override-canary"`, "", 1), - strings.Replace(replacement, `"protocol":"responses"`, `"protocol":"anthropic"`, 1), - strings.Replace(modelOnly, `"model":"model-override"`, `"model":"model-override","x_agents_core":{"harness":"claude_sdk"}`, 1), + strings.Replace(replacement, `"protocol":"responses"`, `"protocol":"unknown"`, 1), + strings.Replace(modelOnly, `"model":"model-override"`, `"model":"model-override","x_agents_core":{"harness":"unknown"}`, 1), strings.TrimSuffix(strings.Replace(body, `"type":"openai_hosted"`, `"type":"none"`, 1), "}") + `,"input":"test"}`, } { call("POST", "/v1/agents/sessions", raw, uuid.NewString(), 400) diff --git a/services/agents-api/internal/store/agent_model_execution_test.go b/services/agents-api/internal/store/agent_model_execution_test.go index 0b8fc607f..794449246 100644 --- a/services/agents-api/internal/store/agent_model_execution_test.go +++ b/services/agents-api/internal/store/agent_model_execution_test.go @@ -102,8 +102,8 @@ func TestAgentModelExecutionAtomicEncryptedSnapshot(t *testing.T) { if err := pool.QueryRow(ctx, "SELECT count(*) FROM agents WHERE tenant_id=$1", tenant).Scan(&count); err != nil || count != 1 { t.Fatal("failed create persisted partial Agent", err) } - if _, err := s.UpdateAgent(ctx, tenant, agent.ID, UpdateAgentInput{Configuration: []byte(`{"x_agents_core":{"harness":"claude_sdk"}}`)}); !errors.Is(err, ErrInvalidInput) { - t.Fatal("incompatible merged provider accepted", err) + if _, err := s.UpdateAgent(ctx, tenant, agent.ID, UpdateAgentInput{Configuration: []byte(`{"x_agents_core":{"harness":"claude_sdk"}}`)}); err != nil { + t.Fatal("Runtime-translatable merged provider rejected", err) } // Provider-only replacement preserves the existing harness. patch, _ := json.Marshal(map[string]any{"x_agents_core": map[string]any{"model_provider": replacement.SafeView()}}) @@ -111,7 +111,7 @@ func TestAgentModelExecutionAtomicEncryptedSnapshot(t *testing.T) { t.Fatal(err) } current, inherited, err = s.GetAgentForSession(ctx, tenant, agent.ID, true) - if err != nil || inherited == nil || *inherited != *replacement || !bytes.Contains(current.Configuration, []byte(`"harness": "codex"`)) { + if err != nil || inherited == nil || *inherited != *replacement || !bytes.Contains(current.Configuration, []byte(`"harness": "claude_sdk"`)) { t.Fatal("provider-only replacement failed", err) } if _, err := withoutKey.UpdateAgent(ctx, tenant, agent.ID, UpdateAgentInput{Configuration: []byte(`{"x_agents_core":{"harness":"codex"}}`)}); err != nil { diff --git a/services/agents-api/internal/store/deployment_model_providers_http_test.go b/services/agents-api/internal/store/deployment_model_providers_http_test.go index c34321d03..e3d3cfd78 100644 --- a/services/agents-api/internal/store/deployment_model_providers_http_test.go +++ b/services/agents-api/internal/store/deployment_model_providers_http_test.go @@ -92,7 +92,7 @@ func TestDeploymentModelProvidersHTTP(t *testing.T) { call("PUT", "/core/v1/harnesses/other/model-provider", coreKey, codexDefault, 404) for _, invalid := range []string{ `{"protocol":"responses","base_url":"http://deployment.example/v1","api_key":"invalid-canary"}`, - `{"protocol":"anthropic","base_url":"https://deployment.example/v1","api_key":"invalid-canary"}`, + `{"protocol":"unknown","base_url":"https://deployment.example/v1","api_key":"invalid-canary"}`, `{"protocol":"responses","base_url":"https://deployment.example/v1"}`, `{"protocol":"responses","base_url":"https://deployment.example/v1","api_key":"invalid-canary","api_key_configured":true}`, `{"protocol":"responses","base_url":"https://deployment.example/v1","api_key":"invalid-canary","context_window":-1}`, @@ -119,6 +119,19 @@ func TestDeploymentModelProvidersHTTP(t *testing.T) { t.Fatal("retrieved view differs") } + for _, protocol := range []string{"anthropic", "chat_completions"} { + bundle := strings.Replace(codexDefault, `"protocol":"responses"`, `"protocol":"`+protocol+`"`, 1) + safe := call("PUT", path, coreKey, bundle, 200) + if text(safe["protocol"]) != protocol || text(safe["harness"]) != "codex" || string(safe["api_key_configured"]) != "true" { + t.Fatal("safe deployment view changed the selected upstream protocol") + } + provider, err := st.DeploymentModelProvider(t.Context(), "codex") + if err != nil || provider == nil || provider.Protocol != protocol || provider.BaseURL != "https://deployment.example/v1" || provider.APIKey != "deployment-canary" { + t.Fatal("cross-protocol deployment bundle did not round trip", err) + } + } + call("PUT", path, coreKey, codexDefault, 200) + // Hosted Sessions freeze the default; later edits never reach them. hostedID := text(call("POST", "/v1/agents/sessions", projectKey, hosted, 201)["id"]) if providerOf(hostedID) != "deployment-canary" { diff --git a/services/agents-api/internal/store/deployment_model_providers_test.go b/services/agents-api/internal/store/deployment_model_providers_test.go index bae4b3662..fb4988757 100644 --- a/services/agents-api/internal/store/deployment_model_providers_test.go +++ b/services/agents-api/internal/store/deployment_model_providers_test.go @@ -26,9 +26,9 @@ func TestDeploymentModelProviderEncryptedAuditedAndReplaced(t *testing.T) { } started := time.Now() ctx := adminaudit.WithSource(t.Context(), adminaudit.Source{CredentialID: "abcd1234", ActorLabel: "console", RequestID: "request-set", TraceID: "trace-set"}) - provider := v1.ModelProviderInput{Protocol: "responses", BaseURL: "https://deployment.example/v1", APIKey: "deployment-key-canary"} - if _, err := s.SetDeploymentModelProvider(ctx, "codex", v1.ModelProviderInput{Protocol: "anthropic", BaseURL: provider.BaseURL, APIKey: "k"}); !errors.Is(err, ErrInvalidInput) { - t.Fatal("protocol unsupported by the harness accepted", err) + provider := v1.ModelProviderInput{Protocol: "anthropic", BaseURL: "https://deployment.example/v1", APIKey: "deployment-key-canary"} + if _, err := s.SetDeploymentModelProvider(ctx, "codex", v1.ModelProviderInput{Protocol: "unknown", BaseURL: provider.BaseURL, APIKey: "k"}); !errors.Is(err, ErrInvalidInput) { + t.Fatal("unknown upstream protocol accepted", err) } if _, err := New(pool).SetDeploymentModelProvider(ctx, "codex", provider); !errors.Is(err, ErrCredentialStorageUnavailable) { t.Fatal("key stored without encryption", err) @@ -55,7 +55,7 @@ func TestDeploymentModelProviderEncryptedAuditedAndReplaced(t *testing.T) { if _, err := NewWithCredentialCipher(pool, other).DeploymentModelProvider(ctx, "codex"); !errors.Is(err, ErrCredentialStorageUnavailable) { t.Fatal("wrong encryption key did not fail closed", err) } - replacement := v1.ModelProviderInput{Protocol: "responses", BaseURL: "https://replacement.example/v1", APIKey: "replacement-key"} + replacement := v1.ModelProviderInput{Protocol: "chat_completions", BaseURL: "https://replacement.example/v1", APIKey: "replacement-key"} if _, err := s.SetDeploymentModelProvider(ctx, "codex", replacement); err != nil { t.Fatal(err) } diff --git a/services/agents-api/internal/store/function_execution_native_test.go b/services/agents-api/internal/store/function_execution_native_test.go index c69f47865..75224a2ed 100644 --- a/services/agents-api/internal/store/function_execution_native_test.go +++ b/services/agents-api/internal/store/function_execution_native_test.go @@ -22,7 +22,7 @@ func TestNativeFunctionExecutionPersistsCallsResultsAndContinuity(t *testing.T) model, output, requests := nativeFunctionModel(t, home) defer model.Close() h.d.Options = func(context.Context, store.Session) (map[string]any, error) { - return map[string]any{"codex_provider": map[string]any{"base_url": model.URL + "/v1", "bearer_token": "synthetic-test-token"}}, nil + return map[string]any{"model_provider": map[string]any{"protocol": "responses", "base_url": model.URL + "/v1", "api_key": "synthetic-test-token"}}, nil } nativeID := "" for index := range 3 { diff --git a/services/agents-api/internal/store/function_public_native_test.go b/services/agents-api/internal/store/function_public_native_test.go index 3fb3b1182..2c74a8664 100644 --- a/services/agents-api/internal/store/function_public_native_test.go +++ b/services/agents-api/internal/store/function_public_native_test.go @@ -26,7 +26,7 @@ func TestNativePublicFunctionExecution(t *testing.T) { model, output, requests := nativeFunctionModel(t, home) defer model.Close() h.d.Options = func(context.Context, store.Session) (map[string]any, error) { - return map[string]any{"enable_features": []any{"multi_agent", "multi_agent_v2"}, "codex_provider": map[string]any{"base_url": model.URL + "/v1", "bearer_token": "synthetic-test-token"}}, nil + return map[string]any{"enable_features": []any{"multi_agent", "multi_agent_v2"}, "model_provider": map[string]any{"protocol": "responses", "base_url": model.URL + "/v1", "api_key": "synthetic-test-token"}}, nil } serverURL, token := nativePublicFunctionServer(t, h, ctx) outputPath, proofPath := filepath.Join(home, "function-output.json"), filepath.Join(home, "public-functions.json") diff --git a/services/agents-api/internal/store/function_stream_native_test.go b/services/agents-api/internal/store/function_stream_native_test.go index b7ae12b72..00659f9c6 100644 --- a/services/agents-api/internal/store/function_stream_native_test.go +++ b/services/agents-api/internal/store/function_stream_native_test.go @@ -25,7 +25,7 @@ func TestNativePublicFunctionStreamHelper(t *testing.T) { }) defer model.Close() h.d.Options = func(context.Context, store.Session) (map[string]any, error) { - return map[string]any{"codex_provider": map[string]any{"base_url": model.URL + "/v1", "bearer_token": "synthetic-test-token"}}, nil + return map[string]any{"model_provider": map[string]any{"protocol": "responses", "base_url": model.URL + "/v1", "api_key": "synthetic-test-token"}}, nil } serverURL, token := nativePublicFunctionServer(t, h, ctx) proofPath := filepath.Join(home, "public-function-stream.json") diff --git a/services/agents-api/internal/store/mcode_public_native_test.go b/services/agents-api/internal/store/mcode_public_native_test.go index 62d712810..ac27f68e5 100644 --- a/services/agents-api/internal/store/mcode_public_native_test.go +++ b/services/agents-api/internal/store/mcode_public_native_test.go @@ -87,11 +87,9 @@ func TestNativeMCodePublicExecution(t *testing.T) { if page, e := h.s.ListTurns(ctx, h.tenant, identity.Session, "", 100, true); e == nil { diagnostic, _ := json.Marshal(page) text := string(diagnostic) - if provider, ok := options["mcode_provider"].(map[string]any); ok { - if opts, ok := provider["options"].(map[string]any); ok { - if key, ok := opts["apiKey"].(string); ok && key != "" { - text = strings.ReplaceAll(text, key, "[REDACTED]") - } + if provider, ok := options["model_provider"].(map[string]any); ok { + if key, ok := provider["api_key"].(string); ok && key != "" { + text = strings.ReplaceAll(text, key, "[REDACTED]") } } _ = os.WriteFile(filepath.Join(home, "failed-turns.json"), []byte(text), 0600) diff --git a/services/agents-api/internal/store/model_protocol_native_test.go b/services/agents-api/internal/store/model_protocol_native_test.go new file mode 100644 index 000000000..73c226d78 --- /dev/null +++ b/services/agents-api/internal/store/model_protocol_native_test.go @@ -0,0 +1,155 @@ +package store_test + +import ( + "context" + "encoding/json" + "io" + "net/http/httptest" + "os" + "os/exec" + "path/filepath" + "testing" + "time" + + v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/device" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/api" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/execution" + "github.com/google/uuid" +) + +// TestNativeModelProtocolPublicExecution is opt-in and never fabricates a model +// response. Run separately for each private engine/protocol configuration. +func TestNativeModelProtocolPublicExecution(t *testing.T) { + python := os.Getenv("OAC_TEST_OFFICIAL_SDK_PYTHON") + binary := os.Getenv("OAC_TEST_NATIVE_DAEMON_BIN") + root := os.Getenv("OAC_TEST_NATIVE_PROOF_DIR") + optionsFile := os.Getenv("OAC_TEST_MODEL_PROTOCOL_OPTIONS") + if python == "" || binary == "" || root == "" || optionsFile == "" { + t.Skip("fixed SDK, native daemon, private model protocol options and proof directory required") + } + raw, err := os.ReadFile(optionsFile) + if err != nil { + t.Fatal("cannot read private model protocol options") + } + var options struct { + Engine string `json:"engine"` + Model string `json:"model"` + Provider v1.ModelProviderInput `json:"model_provider"` + } + if json.Unmarshal(raw, &options) != nil || options.Model == "" || options.Provider.BaseURL == "" || options.Provider.APIKey == "" { + t.Fatal("invalid private model protocol options") + } + switch options.Engine { + case "codex", "claude_sdk", "mcode": + default: + t.Fatal("unsupported native test engine") + } + switch options.Provider.Protocol { + case "anthropic", "responses", "chat_completions": + default: + t.Fatal("unsupported native test protocol") + } + optionsFile, err = filepath.Abs(optionsFile) + if err != nil { + t.Fatal("cannot resolve private model protocol options") + } + h := newDispatchHarness(t) + // Deliberately leave Dispatcher.Options unset. Only the public SDK Session + // creation freezes the deployment default used by the environment:none profile. + if h.d.Options != nil { + t.Fatal("fixture must not override public model provider admission") + } + home, err := os.MkdirTemp(root, "model-protocol-public-") + if err != nil { + t.Fatal("cannot create controlled evidence directory") + } + ctx, cancel := context.WithTimeout(t.Context(), 15*time.Minute) + defer cancel() + worker, err := execution.StartWorker(ctx, h.d) + if err != nil { + t.Fatal("cannot start native execution worker") + } + done := make(chan error, 1) + go func() { done <- worker.Run(ctx) }() + defer func() { + cancel() + select { + case <-done: + case <-time.After(20 * time.Second): + t.Error("native execution worker did not stop") + } + }() + token := uuid.NewString() + auth, err := newTestAuthenticator([]testAPIKey{{OrganizationID: "test", ProjectID: h.tenant, SubjectKind: "service_account", SubjectID: "owner", TokenSHA256: device.HashCredential(token), TenantID: h.tenant}}) + if err != nil { + t.Fatal("cannot create fixture authenticator") + } + handler, err := api.NewHandler(h.s, auth, options.Engine, api.WithExecution(worker), api.WithExecutionPolicy(h.d.Policy), api.WithModelProviderDefaults(func(context.Context, string) (*v1.ModelProviderInput, error) { return &options.Provider, nil })) + if err != nil { + t.Fatal("cannot create public API handler") + } + server := httptest.NewServer(handler) + defer server.Close() + stop := startNativeEngineDaemon(t, h, home, binary, options.Engine) + defer func() { stop() }() + evidence := filepath.Join(home, "public.json") + run := func(stage string) { + command := exec.CommandContext(ctx, python, "../../tests/official_model_protocol_native.py", server.URL, token, optionsFile, stage, evidence) + // Exceptions, SDK HTTP bodies and daemon diagnostics must never be echoed + // into the test log. The script writes only allowlisted proof summaries. + command.Stdout, command.Stderr = io.Discard, io.Discard + if command.Run() != nil { + t.Fatalf("native public model protocol %s failed; controlled evidence: %s", stage, evidence) + } + } + run("initial") + var proof struct { + Session string `json:"session"` + Calls []struct { + Turn string `json:"turn"` + Call string `json:"call"` + Success bool `json:"success"` + } `json:"calls"` + } + raw, err = os.ReadFile(evidence) + if err != nil || json.Unmarshal(raw, &proof) != nil || proof.Session == "" { + t.Fatal("missing controlled public proof") + } + expectedCalls := 3 + if options.Engine == "mcode" { + expectedCalls = 0 + } + if len(proof.Calls) != expectedCalls { + t.Fatal("unexpected public function call count") + } + failed := 0 + for _, item := range proof.Calls { + call, err := h.s.GetFunctionCall(ctx, h.tenant, proof.Session, item.Turn, item.Call) + if err != nil || !call.Applied { + t.Fatal("public function result lacks native delivery acknowledgement") + } + inputs, err := h.s.ListTurnInputs(ctx, h.tenant, proof.Session, item.Turn, 0, 100) + if err != nil || len(inputs) != 2 || inputs[0].Kind != "message" || inputs[1].Kind != "tool_result" { + t.Fatal("public function result input was lost or duplicated") + } + if !item.Success { + failed++ + } + } + if expectedCalls > 0 && failed != 1 { + t.Fatal("missing failed function-result scenario") + } + before, err := h.s.GetSessionExecutionBinding(ctx, h.tenant, proof.Session) + if err != nil || before.NativeSessionID == "" { + t.Fatal("native Session binding missing before restart") + } + stop() + stop = startNativeEngineDaemon(t, h, home, binary, options.Engine) + run("resume") + after, err := h.s.GetSessionExecutionBinding(ctx, h.tenant, proof.Session) + if err != nil || before.NativeSessionID != after.NativeSessionID { + t.Fatal("cold Session continuation changed native history") + } + t.Logf("Native public model protocol acceptance passed (engine=%s protocol=%s); controlled evidence: %s", options.Engine, options.Provider.Protocol, evidence) +} diff --git a/services/agents-api/internal/store/native_environment_test.go b/services/agents-api/internal/store/native_environment_test.go index 824941ad4..6708290eb 100644 --- a/services/agents-api/internal/store/native_environment_test.go +++ b/services/agents-api/internal/store/native_environment_test.go @@ -107,7 +107,7 @@ func TestNativeNoExecutionEnvironment(t *testing.T) { })) defer model.Close() h.d.Options = func(context.Context, store.Session) (map[string]any, error) { - return map[string]any{"enable_features": []any{"multi_agent", "multi_agent_v2"}, "model_verbosity": "high", "web_search": "live", "codex_provider": map[string]any{"base_url": model.URL + "/v1", "bearer_token": "synthetic-test-token"}}, nil + return map[string]any{"enable_features": []any{"multi_agent", "multi_agent_v2"}, "model_verbosity": "high", "web_search": "live", "model_provider": map[string]any{"protocol": "responses", "base_url": model.URL + "/v1", "api_key": "synthetic-test-token"}}, nil } first := h.message("first", "Return an answer.") h.finished(h.run(ctx, first.TurnID), store.TurnCompleted) diff --git a/services/agents-api/internal/store/prepared_dispatch_test.go b/services/agents-api/internal/store/prepared_dispatch_test.go index 8f45995ac..5d611abe8 100644 --- a/services/agents-api/internal/store/prepared_dispatch_test.go +++ b/services/agents-api/internal/store/prepared_dispatch_test.go @@ -163,9 +163,9 @@ func TestSelfHostedProviderReachesOnlyBoundExecutor(t *testing.T) { if frame.DecodePayload(&prepare) != nil { t.Fatal("invalid preparation") } - provider, _ := prepare.Configuration.AgentOptions["codex_provider"].(map[string]any) + provider, _ := prepare.Configuration.AgentOptions["model_provider"].(map[string]any) fixture := store.FixtureModelProvider("codex") - if provider["base_url"] != fixture.BaseURL || provider["bearer_token"] != fixture.APIKey { + if provider["base_url"] != fixture.BaseURL || provider["api_key"] != fixture.APIKey { t.Fatal("bound executor did not receive the frozen provider", prepare.Configuration.AgentOptions) } _ = bystander.conn.SetReadDeadline(time.Now().Add(500 * time.Millisecond)) diff --git a/services/agents-api/internal/store/session_model_execution.go b/services/agents-api/internal/store/session_model_execution.go index d66df3c2c..e12868733 100644 --- a/services/agents-api/internal/store/session_model_execution.go +++ b/services/agents-api/internal/store/session_model_execution.go @@ -1,9 +1,11 @@ package store import ( + "bytes" "context" "encoding/json" "errors" + "io" v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/db/sqlc" @@ -11,19 +13,11 @@ import ( "github.com/jackc/pgx/v5/pgtype" ) -// The embedded provider preserves the original flat ciphertext format. Native -// options exist only in historical snapshots frozen from the retired operator -// options file; they never enter public configuration and are no longer written. -type sessionModelExecution struct { - v1.ModelProviderInput - NativeOptions map[string]any `json:"native_options,omitempty"` -} - func (s *Store) saveSessionModelExecution(ctx context.Context, q *sqlc.Queries, tenant string, session pgtype.UUID, provider *v1.ModelProviderInput) error { if provider == nil { return nil } - raw, err := json.Marshal(sessionModelExecution{ModelProviderInput: *provider}) + raw, err := json.Marshal(provider) if err != nil { return err } @@ -35,33 +29,30 @@ func (s *Store) saveSessionModelExecution(ctx context.Context, q *sqlc.Queries, } func (s *Store) SessionModelExecution(ctx context.Context, tenant, session string) (*v1.ModelProviderInput, error) { - provider, _, err := s.SessionModelExecutionWithOptions(ctx, tenant, session) - return provider, err -} - -func (s *Store) SessionModelExecutionWithOptions(ctx context.Context, tenant, session string) (*v1.ModelProviderInput, map[string]any, error) { tenantID, err := parseID(tenant) if err != nil { - return nil, nil, err + return nil, err } sessionID, err := parseID(session) if err != nil { - return nil, nil, err + return nil, err } ciphertext, err := s.queries.GetSessionModelExecution(ctx, sqlc.GetSessionModelExecutionParams{TenantID: tenantID, SessionID: sessionID}) if err != nil { - return nil, nil, errors.New("session model execution configuration is unavailable") + return nil, errors.New("session model execution configuration is unavailable") } raw, err := s.credentialCipher.OpenModelExecution(ciphertext, tenant, session) if err != nil { - return nil, nil, errors.New("session model execution decryption is unavailable") + return nil, errors.New("session model execution decryption is unavailable") } - var frozen sessionModelExecution - if json.Unmarshal(raw, &frozen) != nil { - return nil, nil, errors.New("invalid stored model execution configuration") + var provider v1.ModelProviderInput + decoder := json.NewDecoder(bytes.NewReader(raw)) + decoder.DisallowUnknownFields() + if decoder.Decode(&provider) != nil || decoder.Decode(new(any)) != io.EOF { + return nil, errors.New("invalid stored model execution configuration") } - if err := frozen.ModelProviderInput.Validate(); err != nil { - return nil, nil, err + if err := provider.Validate(); err != nil { + return nil, err } - return &frozen.ModelProviderInput, frozen.NativeOptions, nil + return &provider, nil } diff --git a/services/agents-api/internal/store/session_model_execution_http_test.go b/services/agents-api/internal/store/session_model_execution_http_test.go index 622d9b6f5..c9f468e01 100644 --- a/services/agents-api/internal/store/session_model_execution_http_test.go +++ b/services/agents-api/internal/store/session_model_execution_http_test.go @@ -3,14 +3,15 @@ package store_test import ( "bytes" "encoding/json" + "net/http/httptest" + "strings" + "testing" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/device" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/api" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/credentialcrypto" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" "github.com/google/uuid" - "net/http/httptest" - "strings" - "testing" ) func TestModelExecutionHTTPWriteOnlyAndStrictAdmission(t *testing.T) { @@ -58,7 +59,19 @@ func TestModelExecutionHTTPWriteOnlyAndStrictAdmission(t *testing.T) { if w := call("POST", "/v1/agents/sessions", strings.Replace(body, "model-http-canary", "changed-key", 1), key); w.Code != 409 { t.Fatal("conflicting credentials accepted", w.Code) } - for _, invalid := range []string{strings.Replace(body, `"protocol":"responses"`, `"protocol":"anthropic"`, 1), strings.Replace(body, `"api_key":"model-http-canary"`, `"api_key":"model-http-canary","unknown":true`, 1), strings.Replace(body, `"type":"openai_hosted"`, `"type":"none"`, 1), strings.Replace(body, `"api_key":"model-http-canary"`, `"api_key":null`, 1)} { + for _, protocol := range []string{"anthropic", "chat_completions"} { + crossProtocol := strings.Replace(body, `"protocol":"responses"`, `"protocol":"`+protocol+`"`, 1) + response := call("POST", "/v1/agents/sessions", crossProtocol, uuid.NewString()) + var created struct{ ID string } + if response.Code != 201 || json.Unmarshal(response.Body.Bytes(), &created) != nil || created.ID == "" { + t.Fatalf("cross-protocol Session rejected: %d", response.Code) + } + provider, err := st.SessionModelExecution(t.Context(), tenant, created.ID) + if err != nil || provider == nil || provider.Protocol != protocol || provider.BaseURL != "https://example.com/v1" || provider.APIKey != "model-http-canary" { + t.Fatal("Session did not freeze its complete upstream bundle", err) + } + } + for _, invalid := range []string{strings.Replace(body, `"protocol":"responses"`, `"protocol":"unknown"`, 1), strings.Replace(body, `"api_key":"model-http-canary"`, `"api_key":"model-http-canary","unknown":true`, 1), strings.Replace(body, `"type":"openai_hosted"`, `"type":"none"`, 1), strings.Replace(body, `"api_key":"model-http-canary"`, `"api_key":null`, 1)} { if w := call("POST", "/v1/agents/sessions", invalid, uuid.NewString()); w.Code != 400 { t.Fatalf("invalid execution accepted: %d %s", w.Code, w.Body) } diff --git a/services/agents-api/internal/store/session_model_options_test.go b/services/agents-api/internal/store/session_model_options_test.go index 1b43ecc43..4415843e8 100644 --- a/services/agents-api/internal/store/session_model_options_test.go +++ b/services/agents-api/internal/store/session_model_options_test.go @@ -3,7 +3,6 @@ package store import ( "bytes" "encoding/json" - "reflect" "testing" v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" @@ -11,9 +10,7 @@ import ( "github.com/google/uuid" ) -// Sessions frozen from the retired operator options file keep their native -// options; new Sessions store only the flat provider bundle. -func TestHistoricalSessionNativeOptionsRemainReadable(t *testing.T) { +func TestSessionModelExecutionStoresOnlyProviderBundle(t *testing.T) { _, pool := testStore(t) cipher, err := credentialcrypto.New(bytes.Repeat([]byte{33}, 32)) if err != nil { @@ -27,15 +24,18 @@ func TestHistoricalSessionNativeOptionsRemainReadable(t *testing.T) { if err != nil { t.Fatal(err) } - flat, frozen, err := st.SessionModelExecutionWithOptions(ctx, tenant, session.ID) - if err != nil || flat == nil || *flat != *provider || frozen != nil { + flat, err := st.SessionModelExecution(ctx, tenant, session.ID) + if err != nil || flat == nil || *flat != *provider { t.Fatal("new Session did not store a flat provider bundle", err) } options := map[string]any{"codex_provider": map[string]any{ "base_url": provider.BaseURL, "bearer_token": provider.APIKey, "wire_api": "responses", "http_headers": map[string]any{"x-deployment-secret": "header-secret-canary"}, }, "mode": "trusted-deployment-mode"} - historical, err := json.Marshal(sessionModelExecution{ModelProviderInput: *provider, NativeOptions: options}) + historical, err := json.Marshal(struct { + v1.ModelProviderInput + NativeOptions map[string]any `json:"native_options"` + }{*provider, options}) if err != nil { t.Fatal(err) } @@ -46,8 +46,7 @@ func TestHistoricalSessionNativeOptionsRemainReadable(t *testing.T) { if _, err := pool.Exec(ctx, "UPDATE session_model_execution SET encrypted_config=$2 WHERE session_id=$1", session.ID, ciphertext); err != nil { t.Fatal(err) } - legacy, legacyOptions, err := NewWithCredentialCipher(pool, cipher).SessionModelExecutionWithOptions(ctx, tenant, session.ID) - if err != nil || legacy == nil || *legacy != *provider || !reflect.DeepEqual(legacyOptions, options) { - t.Fatal("historical native options could not be read", err) + if _, err := NewWithCredentialCipher(pool, cipher).SessionModelExecution(ctx, tenant, session.ID); err == nil { + t.Fatal("retired native options accepted in provider bundle") } } diff --git a/services/agents-api/tests/official_model_protocol_native.py b/services/agents-api/tests/official_model_protocol_native.py new file mode 100644 index 000000000..2c697c023 --- /dev/null +++ b/services/agents-api/tests/official_model_protocol_native.py @@ -0,0 +1,215 @@ +"""Opt-in real-model protocol acceptance through public Sessions and the pinned SDK. + +OAC_TEST_MODEL_PROTOCOL_OPTIONS points to a private JSON file containing engine, +model and model_provider. Evidence contains only scenario markers, resource IDs, +event counts and controlled check names; never provider options or raw errors. +""" +import importlib.metadata +import json +import logging +import os +import sys +import uuid +from pathlib import Path + +sys.dont_write_bytecode = True +logging.disable(logging.CRITICAL) +os.environ.pop("OPENAI_LOG", None) + + +class CheckFailed(Exception): + pass + + +def require(condition, code): + if not condition: + raise CheckFailed(code) + + +def main(): + base, token, options_file, stage, output = sys.argv[1:] + record = {"stage": stage, "checks": [], "calls": [], "runs": []} + + def save(): + path = Path(output) + descriptor = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600) + with os.fdopen(descriptor, "w") as handle: + json.dump(record, handle, indent=2) + + client = None + try: + import httpx2 + from openai import OpenAI + + require(stage in {"initial", "resume"}, "invalid_stage") + settings = json.loads(Path(options_file).read_text()) + engine, model, provider = settings["engine"], settings["model"], settings["model_provider"] + require(engine in {"codex", "claude_sdk", "mcode"}, "invalid_engine") + require(provider["protocol"] in {"anthropic", "responses", "chat_completions"}, "invalid_protocol") + require(isinstance(model, str) and bool(model), "missing_model") + require(isinstance(provider.get("api_key"), str) and bool(provider["api_key"]), "missing_provider_key") + pin = json.loads((Path(__file__).resolve().parents[3] / "contracts/agents-api/upstream.json").read_text()) + distribution = importlib.metadata.distribution("openai") + source = json.loads(distribution.read_text("direct_url.json") or "{}") + require(distribution.version == pin["sdk_version"] and + source.get("vcs_info", {}).get("commit_id") == pin["commit"], "official_sdk_pin_mismatch") + if stage == "resume": + record = json.loads(Path(output).read_text()) + record["stage"] = stage + require(record["engine"] == engine and record["protocol"] == provider["protocol"], "resume_configuration_changed") + else: + record.update(engine=engine, protocol=provider["protocol"], + marker="PROTOCOL-MEMORY-" + uuid.uuid4().hex[:12]) + save() + client = OpenAI(base_url=base + "/v1", api_key=token, max_retries=0, + _strict_response_validation=True, + http_client=httpx2.Client(trust_env=False, timeout=240)) + sessions = client.beta.agents.sessions + + def message(text): + return {"type": "agent.session.input.message", "input": [ + {"role": "user", "content": [{"type": "input_text", "text": text}]}]} + + def answer(sid, expected): + items = [item.to_dict() for item in sessions.items.list(sid, order="asc", limit=100).data] + replies = [item for item in items if item["type"] == "message" and item["role"] == "assistant"] + require(bool(replies), "missing_assistant_answer") + actual = " ".join(part["text"] for part in replies[-1]["content"] if part["type"] == "output_text") + require(expected in actual, "answer_marker_mismatch") + outputs = {item["call_id"]: item for item in items if item["type"] == "function_call_output"} + for call in record["calls"]: + require(call["call"] in outputs, "function_result_not_persisted") + value = outputs[call["call"]] + require((value.get("error") is None) == call["success"], "function_result_error_state_mismatch") + + def run(sid, prompt, scenario, expected=None, tool_result=None, failed=False, cancel=False, creation=None): + events, deltas, handled, cancellation = [], 0, False, False + with (creation if creation is not None else sessions.events.stream(sid, timeout=240)) as stream: + if creation is None: + sessions.events.create(sid, events=[message(prompt)], idempotency_key=str(uuid.uuid4())) + for event in stream: + # Retain event types only, never the event's provider content. + kind = event.type + require(kind not in {"agent.session.failed", "agent.session.turn.failed"}, "native_turn_failed") + events.append(kind) + if kind == "agent.session.turn.output_text.delta": + deltas += 1 + if cancel and not cancellation: + sessions.events.create(sid, events=[{"type": "agent.session.input.cancel"}]) + cancellation = True + if kind == "agent.session.requires_action": + require(engine != "mcode" and tool_result is not None and not handled, "unexpected_function_action") + require(len(event.session.required_actions) == 1, "unexpected_parallel_function_actions") + action = event.session.required_actions[0] + require(action.name == "protocol_lookup", "unexpected_function_name") + result = {"type": "agent.session.input.tool_result", "turn_id": action.turn_id, + "call_id": action.call_id, "success": not failed, "output": tool_result} + if failed: + result["error"] = "Fixture lookup unavailable. Reply PROTOCOL-TOOL-ERROR and do not retry." + sessions.events.create(sid, events=[result], idempotency_key="protocol-" + action.call_id) + record["calls"].append({"turn": action.turn_id, "call": action.call_id, "success": not failed}) + handled = True + if kind == "agent.session.idle": + break + else: + raise CheckFailed("stream_ended_without_idle") + terminal = "agent.session.turn.cancelled" if cancel else "agent.session.turn.completed" + require("agent.session.turn.created" in events and terminal in events, "missing_turn_lifecycle") + require(events.index("agent.session.turn.created") < events.index(terminal) < len(events) - 1, + "turn_lifecycle_order") + require(deltas > 0, "missing_sse_text_delta") + require(handled == (tool_result is not None), "function_action_not_observed") + if cancel: + require(cancellation, "cancel_not_submitted") + current = sessions.turns.list(sid, order="desc", limit=1).data + require(bool(current) and current[0].status == ("cancelled" if cancel else "completed"), + "persisted_turn_status_mismatch") + usage = current[0].to_dict().get("usage") + # Adapters preserve native usage scope. A partial native breakdown + # remains null publicly; protocol conversion must not invent it. + if usage is not None: + require(isinstance(usage, dict) and usage.get("input_tokens", -1) >= 0 and + usage.get("output_tokens", -1) >= 0 and + usage.get("total_tokens") == usage["input_tokens"] + usage["output_tokens"], + "inconsistent_public_usage") + if expected is not None: + answer(sid, expected) + record["runs"].append({"scenario": scenario, "text_deltas": deltas, "completed": not cancel, + "cancelled": cancel, "function_result": handled, + "usage_present": isinstance(usage, dict)}) + record["checks"].append(scenario) + save() + + if stage == "initial": + marker = record["marker"] + agent = {"model": model, "instructions": ( + "Follow the user's exact requested tool calls and remember supplied markers. " + "Only call protocol_lookup when explicitly requested. After a tool result, answer in text and do not repeat the call.")} + if engine != "mcode": + agent["tools"] = [{"type": "function", "name": "protocol_lookup", + "description": "Return the requested fixture value.", + "parameters": {"type": "object", "properties": {"key": {"type": "string"}}, + "required": ["key"], "additionalProperties": False}}] + prompt = "Call protocol_lookup exactly once with key first. Then reply with the text returned by the tool." + else: + agent["instructions"] = "Remember user-supplied markers. Answer in text and do not use tools." + prompt = "Remember " + marker + ". Reply exactly " + marker + "." + record["not_exercised"] = ["public_functions_not_supported_by_mcode", "function_error", "cancel"] + creation = sessions.create(agent=agent, environment={"type": "none"}, input=prompt, stream=True) + first = next(creation) + sid = first.session.id + record["session"] = sid + save() + public = sessions.retrieve(sid).to_dict() + require(provider["api_key"] not in json.dumps(public), "provider_key_exposed_by_public_session") + if engine == "mcode": + run(sid, prompt, "native_protocol_memory_first_turn", expected=marker, creation=creation) + else: + first_value = "PROTOCOL-FIRST-" + uuid.uuid4().hex[:12] + run(sid, prompt, "function_text_first_turn", expected=first_value, + tool_result=first_value, creation=creation) + run(sid, "Call protocol_lookup exactly once with key second. Remember its result and repeat it verbatim.", + "function_text_second_turn", expected=marker, tool_result=marker) + run(sid, "Call protocol_lookup once with key failure. If it fails, reply PROTOCOL-TOOL-ERROR and do not retry.", + "function_error_result", expected="PROTOCOL-TOOL-ERROR", tool_result="Lookup failed.", failed=True) + run(sid, "Do not call tools. Print integers 1 through 10000, one per line. Begin with 1 immediately.", + "cancel_during_sse_text", cancel=True) + run(sid, "Without calling tools, repeat the PROTOCOL-MEMORY marker returned by the second lookup.", + "continue_after_cancel", expected=marker) + else: + sid = record["session"] + run(sid, "Without calling tools, repeat the PROTOCOL-MEMORY marker you remembered earlier, and nothing else.", + "cold_daemon_session_continuation", expected=record["marker"]) + turns = sessions.turns.list(sid, order="asc", limit=100).data + require(len(turns) == (2 if engine == "mcode" else 6), "unexpected_turn_count") + record["passed"] = True + save() + except CheckFailed as failure: + record["failure"] = {"stage": stage, "check": str(failure)} + save() + return 1 + except Exception as failure: + # API and validation exceptions can include private request/response + # bodies. Do not format them or persist their traceback. + record["failure"] = {"stage": stage, "check": "client_or_fixture_exception"} + status = getattr(failure, "status_code", None) + if isinstance(status, int): + record["failure"]["http_status"] = status + trace = failure.__traceback__ + while trace: + if trace.tb_frame.f_code.co_filename == __file__: + record["failure"]["script_line"] = trace.tb_lineno + trace = trace.tb_next + save() + return 1 + finally: + if client is not None: + client.close() + return 0 + + +if __name__ == "__main__": + try: + sys.exit(main()) + except Exception: + sys.exit(1) diff --git a/services/agents-api/tools/microsandbox-provider/go.mod b/services/agents-api/tools/microsandbox-provider/go.mod index 9de6b567f..0c2e4bb9b 100644 --- a/services/agents-api/tools/microsandbox-provider/go.mod +++ b/services/agents-api/tools/microsandbox-provider/go.mod @@ -1,6 +1,6 @@ module github.com/MiniMax-AI-Dev/parsar/services/agents-api/tools/microsandbox-provider -go 1.25.13 +go 1.26.8 require ( github.com/MiniMax-AI-Dev/parsar v0.0.0