From 86347a8acca7781114c1a70dc8e04e3da3576861 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=94=A1=E5=8F=8A?= <522caiji@gmail.com> Date: Sun, 6 Sep 2026 23:25:05 +0800 Subject: [PATCH 1/2] feat: stick multi-turn chats to one account from content Hash a structured first-user fingerprint (text or image, including Anthropic/Responses converters) into the process-local session LRU so later turns reuse the same account without X-CLI2API-Session. --- CHANGELOG.md | 2 + README.md | 4 +- README_EN.md | 4 +- deploy/README.md | 6 +- internal/api/chat.go | 21 +- internal/api/chat_usage_test.go | 49 +++- internal/api/compat.go | 6 +- internal/executor/chat.go | 16 +- internal/executor/session_affinity.go | 20 ++ internal/executor/session_affinity_test.go | 129 +++++++++- internal/translate/compat_test.go | 69 ++++++ internal/translate/session_seed.go | 262 +++++++++++++++++++++ internal/translate/session_seed_test.go | 256 ++++++++++++++++++++ 13 files changed, 815 insertions(+), 29 deletions(-) create mode 100644 internal/translate/session_seed.go create mode 100644 internal/translate/session_seed_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 83d5798..e455e7f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,10 +7,12 @@ Write each change in both `### English` and `### 中文` under `## Unreleased`. ### English +- Keep multi-turn conversations on the same account from the first user message by default, including image-only turns, without requiring `X-CLI2API-Session` - Cache `GET /api/models` for 5 minutes so the console catalog page does not re-hit WorkBuddy or Trae on every load; `?refresh=1` still fetches live. Overview stays uncached. ### 中文 +- 同一段多轮对话默认按首条用户消息（含纯图片）粘到同一个账号，不再需要 `X-CLI2API-Session` - `GET /api/models` 缓存 5 分钟，控制台模型页不再每次都打 WorkBuddy / Trae 目录；`?refresh=1` 仍即时拉取。Overview 不缓存。 ## 0.2.48 - 2026-09-06 diff --git a/README.md b/README.md index 8cabc22..cd5b7f7 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ Base URL: http://127.0.0.1:3010/v1 API Key: <首次启动时生成的 Key> ``` -不指定账号时，调度器自动选择可用账号；需要固定账号时加请求头 `X-Qoder-Account: acc_...`（历史命名，适用于所有 provider）。除 Chat Completions 外，也可使用 Anthropic `POST /v1/messages` 与 OpenAI `POST /v1/responses`；两者要求请求携带完整对话，不支持 `previous_response_id` / `conversation` 服务端续接。需要会话粘性时，可设置 `X-CLI2API-Session`；curl / PowerShell 示例见 [部署说明](deploy/README.md)。 +不指定账号时，调度器自动选择可用账号；需要固定账号时加请求头 `X-Qoder-Account: acc_...`（历史命名，适用于所有 provider）。除 Chat Completions 外，也可使用 Anthropic `POST /v1/messages` 与 OpenAI `POST /v1/responses`；两者要求请求携带完整对话，不支持 `previous_response_id` / `conversation` 服务端续接。同一段对话默认按首条用户消息（含纯图片）粘到同一个账号；也可显式设置 `X-CLI2API-Session`。curl / PowerShell 示例见 [部署说明](deploy/README.md)。 ## 工作方式 @@ -87,7 +87,7 @@ CLI2API 是本地网关：不提供账号、额度或官方 API 服务，不做 - Anthropic `/v1/messages` 与 OpenAI `/v1/responses` 的无状态文本 / 函数工具适配层 - WorkBuddy 每日签到与 token 保活（账号级开关，默认关闭；控制台可立即签到 / 刷新积分） -- 会话粘性路由：通过 `X-CLI2API-Session` 优先复用同一账号，并在失败时按规则切换 +- 会话粘性路由：默认按对话内容（首条用户消息，含纯图片）复用同一账号，也可设置 `X-CLI2API-Session`，并在失败时按规则切换 - 请求历史按账号过滤，以及请求状态、延迟、Token 和用量统计 **长期** diff --git a/README_EN.md b/README_EN.md index f65f881..d08b4ff 100644 --- a/README_EN.md +++ b/README_EN.md @@ -50,7 +50,7 @@ Base URL: http://127.0.0.1:3010/v1 API Key: ``` -Without an account header the scheduler picks a ready account; pin a request with the `X-Qoder-Account: acc_...` header (a historical name that applies to every provider). Anthropic `POST /v1/messages` and OpenAI `POST /v1/responses` are also available; both require the complete conversation in each request and do not support server-side continuation through `previous_response_id` / `conversation`. Use `X-CLI2API-Session` when session-sticky routing is desired. curl / PowerShell examples in the [deployment guide](deploy/README.md). +Without an account header the scheduler picks a ready account; pin a request with the `X-Qoder-Account: acc_...` header (a historical name that applies to every provider). Anthropic `POST /v1/messages` and OpenAI `POST /v1/responses` are also available; both require the complete conversation in each request and do not support server-side continuation through `previous_response_id` / `conversation`. Multi-turn requests stick to the same account from the first user message (including image-only turns) by default; `X-CLI2API-Session` remains an optional override. curl / PowerShell examples in the [deployment guide](deploy/README.md). ## How it works @@ -87,7 +87,7 @@ CLI2API is a local gateway: it does not provide accounts, quotas, or an official - Stateless text and function-tool adapters for Anthropic `/v1/messages` and OpenAI `/v1/responses`; image input where the provider supports it - WorkBuddy daily check-in and token keepalive (per-account opt-in, off by default; console can check in now / refresh credits) -- Session-sticky routing via `X-CLI2API-Session`, with rule-based failover when the bound account cannot serve the request +- Session-sticky routing from conversation content (first user message, including image-only turns), or via `X-CLI2API-Session`, with rule-based failover when the bound account cannot serve the request - Request history filtering by account, plus request status, latency, token, and usage statistics **Longer term** diff --git a/deploy/README.md b/deploy/README.md index d2e8e16..6a21cd6 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -85,8 +85,10 @@ Invoke-RestMethod -Method Post -Uri "http://127.0.0.1:3010/v1/chat/completions" ``` Pin a request to a specific account with the `X-Qoder-Account: acc_...` header -(a historical header name that applies to every provider). Add `X-CLI2API-Session` -when consecutive requests should prefer the same account. +(a historical header name that applies to every provider). Consecutive turns of +the same conversation prefer the same account from the first user message +(including image-only turns); +`X-CLI2API-Session` remains an optional override. ## 5. Configuration diff --git a/internal/api/chat.go b/internal/api/chat.go index 90fc5c6..b460872 100644 --- a/internal/api/chat.go +++ b/internal/api/chat.go @@ -659,7 +659,7 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) { RequestedModel: firstNonEmpty(publicModel, req.Model), }) ctx := executor.WithAllowedProviders(executor.WithRequestID(r.Context(), requestID), identity.AllowedProviders) - if sessionKey := requestSessionKey(r, identity); sessionKey != "" { + if sessionKey := requestSessionKey(r, identity, req); sessionKey != "" { ctx = executor.WithSessionKey(ctx, sessionKey) } w.Header().Set("X-Request-Id", requestID) @@ -706,7 +706,7 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) { } s.finishRequestLog(requestID, started, req, publicModel, upstream.AccountID, firstNonEmpty(upstream.Provider, providerFilter), upstream.Routing, status, ttfb, &stats, relayErr, upstream.AttemptCount) if relayErr == nil { - s.executor.CommitSession(ctx, upstream.Routing, upstream.AccountID) + s.executor.CommitSession(ctx, req, upstream.Routing, upstream.AccountID) } if relayErr != nil { // The upstream answered 200 and failed inside the stream, so the @@ -882,11 +882,18 @@ func writeClassifiedErr(w http.ResponseWriter, err error) { writeErr(w, classified.Status, classified.Code, classified.Message) } -func requestSessionKey(r *http.Request, identity auth.Identity) string { - if r == nil { - return "" +func requestSessionKey(r *http.Request, identity auth.Identity, req translate.ChatRequest) string { + raw := "" + kind := "content" + if r != nil { + raw = strings.TrimSpace(r.Header.Get("X-CLI2API-Session")) + if raw != "" { + kind = "header" + } + } + if raw == "" { + raw = translate.ContentSessionSeed(req) } - raw := strings.TrimSpace(r.Header.Get("X-CLI2API-Session")) if raw == "" { return "" } @@ -894,7 +901,7 @@ func requestSessionKey(r *http.Request, identity auth.Identity) string { if identity.Kind == auth.KindKey && identity.KeyID != "" { namespace = "key:" + identity.KeyID } - sum := sha256.Sum256([]byte(namespace + "\x00" + raw)) + sum := sha256.Sum256([]byte(namespace + "\x00" + kind + "\x00" + raw)) return hex.EncodeToString(sum[:]) } diff --git a/internal/api/chat_usage_test.go b/internal/api/chat_usage_test.go index 02158b8..bca52bf 100644 --- a/internal/api/chat_usage_test.go +++ b/internal/api/chat_usage_test.go @@ -12,6 +12,7 @@ import ( "github.com/caigee-cmd/cli2api/internal/auth" "github.com/caigee-cmd/cli2api/internal/executor" "github.com/caigee-cmd/cli2api/internal/providers" + "github.com/caigee-cmd/cli2api/internal/translate" ) func intPtr(value int) *int { return &value } @@ -58,24 +59,64 @@ func TestRequestSessionKeyRequiresHeaderAndScopesToIdentity(t *testing.T) { withHeader := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil) withHeader.Header.Set("X-CLI2API-Session", "session-a") firstKey := auth.Identity{Kind: auth.KindKey, KeyID: "key-1"} - if got := requestSessionKey(withHeader, firstKey); got == "" || got == "session-a" { + emptyReq := translate.ChatRequest{} + if got := requestSessionKey(withHeader, firstKey, emptyReq); got == "" || got == "session-a" { t.Fatalf("header key = %q", got) } withSameHeader := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil) withSameHeader.Header.Set("X-CLI2API-Session", "session-a") - if requestSessionKey(withHeader, firstKey) != requestSessionKey(withSameHeader, firstKey) { + if requestSessionKey(withHeader, firstKey, emptyReq) != requestSessionKey(withSameHeader, firstKey, emptyReq) { t.Fatal("same header should derive the same opaque key") } withoutHeader := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil) - if got := requestSessionKey(withoutHeader, firstKey); got != "" { + if got := requestSessionKey(withoutHeader, firstKey, emptyReq); got != "" { t.Fatalf("key without header = %q", got) } secondKey := auth.Identity{Kind: auth.KindKey, KeyID: "key-2"} - if requestSessionKey(withHeader, firstKey) == requestSessionKey(withHeader, secondKey) { + if requestSessionKey(withHeader, firstKey, emptyReq) == requestSessionKey(withHeader, secondKey, emptyReq) { t.Fatal("same session header must be isolated by API key") } } +func TestRequestSessionKeyFallsBackToContentSeed(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil) + identity := auth.Identity{Kind: auth.KindKey, KeyID: "key-1"} + first := translate.ChatRequest{ + Model: "glm-5.2", + Messages: []translate.ChatMessage{{Role: "user", Content: "plan the refactor"}}, + } + later := translate.ChatRequest{ + Model: "glm-5.2", + Messages: []translate.ChatMessage{ + {Role: "user", Content: "plan the refactor"}, + {Role: "assistant", Content: "ok"}, + {Role: "user", Content: "continue"}, + }, + } + got := requestSessionKey(req, identity, first) + if got == "" { + t.Fatal("expected content-derived session key") + } + if requestSessionKey(req, identity, later) != got { + t.Fatal("later turn must keep the same content-derived session key") + } + other := translate.ChatRequest{ + Model: "glm-5.2", + Messages: []translate.ChatMessage{{Role: "user", Content: "a different conversation"}}, + } + if requestSessionKey(req, identity, other) == got { + t.Fatal("different first user message must not share a session key") + } + headerReq := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil) + headerReq.Header.Set("X-CLI2API-Session", "explicit") + if requestSessionKey(headerReq, identity, first) == got { + t.Fatal("explicit header must outrank the content seed") + } + if requestSessionKey(req, auth.Identity{Kind: auth.KindKey, KeyID: "key-2"}, first) == got { + t.Fatal("content-derived session keys must stay isolated by API key") + } +} + func TestBuildChatUsagePreservesZeroPromptCacheTokens(t *testing.T) { usage := buildChatUsage(executor.ChatResult{ CacheReadTokens: intPtr(0), diff --git a/internal/api/compat.go b/internal/api/compat.go index a9d37fc..8f2e0b4 100644 --- a/internal/api/compat.go +++ b/internal/api/compat.go @@ -63,7 +63,7 @@ func (s *Server) prepareCompatibilityExecution(r *http.Request, request translat RequestedModel: firstNonEmpty(publicModel, request.Model), }) ctx := executor.WithAllowedProviders(executor.WithRequestID(r.Context(), requestID), identity.AllowedProviders) - if sessionKey := requestSessionKey(r, identity); sessionKey != "" { + if sessionKey := requestSessionKey(r, identity, request); sessionKey != "" { ctx = executor.WithSessionKey(ctx, sessionKey) } return compatibilityExecution{ @@ -135,7 +135,7 @@ func (s *Server) handleAnthropicMessagesStream(w http.ResponseWriter, r *http.Re ttfb := streamTTFB(execution.started, upstream.TTFBMs, stats) s.finishCompatibility(execution, upstream.AccountID, upstream.Provider, upstream.Routing, status, ttfb, &stats, relayErr, upstream.AttemptCount) if relayErr == nil { - s.executor.CommitSession(execution.ctx, upstream.Routing, upstream.AccountID) + s.executor.CommitSession(execution.ctx, execution.request, upstream.Routing, upstream.AccountID) return } if !isStreamClientDisconnect(relayErr) { @@ -203,7 +203,7 @@ func (s *Server) handleResponsesStream(w http.ResponseWriter, r *http.Request, e ttfb := streamTTFB(execution.started, upstream.TTFBMs, stats) s.finishCompatibility(execution, upstream.AccountID, upstream.Provider, upstream.Routing, status, ttfb, &stats, relayErr, upstream.AttemptCount) if relayErr == nil { - s.executor.CommitSession(execution.ctx, upstream.Routing, upstream.AccountID) + s.executor.CommitSession(execution.ctx, execution.request, upstream.Routing, upstream.AccountID) return } if !isStreamClientDisconnect(relayErr) { diff --git a/internal/executor/chat.go b/internal/executor/chat.go index bd17467..5f42dc9 100644 --- a/internal/executor/chat.go +++ b/internal/executor/chat.go @@ -165,11 +165,11 @@ func (e ChatExecutor) stickyEscapeReason(plan *routingPlan) string { // CommitSession binds a successfully completed stream. The executor cannot know // whether an SSE response reached [DONE], so the relay calls this only after it // has finished without an upstream or client error. -func (e ChatExecutor) CommitSession(ctx context.Context, routing, accountID string) { +func (e ChatExecutor) CommitSession(ctx context.Context, req translate.ChatRequest, routing, accountID string) { if routing == routingPin || e.SessionAffinity == nil { return } - e.SessionAffinity.Bind(sessionKeyFromContext(ctx), accountID) + e.SessionAffinity.Bind(resolveSessionKey(ctx, req), accountID) } func itemProvider(item accounts.Item) string { @@ -193,15 +193,15 @@ func itemServesPublicModel(item accounts.Item, publicModel string) bool { return false } -func (e ChatExecutor) prepareRouting(ctx context.Context, prefer, providerFilter, publicModel string) (string, string, string, routingPlan) { +func (e ChatExecutor) prepareRouting(ctx context.Context, prefer, providerFilter string, req translate.ChatRequest) (string, string, string, routingPlan) { prefer = strings.TrimSpace(prefer) providerFilter = strings.ToLower(strings.TrimSpace(providerFilter)) + publicModel := req.Model if prefer != "" { - return prefer, providerFilter, "", routingPlan{Source: routingPin} + return prefer, providerFilter, "", routingPlan{Source: routingPin, PublicModel: publicModel} } - plan := routingPlan{Source: routingPool, SessionKey: sessionKeyFromContext(ctx)} - plan.PublicModel = publicModel + plan := routingPlan{Source: routingPool, SessionKey: resolveSessionKey(ctx, req), PublicModel: publicModel} if plan.SessionKey == "" || e.SessionAffinity == nil || e.Pool == nil { return "", providerFilter, "", plan } @@ -579,7 +579,7 @@ func (e ChatExecutor) ChatNonStream(ctx context.Context, req translate.ChatReque if err != nil { return ChatResult{}, err } - prefer, providerFilter, regionFilter, routing = e.prepareRouting(ctx, prefer, providerFilter, req.Model) + prefer, providerFilter, regionFilter, routing = e.prepareRouting(ctx, prefer, providerFilter, req) excluded := map[string]struct{}{} var lastErr error if regionFilter == "" && prefer != "" && e.Pool != nil { @@ -978,7 +978,7 @@ func (e ChatExecutor) ChatStreamProxy(ctx context.Context, req translate.ChatReq if err != nil { return StreamResult{}, err } - prefer, providerFilter, regionFilter, routing = e.prepareRouting(ctx, prefer, providerFilter, req.Model) + prefer, providerFilter, regionFilter, routing = e.prepareRouting(ctx, prefer, providerFilter, req) excluded := map[string]struct{}{} var lastErr error if regionFilter == "" && prefer != "" && e.Pool != nil { diff --git a/internal/executor/session_affinity.go b/internal/executor/session_affinity.go index 02a18fa..a4152b3 100644 --- a/internal/executor/session_affinity.go +++ b/internal/executor/session_affinity.go @@ -3,9 +3,13 @@ package executor import ( "container/list" "context" + "crypto/sha256" + "encoding/hex" "strings" "sync" "time" + + "github.com/caigee-cmd/cli2api/internal/translate" ) const ( @@ -30,6 +34,22 @@ func sessionKeyFromContext(ctx context.Context) string { return strings.TrimSpace(key) } +func contentSessionKey(req translate.ChatRequest) string { + seed := translate.ContentSessionSeed(req) + if seed == "" { + return "" + } + sum := sha256.Sum256([]byte("content\x00" + seed)) + return hex.EncodeToString(sum[:]) +} + +func resolveSessionKey(ctx context.Context, req translate.ChatRequest) string { + if key := sessionKeyFromContext(ctx); key != "" { + return key + } + return contentSessionKey(req) +} + type sessionBinding struct { key string accountID string diff --git a/internal/executor/session_affinity_test.go b/internal/executor/session_affinity_test.go index 34b1c49..03eb386 100644 --- a/internal/executor/session_affinity_test.go +++ b/internal/executor/session_affinity_test.go @@ -147,7 +147,7 @@ func TestChatStreamProxySessionAffinity(t *testing.T) { t.Fatal(err) } first.Response.Body.Close() - executor.CommitSession(ctx, first.Routing, first.AccountID) + executor.CommitSession(ctx, req, first.Routing, first.AccountID) second, err := executor.ChatStreamProxy(ctx, req, "", "") if err != nil || second.AccountID != "a" || second.Routing != routingSticky { t.Fatalf("second = %+v, err=%v", second, err) @@ -155,6 +155,133 @@ func TestChatStreamProxySessionAffinity(t *testing.T) { second.Response.Body.Close() } +func TestChatNonStreamContentSessionAffinityWithoutExplicitKey(t *testing.T) { + server := func(id string) *httptest.Server { + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + _, _ = io.WriteString(w, `{"model":"glm-5.2","choices":[{"message":{"content":"`+id+`"},"finish_reason":"stop"}],"usage":{"source":"upstream"}}`) + })) + } + a := server("a") + defer a.Close() + b := server("b") + defer b.Close() + + pool := accounts.NewPool(nil, nil) + pool.Upsert(accounts.Item{ID: "a", URL: a.URL, Provider: "qoder", Region: "global", Runtime: "child_process"}) + pool.Upsert(accounts.Item{ID: "b", URL: b.URL, Provider: "qoder", Region: "global", Runtime: "child_process"}) + executor := NewChatExecutor(pool, "") + firstReq := translate.ChatRequest{Model: "glm-5.2", Messages: []translate.ChatMessage{{Role: "user", Content: "plan the refactor"}}} + laterReq := translate.ChatRequest{ + Model: "glm-5.2", + Messages: []translate.ChatMessage{ + {Role: "user", Content: "plan the refactor"}, + {Role: "assistant", Content: "ok"}, + {Role: "user", Content: "continue"}, + }, + } + otherReq := translate.ChatRequest{Model: "glm-5.2", Messages: []translate.ChatMessage{{Role: "user", Content: "a different conversation"}}} + + first, err := executor.ChatNonStream(context.Background(), firstReq, "", "") + if err != nil || first.AccountID != "a" || first.Routing != routingPool { + t.Fatalf("first = %+v, err=%v", first, err) + } + later, err := executor.ChatNonStream(context.Background(), laterReq, "", "") + if err != nil || later.AccountID != "a" || later.Routing != routingSticky { + t.Fatalf("later = %+v, err=%v", later, err) + } + other, err := executor.ChatNonStream(context.Background(), otherReq, "", "") + if err != nil || other.AccountID != "b" || other.Routing != routingPool { + t.Fatalf("other = %+v, err=%v", other, err) + } +} + +func TestChatNonStreamImageOnlyContentSessionAffinity(t *testing.T) { + server := func(id string) *httptest.Server { + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + _, _ = io.WriteString(w, `{"model":"glm-5.2","choices":[{"message":{"content":"`+id+`"},"finish_reason":"stop"}],"usage":{"source":"upstream"}}`) + })) + } + a := server("a") + defer a.Close() + b := server("b") + defer b.Close() + + pool := accounts.NewPool(nil, nil) + pool.Upsert(accounts.Item{ID: "a", URL: a.URL, Provider: "qoder", Region: "global", Runtime: "child_process"}) + pool.Upsert(accounts.Item{ID: "b", URL: b.URL, Provider: "qoder", Region: "global", Runtime: "child_process"}) + executor := NewChatExecutor(pool, "") + image := []any{map[string]any{"type": "image_url", "image_url": map[string]any{"url": "https://example.com/cat.png"}}} + firstReq := translate.ChatRequest{Model: "glm-5.2", Messages: []translate.ChatMessage{{Role: "user", Content: image}}} + laterReq := translate.ChatRequest{ + Model: "glm-5.2", + Messages: []translate.ChatMessage{ + {Role: "user", Content: image}, + {Role: "assistant", Content: "a cat"}, + {Role: "user", Content: "what color?"}, + }, + } + otherReq := translate.ChatRequest{Model: "glm-5.2", Messages: []translate.ChatMessage{{ + Role: "user", + Content: []any{map[string]any{"type": "image_url", "image_url": map[string]any{"url": "https://example.com/dog.png"}}}, + }}} + + first, err := executor.ChatNonStream(context.Background(), firstReq, "", "") + if err != nil || first.AccountID != "a" || first.Routing != routingPool { + t.Fatalf("first = %+v, err=%v", first, err) + } + later, err := executor.ChatNonStream(context.Background(), laterReq, "", "") + if err != nil || later.AccountID != "a" || later.Routing != routingSticky { + t.Fatalf("later = %+v, err=%v", later, err) + } + other, err := executor.ChatNonStream(context.Background(), otherReq, "", "") + if err != nil || other.AccountID != "b" || other.Routing != routingPool { + t.Fatalf("other = %+v, err=%v", other, err) + } +} + +func TestChatStreamProxyContentSessionAffinityWithoutExplicitKey(t *testing.T) { + server := func() *httptest.Server { + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "text/event-stream") + _, _ = io.WriteString(w, "data: [DONE]\n\n") + })) + } + a := server() + defer a.Close() + b := server() + defer b.Close() + + pool := accounts.NewPool(nil, nil) + pool.Upsert(accounts.Item{ID: "a", URL: a.URL, Provider: "qoder", Region: "global", Runtime: "child_process"}) + pool.Upsert(accounts.Item{ID: "b", URL: b.URL, Provider: "qoder", Region: "global", Runtime: "child_process"}) + executor := NewChatExecutor(pool, "") + firstReq := translate.ChatRequest{Model: "glm-5.2", Stream: true, Messages: []translate.ChatMessage{{Role: "user", Content: "plan the refactor"}}} + laterReq := translate.ChatRequest{ + Model: "glm-5.2", + Stream: true, + Messages: []translate.ChatMessage{ + {Role: "user", Content: "plan the refactor"}, + {Role: "assistant", Content: "ok"}, + {Role: "user", Content: "continue"}, + }, + } + + first, err := executor.ChatStreamProxy(context.Background(), firstReq, "", "") + if err != nil || first.AccountID != "a" || first.Routing != routingPool { + t.Fatalf("first = %+v, err=%v", first, err) + } + if _, err := io.ReadAll(first.Response.Body); err != nil { + t.Fatal(err) + } + first.Response.Body.Close() + executor.CommitSession(context.Background(), firstReq, first.Routing, first.AccountID) + later, err := executor.ChatStreamProxy(context.Background(), laterReq, "", "") + if err != nil || later.AccountID != "a" || later.Routing != routingSticky { + t.Fatalf("later = %+v, err=%v", later, err) + } + later.Response.Body.Close() +} + func TestChatNonStreamSessionAffinityRateLimitEscapesSameRegion(t *testing.T) { a := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { w.Header().Set("X-Qoder-Error-Kind", accounts.KindRateLimit) diff --git a/internal/translate/compat_test.go b/internal/translate/compat_test.go index 0096fcf..6a751d5 100644 --- a/internal/translate/compat_test.go +++ b/internal/translate/compat_test.go @@ -56,3 +56,72 @@ func TestTranslateAnthropicToolResultLiftsImages(t *testing.T) { t.Fatalf("image part=%#v", parts[0]) } } + +func TestTranslatedImageOnlyRequestsProduceContentSessionSeed(t *testing.T) { + anthropic, err := TranslateAnthropicMessages(AnthropicMessagesRequest{ + Model: "glm-5.2", + Messages: []AnthropicMessage{ + {Role: "user", Content: json.RawMessage(`[{"type":"image","source":{"type":"url","url":"https://example.com/cat.png"}}]`)}, + }, + }) + if err != nil { + t.Fatal(err) + } + anthropicSeed := ContentSessionSeed(anthropic) + if anthropicSeed == "" { + t.Fatalf("anthropic image-only seed empty; content=%#v", anthropic.Messages[0].Content) + } + anthropicLater, err := TranslateAnthropicMessages(AnthropicMessagesRequest{ + Model: "glm-5.2", + Messages: []AnthropicMessage{ + {Role: "user", Content: json.RawMessage(`[{"type":"image","source":{"type":"url","url":"https://example.com/cat.png"}}]`)}, + {Role: "assistant", Content: json.RawMessage(`[{"type":"text","text":"a cat"}]`)}, + {Role: "user", Content: json.RawMessage(`[{"type":"text","text":"what color?"}]`)}, + }, + }) + if err != nil { + t.Fatal(err) + } + if ContentSessionSeed(anthropicLater) != anthropicSeed { + t.Fatalf("anthropic later turn changed seed:\nfirst=%q\nlater=%q", anthropicSeed, ContentSessionSeed(anthropicLater)) + } + + responses, err := TranslateResponses(ResponsesRequest{ + Model: "glm-5.2", + Input: json.RawMessage(`[{"role":"user","content":[{"type":"input_image","image_url":"https://example.com/cat.png"}]}]`), + }) + if err != nil { + t.Fatal(err) + } + responsesSeed := ContentSessionSeed(responses) + if responsesSeed == "" { + t.Fatalf("responses image-only seed empty; content=%#v", responses.Messages[0].Content) + } + responsesLater, err := TranslateResponses(ResponsesRequest{ + Model: "glm-5.2", + Input: json.RawMessage(`[ + {"role":"user","content":[{"type":"input_image","image_url":"https://example.com/cat.png"}]}, + {"role":"assistant","content":[{"type":"output_text","text":"a cat"}]}, + {"role":"user","content":[{"type":"input_text","text":"what color?"}]} + ]`), + }) + if err != nil { + t.Fatal(err) + } + if ContentSessionSeed(responsesLater) != responsesSeed { + t.Fatalf("responses later turn changed seed:\nfirst=%q\nlater=%q", responsesSeed, ContentSessionSeed(responsesLater)) + } + + otherAnthropic, err := TranslateAnthropicMessages(AnthropicMessagesRequest{ + Model: "glm-5.2", + Messages: []AnthropicMessage{ + {Role: "user", Content: json.RawMessage(`[{"type":"image","source":{"type":"url","url":"https://example.com/dog.png"}}]`)}, + }, + }) + if err != nil { + t.Fatal(err) + } + if ContentSessionSeed(otherAnthropic) == anthropicSeed { + t.Fatal("different anthropic images must not share a seed") + } +} diff --git a/internal/translate/session_seed.go b/internal/translate/session_seed.go new file mode 100644 index 0000000..a607446 --- /dev/null +++ b/internal/translate/session_seed.go @@ -0,0 +1,262 @@ +package translate + +import ( + "bytes" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "strings" +) + +type contentSessionFingerprint struct { + Model string `json:"model"` + Tools string `json:"tools,omitempty"` + Prefix []sessionFingerprintMsg `json:"prefix,omitempty"` + FirstUser sessionFingerprintMsg `json:"first_user"` +} + +type sessionFingerprintMsg struct { + Role string `json:"role"` + Content []sessionContentPart `json:"content"` +} + +type sessionContentPart struct { + Type string `json:"type"` + Text string `json:"text,omitempty"` + Image string `json:"image,omitempty"` +} + +// ContentSessionSeed builds a stable conversation fingerprint from fields that +// stay constant across turns: model, tool definitions, the leading +// system/developer prefix, and the first user message. Later turns append +// assistant/user messages and must not change the seed, or prompt-cache +// affinity would break. +// +// The fingerprint is a hash of role-tagged structured fields, including image +// identifiers on the first user message. An empty result means there is no +// user anchor; callers should leave the request on ordinary pool routing. +func ContentSessionSeed(req ChatRequest) string { + fingerprint, ok := buildContentSessionFingerprint(req) + if !ok { + return "" + } + raw, err := json.Marshal(fingerprint) + if err != nil { + return "" + } + sum := sha256.Sum256(raw) + return hex.EncodeToString(sum[:]) +} + +func buildContentSessionFingerprint(req ChatRequest) (contentSessionFingerprint, bool) { + fingerprint := contentSessionFingerprint{ + Model: strings.ToLower(strings.TrimSpace(req.Model)), + Tools: compactSessionJSON(req.Tools), + } + systemPrefixOpen := true + sawFirstUser := false + for _, message := range req.Messages { + role := strings.ToLower(strings.TrimSpace(message.Role)) + switch role { + case "system", "developer": + if !systemPrefixOpen { + continue + } + parts := sessionContentParts(message.Content) + if len(parts) == 0 { + continue + } + fingerprint.Prefix = append(fingerprint.Prefix, sessionFingerprintMsg{Role: role, Content: parts}) + case "user": + systemPrefixOpen = false + if sawFirstUser { + continue + } + sawFirstUser = true + fingerprint.FirstUser = sessionFingerprintMsg{Role: "user", Content: sessionContentParts(message.Content)} + default: + systemPrefixOpen = false + } + } + if len(fingerprint.FirstUser.Content) == 0 { + return contentSessionFingerprint{}, false + } + return fingerprint, true +} + +func sessionContentParts(content any) []sessionContentPart { + switch value := content.(type) { + case nil: + return nil + case string: + text := strings.TrimSpace(value) + if text == "" { + return nil + } + return []sessionContentPart{{Type: "text", Text: text}} + case []any: + parts := make([]sessionContentPart, 0, len(value)) + for _, item := range value { + parts = append(parts, sessionContentParts(item)...) + } + return parts + case map[string]any: + if part, ok := sessionPartFromMap(value); ok { + return []sessionContentPart{part} + } + return nil + case map[string]string: + converted := make(map[string]any, len(value)) + for key, item := range value { + converted[key] = item + } + if part, ok := sessionPartFromMap(converted); ok { + return []sessionContentPart{part} + } + return nil + case json.RawMessage: + trimmed := bytes.TrimSpace(value) + if len(trimmed) == 0 || bytes.Equal(trimmed, []byte("null")) { + return nil + } + var decoded any + if err := json.Unmarshal(trimmed, &decoded); err != nil { + return nil + } + return sessionContentParts(decoded) + default: + text := strings.TrimSpace(ContentToString(value)) + if text == "" || text == "null" || text == "{}" || text == "[]" { + return nil + } + return []sessionContentPart{{Type: "text", Text: text}} + } +} + +func sessionPartFromMap(block map[string]any) (sessionContentPart, bool) { + if block == nil { + return sessionContentPart{}, false + } + partType := strings.ToLower(strings.TrimSpace(anyString(block["type"]))) + switch partType { + case "text", "input_text", "output_text": + text := strings.TrimSpace(anyString(block["text"])) + if text == "" { + return sessionContentPart{}, false + } + return sessionContentPart{Type: "text", Text: text}, true + case "image_url", "image": + if image := sessionImageFingerprint(block); image != "" { + return sessionContentPart{Type: "image", Image: image}, true + } + return sessionContentPart{}, false + } + if image := sessionImageFingerprint(block); image != "" { + return sessionContentPart{Type: "image", Image: image}, true + } + text := strings.TrimSpace(anyString(block["text"])) + if text != "" { + return sessionContentPart{Type: "text", Text: text}, true + } + return sessionContentPart{}, false +} + +func sessionImageFingerprint(block map[string]any) string { + if url := imageURLFromAny(block["image_url"]); url != "" { + return hashSessionImage(url, "", "") + } + if url := strings.TrimSpace(anyString(block["url"])); url != "" && lookupMap(block, "image_url") == nil && lookupMap(block, "source") == nil { + if strings.ToLower(strings.TrimSpace(anyString(block["type"]))) == "image" || looksLikeImageURL(url) { + return hashSessionImage(url, "", "") + } + } + if source := lookupMap(block, "source"); source != nil { + url := strings.TrimSpace(anyString(source["url"])) + mediaType := strings.TrimSpace(anyString(source["media_type"])) + data := strings.TrimSpace(anyString(source["data"])) + if url != "" || data != "" { + return hashSessionImage(url, mediaType, data) + } + } + return "" +} + +func imageURLFromAny(value any) string { + switch typed := value.(type) { + case string: + return strings.TrimSpace(typed) + case map[string]any: + if url := strings.TrimSpace(anyString(typed["url"])); url != "" { + return url + } + return strings.TrimSpace(anyString(typed["image_url"])) + case map[string]string: + if url := strings.TrimSpace(typed["url"]); url != "" { + return url + } + return strings.TrimSpace(typed["image_url"]) + default: + return "" + } +} + +func lookupMap(block map[string]any, key string) map[string]any { + switch nested := block[key].(type) { + case map[string]any: + return nested + case map[string]string: + out := make(map[string]any, len(nested)) + for nestedKey, nestedValue := range nested { + out[nestedKey] = nestedValue + } + return out + default: + return nil + } +} + +func looksLikeImageURL(url string) bool { + lower := strings.ToLower(url) + return strings.HasPrefix(lower, "data:image/") || strings.HasPrefix(lower, "http://") || strings.HasPrefix(lower, "https://") +} + +func hashSessionImage(url, mediaType, data string) string { + url = strings.TrimSpace(url) + mediaType = strings.TrimSpace(mediaType) + data = strings.TrimSpace(data) + if url == "" && data == "" { + return "" + } + if url != "" && data == "" && !strings.HasPrefix(strings.ToLower(url), "data:") { + return "url:" + url + } + sum := sha256.Sum256([]byte(url + "\x00" + mediaType + "\x00" + data)) + return "data:" + hex.EncodeToString(sum[:]) +} + +func anyString(value any) string { + switch typed := value.(type) { + case string: + return typed + case json.RawMessage: + var text string + if json.Unmarshal(typed, &text) == nil { + return text + } + return strings.Trim(string(bytes.TrimSpace(typed)), `"`) + default: + return "" + } +} + +func compactSessionJSON(raw json.RawMessage) string { + trimmed := bytes.TrimSpace(raw) + if len(trimmed) == 0 || bytes.Equal(trimmed, []byte("null")) || bytes.Equal(trimmed, []byte("[]")) { + return "" + } + var compact bytes.Buffer + if err := json.Compact(&compact, trimmed); err != nil { + return string(trimmed) + } + return compact.String() +} diff --git a/internal/translate/session_seed_test.go b/internal/translate/session_seed_test.go new file mode 100644 index 0000000..c6da7af --- /dev/null +++ b/internal/translate/session_seed_test.go @@ -0,0 +1,256 @@ +package translate + +import ( + "encoding/json" + "testing" +) + +func TestContentSessionSeedStableAcrossLaterTurns(t *testing.T) { + first := ChatRequest{ + Model: "GLM-5.2", + Tools: json.RawMessage(`[{"type":"function","function":{"name":"search"}}]`), + Messages: []ChatMessage{ + {Role: "system", Content: "you are a bot"}, + {Role: "user", Content: "plan the refactor"}, + }, + } + second := ChatRequest{ + Model: "glm-5.2", + Tools: json.RawMessage(`[{"type": "function", "function": {"name": "search"}}]`), + Messages: []ChatMessage{ + {Role: "system", Content: "you are a bot"}, + {Role: "user", Content: "plan the refactor"}, + {Role: "assistant", Content: "here is a plan"}, + {Role: "user", Content: "do step two"}, + }, + } + got := ContentSessionSeed(first) + if got == "" { + t.Fatal("expected content seed") + } + if ContentSessionSeed(second) != got { + t.Fatalf("later turn changed seed:\nfirst=%q\nsecond=%q", got, ContentSessionSeed(second)) + } +} + +func TestContentSessionSeedRequiresFirstUserAnchor(t *testing.T) { + if got := ContentSessionSeed(ChatRequest{Model: "glm-5.2"}); got != "" { + t.Fatalf("model-only seed = %q", got) + } + if got := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{{Role: "system", Content: "identity"}}, + }); got != "" { + t.Fatalf("system-only seed = %q", got) + } + if got := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{{Role: "user", Content: " "}}, + }); got != "" { + t.Fatalf("blank user seed = %q", got) + } + if got := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{ + {Role: "user", Content: " "}, + {Role: "assistant", Content: "ok"}, + {Role: "user", Content: "real question"}, + }, + }); got != "" { + t.Fatalf("blank first user must not fall through to a later user: %q", got) + } +} + +func TestContentSessionSeedKeepsPrefixRoleBoundaries(t *testing.T) { + systemThenUser := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{ + {Role: "system", Content: "shared"}, + {Role: "user", Content: "hi"}, + }, + }) + developerThenUser := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{ + {Role: "developer", Content: "shared"}, + {Role: "user", Content: "hi"}, + }, + }) + if systemThenUser == "" || developerThenUser == "" || systemThenUser == developerThenUser { + t.Fatalf("system and developer prefixes must not collide: system=%q developer=%q", systemThenUser, developerThenUser) + } + oneSystem := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{ + {Role: "system", Content: "a"}, + {Role: "system", Content: "b"}, + {Role: "user", Content: "hi"}, + }, + }) + mergedSystem := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{ + {Role: "system", Content: "ab"}, + {Role: "user", Content: "hi"}, + }, + }) + if oneSystem == "" || mergedSystem == "" || oneSystem == mergedSystem { + t.Fatalf("consecutive prefix messages must keep boundaries: split=%q merged=%q", oneSystem, mergedSystem) + } +} + +func TestContentSessionSeedIgnoresSystemAfterFirstUser(t *testing.T) { + base := ChatRequest{ + Messages: []ChatMessage{ + {Role: "system", Content: "keep"}, + {Role: "user", Content: "hi"}, + }, + } + injected := ChatRequest{ + Messages: []ChatMessage{ + {Role: "system", Content: "keep"}, + {Role: "user", Content: "hi"}, + {Role: "system", Content: "late"}, + {Role: "user", Content: "next"}, + }, + } + if ContentSessionSeed(base) != ContentSessionSeed(injected) { + t.Fatal("system messages after the first user must not change the seed") + } +} + +func TestContentSessionSeedUsesStructuredFieldsNotDelimiters(t *testing.T) { + spoofed := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{ + {Role: "system", Content: "x|first_user=y"}, + {Role: "user", Content: " "}, + }, + }) + if spoofed != "" { + t.Fatalf("delimiter in system plus empty user must not create a seed: %q", spoofed) + } + real := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{ + {Role: "system", Content: "x"}, + {Role: "user", Content: "y"}, + }, + }) + if real == "" { + t.Fatal("expected seed for a real first user message") + } + shifted := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{ + {Role: "system", Content: "x|first_user=y"}, + {Role: "user", Content: "other"}, + }, + }) + if shifted == "" || shifted == real { + t.Fatalf("system delimiter must not collide with a real user field: real=%q shifted=%q", real, shifted) + } +} + +func TestContentSessionSeedUsesMultimodalText(t *testing.T) { + textOnly := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{{Role: "user", Content: "describe this"}}, + }) + got := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{{ + Role: "user", + Content: []any{ + map[string]any{"type": "text", "text": "describe this"}, + map[string]any{"type": "image_url", "image_url": map[string]any{"url": "https://example.com/cat.png"}}, + }, + }}, + }) + if got == "" { + t.Fatal("expected multimodal seed") + } + if got == textOnly { + t.Fatal("image part must change the first-user fingerprint") + } +} + +func TestContentSessionSeedImageOnlyStableAcrossLaterTurns(t *testing.T) { + image := []any{ + map[string]any{"type": "image_url", "image_url": map[string]any{"url": "https://example.com/cat.png"}}, + } + first := ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{{Role: "user", Content: image}}, + } + later := ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{ + {Role: "user", Content: image}, + {Role: "assistant", Content: "a cat sitting on a fence"}, + {Role: "user", Content: "what color is it?"}, + }, + } + got := ContentSessionSeed(first) + if got == "" { + t.Fatal("image-only first user must produce a session seed") + } + if ContentSessionSeed(later) != got { + t.Fatalf("image-only later turn changed seed:\nfirst=%q\nlater=%q", got, ContentSessionSeed(later)) + } + other := ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{{ + Role: "user", + Content: []any{ + map[string]any{"type": "image_url", "image_url": map[string]any{"url": "https://example.com/dog.png"}}, + }, + }}, + }) + if other == "" || other == got { + t.Fatalf("different image must not share a seed: got=%q other=%q", got, other) + } +} + +func TestContentSessionSeedImageOnlyBase64AndAnthropicSource(t *testing.T) { + dataURI := ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{{ + Role: "user", + Content: []any{ + map[string]any{"type": "image_url", "image_url": map[string]any{"url": "data:image/png;base64,abc"}}, + }, + }}, + } + anthropic := ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{{ + Role: "user", + Content: map[string]any{ + "type": "image", + "source": map[string]any{ + "type": "base64", + "media_type": "image/png", + "data": "abc", + }, + }, + }}, + } + if got := ContentSessionSeed(dataURI); got == "" { + t.Fatal("data URI image-only request must produce a session seed") + } + if got := ContentSessionSeed(anthropic); got == "" { + t.Fatal("Anthropic image source must produce a session seed") + } + if ContentSessionSeed(dataURI) == ContentSessionSeed(ChatRequest{ + Model: "glm-5.2", + Messages: []ChatMessage{{ + Role: "user", + Content: []any{ + map[string]any{"type": "image_url", "image_url": map[string]any{"url": "data:image/png;base64,def"}}, + }, + }}, + }) { + t.Fatal("different image payloads must not share a seed") + } +} From 3c556a21b3e23b570f4a38cd62b8694f925e6e02 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=94=A1=E5=8F=8A?= <522caiji@gmail.com> Date: Sun, 6 Sep 2026 23:25:10 +0800 Subject: [PATCH 2/2] feat: allow minor and major bumps in the release workflow Keep patch as the default, but let a dispatch choose minor or major so the next cut can be v0.3.0 instead of another 0.2 patch. --- .github/workflows/release.yml | 36 ++++++++++++++++++++++++++++++++--- CONTRIBUTING.md | 3 ++- 2 files changed, 35 insertions(+), 4 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 26f2dfa..3e88204 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,9 +1,18 @@ name: Release -run-name: Release next patch from ${{ github.ref_name }} +run-name: Release next ${{ inputs.bump || 'patch' }} from ${{ github.ref_name }} on: workflow_dispatch: + inputs: + bump: + description: SemVer component to increment from the latest published release + type: choice + default: patch + options: + - patch + - minor + - major concurrency: group: release @@ -54,16 +63,24 @@ jobs: done echo "Timed out waiting for CI on $GITHUB_SHA" >&2 exit 1 - - name: Calculate next patch version + - name: Calculate next version id: version env: GH_TOKEN: ${{ github.token }} + BUMP: ${{ inputs.bump || 'patch' }} run: | set -euo pipefail if [[ "$GITHUB_REF" != "refs/heads/main" ]]; then echo "Release workflow must run from main, got $GITHUB_REF" >&2 exit 1 fi + case "$BUMP" in + patch|minor|major) ;; + *) + echo "Unsupported bump $BUMP" >&2 + exit 1 + ;; + esac git fetch --tags --force previous_tag="$({ @@ -78,7 +95,20 @@ jobs: else version="${previous_tag#v}" IFS=. read -r major minor patch <<< "$version" - patch=$((patch + 1)) + case "$BUMP" in + major) + major=$((major + 1)) + minor=0 + patch=0 + ;; + minor) + minor=$((minor + 1)) + patch=0 + ;; + patch) + patch=$((patch + 1)) + ;; + esac if ! git merge-base --is-ancestor "$previous_tag" "$GITHUB_SHA"; then echo "$previous_tag is not an ancestor of $GITHUB_SHA" >&2 exit 1 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fb493d1..a2ff418 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -59,7 +59,8 @@ For an end-to-end run, use the Docker Compose flow in `deploy/README.md`. User-facing changes should add matching bullets to `CHANGELOG.md` under `## Unreleased` in both `### English` and `### 中文`. The release workflow copies those notes into the GitHub Release body. Do not create version tags -by hand. +by hand. The workflow defaults to the next patch; choose `minor` or `major` +when the published behavior warrants a series bump. ## Rules