Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 33 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,18 @@
name: Release

run-name: Release next patch from ${{ github.ref_name }}
run-name: Release next ${{ inputs.bump || 'patch' }} from ${{ github.ref_name }}

on:
workflow_dispatch:
inputs:
bump:
description: SemVer component to increment from the latest published release
type: choice
default: patch
options:
- patch
- minor
- major

concurrency:
group: release
Expand Down Expand Up @@ -54,16 +63,24 @@ jobs:
done
echo "Timed out waiting for CI on $GITHUB_SHA" >&2
exit 1
- name: Calculate next patch version
- name: Calculate next version
id: version
env:
GH_TOKEN: ${{ github.token }}
BUMP: ${{ inputs.bump || 'patch' }}
run: |
set -euo pipefail
if [[ "$GITHUB_REF" != "refs/heads/main" ]]; then
echo "Release workflow must run from main, got $GITHUB_REF" >&2
exit 1
fi
case "$BUMP" in
patch|minor|major) ;;
*)
echo "Unsupported bump $BUMP" >&2
exit 1
;;
esac

git fetch --tags --force
previous_tag="$({
Expand All @@ -78,7 +95,20 @@ jobs:
else
version="${previous_tag#v}"
IFS=. read -r major minor patch <<< "$version"
patch=$((patch + 1))
case "$BUMP" in
major)
major=$((major + 1))
minor=0
patch=0
;;
minor)
minor=$((minor + 1))
patch=0
;;
patch)
patch=$((patch + 1))
;;
esac
if ! git merge-base --is-ancestor "$previous_tag" "$GITHUB_SHA"; then
echo "$previous_tag is not an ancestor of $GITHUB_SHA" >&2
exit 1
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@ Write each change in both `### English` and `### 中文` under `## Unreleased`.

### English

- Keep multi-turn conversations on the same account from the first user message by default, including image-only turns, without requiring `X-CLI2API-Session`
- Cache `GET /api/models` for 5 minutes so the console catalog page does not re-hit WorkBuddy or Trae on every load; `?refresh=1` still fetches live. Overview stays uncached.

### 中文

- 同一段多轮对话默认按首条用户消息(含纯图片)粘到同一个账号,不再需要 `X-CLI2API-Session`
- `GET /api/models` 缓存 5 分钟,控制台模型页不再每次都打 WorkBuddy / Trae 目录;`?refresh=1` 仍即时拉取。Overview 不缓存。

## 0.2.48 - 2026-09-06
Expand Down
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ For an end-to-end run, use the Docker Compose flow in `deploy/README.md`.
User-facing changes should add matching bullets to `CHANGELOG.md` under
`## Unreleased` in both `### English` and `### 中文`. The release workflow
copies those notes into the GitHub Release body. Do not create version tags
by hand.
by hand. The workflow defaults to the next patch; choose `minor` or `major`
when the published behavior warrants a series bump.

## Rules

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Base URL: http://127.0.0.1:3010/v1
API Key: <首次启动时生成的 Key>
```

不指定账号时,调度器自动选择可用账号;需要固定账号时加请求头 `X-Qoder-Account: acc_...`(历史命名,适用于所有 provider)。除 Chat Completions 外,也可使用 Anthropic `POST /v1/messages` 与 OpenAI `POST /v1/responses`;两者要求请求携带完整对话,不支持 `previous_response_id` / `conversation` 服务端续接。需要会话粘性时,可设置 `X-CLI2API-Session`;curl / PowerShell 示例见 [部署说明](deploy/README.md)。
不指定账号时,调度器自动选择可用账号;需要固定账号时加请求头 `X-Qoder-Account: acc_...`(历史命名,适用于所有 provider)。除 Chat Completions 外,也可使用 Anthropic `POST /v1/messages` 与 OpenAI `POST /v1/responses`;两者要求请求携带完整对话,不支持 `previous_response_id` / `conversation` 服务端续接。同一段对话默认按首条用户消息(含纯图片)粘到同一个账号;也可显式设置 `X-CLI2API-Session`。curl / PowerShell 示例见 [部署说明](deploy/README.md)。

## 工作方式

Expand Down Expand Up @@ -87,7 +87,7 @@ CLI2API 是本地网关:不提供账号、额度或官方 API 服务,不做

- Anthropic `/v1/messages` 与 OpenAI `/v1/responses` 的无状态文本 / 函数工具适配层
- WorkBuddy 每日签到与 token 保活(账号级开关,默认关闭;控制台可立即签到 / 刷新积分)
- 会话粘性路由:通过 `X-CLI2API-Session` 优先复用同一账号,并在失败时按规则切换
- 会话粘性路由:默认按对话内容(首条用户消息,含纯图片)复用同一账号,也可设置 `X-CLI2API-Session`,并在失败时按规则切换
- 请求历史按账号过滤,以及请求状态、延迟、Token 和用量统计

**长期**
Expand Down
4 changes: 2 additions & 2 deletions README_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Base URL: http://127.0.0.1:3010/v1
API Key: <the key printed on first startup>
```

Without an account header the scheduler picks a ready account; pin a request with the `X-Qoder-Account: acc_...` header (a historical name that applies to every provider). Anthropic `POST /v1/messages` and OpenAI `POST /v1/responses` are also available; both require the complete conversation in each request and do not support server-side continuation through `previous_response_id` / `conversation`. Use `X-CLI2API-Session` when session-sticky routing is desired. curl / PowerShell examples in the [deployment guide](deploy/README.md).
Without an account header the scheduler picks a ready account; pin a request with the `X-Qoder-Account: acc_...` header (a historical name that applies to every provider). Anthropic `POST /v1/messages` and OpenAI `POST /v1/responses` are also available; both require the complete conversation in each request and do not support server-side continuation through `previous_response_id` / `conversation`. Multi-turn requests stick to the same account from the first user message (including image-only turns) by default; `X-CLI2API-Session` remains an optional override. curl / PowerShell examples in the [deployment guide](deploy/README.md).

## How it works

Expand Down Expand Up @@ -87,7 +87,7 @@ CLI2API is a local gateway: it does not provide accounts, quotas, or an official

- Stateless text and function-tool adapters for Anthropic `/v1/messages` and OpenAI `/v1/responses`; image input where the provider supports it
- WorkBuddy daily check-in and token keepalive (per-account opt-in, off by default; console can check in now / refresh credits)
- Session-sticky routing via `X-CLI2API-Session`, with rule-based failover when the bound account cannot serve the request
- Session-sticky routing from conversation content (first user message, including image-only turns), or via `X-CLI2API-Session`, with rule-based failover when the bound account cannot serve the request
- Request history filtering by account, plus request status, latency, token, and usage statistics

**Longer term**
Expand Down
6 changes: 4 additions & 2 deletions deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,10 @@ Invoke-RestMethod -Method Post -Uri "http://127.0.0.1:3010/v1/chat/completions"
```

Pin a request to a specific account with the `X-Qoder-Account: acc_...` header
(a historical header name that applies to every provider). Add `X-CLI2API-Session`
when consecutive requests should prefer the same account.
(a historical header name that applies to every provider). Consecutive turns of
the same conversation prefer the same account from the first user message
(including image-only turns);
`X-CLI2API-Session` remains an optional override.

## 5. Configuration

Expand Down
21 changes: 14 additions & 7 deletions internal/api/chat.go
Original file line number Diff line number Diff line change
Expand Up @@ -659,7 +659,7 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) {
RequestedModel: firstNonEmpty(publicModel, req.Model),
})
ctx := executor.WithAllowedProviders(executor.WithRequestID(r.Context(), requestID), identity.AllowedProviders)
if sessionKey := requestSessionKey(r, identity); sessionKey != "" {
if sessionKey := requestSessionKey(r, identity, req); sessionKey != "" {
ctx = executor.WithSessionKey(ctx, sessionKey)
}
w.Header().Set("X-Request-Id", requestID)
Expand Down Expand Up @@ -706,7 +706,7 @@ func (s *Server) handleChatCompletions(w http.ResponseWriter, r *http.Request) {
}
s.finishRequestLog(requestID, started, req, publicModel, upstream.AccountID, firstNonEmpty(upstream.Provider, providerFilter), upstream.Routing, status, ttfb, &stats, relayErr, upstream.AttemptCount)
if relayErr == nil {
s.executor.CommitSession(ctx, upstream.Routing, upstream.AccountID)
s.executor.CommitSession(ctx, req, upstream.Routing, upstream.AccountID)
}
if relayErr != nil {
// The upstream answered 200 and failed inside the stream, so the
Expand Down Expand Up @@ -882,19 +882,26 @@ func writeClassifiedErr(w http.ResponseWriter, err error) {
writeErr(w, classified.Status, classified.Code, classified.Message)
}

func requestSessionKey(r *http.Request, identity auth.Identity) string {
if r == nil {
return ""
func requestSessionKey(r *http.Request, identity auth.Identity, req translate.ChatRequest) string {
raw := ""
kind := "content"
if r != nil {
raw = strings.TrimSpace(r.Header.Get("X-CLI2API-Session"))
if raw != "" {
kind = "header"
}
}
if raw == "" {
raw = translate.ContentSessionSeed(req)
}
raw := strings.TrimSpace(r.Header.Get("X-CLI2API-Session"))
if raw == "" {
return ""
}
namespace := "console"
if identity.Kind == auth.KindKey && identity.KeyID != "" {
namespace = "key:" + identity.KeyID
}
sum := sha256.Sum256([]byte(namespace + "\x00" + raw))
sum := sha256.Sum256([]byte(namespace + "\x00" + kind + "\x00" + raw))
return hex.EncodeToString(sum[:])
}

Expand Down
49 changes: 45 additions & 4 deletions internal/api/chat_usage_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/caigee-cmd/cli2api/internal/auth"
"github.com/caigee-cmd/cli2api/internal/executor"
"github.com/caigee-cmd/cli2api/internal/providers"
"github.com/caigee-cmd/cli2api/internal/translate"
)

func intPtr(value int) *int { return &value }
Expand Down Expand Up @@ -58,24 +59,64 @@ func TestRequestSessionKeyRequiresHeaderAndScopesToIdentity(t *testing.T) {
withHeader := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
withHeader.Header.Set("X-CLI2API-Session", "session-a")
firstKey := auth.Identity{Kind: auth.KindKey, KeyID: "key-1"}
if got := requestSessionKey(withHeader, firstKey); got == "" || got == "session-a" {
emptyReq := translate.ChatRequest{}
if got := requestSessionKey(withHeader, firstKey, emptyReq); got == "" || got == "session-a" {
t.Fatalf("header key = %q", got)
}
withSameHeader := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
withSameHeader.Header.Set("X-CLI2API-Session", "session-a")
if requestSessionKey(withHeader, firstKey) != requestSessionKey(withSameHeader, firstKey) {
if requestSessionKey(withHeader, firstKey, emptyReq) != requestSessionKey(withSameHeader, firstKey, emptyReq) {
t.Fatal("same header should derive the same opaque key")
}
withoutHeader := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
if got := requestSessionKey(withoutHeader, firstKey); got != "" {
if got := requestSessionKey(withoutHeader, firstKey, emptyReq); got != "" {
t.Fatalf("key without header = %q", got)
}
secondKey := auth.Identity{Kind: auth.KindKey, KeyID: "key-2"}
if requestSessionKey(withHeader, firstKey) == requestSessionKey(withHeader, secondKey) {
if requestSessionKey(withHeader, firstKey, emptyReq) == requestSessionKey(withHeader, secondKey, emptyReq) {
t.Fatal("same session header must be isolated by API key")
}
}

func TestRequestSessionKeyFallsBackToContentSeed(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
identity := auth.Identity{Kind: auth.KindKey, KeyID: "key-1"}
first := translate.ChatRequest{
Model: "glm-5.2",
Messages: []translate.ChatMessage{{Role: "user", Content: "plan the refactor"}},
}
later := translate.ChatRequest{
Model: "glm-5.2",
Messages: []translate.ChatMessage{
{Role: "user", Content: "plan the refactor"},
{Role: "assistant", Content: "ok"},
{Role: "user", Content: "continue"},
},
}
got := requestSessionKey(req, identity, first)
if got == "" {
t.Fatal("expected content-derived session key")
}
if requestSessionKey(req, identity, later) != got {
t.Fatal("later turn must keep the same content-derived session key")
}
other := translate.ChatRequest{
Model: "glm-5.2",
Messages: []translate.ChatMessage{{Role: "user", Content: "a different conversation"}},
}
if requestSessionKey(req, identity, other) == got {
t.Fatal("different first user message must not share a session key")
}
headerReq := httptest.NewRequest(http.MethodPost, "/v1/chat/completions", nil)
headerReq.Header.Set("X-CLI2API-Session", "explicit")
if requestSessionKey(headerReq, identity, first) == got {
t.Fatal("explicit header must outrank the content seed")
}
if requestSessionKey(req, auth.Identity{Kind: auth.KindKey, KeyID: "key-2"}, first) == got {
t.Fatal("content-derived session keys must stay isolated by API key")
}
}

func TestBuildChatUsagePreservesZeroPromptCacheTokens(t *testing.T) {
usage := buildChatUsage(executor.ChatResult{
CacheReadTokens: intPtr(0),
Expand Down
6 changes: 3 additions & 3 deletions internal/api/compat.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ func (s *Server) prepareCompatibilityExecution(r *http.Request, request translat
RequestedModel: firstNonEmpty(publicModel, request.Model),
})
ctx := executor.WithAllowedProviders(executor.WithRequestID(r.Context(), requestID), identity.AllowedProviders)
if sessionKey := requestSessionKey(r, identity); sessionKey != "" {
if sessionKey := requestSessionKey(r, identity, request); sessionKey != "" {
ctx = executor.WithSessionKey(ctx, sessionKey)
}
return compatibilityExecution{
Expand Down Expand Up @@ -135,7 +135,7 @@ func (s *Server) handleAnthropicMessagesStream(w http.ResponseWriter, r *http.Re
ttfb := streamTTFB(execution.started, upstream.TTFBMs, stats)
s.finishCompatibility(execution, upstream.AccountID, upstream.Provider, upstream.Routing, status, ttfb, &stats, relayErr, upstream.AttemptCount)
if relayErr == nil {
s.executor.CommitSession(execution.ctx, upstream.Routing, upstream.AccountID)
s.executor.CommitSession(execution.ctx, execution.request, upstream.Routing, upstream.AccountID)
return
}
if !isStreamClientDisconnect(relayErr) {
Expand Down Expand Up @@ -203,7 +203,7 @@ func (s *Server) handleResponsesStream(w http.ResponseWriter, r *http.Request, e
ttfb := streamTTFB(execution.started, upstream.TTFBMs, stats)
s.finishCompatibility(execution, upstream.AccountID, upstream.Provider, upstream.Routing, status, ttfb, &stats, relayErr, upstream.AttemptCount)
if relayErr == nil {
s.executor.CommitSession(execution.ctx, upstream.Routing, upstream.AccountID)
s.executor.CommitSession(execution.ctx, execution.request, upstream.Routing, upstream.AccountID)
return
}
if !isStreamClientDisconnect(relayErr) {
Expand Down
16 changes: 8 additions & 8 deletions internal/executor/chat.go
Original file line number Diff line number Diff line change
Expand Up @@ -165,11 +165,11 @@ func (e ChatExecutor) stickyEscapeReason(plan *routingPlan) string {
// CommitSession binds a successfully completed stream. The executor cannot know
// whether an SSE response reached [DONE], so the relay calls this only after it
// has finished without an upstream or client error.
func (e ChatExecutor) CommitSession(ctx context.Context, routing, accountID string) {
func (e ChatExecutor) CommitSession(ctx context.Context, req translate.ChatRequest, routing, accountID string) {
if routing == routingPin || e.SessionAffinity == nil {
return
}
e.SessionAffinity.Bind(sessionKeyFromContext(ctx), accountID)
e.SessionAffinity.Bind(resolveSessionKey(ctx, req), accountID)
}

func itemProvider(item accounts.Item) string {
Expand All @@ -193,15 +193,15 @@ func itemServesPublicModel(item accounts.Item, publicModel string) bool {
return false
}

func (e ChatExecutor) prepareRouting(ctx context.Context, prefer, providerFilter, publicModel string) (string, string, string, routingPlan) {
func (e ChatExecutor) prepareRouting(ctx context.Context, prefer, providerFilter string, req translate.ChatRequest) (string, string, string, routingPlan) {
prefer = strings.TrimSpace(prefer)
providerFilter = strings.ToLower(strings.TrimSpace(providerFilter))
publicModel := req.Model
if prefer != "" {
return prefer, providerFilter, "", routingPlan{Source: routingPin}
return prefer, providerFilter, "", routingPlan{Source: routingPin, PublicModel: publicModel}
}

plan := routingPlan{Source: routingPool, SessionKey: sessionKeyFromContext(ctx)}
plan.PublicModel = publicModel
plan := routingPlan{Source: routingPool, SessionKey: resolveSessionKey(ctx, req), PublicModel: publicModel}
if plan.SessionKey == "" || e.SessionAffinity == nil || e.Pool == nil {
return "", providerFilter, "", plan
}
Expand Down Expand Up @@ -579,7 +579,7 @@ func (e ChatExecutor) ChatNonStream(ctx context.Context, req translate.ChatReque
if err != nil {
return ChatResult{}, err
}
prefer, providerFilter, regionFilter, routing = e.prepareRouting(ctx, prefer, providerFilter, req.Model)
prefer, providerFilter, regionFilter, routing = e.prepareRouting(ctx, prefer, providerFilter, req)
excluded := map[string]struct{}{}
var lastErr error
if regionFilter == "" && prefer != "" && e.Pool != nil {
Expand Down Expand Up @@ -978,7 +978,7 @@ func (e ChatExecutor) ChatStreamProxy(ctx context.Context, req translate.ChatReq
if err != nil {
return StreamResult{}, err
}
prefer, providerFilter, regionFilter, routing = e.prepareRouting(ctx, prefer, providerFilter, req.Model)
prefer, providerFilter, regionFilter, routing = e.prepareRouting(ctx, prefer, providerFilter, req)
excluded := map[string]struct{}{}
var lastErr error
if regionFilter == "" && prefer != "" && e.Pool != nil {
Expand Down
20 changes: 20 additions & 0 deletions internal/executor/session_affinity.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,13 @@ package executor
import (
"container/list"
"context"
"crypto/sha256"
"encoding/hex"
"strings"
"sync"
"time"

"github.com/caigee-cmd/cli2api/internal/translate"
)

const (
Expand All @@ -30,6 +34,22 @@ func sessionKeyFromContext(ctx context.Context) string {
return strings.TrimSpace(key)
}

func contentSessionKey(req translate.ChatRequest) string {
seed := translate.ContentSessionSeed(req)
if seed == "" {
return ""
}
sum := sha256.Sum256([]byte("content\x00" + seed))
return hex.EncodeToString(sum[:])
}

func resolveSessionKey(ctx context.Context, req translate.ChatRequest) string {
if key := sessionKeyFromContext(ctx); key != "" {
return key
}
return contentSessionKey(req)
}

type sessionBinding struct {
key string
accountID string
Expand Down
Loading
Loading