Skip to content

Commit 0e00045

Browse files
committed
fix(codearts): 预刷新窗口须宽于轮询周期,短寿命 token 不再恒标「即将到期」
- 新增可热更项 REFRESH_INTERVAL_MINUTES(默认 30,下限 5),build_runner 接线: 此前该形参从未接线、恒用默认 60min,导致封顶 30min 的刷新窗口被整轮跳过, 凭证拖到到期瞬间才刷(实测晚约 26s → 上游 APIG.0602 + 管理台恒标红) - CodeArts refresh_skew_cap_seconds 1800 → 2700(> 30min 周期):轮询粒度下 实际在剩余约 30min 处刷新(每约 90min 一轮),既不贴边也不过密 - 前端 tokenExpiryView 增 issuedAt:寿命 ≤ 预警窗口 2 倍时不再标「即将到期」 (与后端 expiring_tokens 短寿命静音同口径);真过期仍标红 - compose 透传 REFRESH_INTERVAL_MINUTES;同步 README/README.en/TECHNICAL/PROPOSAL(Q69)
1 parent 0f878a3 commit 0e00045

15 files changed

Lines changed: 102 additions & 26 deletions

‎PROPOSAL.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -451,3 +451,4 @@ M0 骨架 → M1a TRAE → M1b CB 基础 → M1.5 CB 完整化 → M2 前端 →
451451
的版权范围。
452452
| Q67 | 收窄后全不可用时区分成因:模型级冷却不再扇出 | 实测故障(2026-10-08 15:53):`qwen3.8-flash` 只有 Qoder 登记(内部原代号 `qfmodel`),Qoder 节点故障 `[FAIL]node:oa_qwen-plus-main Execution failed`(按 Q50 归 MODEL)让两条 Qoder 凭证进 600s 模型级冷却;此后每个请求收窄到 `{qoder}` → 候选空 → 触发 `_select` 的 broader 回退,扇出到**不持有该模型**的渠道。它们别名表没有这个键,外发的仍是原始名 `qwen3.8-flash`(没机会换成对方原代号),必然被拒:CodeBuddy 11102 / TRAE 4001 / zen 401 / kilo 401。实测 6 次轮换全废、客户端拿到 400 `invalid_request`(而非「模型暂不可用,换模型即可」),还给 CodeBuddy 三条凭证各写下 6 小时起步的 BLOCKED 负缓存(`credential_model_cooldowns` 至今可见)。修复:回退前分清「收窄后为什么空」——**账号级原因**(无凭证 / 硬禁用 / 暂停 / 账号冷却)说明目录可能陈旧或降级,仍退回原始候选集重试;**模型级冷却**说明持有方是对的、只是上游暂时不可用,此时放宽即扇出,是纯浪费,故**不扇出**,直接交既有 `_unavailable_text` 出模型级 503(含相似模型建议)。判定抽成 `_model_cooled_only(providers, target, now)`,`_all_model_cooled`(文案分流用)改为复用它,两处口径不再分叉;判定为「纯模型冷却」的前提是候选集非空且每条候选都账号级可选——混入账号级不可用或该上游零凭证时判 False,照旧放宽(目录陈旧兜底语义零变化,已有 3 个回退用例不受影响)。连带修掉的是「扇出噪音」:不再给不认该模型的渠道打上游、不再产生 `invalid_request` 统计、不再写 BLOCKED 负缓存。无 schema / 配置变更 |
453453
| Q68 | CodeArts 凭证生命周期修复(刷新节流 / STS 过期分类 / 一次性票终态 / 到期预警) | 实测故障(2026-10-08):CodeArts 凭证每约 60 分钟被预刷新任务轮转一次,`refresh_token` 是**一次性**的,多烧一次就多一分「被并发消费掉」的风险;一旦烧废,聊天侧收到 `APIG.0602 security token has expired`(HTTP 400),被 `classify_status` 判成 `INVALID`(换 provider 也没用、且不触发冷却),客户端只看到 `model not available on any configured upstream`;同时预刷新任务此后每小时拿同一张废票重试、各刷一条一模一样的 warning,而凭证早已死透。四处修复:① **刷新节流**——`refresh_skew_hours` 默认 24h ≫ STS 寿命 2h,`needs_refresh` 对该渠道**恒为真**,故给 `CodeArtsCredential` 加 `refresh_skew_cap_seconds=1800` 上限,把「一进窗口就每轮都刷」降成「离到期 30min 内才刷」(下界硬约束是预刷新轮询周期,窗口窄于周期会整轮漏过导致凭证静默过期)。② **`APIG.0602` 分类**——`classify_status` 在 `400 → INVALID` 之前识别 `security token has expired`,判 `SOFT`(账号级**可恢复**:下一轮预刷新换上新 STS 即自愈)。不判 `DEAD`:硬禁用会让预刷新任务跳过该凭证(`candidates()` 的 disabled 分支),反而断掉唯一的自愈路径;不判 `INVALID`:语义是「请求本身无效」,换 provider 也没用且不冷却。③ **一次性票终态**——新增 `provider.base.UpstreamReloginRequired`;CodeArts 令牌端点把 `STS5.1806 the refresh token has been used` / `invalid client id` / `InvalidDPoPHeader` / `invalid_grant` 翻译成该异常(`events.relogin_required(body)` 判定,只认上游点名的几类终态,网络/5xx/限流仍走可重试路径),CodeBuddy 401/403 同样归入;`RefreshTask` 捕获后调 `CredentialRepository.mark_relogin_required` 写**可操作**的 `disabled_reason`(「刷新令牌已失效,需重新登录」)并硬禁用 + 清模型级冷却,本轮记 failed、下轮直接跳过——不再每小时刷同样的 warning,管理台按原因提示人工重新登录。④ **到期预警修正**——`_EXPIRY_KEYS` 补 `expiration`(此前漏掉使 CodeArts 的 `token_expires_at` 恒为 0,管理台无到期预览、`token_expiring` 告警对它完全失效);`credential_token_times` 增 `_ISSUED_ONLY_KEYS`,access token 非 JWT 时借**同时签发**的 `refresh_token` JWT 的 `iat`(只借 iat 不借 exp——后者是「登录 +30 天」,会把 2h 临时凭证画成 30 天);`expiring_tokens` 静音「寿命本就短于窗口」的凭证(CodeArts 2h 寿命 vs 默认 24h 窗口会恒真命中,把真告警淹掉;签发时间未知时不猜、按原语义照报)。无 schema / 配置项变更 |
454+
| Q69 | CodeArts 预刷新窗口与轮询周期错配修正 + 短寿命 token 不再恒标「即将到期」 | 实测故障(2026-10-08 二次):用户反馈「CodeArts 显示又要到期」。Q68 把预刷新窗口封顶 30min,但预刷新轮询周期写死 60min(`TaskRunner.refresh_interval_minutes` 默认 60 且 `build_runner` 未从配置接线)——**窗口 ≤ 周期时轮询点会整轮落在窗口之外**,凭证一直拖到**到期瞬间**才刷:实测 `issued 19:07:32 → expires 21:07:32`,刷新发生在 21:07–21:08(晚了约 26s),管理台最后一小时恒标红、刷新瞬间短暂显示「已过期」,上游还回了 `APIG.0602`。两端修复:① **后端**——新增可热更项 `REFRESH_INTERVAL_MINUTES`(默认 30、下限 5)并在 `build_runner` 接线(此前该形参从未接线、实际恒用默认 60),把轮询缩到 30min;`CodeArtsCredential.refresh_skew_cap_seconds` 从 1800 抬到 2700(> 30min 周期)留出余量,轮询粒度下实际在**剩余约 30min** 处刷新(每约 90min 一轮)——既不贴边也不过密。② **前端**——`tokenExpiryView()` 增 `issuedAt` 参数:整段寿命 ≤ 预警窗口 2 倍时该 token 天生在窗口附近(CodeArts STS 寿命 2h、窗口 1h),「即将到期」是常态而非异常,不再标红(**真过期仍标红**,那是刷新失败的信号),与后端 `expiring_tokens` 对短寿命凭证静音同口径;`issuedAt` 缺省时保持旧语义(只看剩余时间),避免未知签发时间时误静音。配置项变更:新增 `REFRESH_INTERVAL_MINUTES`(compose 同步透传);无 schema 变更 |

‎README.en.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ credential pool, unified scheduling, and per-user usage stats.
2828
- **Activity reporting** (CodeBuddy only, **off by default**): `ACTIVITY_REPORT_ENABLED=true` posts one chat-activity event per account per day to keep the growth-center streak alive. The upstream needs a `userId` and silently drops reports without one (HTTP 200 `{"code":0}`, streak unchanged); when the credential has no `user_id`, the gateway falls back to the bearer JWT `sub`. Upstream-internal and may break without notice — not a reliability feature (terms forbid scripted tampering: disqualification + clawback)
2929
- **Pool health endpoint**: `GET /healthz` returns `{status, service, version, credentials:{total,ready,cooling,paused,disabled}}` (unauthenticated) so monitors can alert when the pool is exhausted (`ready=0`: alive but unusable). `GET /health` stays a pure liveness probe. Buckets are mutually exclusive and sum to `total`, using the scheduler's own "selectable" rule
3030
- **Per-key routing policy**: a key can be bound to one provider (`provider_binding`) and/or restricted by source IP (`allowed_ips`, comma-separated IP/CIDR, empty = unrestricted). A bound key requesting a model owned by the other provider gets a 400 naming the real owner. IPs are checked at auth time; `X-Forwarded-For` is **ignored by default** and only honored with `TRUST_PROXY=true`, using the *last* entry (fits exactly one trusted reverse proxy). No per-key quotas / multi-tenancy
31-
- **Runtime settings & background-task view** (*Tasks & Settings* page): 35 settings (default model, blocklist, context-compression knobs, fallback groups, both expiry windows, sticky TTL, irreversible growth actions, growth/probe/catalog intervals, both pacer bounds, the per-channel chat intervals, activity toggle/hour, and the alerting rules) apply immediately without a restart. **DB overrides beat `.env`**; rows are marked "DB override" and can be reset. Startup-only knobs (`APP_SECRET`, `PORT`, `DATA_DIR`, allowlists) are excluded. The same page lists the 8 background tasks with interval, last run, latest result and error — task state is **in-process only** (`GET /api/tasks`, admin, 30s refresh), resets on restart, and a no-op wake-up is not counted as a run
31+
- **Runtime settings & background-task view** (*Tasks & Settings* page): 36 settings (default model, blocklist, context-compression knobs, fallback groups, both expiry windows, sticky TTL, irreversible growth actions, growth/probe/refresh/catalog intervals, both pacer bounds, the per-channel chat intervals, activity toggle/hour, and the alerting rules) apply immediately without a restart. **DB overrides beat `.env`**; rows are marked "DB override" and can be reset. Startup-only knobs (`APP_SECRET`, `PORT`, `DATA_DIR`, allowlists) are excluded. The same page lists the 8 background tasks with interval, last run, latest result and error — task state is **in-process only** (`GET /api/tasks`, admin, 30s refresh), resets on restart, and a no-op wake-up is not counted as a run
3232
- **Token-expiry visibility**: a "token remaining" column, red below `TOKEN_EXPIRY_WARNING_SECONDS`. Expiry comes from explicit `expires_at` and **falls back to the JWT `exp`** — CodeBuddy's token responses carry no expiry (measured), so without the fallback it is always 0 and CodeBuddy tokens would never pre-refresh (they'd only be hard-disabled on a 401). Both missing → `—`, never guessed; `iat` is persisted to `credentials.token_issued_at` for diagnostics but not shown
3333
- **Credit-change log**: the per-credential "credit record" drawer lists the **net change between consecutive quota probes** (`credit_events`, 90-day retention). It does **not** attribute changes to check-in / growth / chat — upstream logs nothing for those calls. The UI says "net change", never "check-in +5". The first probe only records a baseline (`sync`); unchanged balances are skipped; a balance going *unknown* still records a row with no delta, because that is an anomaly worth chasing rather than "no change"
3434
- **Credit in usage stats, and `≈`**: CodeBuddy credit is the **real upstream value**; TRAE's `token_usage` frame carries tokens only, so its credit is **estimated** from official unit prices (with measured overrides where billed cache prices diverge from the published table) and marked `≈`. CodeArts benefit models are estimated 1:1 against the daily pool, also `≈`. The formulas and price tables live in `src/provider/trae/pricing.py` and `src/provider/codearts/units.py`; historical rows are backfilled with `scripts/backfill_trae_credit.py` / `scripts/convert_codearts_credit_unit.py` (`--apply` to write)

‎README.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,7 @@ curl http://127.0.0.1:8000/v1/chat/completions \
183183

184184
几点要知道:
185185

186-
- **令牌刷新是刚性的**:refresh_token 与 `client_id=codearts-agent`、DPoP 私钥三者绑定,且**一次性**——刷新后必须回写新的 refresh_token,否则该账号失效。本服务由 token 预刷新任务自动完成(DPoP ES256/P-256 签名);这也是该渠道的「保活」手段。为降低烧票风险,本渠道的预刷新窗口封顶 30 分钟(全局 `REFRESH_SKEW_HOURS` 默认 24h ≫ 凭证 2h 寿命,不封顶会每轮都刷)。一旦续期凭据被上游作废(票被消费 / 绑定项不符),该凭证会被标记为「需重新登录」并停止自动重试,需重新登录后按状态列的「恢复」解除。
186+
- **令牌刷新是刚性的**:refresh_token 与 `client_id=codearts-agent`、DPoP 私钥三者绑定,且**一次性**——刷新后必须回写新的 refresh_token,否则该账号失效。本服务由 token 预刷新任务自动完成(DPoP ES256/P-256 签名);这也是该渠道的「保活」手段。为降低烧票风险,本渠道的预刷新窗口封顶 45 分钟,且该封顶必须**宽于**预刷新轮询周期(`REFRESH_INTERVAL_MINUTES`,默认 30 分钟)——否则窗口整轮漏过、凭证拖到到期才刷(全局 `REFRESH_SKEW_HOURS` 默认 24h ≫ 凭证 2h 寿命,不封顶会每轮都刷)。一旦续期凭据被上游作废(票被消费 / 绑定项不符),该凭证会被标记为「需重新登录」并停止自动重试,需重新登录后按状态列的「恢复」解除。
187187
- **没有每日签到**:CodeArts 额度是**每日 1000 万免费 token(当日 0 点清零、不累计)**,上游没有每日签到接口,因此本服务不提供签到入口。启动/定时会尝试领取福利 Token(幂等)并探测余额。
188188
- **优先消耗**:每日池用完即弃,故当日剩余被登记为「次日本地 0 点到期」的到期额度,调度器会优先把它排在其它渠道之前——只要 CodeArts 还有额度就先走它,用尽后自动回落。上游按 token 计量,管理台统一显示为**积分**(1 积分 = 10000 token,每日池满额 = 1000 积分);升级前落库的历史数据由 `scripts/convert_codearts_credit_unit.py` 一次性折算。
189189
- **签名**:上游要求华为云 `SDK-HMAC-SHA256`(AK/SK + `X-Security-Token`)。白名单 `CODEARTS_ALLOWED_ENDPOINTS` 必须含 snap 引擎、STS、福利网关与门户四个主机;改 `CODEARTS_API_ENDPOINT` 时同步调整。
@@ -360,6 +360,7 @@ CodeBuddy 成长中心的「连登天数 / 活跃地图」按日统计客户端
360360
| `USERS_FILE` | `secrets/users.txt` | **仅引导期**:老式用户文件路径,启动时一次性导入 SQLite(已存在的用户名不覆盖,幂等)。账号唯一源是 `users` 表 |
361361
| `DATA_DIR` | `./data` | SQLite 与运行数据目录 |
362362
| `QUOTA_PROBE_MINUTES` | `60` | 额度探测周期(下限 1 分钟) |
363+
| `REFRESH_INTERVAL_MINUTES` | `30` | token 预刷新轮询周期(下限 5 分钟);短寿命渠道的预刷新窗口封顶必须**宽于**它,否则窗口整轮漏过、凭证拖到到期才刷 |
363364
| `MODEL_CATALOG_MINUTES` | `30` | 模型目录兜底刷新周期(下限 5 分钟);只影响「没人访问列表时」的保鲜,正常仍按 TTL 随访问刷新 |
364365
| `GROWTH_INTERVAL_MINUTES` | `60` | 成长中心(仅 CodeBuddy)一轮领取的周期;下限 5 分钟 |
365366
| `GROWTH_IRREVERSIBLE_ACTIONS` | `true` | 是否允许成长中心的不可逆动作:抽奖、连登兑换、开 Buddy 盲盒、消耗补登卡。`false` 时仍会领取旅行礼物与任务奖励 |
@@ -419,9 +420,9 @@ CodeBuddy 成长中心的「连登天数 / 活跃地图」按日统计客户端
419420

420421
### 管理台热更(「任务与配置」页)
421422

422-
上表中带「可热更」语义的 35 项可不改 `.env`、不重启,直接在管理台「任务与配置」页修改:
423+
上表中带「可热更」语义的 36 项可不改 `.env`、不重启,直接在管理台「任务与配置」页修改:
423424

424-
`DEFAULT_MODEL`、`MODEL_BLOCKLIST`、`CONTEXT_COMPRESS_ENABLED`、`CONTEXT_COMPRESS_RESERVE_TOKENS`、`CONTEXT_COMPRESS_MIN_KEEP_MESSAGES`、`CONTEXT_COMPRESS_SAFETY_RATIO`、`MODEL_FALLBACK_GROUPS`、`QUOTA_EXPIRY_WINDOW_SECONDS`、`QUOTA_EXPIRY_SECONDARY_WINDOW_SECONDS`、`CONVERSATION_STICKY_SECONDS`、`GROWTH_IRREVERSIBLE_ACTIONS`、`GROWTH_INTERVAL_MINUTES`、`QUOTA_PROBE_MINUTES`、`MODEL_CATALOG_MINUTES`、`CODEBUDDY_CHAT_MIN_INTERVAL`、`ZEN_CHAT_MIN_INTERVAL`、`KILO_CHAT_MIN_INTERVAL`、`QODER_CHAT_MIN_INTERVAL`、`CODEARTS_CHAT_MIN_INTERVAL`、`CODEARTS_MAX_CONCURRENCY`、`CODEARTS_REQUEST_WINDOW_SECONDS`、`PACER_MIN_SECONDS`、`PACER_MAX_SECONDS`、`ACTIVITY_REPORT_ENABLED`、`ACTIVITY_REPORT_HOUR`、`ALERT_ENABLED`、`ALERT_WEBHOOK_URL`、`ALERT_INTERVAL_MINUTES`、`ALERT_SILENCE_MINUTES`、`ALERT_POOL_READY_MIN`、`ALERT_TASK_FAILURES`、`ALERT_TOKEN_EXPIRY_HOURS`、`ALERT_ERROR_RATE_THRESHOLD`、`ALERT_ERROR_RATE_MIN_REQUESTS`、`ALERT_ERROR_RATE_WINDOW_MINUTES`。
425+
`DEFAULT_MODEL`、`MODEL_BLOCKLIST`、`CONTEXT_COMPRESS_ENABLED`、`CONTEXT_COMPRESS_RESERVE_TOKENS`、`CONTEXT_COMPRESS_MIN_KEEP_MESSAGES`、`CONTEXT_COMPRESS_SAFETY_RATIO`、`MODEL_FALLBACK_GROUPS`、`QUOTA_EXPIRY_WINDOW_SECONDS`、`QUOTA_EXPIRY_SECONDARY_WINDOW_SECONDS`、`CONVERSATION_STICKY_SECONDS`、`GROWTH_IRREVERSIBLE_ACTIONS`、`GROWTH_INTERVAL_MINUTES`、`QUOTA_PROBE_MINUTES`、`REFRESH_INTERVAL_MINUTES`、`MODEL_CATALOG_MINUTES`、`CODEBUDDY_CHAT_MIN_INTERVAL`、`ZEN_CHAT_MIN_INTERVAL`、`KILO_CHAT_MIN_INTERVAL`、`QODER_CHAT_MIN_INTERVAL`、`CODEARTS_CHAT_MIN_INTERVAL`、`CODEARTS_MAX_CONCURRENCY`、`CODEARTS_REQUEST_WINDOW_SECONDS`、`PACER_MIN_SECONDS`、`PACER_MAX_SECONDS`、`ACTIVITY_REPORT_ENABLED`、`ACTIVITY_REPORT_HOUR`、`ALERT_ENABLED`、`ALERT_WEBHOOK_URL`、`ALERT_INTERVAL_MINUTES`、`ALERT_SILENCE_MINUTES`、`ALERT_POOL_READY_MIN`、`ALERT_TASK_FAILURES`、`ALERT_TOKEN_EXPIRY_HOURS`、`ALERT_ERROR_RATE_THRESHOLD`、`ALERT_ERROR_RATE_MIN_REQUESTS`、`ALERT_ERROR_RATE_WINDOW_MINUTES`。
425426

426427
要点:
427428

0 commit comments

Comments
 (0)