Skip to content

Add authenticated multi-model alias routing - #45

Open
huangxinV587 wants to merge 1 commit into
mainfrom
branch/model-alias-gateway-20261110
Open

huangxinV587 wants to merge 1 commit into
mainfrom
branch/model-alias-gateway-20261110

Conversation

@huangxinV587

@huangxinV587 huangxinV587 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

现有网关只能在同一模型池中选择后端;新增模型通常需要单独的客户端入口。此变更在既有认证入口前增加模型别名路由,支持每个别名的独立上游、同模型多实例、会话亲和及响应开始前的故障切换。

  • 复用既有 key-store;上游认证材料从 Windows 凭据管理器读取,不向模型转发客户端认证材料。
  • 原入口不可用时,仅列出新增别名并标记旧入口不可用;目录本身不代表后端实时健康。
  • 流式响应保留取消和截断语义;Caddy 切换保留原入口回退,隔离模型特定的 503。
  • 入口增加 256 个请求槽和 512 MiB 总请求体预算;极端大请求并发可能较原入口更早返回 429。

验证:python -m unittest discover -s tests -v(共 259 项,其中 8 项跳过);独立审查及隔离 Caddy 流式、取消、故障和配置竞争检查通过。实际部署与入口切换另行核验。

@huangxinV587
huangxinV587 marked this pull request as ready for review November 10, 2026 16:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant