feat(discovery): fetch credential-scoped Anthropic models

[amondnet]

Summary

Fetch the caller's credential-scoped Anthropic model catalog on each GET /v1/models request instead of relying only on a quickly stale builtin snapshot.

A live re-probe of reference claude gateway 2.1.220 showed that its catalog had added claude-opus-5 within roughly ten days of the previous snapshot. More importantly, three real responses confirmed that the model list varies by credential:

• x-api-key: 11 model IDs (the superset)
• Claude subscription OAuth: 10 model IDs (excluding claude-opus-4-1-20250805)
• Reference apps gateway 2.1.220: 10 model IDs (excluding claude-opus-4-5-20251101)

Because a shared cache could expose one caller's entitlement view to another, discovery remains stateless and uncached.

This PR:

• Adds src/discovery/upstream.rs, which queries the first Anthropic-kind upstream per request, preferring server.default_provider and then declaration order.
• Authenticates discovery with that request's effective credential: the forwarded credential for passthrough, the configured key for api_key, or the first resolvable account for claude_oauth.
• Uses a 2-second timeout, follows has_more pagination, and relays capabilities verbatim.
• Falls back silently to the builtin snapshot when no Anthropic upstream or credential is available, or when the upstream errors, times out, returns an invalid body, or returns an empty list.
• Bypasses AccountPool so discovery does not affect inference selection, cooldown, or quota bookkeeping.
• Lets a live result supersede the snapshot rather than merging it, avoiding re-advertising models the caller cannot use.
• Replaces the 9-entry ID-only builtin catalog with an 11-entry public Anthropic API snapshot containing display names, timestamps, token limits, and real first_id / last_id values.

Milestone / spec

M3 — docs/m3-discovery.md §4 and §8

Also updates docs/gateway-protocol.md, README.md, and all four locales for:

• guides/model-discovery
• reference/configuration
• reference/endpoints
• guides/connect-claude-desktop

Checklist

• cargo build passes (covered by the successful clippy and test builds)
• cargo test passes (1,230 passed, 0 failed with cargo test --all-features --workspace)
• cargo clippy --all-targets -- -D warnings clean (0 warnings with all features enabled)
• cargo fmt --all --check clean
• Source files stay under 500 lines
• English only; matches surrounding style
• Frozen spec in docs/ updated if this change deviates from it
• User-facing docs updated for behavior/config/endpoint/CLI/provider/model changes — README.md / site/ as applicable (wiki/ is generated; don't hand-edit)
• Any new GitHub Action is pinned to a full commit SHA (no GitHub Actions added)

Notes for reviewers

Behavior changes

• /v1/models may now make one upstream request sequence. The previous "no upstream call" guarantee in docs/m3-discovery.md is updated. Each call is capped at 2 seconds and falls back, preserving the documented sub-3-second budget.
• Responses gain optional created_at, max_input_tokens, max_tokens, and capabilities fields. Curated [[models]] entries retain the previous narrow shape.
• auto_include_builtin_models = false remains strictly curated and now also suppresses the upstream call.
• The fallback snapshot intentionally over-advertises because it has no caller credential to test. This preserves the existing policy that entitlement gaps surface as runtime errors rather than discovery-time probes.

Credential handling

The implementation removes a duplicate x-api-key when an sk-ant-oat… bearer credential is forwarded. Please review the passthrough, configured API-key, and Claude OAuth resolution paths closely.

Tests

The discovery suite adds 15 wiremock-backed tests covering:

• Credential forwarding and complete field mapping
• No upstream call when passthrough has no caller credential, asserted through received_requests
• Removal of a duplicated x-api-key beside an sk-ant-oat… bearer
• Configured api_key injection
• has_more pagination
• Fallback on upstream errors and empty lists
• End-to-end confirmation that a live list supersedes the builtin snapshot

Open question

The upstream lookup currently runs whenever an Anthropic-kind upstream is configured, with no opt-out. A slow or unreachable Anthropic upstream can therefore add up to the 2-second timeout to every /v1/models request before fallback. If deployments need explicit control, an opt-out config key can be added as a small follow-up.

---

Summary by cubic

Serve a live, credential‑scoped Anthropic model list on GET /v1/models with a single 2 s overall timeout and a snapshot fallback. Live fetch runs only when the server.default_provider is Anthropic‑kind; otherwise we serve the 11‑model snapshot with upstream metadata and populated first/last ids.

• New Features

  • Query only the Anthropic default provider using its auth mode: passthrough (forwards caller token; never replays gateway auth), api_key (configured key), or claude_oauth (first resolvable, non‑disabled account from the effective set). Stateless; no AccountPool. Relay capabilities, follow has_more pagination (max 5 pages), and drop a duplicated x-api-key beside an sk-ant-oat… bearer. auto_include_builtin_models = false returns only curated entries and suppresses the upstream call.

• Bug Fixes

  • Bind discovery to the default provider so only routable Anthropic ids are advertised; use the snapshot when the default provider is not Anthropic‑kind. Enforce one 2 s timeout across credential resolution and all pages; fall back on upstream errors, timeouts, empty lists, unusable cursors, or incomplete pagination. Log upstream failures at warn. Minor refactor: clarified the consumed‑bearer check and removed a page‑limit allocation.

^{Written for commit 61edb47. Summary will update on new commits.}

Demo

[gemini-code-assist]

Code Review

This pull request implements live model discovery from the Anthropic upstream, enabling credential-scoped model lists with a fallback to an updated builtin snapshot. Feedback on the changes suggests optimizing URL formatting and query string allocations outside the pagination loop, and raises a concern regarding potential concurrent OAuth token refresh race conditions when bypassing the AccountPool.

[codecov]

Codecov Report

❌ Patch coverage is 93.26599% with 20 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/discovery/upstream.rs	89.13%	20 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 11 untouched benchmarks

---

_{Comparing amondnet/auto_include_builtin_models-2 (61edb47) with main (f4cd619)}

[greptile-apps]

Greptile Summary

This PR adds live, credential-scoped Anthropic model discovery with bounded pagination and snapshot fallback.

• Fetches the catalog from the Anthropic-kind default provider using passthrough, configured API-key, or Claude OAuth credentials.
• Relays upstream model metadata and replaces the builtin catalog with an expanded 11-model snapshot.
• Aligns live discovery with default-provider routing and updates discovery documentation and localized guides.

Confidence Score: 3/5

The PR is not yet safe to merge because Claude OAuth discovery can advertise a catalog from a different account than inference will use.

Discovery still selects the first resolvable enabled OAuth account, while inference applies pool priority, quota, cooldown, rotation, and session selection; because catalogs are credential-scoped, a multi-account deployment can expose models that the serving account cannot use or omit models that it can.

Files Needing Attention: src/discovery/upstream.rs, src/adapters/anthropic/mod.rs

Important Files Changed

Filename	Overview
src/discovery/upstream.rs	Implements bounded live catalog fetching and default-provider alignment, while the previously reported mismatch between first-account discovery and pooled inference selection remains.
src/discovery.rs	Integrates live results with curated models, snapshot fallback, expanded metadata, deduplication, and populated envelope cursors.
src/adapters/anthropic/mod.rs	Exposes existing OAuth-header normalization helpers for reuse by discovery without changing their behavior.
src/auth/inbound.rs	Exposes bearer-token parsing within the crate so discovery can avoid forwarding credentials consumed by gateway authentication.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Discovery as GET /v1/models
    participant Auth as Credential resolution
    participant Upstream as Default Anthropic provider
    Client->>Discovery: Request model catalog
    Discovery->>Auth: Resolve effective credential
    alt Credential and Anthropic default available
        Auth-->>Discovery: Upstream credential
        Discovery->>Upstream: GET /v1/models (2 s overall deadline)
        Upstream-->>Discovery: Credential-scoped pages
        Discovery-->>Client: Curated entries + live catalog
    else Resolution or upstream request fails
        Discovery-->>Client: Curated entries + builtin snapshot
    end

Loading

_{Reviews (7): Last reviewed commit: "refactor(discovery): name the consumed-b..." | Re-trigger Greptile}

[amondnet]

@greptileai review

[amondnet]

/gemini review

[amondnet]

@greptileai review

[gemini-code-assist]

Code Review

This pull request implements live model discovery for the Anthropic upstream, querying the default provider using its configured authentication mode (passthrough, API key, or OAuth) and falling back to an updated offline snapshot of 11 models if the call fails or times out. It includes extensive documentation updates across multiple languages and robust test coverage. The review feedback recommends minor optimizations and readability enhancements, specifically using a static string literal for the page limit query parameter to avoid allocations, and extracting complex boolean logic into a descriptive variable to clarify when a bearer token is considered consumed.

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
93.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud