Repository navigation
Description
Summary
Constructing or validating an LLM configured with an openhands/* model can synchronously call the managed LiteLLM proxy's GET /v1/model/info endpoint even when no API key is available.
The SDK correctly omits the Authorization header when the key is empty, but it still sends the request. The managed proxy returns 401, the SDK silently falls back to static LiteLLM metadata, and production accumulates roughly 14K–30K unauthenticated model-info responses per day.
For the known managed OpenHands proxy, a request without credentials cannot return key-scoped proxy metadata and should be skipped.
Current behavior
The initialization path is:
LLM._post_init()LLM._init_model_info_and_caps()get_litellm_model_info()_get_model_info_from_litellm_proxy()httpx.get(f"{base_url}/v1/model/info", headers=headers)
For a missing key, headers remains empty but step 5 still runs.
This commonly occurs during startup, profile loading, validation, or deserialization—not only before a completion. The process-local LRU cache reduces identical calls, but pod restarts, hourly cache keys, concurrency, and key/model cardinality still produce substantial traffic.
PR #1022 previously fixed a blank Authorization: Bearer header by omitting it. That changed the failure from 422 to 401 but retained the request. PR #1148 added caching, but the lookup remains eager during LLM initialization.
Expected behavior
When all of the following are true:
- the model resolves to the managed OpenHands LiteLLM proxy;
- the proxy URL is the default or recognized managed OpenHands proxy, rather than an arbitrary custom proxy; and
- no API key is available;
the SDK should skip /v1/model/info discovery and use its existing static LiteLLM metadata fallback without making an HTTP request.
Unauthenticated discovery must remain supported for generic litellm_proxy/* configurations and custom proxy URLs, because private proxies may intentionally expose model information without authentication.
Authenticated managed-proxy discovery must also remain unchanged because it supplies key-scoped aliases and capability/token-limit overrides.
Suggested implementation
- Preserve enough pre-translation context to distinguish the managed OpenHands provider from a generic LiteLLM proxy.
- Before the HTTP request, return
Nonewhen the target is the managed OpenHands proxy and the resolved API key is empty. - Preserve the current static fallback.
- Consider explicitly handling non-2xx responses instead of treating arbitrary error JSON as an empty model list, without propagating discovery failures from initialization.
- Coordinate with draft PR fix(llm): bound model-info discovery with a timeout to prevent deadlocks #3912, which adds model-info timeouts but does not address missing credentials or 401 volume.
Acceptance criteria
- Constructing
LLM(model="openhands/<model>")without an API key and without a custom base URL performs no model-info HTTP request. - A recognized managed OpenHands proxy URL without an API key also skips the request.
- An authenticated managed OpenHands configuration still calls
/v1/model/infowith its Bearer token and applies returned overrides. - A custom or authless
litellm_proxy/*configuration without an API key can still query model info. - A custom
openhands/*proxy URL is not incorrectly classified as the managed production proxy. - Regression tests cover each case above.
Related work
- step-3.7-flash: vision support not detected at runtime (LiteLLM proxy /v1/model/info lookup returns None) #3495 documents silent model-info lookup failure affecting capability detection, but its authenticated eval-proxy case has different behavior and acceptance criteria.
- Fix error in 422 logs due to blank bearer token #1022 removed the blank Bearer header while retaining missing-key discovery.
- Fix performance bottleneck in LLM initialization (~1.2s delay) #1148 introduced the current model-info cache.
- fix(llm): bound model-info discovery with a timeout to prevent deadlocks #3912 adds a timeout to eager discovery.
- [Bug]: Prevent budget enforcement from blocking model discovery endpoints (allow accessing free models) BerriAI/litellm#27923 and fix(proxy): don't enforce budgets on model-discovery / info routes (#27923) BerriAI/litellm#29483 address the separate problem of budget enforcement blocking authenticated discovery. That fix ships in LiteLLM v1.89.0+ and does not change missing-key 401 behavior.
Activity
all-hands-bot commentedon Jul 15, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was created by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Jul 16, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was created by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Jul 18, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was created by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Jul 19, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was created by an AI agent (OpenHands) on behalf of the repository maintainers.
all-hands-bot commentedon Jul 20, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was generated by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Jul 22, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was generated by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Jul 24, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was created by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Jul 29, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was created by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Jul 31, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you please take a look and add the appropriate priority label when you have a chance?
This comment was created by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Aug 2, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was posted by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Aug 3, 2026
[Automatic Post]: This issue has been waiting for triage. @xingyaoww @neubig @enyst, could you
please take a look and add the appropriate priority label when you have a chance?
This comment was created by an AI agent (OpenHands) on behalf of the user.
all-hands-bot commentedon Sep 4, 2026
[Automatic Post]: This issue was automatically closed due to 53 days of inactivity. We do this to help keep the issue tracker manageable and focused on active issues. If this is still relevant, feel free to reopen it.
Generated by OpenHands automation on behalf of the user.