Area
Dashboard
What are you trying to accomplish?
When a provider exposes or is configured with many models, I want to quickly verify which individual models can actually handle requests and compare their response latency before selecting or routing traffic to them.
What prevents this today?
The current provider test only shows whether a provider-level connection is usable. A successful provider check does not tell me whether every model is available, authorized for the account, compatible with the provider endpoint, or blocked by a model-specific quota or configuration problem.
Today I have to select and invoke models one at a time, then inspect the request result or logs. This is slow and error-prone for providers with large model catalogs.
What should OpenCodex do?
Add a per-model availability check to the Dashboard's model or provider view.
The user should be able to check all models for one provider, or retry a single model, and see:
- available, unavailable, checking, and not-yet-checked states;
- latency for successful checks;
- a concise model-specific failure reason;
- the time of the last check.
The bulk check should use bounded concurrency and be manually triggered by default so it does not unexpectedly consume excessive quota, cost, or rate limits.
Example usage or interface
Provider: example-provider [Check all models]
Model Status Latency Last checked
model-a Available 420 ms just now
model-b Unavailable — just now
model-c Checking... — —
model-d Not checked — —
Selecting an unavailable model could show a sanitized reason such as model_not_found, permission_denied, rate_limited, or incompatible_endpoint, with a retry action.
An optional CLI equivalent could be:
ocx models check --provider example-provider
Alternatives or workarounds
I can manually send a request with each model, inspect logs, or call the provider's model-list endpoint. The first approach does not scale, and model discovery only proves that an ID was listed; it does not prove that inference succeeds for the current account and endpoint.
Additional context
A sanitized startup example shows the current ambiguity:
[opencodex] Provider model discovery for "deepseek" omitted configured model ids;
dropping them from the authoritative live catalog: deepseek-chat, deepseek-reasoner.
This reports a catalog discrepancy, but it does not provide a per-model health result or latency. Related UX and health discussions include #1062, #1146, and #753.
Checks
Area
Dashboard
What are you trying to accomplish?
When a provider exposes or is configured with many models, I want to quickly verify which individual models can actually handle requests and compare their response latency before selecting or routing traffic to them.
What prevents this today?
The current provider test only shows whether a provider-level connection is usable. A successful provider check does not tell me whether every model is available, authorized for the account, compatible with the provider endpoint, or blocked by a model-specific quota or configuration problem.
Today I have to select and invoke models one at a time, then inspect the request result or logs. This is slow and error-prone for providers with large model catalogs.
What should OpenCodex do?
Add a per-model availability check to the Dashboard's model or provider view.
The user should be able to check all models for one provider, or retry a single model, and see:
The bulk check should use bounded concurrency and be manually triggered by default so it does not unexpectedly consume excessive quota, cost, or rate limits.
Example usage or interface
Selecting an unavailable model could show a sanitized reason such as
model_not_found,permission_denied,rate_limited, orincompatible_endpoint, with a retry action.An optional CLI equivalent could be:
Alternatives or workarounds
I can manually send a request with each model, inspect logs, or call the provider's model-list endpoint. The first approach does not scale, and model discovery only proves that an ID was listed; it does not prove that inference succeeds for the current account and endpoint.
Additional context
A sanitized startup example shows the current ambiguity:
This reports a catalog discrepancy, but it does not provide a per-model health result or latency. Related UX and health discussions include #1062, #1146, and #753.
Checks