Skip to content

Add per-model availability checks and latency display #1765

Description

@jmzhang1911

Area

Dashboard

What are you trying to accomplish?

When a provider exposes or is configured with many models, I want to quickly verify which individual models can actually handle requests and compare their response latency before selecting or routing traffic to them.

What prevents this today?

The current provider test only shows whether a provider-level connection is usable. A successful provider check does not tell me whether every model is available, authorized for the account, compatible with the provider endpoint, or blocked by a model-specific quota or configuration problem.

Today I have to select and invoke models one at a time, then inspect the request result or logs. This is slow and error-prone for providers with large model catalogs.

What should OpenCodex do?

Add a per-model availability check to the Dashboard's model or provider view.

The user should be able to check all models for one provider, or retry a single model, and see:

  • available, unavailable, checking, and not-yet-checked states;
  • latency for successful checks;
  • a concise model-specific failure reason;
  • the time of the last check.

The bulk check should use bounded concurrency and be manually triggered by default so it does not unexpectedly consume excessive quota, cost, or rate limits.

Example usage or interface

Provider: example-provider                         [Check all models]

Model                       Status          Latency       Last checked
model-a                     Available       420 ms        just now
model-b                     Unavailable     —             just now
model-c                     Checking...     —             —
model-d                     Not checked     —             —

Selecting an unavailable model could show a sanitized reason such as model_not_found, permission_denied, rate_limited, or incompatible_endpoint, with a retry action.

An optional CLI equivalent could be:

ocx models check --provider example-provider

Alternatives or workarounds

I can manually send a request with each model, inspect logs, or call the provider's model-list endpoint. The first approach does not scale, and model discovery only proves that an ID was listed; it does not prove that inference succeeds for the current account and endpoint.

Additional context

A sanitized startup example shows the current ambiguity:

[opencodex] Provider model discovery for "deepseek" omitted configured model ids;
dropping them from the authoritative live catalog: deepseek-chat, deepseek-reasoner.

This reports a catalog discrepancy, but it does not provide a per-model health result or latency. Related UX and health discussions include #1062, #1146, and #753.

Checks

  • I searched existing issues and documentation.
  • This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
  • I removed secrets and personal data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestguiDashboard, tray, settings UI

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions