Skip to content

Fall back to /v1/models when a server has no /health endpoint - #62

Merged
PTFOPlayer merged 1 commit into
masterfrom
feat/health-check-models-fallback
Oct 5, 2026
Merged

PTFOPlayer merged 1 commit into
masterfrom
feat/health-check-models-fallback

Conversation

@PTFOPlayer

Copy link
Copy Markdown
Owner

OpenAI-compatible gateways and llama.cpp-style servers frequently ship without a /health route, so the startup check failed with a bare "404 (no /health endpoint)" warning even when the backend was perfectly usable — leaving --skip-health-check as the only way to silence it.

health_check() now treats an HTTP 404 on /health as "endpoint missing" rather than "server unhealthy" and probes /v1/models instead: a non-empty model list proves the backend is reachable and serving, so it counts as a successful check. An empty list or a failing model request still reports an error naming both endpoints. Other /health statuses (5xx, 401, transport failures) are surfaced unchanged — the fallback never masks an endpoint that exists but is unhappy.

  • add fetch_models_raw(): strict variant that separates "answered with an empty list" from "request could not complete", which the health check needs to interpret the result. fetch_model_list() becomes a thin wrapper, keeping its lenient contract so model auto-selection is untouched
  • add models_url(), normalising a trailing /v1 the same way chat_url() does
  • cover all six paths with unit tests against a minimal in-test HTTP mock (tokio TcpListener, no new dependencies)
  • document the fallback in the Provider trait, the skip_health_check setting, the --skip-health-check help, README and TINYHARNESS.md

Ollama needs no change (its health check already is a model listing) and Sockudo is left alone: its /up/ route reports tunnel status, not OpenAI compatibility.

OpenAI-compatible gateways and llama.cpp-style servers frequently ship
without a /health route, so the startup check failed with a bare
"404 (no /health endpoint)" warning even when the backend was perfectly
usable — leaving --skip-health-check as the only way to silence it.

health_check() now treats an HTTP 404 on /health as "endpoint missing"
rather than "server unhealthy" and probes /v1/models instead: a
non-empty model list proves the backend is reachable and serving, so it
counts as a successful check. An empty list or a failing model request
still reports an error naming both endpoints. Other /health statuses
(5xx, 401, transport failures) are surfaced unchanged — the fallback
never masks an endpoint that exists but is unhappy.

- add fetch_models_raw(): strict variant that separates "answered with
  an empty list" from "request could not complete", which the health
  check needs to interpret the result. fetch_model_list() becomes a
  thin wrapper, keeping its lenient contract so model auto-selection is
  untouched
- add models_url(), normalising a trailing /v1 the same way chat_url()
  does
- cover all six paths with unit tests against a minimal in-test HTTP
  mock (tokio TcpListener, no new dependencies)
- document the fallback in the Provider trait, the skip_health_check
  setting, the --skip-health-check help, README and TINYHARNESS.md

Ollama needs no change (its health check already is a model listing)
and Sockudo is left alone: its /up/<app> route reports tunnel status,
not OpenAI compatibility.
@PTFOPlayer
PTFOPlayer merged commit d907e04 into master Oct 5, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant