Repository navigation
Requesting retry logic when model is overloaded #164
Description
I'm seeing:
[WARN] 5 concept(s) planned but only 4 written for foo (ServiceUnavailableError).
summary-rewrite.................openkb.agent.compiler WARNING: summary-rewrite failed for foo:
litellm.ServiceUnavailableError: GeminiException - {
"error": {
"code": 503,
"message": "This model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later.",
"status": "UNAVAILABLE"
}
}
Which apparently looks like I lost information? Not sure if there's a retry, but it looks like a retry isn't happening?
Hope this helps!
Activity
KylinMountain commentedon Jul 13, 2026
You can get this today without waiting on a release — OpenKB already forwards litellm's own retry setting through the
litellm:passthrough block. In.openkb/config.yaml:litellm uses
num_retriesas the default retry count for every LLM call (it's applied as a module-level global), so transient failures — Gemini503, Anthropic529 overloaded, etc. — get retried automatically across compile, query, and chat, with exponential backoff.