[antonwolfy]

Summary

Follow-up to the review of #709, where two threads left open that a credentials error under a catch-all class is judged transient, so a routing run keeps calling and writing failed rows (review, review). No reply was posted on either; the PR merged with both open.

On main, is_retryable(APIConnectionError("Unable to locate credentials")) is True (the class is a litellm catch-all and _NON_RETRYABLE_MESSAGES does not hold the text), so is_run_wide returns False at once and model_routing makes every call and writes a failed row for each. The Bedrock mapping raises AuthenticationError for that exact text, so the reported #677 path was covered; the gap is any other path that wraps the text in a catch-all class. litellm's Bedrock mapping also leaves the other botocore credential errors as a bare APIConnectionError (checked by passing each botocore exception through litellm.exception_type with custom_llm_provider="bedrock"), and those were retried three times too:

botocore error	text	class litellm raises	is_retryable on main
NoCredentialsError	Unable to locate credentials	AuthenticationError (Bedrock); the same text under a catch-all is the review case	False; True under a catch-all
NoAuthTokenError	Unable to locate authorization token	APIConnectionError	True
PartialCredentialsError	Partial credentials found in env, missing: ...	APIConnectionError	True
ProfileNotFound	The config profile (x) could not be found	APIConnectionError	True
UnauthorizedSSOTokenError	The SSO session associated with this profile has expired ...	APIConnectionError	True

Changes

• dokimi/llm.py: _NON_RETRYABLE_MESSAGES gains unable to locate credentials and the four texts above, so is_retryable is False for them under APIConnectionError, APIError and InternalServerError, and is_run_wide is True.
• dokimi/novelty/routing_stop.py: _NO_CREDENTIALS was _NON_RETRYABLE_MESSAGES plus that one entry; it is dropped and is_run_wide reads _NON_RETRYABLE_MESSAGES directly.
• docs/skill-novelty.md: the permanent-error paragraph names the AWS cases.
• Tests: tests/test_llm_retry_credentials.py adds the five AWS texts to the credential cases (not retried: one request, no sleep); tests/test_novelty_routing_estimate_and_stop.py adds a routing run whose provider raises the credentials error (and a missing profile) under an APIConnectionError-named class: one call, reason permanent_error; and a connection reset under the same class is still a failed row per call.
• Changelog fragment (Fixed).

Risk

is_retryable is shared by the retry wrappers (LLM.ask in dokimi.llm, the novelty/batching.py callers and the CLI error reader), so those five texts now fail on the first attempt instead of after 2 + 4 + 8 s of backoff. A retry cannot supply credentials, a profile or a refreshed SSO session, so no retry is the right outcome. Other messages under the catch-alls, and a transient APIConnectionError("Connection error."), are retried as before. Not added, because they can be transient: TokenRetrievalError and CredentialRetrievalError (a remote token or metadata endpoint failing), SSOTokenLoadError, NoRegionError (not a credential).

Testing

On a Linux review host, at dad1dc2 (on main b4bcfa4):

• make ci-local: 31 passed, 2 failed, 14 skipped. 20407 tests passed, 19 skipped; changed-line coverage 100% (1 line measured). The two failures are the host's: static / Install gitleaks (the download returned HTTP 504) and static / Scan commits (gitleaks missing, exit 127). Skipped: the matrix legs on other Pythons, semgrep, conda-forge, fuzz, deploy and the other steps that need a runner or network.
• Repro: with APIConnectionError-named exceptions and the real litellm classes, the new tests fail at main's code (7 failures: five credential cases, two routing cases) and pass here; the plain connection reset test passes on both.
• Mutant: dropping unable to locate credentials from _NON_RETRYABLE_MESSAGES fails the routing test and the retry test for that text.
• Merged together with fix/url-extraction-delimited, test/leftovers-2026-11-14 on main b4bcfa4: pytest -n auto 20429 passed, 19 skipped, mypy clean (420 source files), ruff check clean.
• Not run: Python 3.11, 3.12 and 3.14 (this host runs one version), gitleaks, a real Bedrock call.

Revert

A plain revert restores the old behaviour. Nothing is stored.

Checklist

• Tests added/updated and pytest passes
• ruff, mypy, and schema checks pass
• Docs updated (or N/A) — see the docs-drift checklist in CONTRIBUTING.md
• Self-reviewed per "Before opening a PR" in AGENTS.md
• No ad-hoc run artifacts committed
• New files carry the SPDX license header (or are exempt in .github/.licenserc.yaml)

[exai-bot]

✅ OpenCodeReview: Review complete: 0 finding(s) across 6 selected item(s).

CI at dad1dc2: 25 passing, 3 skipped.

Review details

• Commit reviewed: dad1dc2 against merge base b4bcfa4
• Coverage: 6 files in 2 groups
• Engine: OpenCodeReview v1.13.5-exai.4.8f73eada, us.openai.gpt-6.1-sol
• Usage: 109k tokens (66k from cache), 11s
• Tool calls: 15 (8 file_read, 2 ci_status, 2 code_search, 2 file_read_diff, 1 local_check_results)
• Run-wide LLM credentials and retry handling: 3 files
• Novelty routing stop behavior and documentation: 3 files

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.71%. Comparing base (b4bcfa4) to head (dad1dc2).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #751      +/-   ##
==========================================
- Coverage   95.71%   95.71%   -0.01%     
==========================================
  Files         414      414              
  Lines       55310    55309       -1     
  Branches     9509     9509              
==========================================
- Hits        52941    52940       -1     
  Misses       1466     1466              
  Partials      903      903              

Files with missing lines	Coverage Δ
dokimi/dokimi/llm.py	98.77% <100.00%> (ø)
dokimi/dokimi/novelty/routing_stop.py	100.00% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[vchamarthi]

Ready to merge at dad1dc2: Adds the AWS credential texts to _NON_RETRYABLE_MESSAGES so they are permanent under litellm's catch-all classes, and makes is_run_wide read the same list. It does what it says; nothing to fix.

Verified

• Retry of aws texts: The five AWS credential texts under APIConnectionError are not retried on the head and are retried on main; removing unable to locate authorization token fails the retry test for that text.
• Routing stop: model_routing with a catch-all-named error carrying Unable to locate credentials or a missing config profile makes one call and stops with permanent_error; connection reset under the same class is still one failed row per call.
• Merge and suite: The head merges onto main with no conflict and the whole local run passes.