Skip to content

security: drop guessable LITELLM_MASTER_KEY fallback on gateway clients#116

Merged
AlienWalker1995 merged 1 commit into
mainfrom
security/client-litellm-key-guards
Jul 25, 2026
Merged

security: drop guessable LITELLM_MASTER_KEY fallback on gateway clients#116
AlienWalker1995 merged 1 commit into
mainfrom
security/client-litellm-key-guards

Conversation

@AlienWalker1995

Copy link
Copy Markdown
Owner

Summary

Follow-up to PR #111 (model-gateway edge-auth hardening). The gateway now refuses a guessable master key, but downstream clients still fell back to the literal local when authenticating to it. Post-hardening that means an unset key is sent as a guessable local (or the client is silently locked out with a 401) instead of failing loud. This removes every guessable local default on the client side — an unset key now yields an empty bearer the gateway rejects loudly (security + correctness).

Verification — each client IS provided the real key (checked before removing)

LITELLM_MASTER_KEY is a CORE_SECRET_KEY (ordo/render.py:35), always rendered into secrets.env. The stack up and the per-service recreate both pass --env-file .env --env-file secrets.env (services/ops-api/compose_recreate.py — its docstring states: "Without secrets.env, ${LITELLM_MASTER_KEY}… go UNSET"), so:

  • comfyui (wants_secrets: true), open-webui (wants_secrets: true), v1-parity dashboard, n8n — receive the real value via compose ${LITELLM_MASTER_KEY} interpolation at up-time.
  • hermes/agent — service rendered secrets=Trueenv_file: [.env, secrets.env], so the entrypoint's runtime ${LITELLM_MASTER_KEY} expansion reads the real key in-process.

None genuinely depended on the :-local fallback; all removals are safe.

Sites fixed

File Before After
services/comfyui/plugin.yaml LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:-local} ${LITELLM_MASTER_KEY}
services/open-webui/plugin.yaml OPENAI_API_KEY: ${LITELLM_MASTER_KEY:-local} ${LITELLM_MASTER_KEY}
services/open-webui/plugin.yaml RAG_OPENAI_API_KEY: ${LITELLM_MASTER_KEY:-local} ${LITELLM_MASTER_KEY}
services/v1-parity/dashboard.yaml MODEL_GATEWAY_API_KEY: ${LITELLM_MASTER_KEY:-local} ${LITELLM_MASTER_KEY}
hermes/entrypoint.sh model.api_key "${LITELLM_MASTER_KEY:-local}" : "${LITELLM_MASTER_KEY:?…}" fail-loud guard + ${LITELLM_MASTER_KEY}
services/automation/plugin.yaml (n8n) OPENAI_API_KEY: local (bare literal) ${LITELLM_MASTER_KEY}

Beyond the reported set: the n8n site (automation/plugin.yaml) hardcoded a bare local (not the :-local pattern), so it wasn't in the batch-2 list — but it's the same class of guessable gateway-client key and is fixed here. Verified it interpolates the real key from secrets.env at up-time (interpolation is CLI---env-file based, so it works despite n8n lacking wants_secrets).

Validation

  • grep for LITELLM_MASTER_KEY:-local / :- local across the repo → zero remain (client and gateway code).
  • python -m ordo --source out/ordo.yaml render → succeeds. New docker-compose.yml sha 6450f3f0 (baseline 727e332d — already drifted from the audit-quoted 3f63588d because out/ carries the operator fable pin). grep -i "master.*local\|LITELLM.*local\|API_KEY: local" on the new compose → none; all client key lines render as bare ${LITELLM_MASTER_KEY}.
  • ruff check . → clean. pytest tests/ -q564 passed, 4 skipped (no test asserted the old client env strings).

Out of scope (surfaced, not changed here)

  • services/v1-parity/dashboard/app.py:202os.environ.get("MODEL_GATEWAY_API_KEY", os.environ.get("LITELLM_MASTER_KEY", "local")). Baked into ordo/dashboard-v1:latest (needs an image rebuild, not a next-deploy manifest change); the "local" default is dead in practice because MODEL_GATEWAY_API_KEY is always injected. Recommend a follow-up dashboard-image PR to drop it.
  • scripts/smoke_test.sh / scripts/smoke_test.ps1 — manual diagnostic probes default to 'local'; operator tooling, not a running client credential.

🤖 Generated with Claude Code

PR #111 hardened the model-gateway to refuse a guessable master key.
Downstream clients still fell back to the literal `local` when calling the
gateway, so an unset key now silently locks the client out (401) instead of
failing loud. Remove every guessable `local` default on the client side; an
unset key now yields an empty bearer the gateway rejects loudly.

Each client is provided the real key: LITELLM_MASTER_KEY is a CORE_SECRET_KEY
rendered into secrets.env, and the stack (and per-service recreate) is brought
up with `--env-file .env --env-file secrets.env`, so `${LITELLM_MASTER_KEY}`
interpolates the real value at compose-up (comfyui/open-webui/dashboard/n8n)
and reaches the hermes/agent container in-process via its secrets.env env_file.

Sites fixed:
- services/comfyui/plugin.yaml          LITELLM_MASTER_KEY  ${..:-local} -> ${..}
- services/open-webui/plugin.yaml (x2)  OPENAI_API_KEY / RAG_OPENAI_API_KEY
- services/v1-parity/dashboard.yaml     MODEL_GATEWAY_API_KEY
- hermes/entrypoint.sh                  model.api_key: add `:?` fail-loud guard, drop :-local
- services/automation/plugin.yaml       n8n OPENAI_API_KEY: hardcoded `local` -> ${LITELLM_MASTER_KEY}
  (bare-literal sibling of the same hole, found by grep beyond the reported set)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@AlienWalker1995
AlienWalker1995 merged commit 35b3032 into main Jul 25, 2026
3 checks passed
@AlienWalker1995
AlienWalker1995 deleted the security/client-litellm-key-guards branch July 25, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant