Skip to content

fix: reconcile LLM API keys after out-of-band llmPorts edits (PATCH route + load-time) - #76

Open
sethforprivacy wants to merge 1 commit into
MiaAI-Lab:mainfrom
sethforprivacy:fix/llm-api-key-reconcile
Open

fix: reconcile LLM API keys after out-of-band llmPorts edits (PATCH route + load-time)#76
sethforprivacy wants to merge 1 commit into
MiaAI-Lab:mainfrom
sethforprivacy:fix/llm-api-key-reconcile

Conversation

@sethforprivacy

Copy link
Copy Markdown

Follow-up to #25/#26 (per-port LLM API keys + syncLlmApiKeysToPorts), rebased on main at v1.8.6 (cc44d35).

Problem

syncLlmApiKeysToPorts is called only from the two llm-ports PUT routes, so two paths still rename llmPorts underneath the encrypted secrets store:

  1. PATCH /api/sparks/:id with llmPorts in the body — updates the spark, never syncs the keys.
  2. A direct sparks.json edit (config changed in the container or on disk) — never goes through a route at all.

Either way the key stays stored on the old port while the port the probe now uses has none, so the LLM collector probes the backend keyless. On a keyed vLLM backend that is a hard 401, silently, until someone notices and re-enters the key in the UI.

We hit path 2 moving one lane from :8888 to :8899 on a monitored node: 1,394 keyless GET /v1/models → 401 in a single hour from the dashboard, with the dashboard reporting the node's LLM panel as unavailable rather than as an auth failure. The other two monitored nodes, untouched, stayed green throughout — which is what made it non-obvious.

Fix

PATCH route — when the body carries llmPorts (validated the same way as the PUT /api/sparks/:id/llm-ports route), capture the pre-update ports and run syncLlmApiKeysToPorts after updateSpark, then restart the monitor from the re-read spark so the collector picks up the moved key. Bodies without llmPorts are untouched.

Load-time reconcileSparkRegistry reconciles once during load, deliberately conservative:

  • Exactly one orphaned keyed port and exactly one configured port without a key → the rename is unambiguous, so move the key and warn what was moved.
  • Any other mismatch shape → warn only, change nothing.
  • Load never prunes or deletes stored key material, in any shape. A config edit should not be able to destroy secrets on the next restart.

Warnings carry port numbers only, never key values. On our fleet the reconcile is quiet except for warn-only lines about pre-existing nodes that have ports configured but no key provisioned — no mutation, correct behavior.

Tests

New server/sparks/__tests__/SparkRegistry.llmApiKeys.test.js (8 tests, node:test, synthetic key, temp SPARKS_JSON_PATH / SPARKS_SECRETS_PATH, registry constructed through its real load path so the reconcile provably runs): the move shape, the migration warning text, the ambiguous shape both retaining the key and warning without leaking it, the aligned shape staying silent, and the syncLlmApiKeysToPorts rails (rename move, no-op, prune-on-removal).

npm ci && npm test on this branch: 177 tests, 176 pass, 1 fail — the failure is server/collectors/__tests__/showcasePrompts.test.js, which asserts on a pickShowcasePrompts("structural" source string that DecodeBench no longer uses. It fails identically on unmodified main at cc44d35 (7/8 in that file) and looks like what #67 is for. Everything else, including all 8 new tests, is green.

Deployed from this diff on a 3-node fleet since 2026-09-01; probes read auth=keyed on the renamed port with no manual key re-entry.

🤖 Generated with Claude Code

…oute + load-time)

MiaAI-Lab#25/MiaAI-Lab#26 added per-port LLM API keys and syncLlmApiKeysToPorts, but the sync
is only called from the two llm-ports PUT routes. Two paths still rename
llmPorts underneath the encrypted secrets store:

- PATCH /api/sparks/:id carrying llmPorts
- a direct sparks.json edit (config edited in the container / on disk)

Either leaves the key stored on the old port while the port the probe now
uses has none, so LLM probes hit the backend keyless and 401 until someone
re-enters the key by hand.

- PATCH /api/sparks/:id: capture prev llmPorts when the body carries
  llmPorts (validated as in the PUT route) and run syncLlmApiKeysToPorts
  after the update, then restart the monitor from the re-read spark.
- SparkRegistry: reconcile once at load. Only the unambiguous shape —
  exactly one orphaned keyed port and exactly one configured port without a
  key — moves the key; every other mismatch warns and changes nothing. Load
  never prunes stored key material.
- tests: load reconcile move / retain / aligned shapes plus the
  syncLlmApiKeysToPorts rails, with a synthetic key and temp
  SPARKS_JSON_PATH / SPARKS_SECRETS_PATH.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@sethforprivacy
sethforprivacy force-pushed the fix/llm-api-key-reconcile branch from c3fefed to 754aabb Compare September 2, 2026 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant