[JeniaJitsev]

Problem

registerLLMService builds an llm worker's identity groups from the backend's /v1/models (model=<id>). They are not part of the configuration. When StartServiceHealthRevalidation restores a service it had withdrawn, it re-advertises it with viper.GetStringSlice("service.identity_group") only. For a typical llm worker that list is empty. The worker comes back ready but carries no model groups, so model-based routing does not reach it, and nothing repairs that short of restarting the worker.

Observed

On 2026-11-06, on a private mesh at f740781, two workers fronted vLLM engines through an SSH-tunnelled loopback bridge. Both failed three consecutive health probes during a stretch of slow logins on the remote side, and both were withdrawn ("failed 3 consecutive health checks; withdrawing advertisement"). Once the backends answered again, the head's /v1/dnt/table listed both workers ready with an llm service whose identity_group was empty. The gateway's model list no longer contained their models. Restarting only the worker processes (backends untouched) re-registered them with model=<id> groups within about 30 s, and inference worked again.

Fix

• The model fetch moves out of registerLLMService into fetchLLMIdentityGroups(port, attempts, interval). Behaviour there is unchanged: 10 attempts, 3 s apart, loud on final failure.
• On restore, restoredService re-reads /v1/models once for an llm service (it has just passed a health check). If that read fails, it uses the identity groups the service held when it was withdrawn, recorded by the loop before removeLocalService. Configured groups are kept in both cases. Non-llm services behave as before.
• The fetch sits behind a package-level variable (llmIdentityGroupsFetcher) so the restore path can be tested without a network.

Tests

go test ./internal/protocol/... and go vet ./internal/protocol/ pass with Go 1.27.3, and go build ./... succeeds. The baseline on upstream main (4b6e90f) passed before the change. New tests:

• TestRestoredService: llm restore uses the fetched groups; configured groups are merged; a failed fetch falls back to the groups held at withdrawal; a non-llm service keeps its configured groups and is never probed.
• TestFetchLLMIdentityGroups: parses /v1/models from an httptest server and returns an error once it is gone.
• TestLocalServiceIdentityGroups: returns a copy, or nil for an unknown service.

Separate observation, not addressed here

After a worker restart, the head sometimes did not pick up the worker's node-table row for several minutes. The worker listed itself ready with its services, while the head's /v1/dnt/table lacked it, and stopping the worker for 20 s or 70 s before starting it did not help. A POST /v1/dnt/_node with body {} on the worker, which forces UpdateNodeTable and a fresh store.Put, made the row appear at the head within 5 s. I have not traced the cause; it may deserve its own issue.

🤖 Generated with Claude Code