Gate server readiness on initial datasource discovery#224
Merged
Conversation
Contributor
🐼 Smoke eval —
|
| question | result | tokens | tools |
|---|---|---|---|
forky_node_coverage |
✅ | 12,256 | 3 |
tracoor_node_coverage |
✅ | 12,785 | 3 |
mainnet_block_arrival_p50 |
✅ | 17,538 | 15 |
list_datasources |
✅ | 11,467 | 1 |
block_count_24h |
✅ | 17,134 | 13 |
missed_slots_24h |
✅ | 14,764 | 6 |
🔭 Langfuse traces (6 runs; ⚠️ = failed)
The report walks this branch's commits against the master baseline and the most recent release. A self-contained copy is in the run's eval-smoke-* artifact.
The server's /ready endpoint returned 200 as soon as the HTTP listener was up, before the proxy completed its first datasource discovery, and 'panda init'/'server start' waited on the equally-unconditional /health — so they declared the server ready while /api/v1/datasources was still empty. Track a per-proxy discovery flag exposed via proxy.Service.Ready() (the router reports its primary external proxy, or all local proxies when there is no external one), gate /ready on it, and switch the init/start wait to poll /ready. /health stays unconditional for liveness. Ready() also returns true when discovery is blocked waiting for the user to authenticate: the server is up and can do no better unattended, so a yet-to-auth user does not wedge the wait at 503. Only transient discovery failures keep it not-ready.
The semantic-search index build stays on the startup path (synchronous), but now logs document-level progress so operators can watch it advance in 'panda server logs'. The remote embedder reports documents embedded as it works (per cache phase and per batch) via an OnProgress callback, and searchruntime attributes that to the current stage, logging e.g. "Embedding search index stage=runbooks embedded=450 total=1200".
samcm
force-pushed
the
feat-server-readiness-gate
branch
from
af27bbe to
1db6c4d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two server-startup improvements (rebased onto master after #228/#222 merged).
The server's
/readyreturned 200 as soon as the HTTP listener was up — before the proxy completed its first datasource discovery — andpanda init/server startwaited on the equally-unconditional/health, so they reported ready while/api/v1/datasourceswas still empty. This tracks a per-proxy discovery flag exposed viaproxy.Service.Ready()(the router reports its primary external proxy, or all locals when there's no external one), gates/readyon it, and switches the init/start wait to/ready;/healthstays for liveness. Crucially,Ready()also returns true when discovery is blocked waiting for the user to authenticate — the server is up and can do no better unattended, so a yet-to-auth user never wedges the wait at 503; only transient discovery failures keep it not-ready.It also logs document-level progress of the (still synchronous) search-index build, so operators can watch it advance in
panda server logs: the embedder reports documents embedded as it works and searchruntime attributes it to the current stage, e.g.Embedding search index stage=runbooks embedded=450 total=1200".