Repository navigation
fix(code-index): resume a successor's graph seat after restart - #2897
Closed
devin-ai-integration[bot] wants to merge 2 commits into
Closed
devin-ai-integration[bot] wants to merge 2 commits into
devin-ai-integration[bot] wants to merge 2 commits into
Conversation
A SIGTERM after text publication could leave the graph head on the prior generation. Distinguish an interrupted active generation from a stale retained manifest by checking the durable publication pointer. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
ScriptedAlchemy
added a commit
that referenced
this pull request
Oct 2, 2026
…2927) e0fb1eb (#2842) made retained graph-head recovery treat any head on another generation as an in-flight successor that owns the head, and stopped replaying that retained manifest for good. When the durable publication pointer names the retained manifest, the head is older: the manifest is the newest publication and its graph never seated. A stale ignored-dependency admission that loses to a competing reconcile leaves exactly that state, so the competing generation never served. Recovery now reads the publication pointer and resumes the replay when it names the retained manifest. The #2797 case, a stale retained owner behind a newer pointer, keeps the skip. The production change is the one in #2897, applied unchanged. Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Owner
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
current. Before this change, graph serving stayedpendingand status stayedwarming/indexingwith no end.Motivation
Refs #1226 (restart after SIGTERM mid-refresh rebuilds or never finishes). This builds on current master, after #2851 withdrew the parent-carried lexical artifact path. Nothing from that path comes back.
Root cause, from a kill/restart probe and the daemon log:
bulk_commit 22/98, the restarted daemon continues at22/98, so text receipts are already durable per batch.registry/mount.rs, added in e0fb1eb for code-index: store-lock contention spins follow-up reconciles; graph-head conflict forces full rebuild #2797) compares the retained manifest (the successor) with the durable graph head (still the older generation). It hitscode_graph.recover_verified_snapshot_from_head.generationand reads the mismatch as "an in-flight successor owns the head". It then setsgraph_head_conflict_generation, and every later retained pass skips withgraph_head_belongs_to_another_generation. Nothing else ever publishes that generation's graph:To tell the two cases apart: a graph head is published only for a generation that the durable publication pointer has already named, and the pointer only moves forward. So if the retained manifest is the pointer's generation, a head on any other generation is older, which means this generation's own graph publication was interrupted. The #2797 case is different: the worker holds a stale retained owner while the pointer already names a newer successor. That case keeps the existing skip.
Changes
code_index_scheduler/graph_activation.rs: addsCodeIndexWorktreeSchedulerV1::names_active_publication(&CodeGenerationId) -> Result<bool, CodeIndexSchedulerErrorV1>, which reads the durable publication pointer. Errors stay typed.code_index_scheduler/registry/mount.rs: the retained-recoveryspawn_blockingreturns(replay_binding, retained_names_active_publication). A new arm comes before the existing conflict arm:registry.rs: doc comment ongraph_head_belongs_to_another_generationchanged so it no longer overstates what the conflict means.tests/daemon_suite/indexing_lifecycle_test.rs: new testsigterm_after_successor_publication_resumes_its_unfinished_text_and_graph. The journey:0 < completed_files < total_files).currentis the same successor (nothing was re-extracted or resealed), graph serving isready, a batch symbol (cancellation_probe_0095_127) resolves, and the originallifecycle_main_symbolstill resolves fromsrc/lib.rs.No timeout was raised and no test bound was loosened (the existing
RECEIPT_TIMEOUTis used). SQLite mmap is unchanged.Test plan
Red before the fix (CLI built from unmodified production code): the new test failed after 47.40 s with
restart did not make the retained successor current. Status showedcode_graph_serving.state: "pending"while the successor's text was at98/98.Green after the fix:
cargo fmt --all -- --checkcargo clippy -p tracedecay-code-index-runtime --all-targets -- -D warningscargo clippy -p tracedecay --all-targets --features test-helpers,test-transport -- -D warningscargo build -p tracedecay-cli --bin tracedecaycargo test -p tracedecay --test daemon_suite --features test-helpers,test-transport --no-runtarget/debug/deps/daemon_suite-<hash> --exact indexing_lifecycle_test::sigterm_after_successor_publication_resumes_its_unfinished_text_and_graph1 passed; 0 failed, 3 of 3 runstarget/debug/deps/daemon_suite-<hash> --exact indexing_lifecycle_test::mounted_incremental_lifecycle_preserves_only_complete_compatible_generations1 passed; 0 failedcargo test -p tracedecay-code-index-runtime --lib code_index_scheduler::registry52 passed; 0 failed; 513 filtered outcargo nextest run --workspace --no-fail-fastpasses (not run locally; only the affected suites above were run)cargo clippyhas no new warnings (touched crates,-D warnings)Not covered here: a kill that lands before the pointer moves (during extraction or seal). In that case nothing about the successor is durable yet (staged segments are temporary files that get removed), so the restart rebuilds that batch from the retained complete generation, as it did before this change.
Checklist
CHANGELOG.mdupdated (generated from conventional commits).envfiles includedLink to Devin session: https://app.devin.ai/sessions/c35915f46ae44002baff5733673cb8bf
Open in Devin Desktop: https://app.devin.ai/desktop/session/c35915f46ae44002baff5733673cb8bf?variant=devin
Requested by: @ScriptedAlchemy
Local verification (land-devin-prs): not landed
Rebased on master 6f99af4; CLI built from each variant (
cargo build -p tracedecay-cli --bin tracedecay,TRACEDECAY_TEST_BINset),daemon_suite --exact indexing_lifecycle_test::sigterm_after_successor_publication_resumes_its_unfinished_text_and_graph:successor never exposed partially applied text progress(never reaches the restart).The final status in the failure shows the successor already
current,progress.completed_files: 98of 98 after ~2.5 s, so no 25 ms status poll observed0 < completed < totalon this host. The test races the product for an intermediate state and is not falsifiable as written: the fix has no fail-before/pass-after evidence here. It needs a deterministic interruption point for the successor's text (for example the restart test reaching the same state through the real overflow/cancel path at a receipt it can wait on, rather than a progress window) before it can land.