Skip to content

fix(code-index): fence stale progress boundaries - #4800

Open
ScriptedAlchemy wants to merge 81 commits into
masterfrom
fix/4508-progress-boundaries-ci-clean
Open

ScriptedAlchemy wants to merge 81 commits into
masterfrom
fix/4508-progress-boundaries-ci-clean

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Indexing progress

Refs #4508. The daemon records observed capture, parsing, resolution and native graph writes for a particular generation and writer attempt. Durable text pages and verified compact graph segments have separate counters from rows accepted by a builder. Progress is fenced by writer and generation identity across retries, restarts, and discarded publications. The CLI, MCP and dashboard use that distinction for stall diagnostics, freshness and progress display.

Both manifest and streamed native writers now report accepted entity and relation windows. Replaying an already-installed segment does not count it twice. The dashboard retains the completed generation’s committed work, and a rebuilding search with no work evidence reports progress=unreported.

Verification

On 394be3dd46, the macOS Bazel CLI suite passed all 421 tests, including 27 daemon precondition cases. The previously failing real daemon cold-start and restart test passed, and the graph/store tests observed both kinds of rows before the single verified segment and no events on replay. On the preceding changes, seven MCP graph-dispatch tests, three lexical-anchor tests, 49 dashboard DOM tests and 94 extension tests passed. A prior Linux run exceeded the lexical-anchor fixture’s 20-second publication budget under load. The complete macOS MCP suite at 394be3dd46 also passed all 673 tests in 789 seconds, including that lexical-anchor case. Hosted verification is pending.

Review status

This replaces closed #4795. Keep #4508 open pending independent review, the hosted Linux run on the current head, and acceptance on master for cold indexing, interrupted work, restart and completed progress. The separate draft hold on #4776 remains in effect.

At 416adbdb22, the pinned graph-db rustfmt gate and the native staged compact-copy test passed. The regression observes actual row windows on interruption and verifies the completed segment before publication. The preceding repository-gate failure reported formatting in two assertions added after the prior broad test run.

At 6b4ff93636, repository rustfmt and 45 distinct macOS Bazel cases passed. The tests included all 35 CLI index cases, eight sparse/full construction cases and two daemon publication cases. The registered attempt stopped at durable handoff while retaining the last recorded work and rejecting late callbacks. The CLI regression kept stall detection active for unfinished construction and let healthy durable publication proceed past the simulated stall window. All 49 DOM tests across IndexFreshness, CodeIndexNativeProgress and CodeIndexPipeline also passed. Hosted Linux, the earlier history-reconciliation hold and master acceptance for #4508 remain open.

At 0dda36d127, the repository rustfmt gate, strict Clippy on the shipped CLI and CLI unit target, all 38 CLI indexing tests, and all three native graph progress tests passed on macOS. The latter exercised accepted encoding work, durable page and segment receipts, read-only activation, and fencing of stale callbacks after restart.

Historical branch reconciliation

The original published head and its counterpart in the current branch both point to the complete repository tree 068ad9b3b09a533f498161bc6783c5882a79265b. Starting after common ancestor 091bf9d86c, every one of the 23 commits in the original sequence has the same full tree and stable patch ID as the corresponding commit in the current sequence. The only difference reported within those paired commits by git range-diff is the shortened header of the first commit, de2d6134ef → b2c4997fb4. The 22 changed file blobs at the previously reported 0651f1dbed head came from two subsequent commits, a 21-file formatting change (36b58e3f9f) and a one-line borrowed-resolution-segment correction (0651f1dbed).

The original commit identities remain on a divergent branch of history. All original product changes survived at the matched head, and git cherry d424073ee2 609aff4401 found zero original-only patches. The current head d424073ee2 descends from the identical tree through 19 additional commits.

The owner’s explicit merge hold remains in effect. Keep this PR and #4508 open pending owner disposition of the historical divergence, current-head review and CI, and live master acceptance for daemon cold indexing, interruption, stall detection, restart recovery and completed progress.

At abe2b55785, independent review of the shared polling classifier and its two dashboard callers confirmed that an active preseal/native successor continues polling past a retained terminal refusal, an idle terminal park falls back, and an empty loading scope polls for its first mount. All 54 DOM tests across IndexFreshness, CodeIndexPipeline and CodeIndexNativeProgress passed, as did the dashboard TypeScript check. The owner reiterated the merge hold on this head.

Refs #4508

Reapply PR #4795 without changing its existing branch or commits.
The previous immutable history failed the 72-character commit header gate.
@changeset-bot

changeset-bot Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 7218ed2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Refs #4508

Reject foreign daemon, producer and sealed-source identity reports,
even after a verified SQLite staging restart clears committed counters.
Add a nonzero regression for the production progress authority.

Copy link
Copy Markdown
Owner Author

Taking the #4508 producer implementation on this existing branch, starting from exact head 29c1258. File ownership for this pass: pre-seal capture/parse observation and its separately typed attempt authority, native graph writer progress, and the corresponding freshness/status consumers and regressions. Maintenance/sweep owners: please do not duplicate or rewrite this work; preserve all commits and use expected-head nonforce fast forwards. #4616 owns sealed-store reader/receipt/retention changes; I will keep graph writer changes separate and reconcile any overlap against the exact current heads. Existing boundary tests are not completion evidence. Refs #4508; no closure/merge claim, and #4776/Kev stays draft and #4763 stays read-only.

ScriptedAlchemy commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner Author

Published commit 9f3bfe04cbac4dee01979cd44791d57c5a68b99c directly to this PR, fast-forwarding exact head 29c1258b8f491def3cf34c5cd6e4b4a6b5e194f6 with expected_sha and force=false.

Source changes

  • dashboard/src/workspaces/observatory/CodeIndexPipeline.tsx: keep active polling when rebuild_in_flight is true or immutable-artifact restore_progress exists, even when the serving envelope/previous report is already ready. useObservatoryReads therefore keeps its existing 1-second active interval instead of prematurely falling back to 30 seconds.
  • An in-flight build without a committed report now renders “code-index build in progress; progress not yet reported”, not “no active code-index build.” It does not manufacture a generation, phase, percentage, or completed-work counter.
  • Added CodeIndexPipeline.dom.test.tsx: eleven polling cases plus two React rendering journeys for unreported → committed → ready → retired progress and bounded restore → restored state. Existing tests are unchanged.

Verification performed

The original source was reconstructed byte-exactly and matched GitHub blob 5a3704cdfe5ec3b429fb0b9086e7acc67f35c0b9. A Node test harness executed the actual TypeScript-transpiled exported polling predicate, with unused UI imports guarded against invocation: 11 checks, 4 failing before; 11 passing, 0 failing after. This is focused predicate execution, not a React renderer or full Vitest run. Both changed TSX files also passed syntax transpilation. Published source/test blobs match the locally checked bytes (571d7eb4…, bc61e12c…). The final remote compare is one ordinary commit and exactly these two files (12 source insertions/2 deletions plus the new regression file).

Still required: run pnpm --dir dashboard test src/workspaces/observatory/CodeIndexPipeline.dom.test.tsx, dashboard typechecking, and native Bazel checks for the PR's Rust changes. This container has no Bazel and cannot resolve GitHub for a full checkout/dependency installation; no native Rust pass, full Vitest pass, DOM pass, or independent review is claimed. Exact-head hosted CI is queued.

Backend capture/parse and direct-seal/fallback graph producers remain owned by the active implementation pass above; this commit does not touch those files or claim to complete them. Refs #4508 only. Keep the issue/PR open pending full acceptance; #4776/Kev and #4763 remain untouched.

Keep active polling for in-flight rebuilds and lexical artifact restores,
even when the serving envelope or previous generation is already ready.
Do not render an in-flight build without a report as an idle pipeline.

Add polling regressions and DOM transition coverage for unreported work,
committed progress, completion, and bounded artifact restoration.

Refs #4508
Carry daemon/producer epochs, generation, last-progress time, committed text counters and blocked reason into MCP search and graph-read summaries. Distinguish absent progress from reported zero work without inventing pre-seal authority. Add nonzero rendering regressions for repeated reads, blocked recovery and restart/remount epoch scope.

Refs #4508. Producer instrumentation and cold/restart end-to-end acceptance remain incomplete; no Bazel execution is claimed.

Copy link
Copy Markdown
Owner Author

Published 0fc10fe as a non-force fast-forward with expected head 9f3bfe0, preserving the concurrent dashboard commit and all earlier commits.

This is a narrow production MCP propagation fix, NOT completion of #4508. Search and graph-read summaries previously dropped the daemon's progress authority, epoch, last-work timestamp, committed counters and blocked reason. They now carry generation + daemon/producer incarnation, progress epoch, the original last_progress_micros, committed TEXT artifact counts/bytes and the typed blocked reason. Text counters are explicitly labelled as text, not native graph writes. An in-flight rebuild without a producer report renders progress=unreported rather than inventing a pre-seal generation/count/clock.

Four new tests exercise the actual search_freshness/freshness_lines renderers with nonzero counters: cold_unreported_work_has_no_fabricated_progress_authority; repeated_reads_do_not_turn_elapsed_time_into_progress; blocked_and_recovered_reports_preserve_work_and_name_the_real_phase; restart_and_remount_keep_epoch_scope_visible. Existing coverage and restore/park tests are retained. These are renderer regressions, not cold-build/restart end-to-end proofs.

Validation: diff inspected against exact parent; no Rust build/test/check/codegen has been run locally. This session has no Bazel/Bazelisk executable or checkout-capable network path (git fails proxy DNS), and the public download path also failed. No Cargo, alternate Rust toolchain invocation, generated-contract hand edits, infrastructure changes, or test bypasses were used. Exact-head native CI run 42439812291 is queued; no pass is claimed.

Independent PR-maintenance review requested for 0fc10fe, especially text-vs-graph attribution, owner-scoped epochs, absence of read-time clock fabrication, and the renderer assertions. Keep this PR unmerged: separately typed pre-seal capture/parse production instrumentation, direct native graph durable-write instrumentation, and cold/interrupted/restarted end-to-end acceptance are still unfinished. The earlier unreferenced producer-draft commit 95aac68 is incomplete and must not be adopted; no producer draft was pushed onto this branch. Refs #4508. #4776/Kev remains draft; #4763 untouched.

Refs #4508. Keep pre-seal work under separately typed attempt authority, not a fictional sealed generation. Capture on clean-HEAD and dirty paths; instrument actual staged parser-backed file work; reject stale, duplicate and delayed callbacks. Durable native-graph work, codegen/wire and restart e2e remain open.

Copy link
Copy Markdown
Owner Author

Published 091bf9d86c8e7dbc1e9bd661fe60a06ea10366ae on this existing PR by expected-head non-force fast-forward from 0fc10fed2121688077c5427072b495ee0f393690 (all prior boundary, dashboard and MCP commits preserved). Refs #4508; partial producer work, NOT acceptance or a close/merge signal.

  • Introduced CodeIndexPresealWorkV1 (capture vs parser-evidence materialization) and a separately typed attempt slot keyed by daemon incarnation, producer incarnation and monotonic pass sequence. The slot has no fictional generation ID or sealed digest. Only actual positive producer events move work counts or last_progress_micros; duplicate/out-of-order callbacks cannot refresh a stalled attempt, and cancellation/retry/Drop fences an old producer.
  • Recorded successful real capture decisions inside the parallel dirty-worktree worker, rather than after the whole batch, and inside the clean-HEAD Git-tree visitor, which is the primary cold path. Clean traversal has no invented total. Reused rows are not counted as new capture.
  • Recorded parser-backed file materialization only after its segment has successfully entered preparation, explicitly not as a durable text artifact or native graph commit. Reused parser evidence is not labelled a fresh parser invocation.
  • Added two nonzero slot regressions (no heartbeat/rollback, stale retry and incarnation fencing) and one real two-file cold reconcile regression asserting positive capture + parser-evidence observations.

Not claimed: no local Rust/Bazel execution (this environment lacks Bazel/Bazelisk and checkout-capable network); exact-head CI run 42447419869 was queued at last check. The attempt slot is not yet surfaced through the freshness wire, CLI/MCP or dashboard. Native durable graph writer/page progress and end-to-end restart/resume, stalled and blocked/recovered transitions are still required, as are hosted Bazel tests and independent review. Do not treat these three tests as #4508 completion. #4776 remains draft; #4763 was untouched.

Refs #4508. Distinguish real WAL-settled page transactions from adopted receipts; direct seal is one compact artifact, not invented durable pages. Add nonzero persistent write/replay/close-reopen regression. Status wiring and restart E2E remain open.

Copy link
Copy Markdown
Owner Author

Follow-up producer commit de2d6134efac295d253b915424fd7a4976ac04ae has now been published on the same #4800 branch, fast-forwarded from exact head 091bf9d86c8e7dbc1e9bd661fe60a06ea10366ae with expected_sha and force=false.

This targets the actual native graph writers, not project_graph_publication_phase's read-side projection:

  • Native fallback staging pages now yield a typed Applied vs Reused transaction verdict from the real run_gated_batch write path. The graph_native_stage_page_applied event is emitted only after an actual row+receipt transaction completes, including WAL settlement for persistent stores; replaying an already-applied receipt emits no fake write event. It carries real page ordinal/kind, mutation and live-byte counts, and whether WAL sync applies.
  • The dependency-free direct-seal path does not write pages. It emits graph_native_compact_segment_verified only after its single compact container is installed and reopened with recovered-digest proof; no fabricated per-page progress during encoding.
  • Added a real nonzero persistent-graph regression which applies an actual native page, verifies replay returns Reused instead of Applied, then closes/reopens the WAL-backed store and proves the page receipt and rows survived.

This is real producer instrumentation, not yet the progress-status pipeline. Neither the native events nor the new separately typed pre-seal attempt slot are wired to freshness/CLI/MCP/dashboard yet. No local Bazel/Rust test has been executed; wait for an actual exact-head hosted run, not queued/skipped jobs. Full cold + restart/resume, stalled, blocked/recovered end-to-end evidence remains required. Refs #4508, not Fixes; keep PR and issue open, and do not touch #4776 or release #4763.

Refs #4508. A concurrent verified segment may be adopted after a losing staging install; only the writer that actually installed a new directory emits the native durable-segment event. Add real sealed-container collision, cleanup, and verified-reopen regression. Status wiring and interrupted-restart acceptance remain open.

Copy link
Copy Markdown
Owner Author

Published follow-up b329b67 on this same PR with a checked non-force fast-forward from exact parent de2d6134. Refs #4508 (partial).

Native writer correctness: the compact sealed-store path previously emitted graph_native_compact_segment_verified even when this writer lost the directory-install race and merely reopened another writer’s verified artifact. The new install verdict suppresses that false new-write observation while preserving the required digest-verified adoption and cleanup. Added concurrent_sealed_install_candidate_does_not_count_as_new_write, a nonzero real-container fixture that installs the winning segment, exercises an actual losing rename into the nonempty destination, verifies loser staging cleanup, and reopens the real winner with the expected recovered digest and nonzero entity/relation counts.

Scope coordination: this touches only the native compact writer-install/reporting block of sealed_store.rs; #4616/#4804 owns bounded receipt reads, read-side cleanup and retention. No changes to those reader/retention sites, no conflict rewrite or dropped commits. Independent review requested for the race outcome, platform-specific directory rename behavior, and whether this remains compatible with #4804.

Not acceptance: no local Bazel/Bazelisk or checkout-capable path; no Rust compile/format/test result claimed. Exact-head CI 42450800548 was queued when last sampled. Pre-seal typed attempt and native durable events still require daemon → CLI/MCP/dashboard status wiring and real cold/interrupted-restart/stall/blocked/recovered end-to-end tests. Keep #4508 and this PR open. #4776/Kev remains draft; #4763 untouched.

Copy link
Copy Markdown
Owner Author

Newly confirmed CI blocker — exact #4800 head b329b671 (2026-11-08)

Hosted Repository gates job 128639401232 executed and failed commitlint, rather than merely being queued. Its output: header must not be longer than 72 characters ... current length is 88 [authored-header-max-length]. This is the existing immutable de2d613 subject, feat(graph): observe genuinely committed native stage pages and verified compact segment (88 characters). A follow-up additive commit does not repair an earlier commit checked individually.

Unblocking this mandatory gate normally requires replacing that historical commit with a compliant-header commit (which would rewrite history), or publishing its content on a clean replacement lineage (which would duplicate this active PR). Both violate the explicit preserve all commits / no rewrite / no duplicate constraints. I have not force-pushed, modified policy, bypassed checks, discarded any commit or created another PR. This is a hard acceptance blocker that requires a policy decision outside the current authorization; later additional code changes cannot make this already-published header pass. The benchmark harness self-tests succeeded, but no native Bazel test or full #4508 end-to-end acceptance is established. Issue #4508 and PR #4800 remain open; protected draft #4776 and release #4763 remain untouched.

Refs #4508

Carry the real capture and parse attempt across the mounted scheduler,
canonical status wire, CLI wait display and MCP search. Preserve attempt
owner, epoch and work timestamp independently of sealed generation data.
Add nonzero wire and renderer regressions. No generated-contract edits.

Copy link
Copy Markdown
Owner Author

Published 2dc0150edc57 onto the existing #4800 branch using expected_sha=b329b671… and force=false, preserving all ten prior commits. Refs #4508 (partial). This connects the already-produced, separately typed pre-seal capture/parse attempt to the actual mounted registry's lock-independent freshness read. The canonical Rust wire now carries the attempt's daemon/producer incarnation, sequence, phase, completed/total source decisions, progress epoch, original work timestamp, and active state. CLI init --wait and MCP graph/search summaries show those observed fields without inventing a sealed generation, durable text commit, graph page, or polling heartbeat. Three focused nonzero wire/renderer tests were added, but none has been Bazel-executed here.

I also checked master, current root AGENTS.md (repository tree contains no nested AGENTS.md), paginated issue comments, PR discussion, reviews (none), the exact earlier CI jobs, and competing #4508 PRs. The preexisting immutable 88-character de2d613 header still fails mandatory commitlint: the preservation/no-rewrite/no-duplicate instructions preclude correcting it in this lineage. Generated dashboard contract outputs must be regenerated with the mandated Bazel codegen, not hand edited; the new field is on the Rust/daemon/MCP/CLI wire but the React dashboard panel is not yet updated. Native graph writer events still need canonical status observations, and cold/restart/stall/blocked/recovered integration acceptance remains unfinished. Exact-head CI was queued when inspected.

Independent review requested on the public wire change, whether pre-seal attempt expiration correctly fences old owners, correct graph-vs-text attribution, and all fully mounted paths. #4616/#4804 reader-retention ownership is untouched; #4776/Kev remains draft and #4763 release remains read-only. Do not merge or close on this partial evidence.

devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Owner Author

Continuing #4508 on exact head 2dc0150. Taking the CLI first-generation wait implementation and its behavioral tests: the live wait currently resets its stall timer merely because rebuild_in_flight/Restoring/Verifying stays true, even when all producer counters are frozen. I am replacing that activity-as-progress inference with observed work tracking, retaining typed refusals and leaving daemon work running when the client stops waiting. Maintenance/sweep owners: preserve these edits and all existing commits; no duplicate/rewrite/force update. Native graph callback/status integration and Bazel-generated dashboard contracts remain separate unfinished acceptance, not made complete by this slice. #4804 owns sealed receipt/retention hunks, #4796 owns graph source reader work; neither is changed by this CLI slice. Refs #4508 only; #4776 remains held draft and #4763 read-only.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published additive production/test fix e514005ffe2a on the sole active #4800 branch.

Verified defect / source-owned correction: The latest inline review correctly identified that an inactive preseal snapshot survived after its reconcile completed, so a later source wake could set rebuild_in_flight=true before a new attempt began and falsely present the prior generation's resolution counters as current work. The real Rust CodeIndexPresealAttemptGuardV1 now leaves the last producer-observed counters visible only while that guard still owns the durable-publication handoff. Its Drop retires the exact identity-matched slot on successful exit, cancellation, panic, or retry. The identity fence prevents a late old guard from clearing a newer producer. finish still closes the observer at handoff without forging a new work timestamp. No generated contract, synthetic generation, heartbeat, page, phase, or counter was added.

Falsifiable source regressions: (1) real capture→parse→resolution observations remain visible after finisher() but vanish after guard exit, before a new source wake/admitted attempt; (2) a superseded guard's late drop/callback cannot erase the next active attempt. Updated the existing handoff test to assert retirement at exit. Existing Dashboard handoff DOM tests remain intact; no assertions or budgets weakened.

Independently read PR+branch at 36174dc immediately before the write, Git blob/tree/one-parent commit, compare ahead 1 / behind 0 with just attempt_progress.rs (+78/-3), update_ref(expected_sha=36174dc, force=false) succeeded, and independent post-write PR/ref/parent readbacks all match e514005. Historical published 609aff4 lineage remains externally divergent (ahead 98, behind 23), so HOLD MERGE pending authorized reconciliation.

Please independently review the real guard lifetime and late-drop identity test, plus exact-head hosted Bazel Rust/JS jobs. Tests were not executed locally; new-head CI and Plugin Validation are queued, not acceptance. Physical cold/SIGKILL/native WAL page interruption+restart daemon→CLI/MCP/dashboard, stalled/blocked/recovered E2E, exact-head nonzero Bazel/DOM, native macOS where available, independent approval, and master acceptance remain required. Keep issue #4508 and PR #4800 OPEN/UNMERGED; Refs #4508 only. No reviewer assignments or protected PR/release changes.

Copy link
Copy Markdown
Owner Author

Refs #4508 — exact-head hosted CI failure repaired on sole open PR #4800. On prior e514005ffe2a, Repository gates job ran 8 nonzero policy tests successfully but Bazel //:rustfmt_check failed in attempt_progress.rs:228 on one extra blank line before completed_handoff_does_not_reappear_during_a_later_source_wake. The failure is a verified source-format violation, not a proven Rust production-test failure.

Published 15743f70763f, removing only that blank line. No assertions, callbacks, durable progress claims, generated code, test targets, budgets, or CI policy were changed. GitHub blob/tree/one-parent commit; old...new compare ahead 1 / behind 0, one Rust source/test file (0 additions, 1 deletion); branch update with exact prior expected_sha=e514005, force=false succeeded. Independent PR+Git-ref+parent verification agrees on the new head.

New exact-head CI and Plugin Validation are queued: no new-head nonzero Bazel Rust, DOM, Clippy or generated-contract acceptance is claimed. The new head still diverges from original personally published 609aff4 ancestry (ahead 99, behind 23); additive repair does not reconcile history or authorize merge. Please independently review the native lifecycle correction and its two tests, and confirm current-head Bazel verification. Continue HOLD MERGE; leave #4508 and #4800 open with Refs #4508 pending physical cold/SIGKILL/restart+staging-page recovery, stall/blocked→recovered journeys, independent approval and master acceptance.

Copy link
Copy Markdown
Owner Author

Refs #4508 — new exact-head Bazel dashboard failure repaired on sole open PR #4800.

On prior 15743f70763f, Dashboard job ran 2,289 nonzero Vitest tests: 2,272 passed, 17 failed (one dashboard target failed; eight other frontend Bazel test targets passed, including frontend typechecks). The first failed case, does not let a fresh sealed predecessor clock disguise a frozen capture successor, timed out after 5 seconds because it used Testing Library's asynchronous screen.findByText while Vitest fake timers were enabled. Subsequent failures in the same test file showed empty DOM / mismatched fetch counts; those may be cascade contamination, not independently proven production bugs. Repository gates, strict Clippy, benchmark/change-scope and feature gates passed; Linux Rust fleet was still running at inspection. Dashboard failure artifact.

Published fc114119e824: the regression now uses Vitest's fake-timer-aware vi.waitFor to await the actual rendered capture · 8 / 20 items, preserving the authentic 7→8 transition and all existing 4-second quiet / 1-second recovered polling and HTTP-count assertions. No timeout extension, assertion removal, production change, generated-code editing, or CI bypass. Exactly one owned React DOM test file (+5/−1); GitHub one-parent commit, compare ahead 1/behind 0, exact old-head leased update_ref(force=false), independently verified PR+Git branch+parent at fc114119.

New exact-head CI and Plugin Validation are pending; no new-head nonzero Bazel Rust or DOM pass is yet claimed. Please independently review the fake-timer/wait behavior and the lifecycle/physical acceptance; no person assigned or messaged.

HOLD MERGE: original personally published 609aff4 ancestry remains externally divergent (a normal append does not restore it). Keep #4508 and #4800 open and unmerged with Refs #4508 until physical cold/SIGKILL, durable-stage-page restart, stalls, blocked/recovered, exact-head nonzero CI, independent approval, authorized history disposition and master acceptance.

Copy link
Copy Markdown
Owner Author

Refs #4508 — new additive producer-aware CLI fix published as a151169fcc51 on the sole active #4800 branch.

Concrete production defect: init --wait selected the retained text bulk_commit marker ahead of the independently active successor preseal capture/parse marker. During a quiet retained text writer, genuine successor parse callbacks were ignored and the 120-second silence window could falsely refuse a progressing first generation. In addition, its global owner ordering could suppress authentic older-producer text commits when a newer preseal writer coexisted on the same daemon.

Source fix: keep the committed-text marker for its own durable receipts but also inspect the separately identity-fenced live preseal marker in the same status read. The progress window advances only when either producer's actual completed units advance; duplicate epochs, phase-only updates, frozen counters and old-daemon replay cannot renew it. The global cross-writer fence now rejects older daemon incarnations, while each producer kind still enforces its own owner/epoch/counter fence. No synthetic generation, timestamp, digest, heartbeat, phase projection or durable page.

Falsifiable native regressions: (1) text counters frozen → actual capture→parse work → text receipts resume, including concurrent different producer incarnations; wait must reach readiness, not falsely stall; (2) duplicate reports from both concurrent producers still cause an actual timeout before a later Ready response. This changes only crates/tracedecay-cli/src/commands/index.rs (+118/-19), no generated output or weakened assertions. GitHub one-parent commit from fc114119, compare ahead 1/behind 0, leased update_ref(expected_sha=fc114119, force=false), independent post-commit PR/branch/parent readbacks verified.

On superseded fc114119, Dashboard Bazel job completed SUCCESS and repository gates/Clippy completed SUCCESS. Linux Rust was still active when superseded and is not current-head acceptance; the new exact-head CI and Plugin Validation are queued. Tests were not run locally; no current-head Bazel Rust/JS green is claimed. Please independently review the concurrent-authority wait logic and verify nonzero current-head CI, physical daemon cold/SIGKILL/page-restart and blocked/recovered journeys. Historical original 609aff4 ancestry remains externally divergent; HOLD MERGE and keep #4508/#4800 open with Refs #4508. No people assigned or messaged.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published additive production and regression fix 8635be2 on the existing #4800 branch, verified expected-head a151169 non-force fast-forward and independent post-write PR/ref readbacks. One changed source/test file, +52/-1, no generated artifacts or changed timeout budgets.

Reviewed failure mode: init --wait already retained independent text and pre-seal watermarks when text BulkCommit was selected, but when an earlier generation's native graph writer was active it selected only its native marker. A quiet/frozen old native writer could therefore mask genuinely advancing successor capture/parse and trigger a false no-work timeout. The existing native-fallback logic had no concurrent preseal observation.

Production change: while a native graph marker is the primary work authority, also record the separately identity-fenced active pre-seal marker. Each marker is independently validated for real counter advancement. Neither stale epochs nor status polling supply work.

Two falsifiable Rust cases: a real serialized status wait loop with a silent native writer and advancing capture→parse must continue to a Ready receipt (the old implementation refuses); repeated native and pre-seal observations with no new work must still refuse before a later Ready receipt. All existing tests/assertions retained. Request independent review of graph/pre-seal concurrency and restart/identity fencing.

No local Bazel execution is possible here; new-head native Rust/Clippy CI is not claimed passing. The preceding head's ordinary Linux jobs remained queued; no historical lineage reconciliation, approval, merge, or issue closure is claimed. Original 609aff4 ancestry remains externally diverged, still a merge hold.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published additive real source/test repair 2ad500048c4b on the sole existing #4800 branch. Independent PR + Git branch matched exact parent 8635be2fd777 before the update, GitHub blob/tree/ONE-parent commit, compare ahead 1/behind 0 with exactly two owned Rust files (+59/-36), update_ref(expected_sha=8635be2, force=false) succeeded. Independent post-write PR/ref/commit-parent all match 2ad5000 OPEN, UNMERGED; no force/rewrite, retarget, duplicate branch/PR or peer history drop. Externally rewritten 609aff4 historic ancestry is NOT reconciled: HOLD MERGE.

Verified exact-head hosted failures on predecessor 8635be2: Repository gates failed Bazel //:rustfmt_check on a two-line import layout in crates/tracedecay-cli/src/commands/index.rs (all eight nonzero repository policy tests passed). Clippy failed with method observe is never used: the new concurrency path always called the multi-marker method, so the single-marker method was used only in tests. The production waiter now uses observe for actual single-writer observations and observe_with_concurrent_preseal only when an independent live preseal writer exists. This restores the production caller without disabling warnings, changing stall budgets or inventing work. Import layout fixed exactly per hosted rustfmt output.

Real Linux Rust regression: Linux fleet, archived Bazel artifact bazel-test-logs-Linux-fleet ID 13722123396, ran 702 passed / 2 failed in the code-index runtime unit test. The two regressions in tests/code_index_scheduler/reconcile.rs still asserted that a preseal observation remained available after the corresponding real reconcile guard was dropped, contradicting the prior production lifecycle correction that retires that live slot at reconcile exit to prevent old-generation counts reappearing in a successor. Instead of leaving the snapshot alive or weakening the lifecycle fence, the cold physical test now reads authentic captured-file counters, epoch, timestamp and active phase during reconcile_now_with_capture using the existing production capture path, then explicitly asserts retirement after publication. The real retained-restart test still checks measured captured-file progress during ownership, verifies retirement on drop, and verifies a second real capture is a separate, later-sequence, genuinely observed attempt and also retires. No synthetic producer callbacks, generated contract edits or time-budget increases.

Other Linux failures in the same exact-head artifact: hooks_lsp_suite 28 passed/1 timeout (spool-drain receipt deadline) and runtime_acceptance_suite 140 passed/1 failure (daemon LSP acknowledge DeadlineExceeded). Those files are outside this lane's ownership; do not hide or classify them as #4508 fixes without independent diagnosis/owner coordination. Dashboard SUCCESS. Native macOS SKIPPED, not acceptance.

Request independent review of the production single-/dual-marker dispatch and real in-guard physical observation/restart assertions, plus exact-head hosted Bazel Rust and JS. New CI and Plugin Validation are queued/pending: no current-head passing claims. Still need actual SIGKILL before seal, native durable-page interruption+resume, no-work/blocked→recovered end-to-end via daemon→CLI/MCP→generated dashboard/DOM, independent APPROVED review, authorized history disposition and MASTER acceptance. Keep #4508 and #4800 OPEN UNMERGED, Refs #4508 only.

Copy link
Copy Markdown
Owner Author

Refs #4508 — new additive production/test correction published as 29ac2e51e1d0 on sole open PR #4800.

Verified source issue: init --wait correctly tracks concurrent preseal and retained text work independently, but FirstGenerationStall::of still treated an old generation's blocked BulkCommit as a live refusal even while a newly admitted preseal capture/parse producer progressed. The unconditional old-text blockage could stop a healthy successor after the stalled idle window. A currently active preseal attempt has no sealed generation identity yet; a sealed BulkCommit reading with generation_id == latest_generation_id belongs to the previously sealed producer, not the active preseal attempt.

Correction: where the old sealed generation is retained and a real preseal guard is active, treat its blocked post-source-scan text reading (including BulkCommit) as an old writer's refusal, not as the successor's refusal. Preserve genuine SourceScan blocks, same-generation native graph blocked diagnostics, no-writer detection, the per-producer work-clock counters and unchanged 120-second window.

Falsifiable native CLI regressions: a real status wait sequence with retained blocked text and actual successor capture→parse advances must reach Ready; duplicate blocked text + frozen capture must still produce a no-work refusal before any later Ready. Both would fail appropriately without the fix; no timeouts, existing assertions, generated contracts or CI policy weakened.

GitHub independently verified pre-update PR+Git ref 76f083f6, GitHub one-parent blob/tree commit, compare ahead1/behind0 changing exactly crates/tracedecay-cli/src/commands/index.rs (+57/-4), update_ref(expected_sha=76f083f6, force=false) success and independent postwrite PR/ref/commit parent all at 29ac2e51. Review requested through this PR comment only; no reviewer assigned. Exact-head Bazel CI is queued; no Rust/DOM success claimed. Original 609aff4 ancestry remains externally divergent, and owner-authorized history disposition, physical interruption/recovery and independent approval remain missing. HOLD MERGE; keep Refs #4508, issue and PR open.

Exercise real task abort and panic unwinding of the preseal guard,
including inactive publication handoff, late callbacks and same-producer
retry isolation. Keep imports explicit and retain all existing tests.

Refs #4508
Do not let a prior completed native row stream or verified segment hide
a separate live preseal attempt. Preserve unknown final-container timing
without treating elapsed status reads as production work.

Refs #4508

Copy link
Copy Markdown
Owner Author

Refs #4508 — published production correctness fix b19884d9a140 to the existing #4800 branch using GitHub create_blob/tree/one-parent commit and update_ref(expected_sha=4af29660a8e5, force=false). Independent PR/branch/raw-commit readbacks all agree; compare is ahead 1 / behind 0, only crates/tracedecay-cli/src/commands/index.rs (+99/−1). All current ancestors and peer work remain preserved. No rewrite, merge, retarget, extra branch/PR, generated edit or bypass. The previously rewritten historical 609aff4 ancestry is not restored; HOLD MERGE.

Actual production defect: FirstGenerationWorkMarker::of correctly treats a completed compact row stream or CompactSegmentVerified as an unknown final single-container write, but returned None before checking an independently active successor preseal attempt. During an older native graph's final Grafeo write, genuine successor capture/parse work was thus discarded and a frozen successor could never reach the no-work refusal. The current fix continues to leave the final native write un-timed when it is the only writer, while selecting the separately owned active preseal marker when present. Only that producer's actual callbacks can renew the wait.

Three falsifiable Rust regressions: (1) complete entity+relation windows and verified graph segment do not conceal the active successor's exact capture owner/count/epoch, but retiring the attempt restores the unknown final native write; (2) serialized init --wait accepts advancing capture→parse behind old final native encoding and reaches an actual Ready receipt; (3) duplicate frozen preseal observations behind the same graph boundary refuse before the scripted later Ready. Existing timeout budget, tests and assertions remain unchanged.

Verification still required: These new tests have not been executed locally; exact-head hosted CI and Plugin Validation are queued. Please independently review the graph-final/preseal authority distinction and require nonzero Bazel Rust/JS/rustfmt/Clippy execution, physical cold/SIGKILL and native durable-stage-page restart/recovery, blocked↔recovered/stalled daemon→CLI/MCP→generated-dashboard DOM journeys, and approval before acceptance. All 11 published inline review threads are currently resolved; no independent approving review. Keep #4508 and #4800 OPEN, UNMERGED, Refs only. No reviewer assigned.

Record genuine retained lexical page receipts alongside native graph
encoding and successor preseal work. Frozen clocks and epoch-only updates
remain stall-observable; leave final compact verification uninstrumented.

Refs #4508

Copy link
Copy Markdown
Owner Author

Refs #4508 — published the production wait-progress correction 10dc2fc7ce91 to this sole existing #4800 branch with a one-parent expected-head force=false fast-forward. Independent PR and Git ref readbacks match the published commit; compare from b19884d9 is ahead 1 / behind 0, only crates/tracedecay-cli/src/commands/index.rs (+144/-9). No force, rewrite, retarget, duplicate branch/PR, generated edit or merge. The historical 609aff4 ancestry is still not restored; HOLD MERGE.

Concrete producer-observation bug: init --wait retained independent preseal and native graph watermarks, but when an active native writer had already produced at least one row boundary, it selected only the native marker (and optional preseal). A separately owned retained lexical BulkCommit could commit genuinely new durable pages while the native writer was quiet; those receipts were dropped, causing a false 120-second no-work refusal. Likewise, a final Grafeo container write could overlap a quiet successor preseal guard and real text receipts; the preseal marker hid the receipts. The waiter now records authentic lexical BulkCommit progress alongside an independent native/preseal marker, each under its own existing owner/epoch/counter fence. It still considers a final monolithic Grafeo write alone uninstrumented, does not project phases, and never credits timestamp/epoch-only changes as completed work.

Three falsifiable native CLI regressions: (1) frozen partial native encoding + advancing independent text receipts reaches Ready; (2) completed native row windows + quiet active successor preseal + advancing text receipts reaches Ready without timing the final write; (3) all three writers publishing only new epochs/timestamps but no counts refuse before a scripted later Ready. Existing budget, tests, real durable attribution, and assertions are unchanged.

Verification: exact new-head Bazel workflow and Plugin Validation are queued; no new-head nonzero Rust/JS/rustfmt/Clippy or macOS pass is claimed, and no local Bazel/pnpm checkout was available. No artifacts yet. Please independently review the simultaneous text/native/preseal authority selection, all three regressions, and current-head CI. Physical cold/SIGKILL/native WAL-page interruption and restart/resume through daemon → CLI/MCP → Bazel-generated dashboard/React DOM, blocked/recovered/no-work E2E, human approval, historical disposition, and MASTER acceptance remain outstanding. Keep #4508 and #4800 OPEN, UNMERGED, Refs #4508 only. No people assigned or contacted.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published 6edb4af07692 to the sole existing PR #4800 on fix/4508-progress-boundaries-ci-clean.

Concrete CLI correctness: init --wait treated an authoritative terminal convergence park as “no reconcile pass” when its typed status contained the actual failure cause and operator remediation. Worse, a stale rebuild_in_flight=true bit could suppress that diagnosis entirely. On an immediate wait=unavailable response, the CLI returned only the short availability code, dropping the daemon's separate Parked cause/remedy. The source fix consumes the existing canonical typed StatusCodeIndexFreshnessV1::Parked reading, preserves its cause, remedy and retry-on-wake flag in the typed unavailable refusal, and classifies terminal parks truthfully during timed-out slices. A separately active successor preseal/native writer still overrides the retained park; recoverable parks are not prematurely classified terminal. No new schema, generated output, heartbeat, progress epoch, durable claim or timeout budget.

Three falsifiable regressions: (1) serialized wait=unavailable with a real-shaped typed parked status must return the exact cause/remedy/retry class before a later scripted Ready; (2) terminal park plus a stale rebuild flag must time out with the parked cause instead of silently continuing or reporting NoPass; (3) real successor-admitted work must clear an old parked classification and reset the idle window, which can expire again only after renewed continuous silence. Existing blocked/recovered, native/preseal and wait assertions are retained.

Verified independent PR+Git branch 10dc2fc7 immediately before each Git object/write and the CAS; one-parent GitHub blob/tree/commit, compare ahead 1 / behind 0, exactly one owned CLI source/test file (+149/-4), update_ref(expected_sha=10dc2fc7,force=false) SUCCESS. Independent post-write PR+branch+commit parent agree at 6edb4af07692ae0ae861f17f56b7fc04c83ab2d6; PR remains open/unmerged. This preserves the current history only; the original published 609aff4 lineage is still externally rewritten: HOLD MERGE.

Please independently review the terminal parked-vs-live-successor precedence, actual typed status deserialization/refusal, and the new tests. No person assigned or messaged. The change has not been Bazel-compiled/tested locally (no checkout/Bazel). Exact-head CI is queued, not a pass. Full live 6,136-file cold/SIGKILL, durable WAL stage interruption+recovery, stalled and blocked→recovered daemon→CLI/MCP→generated dashboard DOM, nonzero exact-head Rust/JS/Clippy/codegen, independent approval, owner-authorized history disposition and master acceptance remain required. Keep #4508 and #4800 OPEN UNMERGED, Refs #4508 only.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published a new production correctness and regression fix as 78d33055e8b0 on the sole existing #4800 branch.

Real defect: during final native graph publication, FirstGenerationWorkMarker::of deliberately suppresses a completed compact row-stream/verified-segment marker because Grafeo's final container write is monolithic and uninstrumented. However, the early return also discarded a separately owned lexical BulkCommit when no successor preseal attempt was active. Real durable text page receipts therefore could not renew init --wait; phase-only polls also could not detect that the lexical writer was genuinely frozen. The earlier fix covered this overlap only when an active preseal marker existed.

Source fix: at completed native encoding and CompactSegmentVerified, select an authentic active preseal observer if present, otherwise the independently owned lexical BulkCommit receipt marker. If neither writer has an eligible observed marker, the final Grafeo write remains unknown, not a synthetic stall. The existing owner/epoch/counter fences still reject metadata-only clock changes. No fake graph page, timestamp, digest, generation, heartbeat or durable claim.

Falsifiable regressions: (1) across both fully encoded native rows and digest-verified native segment phases, real serialized status-loop responses with actual advancing lexical committed pages/bytes must reach Ready, and removing the text writer restores the deliberately unknown final write; (2) when only native/text epochs and text timestamps change without completed counters, the loop must refuse before a later scripted Ready. Both fail against the prior early-return branch. Existing assertions, timeouts, generated contracts, CI and other ownership lanes are unchanged.

Before EACH Git object mutation and the branch CAS, independent PR and raw branch refs matched exact old head 6edb4af07692. Created Git blob/tree/ONE-parent commit; compared exactly ahead 1/behind 0 with only crates/tracedecay-cli/src/commands/index.rs (+91/-4), and update_ref(expected_sha=6edb4af,force=false) succeeded. Fresh independent PR, branch and raw commit confirmed new head and its sole parent. Historical rewritten 609aff4 lineage is still not restored; HOLD MERGE.

Please independently review this physical-observation priority and both new CLI test cases. Exact-head hosted CI is queued at publication, with no current-head nonzero Bazel Rust/JS validation or native macOS proof. Full live cold/SIGKILL, durable fallback stage-page resume, stalled and blocked/recovered daemon→CLI/MCP→generated-dashboard DOM acceptance, independent APPROVED review, history disposition and master acceptance remain missing. Keep PR #4800 and issue #4508 OPEN/UNMERGED, Refs only; no reviewer assignments.

devin-ai-integration[bot]

This comment was marked as resolved.

Keep independent lexical watermarks without treating a retained BulkCommit
snapshot as silence evidence for the uninstrumented final graph write.
Preserve active preseal stall detection and require frozen text to refuse
once the native writer retires. Repair the reported Rust formatting.

Refs #4508
Apply exact formatting required by the current-head repository gate
without changing the CLI wait semantics or weakening the regressions.

Refs #4508

Copy link
Copy Markdown
Owner Author

Refs #4508 — published 7218ed2638b0 on the existing branch to repair the exact-head Bazel //crates/tracedecay-cli:rustfmt_check_2024 failure from Repository gates.

I read the failing job log and applied its four exact formatting hunks in crates/tracedecay-cli/src/commands/index.rs (imports, parked diagnosis write!, frozen-text assertion, and parked-cause assertion). No production behavior, tests, budgets, generated files, or CI rules were changed. GitHub verified an exact one-parent non-force fast-forward from e880e8e8: ahead 1 / behind 0, only that file (+14/-5). Independent PR, Git-ref and raw-commit readbacks match the new SHA.

Please independently review the final-native/lexical silence handling and the new regression coverage alongside this formatter correction. New-head hosted CI has not yet supplied nonzero Bazel Rust/JS/Clippy/rustfmt evidence. The physical cold/SIGKILL/restart/durable-page journey and historical lineage disposition remain outstanding. This is not a merge or a claim of full #4508 acceptance; keep the PR and issue open.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant