Repository navigation
fix(graph): verify sealed retirement and preserve retry state - #4819
ScriptedAlchemy wants to merge 72 commits into
Conversation
Refs #4616 Combine the proven bounded receipt/ownership cleanup fixes with post-unlink collection-state ordering and compatible filesystem regressions. Replace the old separately reviewed PR history without rewriting either branch, and preserve all exclusion and fail-closed checks.
|
Refs #4616 Reconcile all eleven formatting deltas from the exact-head Bazel repository-gates failure without modifying retirement behavior or tests.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Reviewed commit 65bd98f against its parent and the real sealed-build call path. The pre-change missing/malformed receipt branch could delete foreign real-directory contents; the revised path refuses those entries and tests negative preservation plus subsequent legitimate recovery. The pre-existing sealed-root symlink test similarly proves no foreign staged writes and positive recovery. Scope is one graph-db source file, no CI weakening. This is partial #4616 safety work only: parent identity and post-check pathname races remain unresolved; Bazel/native verification is pending, so this is not an approval or merge recommendation.
Repair three hosted automation_runner_test failures by supplying the configured executable identity required by the production prefilter. Use the same identity for the prefilter and retained runner, preserving manual-schedule, exact-ledger, live-lock and lock-race assertions. Add a production-prefilter regression for unconfigured refusal, deduplication and same-occurrence recovery after binding, with zero backend calls and no effectful terminal. Do not relax backend admission, change timeouts, or alter the graph publication changes. Refs #4532
Refs #4616 Use the strict generation/digest-bound retirement authority on failed recovery adoption and post-install reopen failures. Preserve unverified artifacts instead of path-deleting them. Cover missing, corrupt, wrong- digest and owned receipts plus unrelated sibling preservation.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Reviewed the exact 882d642 diff against its predecessor and the production sealed build/install path. Previously, std::fs::rename(staging, generation) could silently replace an empty existing real directory on Linux/macOS. The new production call uses the existing capability-relative rename_noreplace authority via a no-follow sealed-root/staging helper. The native regression constructs exactly that empty occupied destination, proves refusal and staging preservation, then proves successful installation after the destination is removed. A separate filesystem regression proves that a symlinked sealed root or staging entry cannot redirect publication into a foreign tree. Two source files changed; no test or CI gate was weakened.
Remaining safety boundary: ambient store ancestors are not yet pinned to the original store-owner identity, and staging writes remain pathname-based. A real directory substituted between staging creation and subsequent writes or between the staging handle check and rename requires a separate owner-bound I/O fix. This is a partial Refs #4616 fix only, not an approval to merge/close. Hosted exact-head CI is pending; no Bazel/native macOS pass claimed.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — reviewed exact commit e2f3371c against its parent and the real recovery path (open_sealed_generation_store_if_present -> discard_failed_sealed_adoption -> ownership-checked retirement). The previous restart_recovery_adopts_or_discards_the_on_disk_artifact assertion demanded removal of a tampered/unverified receipt, contradicting the new fail-closed ownership contract and causing a native regression failure once this PR's safety code executes. The repaired test checks all three outcomes of the real recovery journey: no sealed adoption, byte-for-byte preservation of the unverified on-disk receipt/directory, and successful reads from staging rows. It still proves positive sealed adoption before tampering. One test file, no behavior or CI weakening; exact-head nonforce FF verified. This is partial, not acceptance of all #4616 or approval to merge. Bazel/native macOS results remain outstanding; CI run 42540628881 is queued.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — reviewed exact commit b568770d against its parent and build_or_open_sealed_store's production stage/create/install path. The previous unconditional remove_child_directory_nofollow was capability-relative but authenticated no ownership: a pre-existing real .staging-<generation> directory was recursively erased before its contents were examined. The new exclusive create_dir refuses occupied staging (including symlinks), with a native regression that checks byte-exact foreign-sentinel preservation, no artifact installation, and successful verified sealing after fixture cleanup. Only sealed_store.rs changed; no gate, timeout, release, or other PR was modified.
This is a narrowly reviewed safety repair, not approval for full #4616 acceptance. Outstanding separately: the store-open abandoned-staging sweeper still deletes by filename pattern without an ownership receipt, and store-ancestor/staging-path substitution remains unsafe without a stable owner capability. This head has no Bazel/native macOS pass. Confirm exact-head CI and nonvacuous Bazel graph-db tests before any merge.
Refs #4616 Do not treat a staging-shaped name as deletion authority. Retire only staged children with bounded matching sealed-generation receipts through held no-follow capabilities, preserving unknown or substituted contents. Exercise real store reopen for owned cleanup and unverified preservation.
Refs #4616 A copied receipt with rewritten physical namespace must not authorize recursive cleanup of an unrelated staging directory. Exercise the forgery and malformed-receipt refusals through real store reopen.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independently reviewed the exact two-commit staging-sweep repair 5de9d92c..0bf8a868 against the prior b568770d and the actual store-open/build/receipt paths.
Previously sweep_abandoned_sealed_staging handed every filename-shaped .staging-<hex> child to a no-follow receipt-free recursive remover. A foreign real directory with an arbitrary payload was therefore deleted on store open. The change reuses the existing held-directory, no-follow, bounded-receipt collector; it rejects unreceipted/malformed/oversized/linked entries and binds the receipt's logical namespace + projection + generation to its canonical physical namespace before authorizing deletion. An interrupted seal with a matching real receipt still reclaims. No changes to store admission, CI, model work or other branches.
The real graph-store reopen regression exercises positive reclamation plus preservation of exact foreign payloads, missing receipt, wrong physical name, forged/copied receipt, malformed/oversized receipt, symlinked directory and installed serving continuity. This is behavioral evidence in the source tree, not an executed test result. CI is queued at https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42552653839; Bazel/native results remain unproven. Remaining #4616 root risks include trusted ancestor identity before capability acquisition, path substitution during staging writes, unrecoverable pre-receipt interrupted staging, N>3 end-to-end retirement and master acceptance. This review is a scope/safety review, not approval to merge.
Keep accepted master tests and isolate sealed-generation retirement changes. Refs #4616
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independently reviewed the published bd4f39eb regression correction and the accepted master synchronization 95e8a8b6.
Root cause: the previous direct_build_recovers_from_interruption_and_matches_the_staged_build test created an unreceipted occupied .staging-<generation> containing a torn container and required the builder to erase it and rebuild. The production builder now deliberately refuses that name collision (ownership cannot be established), so that pre-existing test was both failing and demanding unsafe behavior.
The revised native test asserts the precise GraphDbError::Unavailable refusal, byte-exact preservation of the torn file, absence of an installed generation, then removes the test fixture and proves direct/staged/parallel payload equivalence and successful install. No production guard is relaxed.
The merge commit has two exact parents: existing PR head bd4f39eb and accepted master 83694e00. The resulting tree is byte-identical to its PR parent; master comparison now contains only six graph-db/private-fs files, with the previously overlapping automation-runner test file absent. Both updates were nonforce expected-old-head fast-forwards. This review is source-level evidence, not a Bazel/native-test pass. Exact-head CI run 42554042152 is queued; do not merge until CI and independent acceptance are proven.
Refs #4616 Preserve foreign row-spill directories on store-open sweeping and drop. Persist bounded owner receipts before writing rows; reclaim only from receipt-verified held directories, with native crash/replacement tests.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independent source/diff review of exact one-parent commit f1f880fb899921d20a251a662ff51b2553a02379, based on 9f04bede. This is a partial ownership-safety repair, not approval to close #4616 or merge absent CI.
Root cause: SpillDirectory::prepare returned a TempDir whose pathname-based destructor remained armed during exclusive receipt open/write/sync. A receipt creation/persistence error could drop the object after a concurrent replacement of its staging pathname with a foreign real directory, recursively destroying foreign contents. The earlier install-path fix only disarmed cleanup before rename, so did not cover receipt failures.
Fix: persist_pending_receipt now disables TempDir cleanup before its first fallible receipt operation. New private-fs write_exclusive_pending_receipt_nofollow opens a held no-follow directory and writes the exclusive producer receipt relative to that handle. On write/sync failure it preserves the original error, unlinks the marker only when its Unix dev/inode still match the exclusively created file, then attempts only a nonrecursive removal of that held directory. A foreign sibling prevents removal; a failed exclusive marker open preserves the directory.
Native behavioral regression: failed_pending_receipt_open_preserves_substituted_directory physically moves the producer's pending directory, substitutes a real directory containing a sentinel plus a foreign marker, exercises the real receipt-persistence path, and requires both foreign bytes and the original moved directory survive. The existing isolated file-size-limit/EFBIG test continues to require zero abandoned directories after an owned marker write failure, and normal publication/drop coverage remains.
Qualification: exact-old-head expected_sha=9f04bede, force=false; GitHub compare independently shows one commit ahead, none behind and only the two issue-owned Rust paths (+120/-12). Hosted CI on this exact head is queued; there is no local checked-out Bazel/native runner and no Rust test pass is claimed. No Cargo was run. The remaining gap is atomic ownership binding across initial TempDir creation and directory capability acquisition, including hostile source swaps at publication, plus full scheduled N>3 retirement under sustained writers and on-master Doctor/report acceptance. Protected draft #4776, model branches, release, peer owner branches and CI remain unchanged.
Refs #4616 The sealed census charged a shared inode to the first directory visited. A superseded generation could therefore claim live-head physical bytes. Transfer hard-link charges to the stronger verified category regardless of read_dir order, retaining bounded metadata-only accounting. Prove the adverse visitation order using real native hard links and the production census.
Refs #4616 Keep the native test's method chain inside the repository rustfmt width. No production behavior or test assertion changes.
Refs #4616 State that only multi-linked inode identities consume deduplication metadata, and explain the priority ordering used by the census. No behavior or test assertion changes.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independent source/diff review of commits 72d885b0, 8b1b0620 and 94b5d9fc on exact PR head 94b5d9fcc68f0242617e5e3a92acc92e1f840405.
Root cause: census_receipted_directory_bytes used a global HashSet<(dev, ino)>, charging the first directory to enumerate a multiply linked inode. read_dir has no semantic ordering: a superseded sealed generation could claim the same physical cold-base bytes belonging to a later verified head. That makes Doctor's live-vs-reclaimable sealed footprint inaccurate even when the overall total is right.
Fix: Retain the original capability-held, no-follow, depth-first and bounded-receipt traversal, but record the charged category + byte count for multiply linked inodes. When a stronger class (the caller's Head < Superseded < AbandonedStaging < Unrecognized) appears later, debit the previous category and credit the stronger one exactly once. A failed prior accounting lookup/underflow surfaces InvalidData, not a silent success. No raw file contents or second tree pass is needed; only multi-linked inode identities use deduplication memory.
Native filesystem regressions: create real hard-linked payloads with distinct on-disk receipts, deliberately call the production walker in the adverse Superseded-then-Head order (independent of OS readdir order), and require shared bytes charged only to the live head. Also invoke the public census API and verify both categories, no unknown entries and persisted payload bytes. The previous HashSet implementation fails the deliberate-order assertion. The two follow-ups normalize rustfmt-like expression wrapping and clarify the memory invariant only.
Validation/holds: GitHub confirms three nonforce exact-old-head fast-forwards, +93/-6 across exactly one issue-owned capability_dir.rs file and no peer branches. Hosted CI is pending at exact head; Bazel/native macOS execution is unavailable locally, so no Rust or Bazel pass is claimed. This is partial, not a merge or closure approval: automatic N>3 retirement under a real retention pass, writer fairness (#4813), stable ancestor/staging ownership, Doctor guidance (#4799), and integrated incident-scale reporting still require acceptance proven on master. Protected draft #4776 and Kev/model PRs remain held; no CI policies were changed.
Refs #4616 Bound shared-inode metadata and reject a census as partial before its allocation scales indefinitely with millions of hard-linked files. Use fallible reservation and a filesystem regression for the refusal and the normal complete census. No CI or release settings changed.
Refs #4616 The bounded Unix inode census leaves its budget unused on other targets. Retain the existing non-Unix no-op binding to keep strict lint clean.
Refs #4616 Attach the invariant documentation to the exported census API, not to the shared-inode memory ceiling.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — reviewed the exact nonforce 4-commit delta from 94b5d9fcc68f to d7bbee1d5517 against the code paths and the preceding hosted Bazel failure.
- CI root cause: repository gates on run #42732668837 failed the two crate rustfmt targets on precisely three hunks in
capability_dir.rs,generation/spill.rs, andruntime/tests.rs. Commitb23f36fbapplies those hunks without changing semantics or weakening tests. - OOM-bound source fix:
census_receipted_directory_bytespreviously retained every distinct multiply-linked inode in aHashMapwith no bound. Large shared stores could grow this metadata indefinitely.bca1e23acaps it at 65,536 identities, uses fallible reservation, and returns typedOutOfMemoryrather than asserting a complete size census when coverage becomes unknowable. The existing Doctor caller maps the census error toPartial, not a complete zero. Follow-ups3c4fab3bandd7bbee1dpreserve portable compilation and the exported API documentation. - Behavioral regression: the native Unix test creates three distinct actual hard-linked inodes, exercises the production walker with a two-identity budget and requires a typed refusal, then exercises the public entrypoint and requires all eight payload/receipt bytes and both link names to survive. A no-op limit or a missing walk cannot satisfy both outcomes.
- Review boundaries: this limits only shared-inode bookkeeping, not full storage report memory use or the acceptance-scale ~70-generation/15-GB journey. The new exact-head Bazel workflow is queued; there is no local checked-out Bazel runner, so no fresh Rust pass is claimed and Cargo was not used. Further work on Doctor's
doctor_kernel.rsoverlaps active #4794 and was intentionally not edited.
Keep #4616 open pending full scheduled N>3 retirement, writer fairness, ownership-race resolution and joint acceptance on master. No changes to protected draft #4776, Kev/decision-model PRs, release #4763, peer branches or CI settings.
Refs #4616 When a registered sealed store's root or generation directory was moved before receipt-verified deletion, no-follow cleanup returned Ok(false). That was mistaken for successful retirement, uninstalling its reader and marking generation state collected while durable bytes survived elsewhere. Only unregistered disk-only retries may treat an absent path as idempotent. A real sealed store regression moves the root and child independently, requires typed refusal and retained reader, then restores and reclaims both.
Refs #4616 Keep the new typed missing-artifact retirement branch unchanged.
Refs #4616 The native directory-rename regression uses Unix unlink/rename semantics. Keep cross-platform product behavior unchanged.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independent source-path review of the three exact-old-head, nonforce commits 6027cbac, 9c3dfc48, and b9798d71 against prior PR head d7bbee1d. This is a partial safety fix, not approval for full issue acceptance.
Root cause: remove_receipted_child_directory_nofollow returns Ok(false) if the sealed root or child is missing. The registered-reader retirement path previously mapped that to success and uninstalled its reader. A root or generation directory moved aside immediately before the cleanup acquired its capability could therefore leave durable sealed bytes elsewhere while the generation was considered retired. Merely checking a pathname again would not establish deletion.
Fix: retire_sealed_directory_checked now treats Ok(false) as a typed refusal when called with the registered store's exact expected digest, retaining the reader and retry authority. The unregistered disk-only path (None) remains idempotent on an absent directory. Other ownership checks and no-follow removal are unchanged.
Nonvacuous native regression: build an actual verified sealed generation, register its reader in a real persistent parent graph database, relocate (a) the entire sealed root and (b) only its generation directory. Each retirement must refuse, retain the exact reader and leave the relocated durable receipt byte-for-byte intact. Restore the directory and retry: the real retirement must remove both the artifact and registry entry; a further disk-only notification remains idempotent. The prior implementation would return Ok on the first attempt and fail these assertions.
GitHub compare confirms one owned Rust file only (+95/-3) across three nonforce exact-old-head commits. No changes to other issue owners, held #4776 / Kev / model PRs, release #4763 or CI gates. Bazel/native macOS execution is unavailable in this environment; exact-head hosted CI #42752083614 is queued, not passing evidence. The unresolved arbitrary 65,536 shared-inode threshold review, genuine ancestor-owner anchoring and combined automatic N>3/Doctor/large-store master acceptance remain blockers.
Refs #4616 The moved-sealed-root regression calls the production receipt-verified retirement helper from its nested test module without importing it. Both Linux Bazel graph-db unit compilation and strict Clippy fail with E0425 before the regression executes. Import the actual helper and keep the native relocation, refusal, restoration, and idempotency assertions.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independent exact-head review of 1f3e3364cb08 against b9798d7169bc and hosted Bazel CI #42752083614. Both the Linux test-binary build and strict Clippy failed with E0425 in sealed_store.rs:4201: the new registered-reader missing-root/child regression invoked the production retire_sealed_directory_checked helper without importing it into nested build_tests. The commit adds only that existing sibling-module import, preserving the full native behavioral journey (relocation refusal, durable receipt preservation, restored successful reclaim, idempotent disk-only repeat). GitHub verifies one nonforce expected-old-head fast-forward, one Rust file (+3/-3 import wrapping), zero other files and no changes to tests, production deletion policy, CI, or peer branches. The new exact-head CI is https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42764317492 and is pending; no local Bazel is available, no Cargo run or passing Rust test claimed. This is a compile blocker repair, not complete #4616 acceptance or merge approval.
Refs #4616 Unit tests never executed while compilation failed, so three assertions still described superseded contracts. A registered sealed generation whose artifact is absent is now an unconfirmed retirement (typed refusal), with only a disk-only retry idempotent. A close retry after a poisoned close stays DurabilityUncertain, not Closed, and the engine is still verified released. A prepared pending row spill survives drop by design, so the live-producer test installs it before checking name reuse. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Refs #4616 A sealed store with many disjoint hard-link pairs exhausted the fixed shared-inode budget although only one dedup identity was unresolved at a time. Track the remaining filesystem-reported links and discard an identity after the final alias. Preserve ownership-priority byte transfer and the fail-closed bound for links outside the scanned store. Native filesystem regression walks four disjoint hard-link pairs with a one-entry unresolved budget; the old implementation fails on pair two.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independent source/diff review of exact commit 6939336af6c195e05642b6d94e20cc15f57abec7 against 736779337edb in PR #4819.
Verified failure mechanism: The sealed census retained an inode identity for the entire walk whenever its link count exceeded one, even after every alias had been traversed. Four independent two-link groups in separate nested columns therefore exhausted a one-entry budget at group two, although at most one group was unresolved at a time. This is a false partial-coverage result, not a real heap-pressure condition.
Focused repair: The entry now carries its remaining filesystem-reported alias count and is evicted when that count is exhausted. A previously visited lower-priority generation still transfers its physical byte charge to a later live head before the entry is released. External aliases not fully seen remain accounted for and continue to trigger the existing bounded, typed partial-coverage refusal.
Nonvacuous regression: A real Unix filesystem fixture constructs four independent two-link groups in four subdirectories, then invokes the actual production census with a one-identity budget, requires complete coverage of exactly 21 payload/receipt bytes, and checks that both hard-link paths retain their contents. The previous algorithm fails on the second group. The existing priority-transfer and external-alias budget tests are preserved; the former now verifies the exhausted identity was evicted.
GitHub compare independently confirms exactly one source/test Rust file (+59/-6), one expected-old-head nonforce commit, and no CI or peer-branch modifications. This is static source review, not a Bazel/native test pass: no local checkout or Bazel binary is available and hosted CI is queued. The 65,536 ceiling's measurement justification, mutable-filesystem census races, scheduled N>3 retirement, and combined acceptance on master remain open. This review is not approval to merge or close #4616.
Compare corruption census paths with the physical canonical path selected by the registry, including macOS temporary roots reached through /var aliases. Apply the pinned formatter to the new hard-link census branch. Bazel: graph-db 198 passed with one pre-existing ignored; private-fs 49 passed; repository rustfmt and strict Clippy on both unit targets passed. Concurrent-publication census correctness remains a separate unresolved finding.
A filesystem link count does not close an inode group. Layer publication can add aliases after an observed group completes, including files originally visited with one link. Retain bounded inode evidence for the whole census and preserve priority transfer; refuse incomplete coverage when that evidence fills. Two native regressions publish real layered artifacts between visits of one census. Both reproduced a 42,246-byte excess charge before the fix and now match independent unique-byte and category accounting. Keep the disjoint-pair fixture with exact accounting at sufficient capacity and typed refusal below it. Validation: graph-db unit suite 200 passed, one pre-existing ignored; private-fs unit suite 49 passed; strict Bazel Clippy and pinned rustfmt checks pass for both. The 65,536-entry bound remains unmeasured; this does not establish retirement or CPU incident acceptance.
Refs #4616 Keep deterministic concurrent hard-link publication coverage in the filesystem census's private unit tests instead of exporting a feature-gated production API. Retain real layered-generation coverage through the public census, and move Unix-only metadata imports to module scope.
Refs #4616 Keep the private concurrent-publication regression compilable by placing the filesystem metadata trait in the module that owns that test, rather than the unrelated marker-replacement tests.
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| let observed = if kind == 0 { | ||
| census.head_bytes | ||
| } else { | ||
| census.superseded_bytes | ||
| }; | ||
| assert_eq!( | ||
| observed, bytes, |
There was a problem hiding this comment.
🟡 Layered census generation counts go unchecked
For layered stores, census_sealed_store now passes this regression when generation counts are wrong but byte totals match. The test no longer asserts head_count or superseded_count for the published generations.
Learn more
The graph-store census reports generation counts separately from physical byte totals. This regression creates one live generation and one or two older generations, but only compares the byte fields after publication. A classification change can leave those totals correct while returning wrong counts. The lower-level filesystem census test verifies synthetic receipt categories, not the graph-store counts for these real receipts.
Example: With initial_layers = 1, a census reporting one head and one superseded generation instead of two superseded generations passes both byte comparisons.
Recommended fix: Assert census.head_count == 1 and census.superseded_count == 1 + initial_layers alongside the physical-byte checks.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Addressed in 6ac18619: the real layered-generation census now asserts exactly one head, 1 + initial_layers superseded generations, and zero unrecognized entries, in addition to comparing both physical-byte totals against independent inode accounting. The previously removed test-only public port remains absent. Exact-head Bazel CI is queued, so this is a source fix, not a passing-test claim; leave thread open pending validation.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independent source/diff review of nonforce expected-head commits cc772b3 and 80977b3 on PR #4819 (exact head 80977b3).
Confirmed repair: The former capability_dir::test_support::census_in_entry_order was a feature-gated public library API with no production caller, contrary to the repository rule against test-only production ports. It is now deleted. The scheduled hard-link publication regression executes the actual private census_receipted_children algorithm in its owning #[cfg(test)] module, never through a public test API. It constructs a real on-disk inode, visits one or both original hard links, publishes a new hard link while the census iterator advances, and requires single physical charging transferred to the later live head (8 live bytes; 3 or 6 bytes of old receipts). Both new link-count variants are asserted; the earlier nlink-based eviction implementation double-counted them.
Preserved production integration: Graph-db still builds authentic compact and layered sealed stores, publishes one or two layers, and compares census_sealed_store head/retired bytes against a filesystem inode inventory. Unix MetadataExt imports moved to module scope; follow-up 80977b3 places the trait in the private-fs tests module rather than the separate replacement-test module.
Scope and limitations: Exactly two issue-owned Rust files; no production census behavior, CI, dependencies, protected drafts, other owner files, or release changes. GitHub compare: 2 commits ahead / 0 behind. Static source review only: repository Bazel/native macOS tests unavailable locally; the hosted exact-head run is pending. This is not an approval to merge or close #4616. Scheduled N>3 automatic retirement under sustained writers, stable ancestor ownership, Doctor on master, and incident-scale OOM-safe reporting remain unproven.
Refs #4616 The real layered-generation regression must require a single live head, all retired siblings and no unrecognized artifacts, not only byte totals.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independent exact-commit review of 6ac1861 against 80977b3. The three added assertions restore the layered regression's generation-count contract: exactly one verified head, exactly 1 + initial_layers superseded generations and no unrecognized entries, while preserving the independently computed inode-byte comparisons for both classes. This directly addresses the new review finding without reinstating the removed test-only public API. One Rust test file, +3 lines, no CI or production logic change. Hosted exact-head CI #42777493212 is queued; no native/Bazel pass claimed, and this is not a merge or issue-closure approval.
Root cause and scope
Refs #4616 — partial ownership/OOM safety and collection-state repair only. This is a nonforce, clean-history replacement for #4804 and #4818, which could not be combined for CI because #4804 contains an immutable >72-character commit header and its Linux graph-db test disagrees with the independent strict-retirement error/receipt behavior. The original branches and commits remain intact.
Exact provenance / scope
Single commit
c8a1effe74cbbc15ae8fcf754cc5817924b58310, based onmaster91c72aba7750a398dd28e83d64bf0c6d6fef4172. Five paths only: the four changed by #4804 pluscrates/tracedecay-graph-db/src/generation_runtime.rsfrom #4818. The branch was created at the exact base and advanced usingexpected_shawithforce=false; neither original head was rewritten. No changes from the held mixed draft #4776, Kev work, release #4763, other issue owners, production or infrastructure.Validation and gaps
No local Bazel/native macOS run is claimed (no checked-out workspace or Bazel in this execution environment). Require hosted repository/commit gates,
bazel test //crates/tracedecay-graph-db:unit_test //crates/tracedecay-graph-db:graph_db_suite //crates/tracedecay-private-fs:unit_test --test_output=errorsas applicable to available targets, explicit nonzero test execution, and independent source review before merge. CI on the old #4804 head failed repository commit messages and a graph-db unit test; this clean-history joint branch addresses both reasons but does not establish a pass before CI runs.Remaining #4616 acceptance: complete N>3 end-to-end sealed retirement under actual retention and busy writers, stable owner identity before directory capability acquisition, integration with unrelated unsafe artifacts (#4752 is merged), Doctor reclaim (#4799), and large-store report full/partial memory behavior (#4809), all proven on master. Keep #4616 open. Kev/decision-model PRs and mixed #4776 must remain draft and never merge/auto-merge; release #4763 read only.
Review verification
At
35abfc8c0b, repository rustfmt and 39 macOS sealed-store cases passed through Bazel (17 unit and 22 integration). Two existing opt-in measurement cases were skipped. The run exercised the corrected direct-build interrupted-staging case, byte-exact retention of a foreign directory, restart adoption, refusal of an altered receipt and subsequent verified reconstruction. The previous head failed to build two staging assertions because their module omitted the canonical receipt filename, alongside the reported formatting hunks. Hosted checks and incident-scale acceptance for #4616 remain pending.Row-spill receipt failure and recovery
The marker persistence review is addressed in
05c18a3c70. A spill owner writes and syncs its receipt in a temporary sibling directory, then uses the filesystem's atomic no-replace operation to publish the complete directory. A synchronous receipt failure releases the staging directory. Occupied destinations retain their contents. Drop and store-open recovery continue to require the exact owner receipt before reclaiming a published directory.An isolated native child under a zero-byte file-size limit reached the real receipt write and received EFBIG. The original constructor left one directory behind. The corrected constructor left zero. An occupied destination remained intact and a subsequent creation succeeded. The store-open integration now creates the abandoned and live spills with the production constructor.
Native macOS Bazel passed 20 affected unit tests and 24 integration cases, with two existing measurement probes ignored. The repository formatting build and Clippy for both targets passed. Hosted checks and the existing acceptance hold remain pending.
The producer-exit follow-up is addressed in
d13622cb45. A pending spill now has a bounded name derived from its producer PID and exact receipt. Store-open recovery verifies that receipt through the held child directory before reclaiming a foreign producer's unpublished spill. A missing or mismatched receipt preserves the directory for inspection.The native regression exits a child process after receipt sync and before rename. Recovery removed the completed pending spill and retained both a live producer's staging and a mismatched staged directory. The final head passed all 21 affected unit cases, 24 integration cases, repository formatting and Clippy on both targets. The two existing measurement probes remain ignored.
The PID-reuse review is addressed in
ded88e87c5. Published names, pending names and marker receipts now bind the producer's numeric PID and per-process random nonce. A restarted producer with the same PID and sequence can create its spill while the old one remains. Store-open recovery checks the held directory's complete receipt, reclaims the previous incarnation and preserves the live incarnation's work.Native Bazel passed 22 graph unit cases, 24 graph integration cases and eight filesystem-witness cases. This includes the same-PID regression, real child exit after preparation and a real published child spill followed by a remount that preserves a live peer attachment. Repository formatting and Clippy for the three affected test targets passed.
At
552b5c7fc6, a failed unlink of a verified directory still permits cleanup of independent siblings under the same held root. The new native regression returned the original unlink error and removed the second verified sibling. Restoring the preceding implementation left that sibling unvisited and failed the regression. The focused macOS Bazel runs passed 15 private filesystem tests, 22 graph unit tests and 24 graph integration tests, plus strict private-filesystem Clippy and repository rustfmt.Current receipt verification
At
e60c8940c5, the reused-PID regression binds the old producer's receipt to its own canonical published name and verifies that only that incarnation's pending directory is removed. The store rebuild integration writes receipt/named-directory pairs using the persisted JSON identity for both the old and current graph generations. The preceding fixtures failed once production began rejecting mismatched pairs.On native macOS all 160 enabled graph integration cases and all 42 private filesystem cases passed. Of 183 enabled graph unit cases, 182 passed on this exact head, including the prior-incarnation and noncanonical-receipt cases. The remaining unrelated registry fixture expected the macOS
/varalias while production returned its canonical/private/varpath. Applying only the existing #4828 registry fixture repair in an isolated comparison made all 183 graph unit cases pass. The registry fixture is unchanged in this PR. Repository formatting and strict Clippy for both graph targets passed. The existing independent acceptance requirement remains outstanding.