Repository navigation
fix(sqlite): publish complete stores and recover interrupted initialization - #4805
ScriptedAlchemy wants to merge 64 commits into
Conversation
|
|
Review/validation hold: This PR is marked ready solely so the existing PR CI can execute on its exact head. Please do not merge until nonzero hosted Bazel tests, native macOS/Windows SQLite sidecar and locking behavior, and the preserved-failure/retry tradeoff are reviewed. The default PR workflow skips native macOS; a skipped check is not platform acceptance. The cross-process replacement and initial sidecar refusal tests are intended to be exercised by |
|
Refs #4513 — follow-up fix published on this same issue-specific PR at New data-loss path on retry: Patch: both Independent native SQLite external read-only subprocess / concurrent writer probes also passed 32 nonzero Linux assertions, 8 each in rollback and WAL modes (reader result and quick_check). Neither those probes nor this review count as repository Bazel, hosted CI, native macOS or Windows qualification. New-head hosted CI must run nonzero Not complete acceptance: an initialization that got far enough to write a valid database header can still leave an incomplete family, and post-admission POSIX pathname replacement lacks a stronger private ownership/VFS fence. Daemon-level external |
Refs #4513 Remove the stale discard_created call left after unsafe pathname cleanup was deleted. Abort now preserves the database and all sidecars, releases shared descriptors only after normal worker teardown, and transfers a lost-file initialization pin to the existing deferred worker cleanup. Reject initialization publication once drain or close has begun. Add real runtime regressions for abort/read-back/stale handles, separate- process replacement of the database and WAL/SHM/journal after worker close, idempotent abort, and a lost writer held across initialization abort. These Rust tests still require nonzero Bazel and native macOS execution; no Cargo or direct Rust invocation was used.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4513 — code review of exact head 1738c8e1de983e062053fb2544c62f3a9c222300.
Three actionable safety findings are attached inline: (1) invalid-header rejection closes an independent raw database descriptor and can release another local SQLite connection's POSIX locks; (2) the SQLite magic does not establish completed initialization or authorize reusing a preserved sidecar family; (3) sidecar reservations do not fence a replacement between reservation and SQLite's first access.
I reproduced the mechanisms in disposable Linux fixtures with native SQLite 3.46.1: the rejected-header/raw-close schedule changed an independent subprocess writer from SQLITE_BUSY to admitted before the original transaction ended; a separate-process post-reservation replacement lost sentinel bytes for each of WAL/SHM/journal; and a header-bearing but otherwise uninitialized WAL database accepted a retry that overwrote/deleted each sentinel. These are standalone mechanism reproductions, NOT execution of this repository's Rust/Bazel tests, and not macOS/Windows qualification.
Keep the removal of unsafe pathname-based cleanup. The next change needs production-path regressions through attachment startup/failure/retry, rather than only calling repository_start_failure directly. Require nonzero Bazel results on the resulting exact head, plus native macOS locking/sidecar coverage and relevant Windows coverage. This review is not an approval or a claim of master completion. Keep #4513 open; ordinary merging remains with the PR-maintenance owner. Do not merge or modify held draft #4776 or release #4763.
| .map_err(|_| OpenedDatabaseFileError::Reserve)?; | ||
| let journal = reserve_new_sidecar(&sidecar_path(path, "-journal")) | ||
| .map_err(|_| OpenedDatabaseFileError::Reserve)?; | ||
| opened.created_sidecars = Some(Arc::new(Mutex::new(Some([wal, shm, journal])))); |
There was a problem hiding this comment.
[P1] Retained reservation descriptors do not protect the sidecar directory entries used by SQLite.
After these O_EXCL creations succeed, another process can atomically replace just sessions.db-wal (or -shm/-journal) while leaving the main database unchanged. The retained handle still pins the old sidecar inode, but the worker opens the replacement by pathname; main-file identity/HAS_MOVED checks do not detect that sidecar-only substitution.
A disposable Linux/SQLite 3.46.1 probe reproduced foreign sentinel loss for all three suffixes after a separate subprocess performed exactly that replacement and before the first WAL initialization. Holding the reservation fds throughout did not prevent it.
The new cross-process test swaps files and calls repository_start_failure directly, so it proves the removal of explicit unlink cleanup, not that SQLite startup preserves replacements. Add the sidecar-only replacement schedule through actual attachment startup, including failure/retry. Establish an ownership boundary that remains valid during SQLite's sidecar operations; another pre-open stat/recheck or an advisory lock ignored by the replacing process is not a race-safe fix.
Apply the exact rustfmt changes reported by hosted Bazel Repository gates without touching SQLite behavior or other PR files. Refs #4513
|
Refs #4513 — repaired the exact four formatting differences reported by the failed Repository gates job at the peer's head |
Avoid opening a raw database descriptor when an existing attachment cannot contain the SQLite header. Closing that descriptor could unlock another live SQLite writer; cover the refusal with a cross-process test. This does not fix header-bearing incomplete stores or sidecar swaps. Refs #4513
SQLite's minimum initialized database occupies one 512-byte page. Refuse header-bearing shorter files without opening a raw Unix handle; otherwise they bypass magic-only admission and can release live locks. Refs #4513
|
Refs #4513 — published narrow POSIX-lock preservation follow-ups on this PR:
Independent Linux system SQLite 3.46.1 probe confirmed the pre-fix raw-close lock loss and 5 nonzero metadata-only mitigation assertions; these are not repository Rust/Bazel passes. Hosted exact-head Bazel Scope is deliberately partial: this does not make invalid-header refusal safe for files >=512 bytes; a raw pin still closes on those errors. It also does not prove a header-bearing store completed initialization or protect the sidecar names against noncooperating POSIX replacement after reservation. Keep the earlier three P1 review threads open and PR on review hold. Require an ownership-safe lifecycle and daemon+external |
Existing-mode admission must not open sidecars for databases whose file size cannot be a whole number of SQLite pages. Refuse those paths using metadata before opening a descriptor, preserving foreign journal entries and avoiding POSIX lock-loss on the refusal path. Add production attachment regressions for truncated header-bearing files and WAL, SHM, and journal sentinel preservation. Refs #4513
|
Refs #4513 — published Additional defensive admission fix: an existing SQLite DB must consist of complete pages; every legal SQLite page size is a multiple of 512 bytes. The prior >=512-byte check admitted header-bearing files of size 513 or 1025. Those could still reach the SQLite worker and allow it to delete or change unrelated journal/SHM sidecars despite the database being incomplete. Admission now rejects non-512-aligned lengths using filesystem metadata before opening a raw descriptor (avoiding an additional POSIX close on this refusal path), on all supported platforms. Rust regressions added: direct existing-header tests for 513/1023/1025 bytes, plus a test through real Independent Linux SQLite 3.46.1 mechanism probe (NOT Bazel/Rust tests): of 6 unprotected invalid-page-length cases (2 lengths x 3 suffixes), 2 journal sentinels were deleted, and 6 metadata-only refusal cases preserved all original sentinels. The probe did not demonstrate every suffix is destroyed in this specific invalid-header schedule; it did confirm a real journal-loss path. Still outstanding: this is only a necessary structural guard, not proof of completed initialization or sidecar ownership. Page-aligned but invalid or partially initialized DBs, a header-read failure after a fresh raw pin, and post-reservation pathname replacement remain unresolved. Native macOS/Windows, full external |
Match the pinned Bazel rustfmt_check_2024 output for the refused empty-attachment writer-lock regression without changing its behavior. Refs #4513
|
Refs #4513 — fixed the exact-head hosted Bazel Repository gates rustfmt failure on #4805 without changing behavior. Commit Independent disposable Linux SQLite 3.46.1 checks this run reproduced the successful-init overwrite of pre-existing The existing remaining holds are unchanged: full-page malformed/retry lock safety, complete initialization/sidecar ownership, post-reservation substitution, ordinary external |
|
Refs #4513 — published |
|
Refs #4513 — issue-specific follow-up published on #4805 (non-force, expected-old-head fast-forwards retaining prior and peer commits): Specific fix: previously a page-aligned malformed database with a bad SQLite header could be opened via Independent disposable Linux SQLite 3.46.1 mechanism checks passed: 6 malformed aligned-header/sidecar preservations and 10 external writer-lock/immutable-probe assertions, plus a separate direct POSIX-fcntl schedule demonstrating exclusion survives SQLite immutable refusal whereas an independent raw open/close releases it. These checks are NOT the Rust regressions, repository Bazel, or native macOS results. Validation hold: exact-head CI run 42465671210 was queued when checked. Require nonzero |
|
Refs #4513 — published issue-specific formatting repair The preceding exact-head CI run 42465671210 failed Repository gates at Keep #4513 open; ordinary PR merging is owned by the existing maintenance lane. Held draft #4776 and release #4763 are untouched. |
Refs #4513. The prospective SQLite initializer now owns empty -journal reservations as well as WAL/SHM. Require the scoped reset tests to account for those exact owned files and retire the journal in the shipped-store fixture before attempting a new installation. The hosted Linux fleet at b6c7cd3 reproduced the old assertions and leftover journal in the test fixture; these assertions remain strict.
|
Refs #4513 — current-head Linux Bazel regression triage and narrow test repair. The exact PR head
Published focused commit Standalone Linux SQLite 3.46.1/Python subprocess probes passed 40 nonzero checks for WAL reader/writer exclusion and committed-data visibility, integrity/restart, occupied journal refusal and four-member scoped removal. These are not repository Rust/Bazel tests or native macOS qualification. The existing cross-process canonical namespace/sidecar race under a noncooperating same-UID writer is still an independent acceptance blocker. |
Refs #4513. Treat prospective schema finalization as a serialized writer barrier rather than a best-effort try_send. Deterministic bounded-channel regressions cover backpressure, FIFO admission, completion replies, and closed-writer refusal without relaxing ordinary SQL admission.
|
Refs #4513 — published Fix: Regression added: two deterministic, nonzero-assertion Rust tests in Exact-head hosted CI is queued, not green. No local Bazel runner is available, and no Rust test pass is claimed for this new head. The three global-db schema-unpublished fixture failures, canonical SQLite pathname/sidecar ownership race, and final native macOS/Windows and daemon/external |
|
Refs #4513 — published The previous hosted Linux Bazel run reported 497/500 global-db tests passing, with three failures where test fixtures manufactured an existing schema-unpublished SQLite namespace then expected registered runtime admission to silently claim it. Production's strengthened physical attachment correctly refuses that ambiguity. This was an obsolete fixture/contract, not grounds to weaken the sidecar ownership guard. The patch:
The commit was a one-parent expected-old-head, nonforce fast-forward from Verification status: the current exact-head hosted CI is pending, so these Rust changes are not yet Bazel-verified. Independent Linux SQLite 3.46.1/Python child-process probes passed eight nonzero assertions for live WAL reader visibility, uncommitted isolation, restart integrity and unchanged unclaimed sidecar bytes; those are not repository Bazel or native macOS tests. No issue closure or merge. The known canonical SQLite pathname/sidecar namespace race under a noncooperating same-UID process remains an independent acceptance blocker; full native macOS, Windows, daemon/external-system- |
|
Refs #4513 — new exact-head hosted Bazel finalization blocker on #4805 Linux fleet job 129256248869 failed 2 of 183 Bazel test targets (181 passed):
Independent SQLite mechanism proof, not TraceDecay Bazel: with SQLite 3.46.1 on Linux, a separate process held an active WAL read transaction while the writer committed a new row. Required repair/evidence: distinguish writer-channel admission from SQLite checkpoint-busy results (record the checkpoint result and actual live reader/lease blockers in repository-native diagnostics), arrange a safe publication barrier that waits for/ends the owning initialization readers without violating worker ordering or WAL durability, and add a deterministic real registered-runtime Rust regression with an overlapping SQLite reader and finalization/recovery, including WAL/SHM/journal preservation and retry. Verify using Bazel, then native macOS and Windows; keep test semantics and existing timeouts intact unless a measured irreducible budget justifies a separate change. The separately acknowledged canonical pathname/sidecar ownership race is still an independent blocker.
|
The schema writer must retain its original reservations while all staging reader workers exit. Retire both reader lanes before the TRUNCATE barrier, so live snapshots cannot turn a complete schema into a false BUSY refusal. Add a real repository attachment regression for a pinned WAL snapshot, reader admission fencing, retained writer, and committed SQLite integrity. Refs #4513
|
Refs #4513 — published the initialization reader-drain repair at Exact-head failure addressed: Linux Bazel fleet job 129256248869, with its archived per-target test logs, reports Repair: Prospective schema installation now fences general and reserved-health reader admission, waits for active/limbo snapshots to settle, and joins the staging reader workers on the blocking pool before asking the retained original SQLite writer to stamp and Regression: Added a real Rust repository-attachment test that pins a SQLite read snapshot, writes a second committed WAL row, confirms reader finalization waits for the snapshot, refuses reopened reads after the fence, preserves the writer until finalization, and checks both committed rows plus Verification boundary: Eight nonzero standalone Linux SQLite 3.46.1 assertions passed against an independent-process |
Match Bazel's Rust 2024 rustfmt gate for the reader-drain repair. Refs #4513
|
Refs #4513 — published The previous exact-head hosted run failed the Bazel repository gate at The GitHub compare confirms one-parent nonforce, expected-old-head update ( Independent local Linux SQLite 3.46.1 probes passed 25 nonzero assertions with an external read-only subprocess, live WAL writer, committed-row visibility, restart integrity, and
|
|
Refs #4513 — new exact-head hosted Linux Bazel CI blockers, with source attribution still unproven. At PR head
Repository gates, Clippy, feature gates, Dashboard, benchmark self-tests all passed for this exact head. These two failures have not been independently reproduced in isolation and neither is presently attributed to the SQLite patch; avoid speculative source edits or increased timeouts. The Linux summary job later completed failure solely because the Linux fleet failed (it did not run a second test suite); Windows is still independently running, and macOS was skipped. Archived Bazel logs were uploaded. Independent local Linux SQLite 3.46.1 subprocess/filesystem reproduction passed 30 nonzero assertions: read-only readers observed only committed rows across three live-WAL write transactions, reopening passed integrity, all three occupied sidecar reservations refused The untrusted same-UID canonical sidecar namespace still lacks a proved lifetime ownership boundary, so #4513 remains open, and even green CI would not close acceptance. |
|
Refs #4513 — new POSIX WAL/SHM lock-loss exposure to qualify in PR #4805 (reviewed exact head The new-store Six isolated native Linux SQLite 3.46.1 trials, 36 nonzero assertions, actual separate-process writer: Actionable acceptance work: audit whether an independent same-process SQLite handle can overlap No source commit, CI/test relaxation, merge, or issue closure was made in this investigation. The previously known canonical basename replacement race remains unresolved; #4513 stays open. |
Transfer creation pins into publication and close all raw main/sidecar verification handles before clearing the admission receipt. Apply the same ordering to prepared publication and published-family recovery. Add deterministic Unix subprocess regressions that admit a real SQLite connection at the completed-handoff event and verify rollback/WAL read locks, independent WAL writer exclusion, release, and restart integrity. Preserve refusal when cold recovery lacks durable allocation generations. Standalone Linux SQLite 3.46.1 mechanism checks passed; repository Bazel and native macOS qualification remain required on this exact commit. Refs #4513
|
Refs #4513 — published a production handoff repair and four Unix regression tests in RepairThe raw-SHM report led to a concrete ordering defect in the publisher: Fresh publication now transfers creation handles into the same owned collection as the verification handles. That entire collection is closed before the receipt is cleared. Prepared publication and already-published recovery apply the same order. Identity/content checks, no-replace publication, foreign-file preservation, durable receipt handling, and refusal on unproven allocation generations are unchanged. The completed-handoff tracing event now records the actual released boundary. Regression coverageThe new Evidence and validation hold
Required exact-head checks: Confirm nonzero execution of the four behavioral regressions, and run the same affected targets natively on macOS using the existing workflow. The available connector exposes job reads/reruns but no workflow-dispatch action; no workflow files or infrastructure were altered. This repairs the publication/pin-lifetime window, not arbitrary same-user mutation of the active canonical SQLite namespace. That separate P1 limitation, current-head native qualification, daemon acceptance, and verification on master remain open. No approval, merge, or issue closure; ordinary merges remain with PR maintenance. #4776/model work and release #4763 are untouched. |
|
Refs #4513 — published a narrow hosted Bazel Rust 2024 formatting repair on the existing issue-specific PR #4805: The previous exact-head CI run 42703851238 failed One-parent, issue-focused commit; the branch advanced by expected-old-head ( A separate disposable Linux SQLite 3.46.1 subprocess baseline passed 57 nonzero assertions: read-only SQLite processes observed every completed WAL commit, restart/integrity checks passed, and The existing P1 canonical sidecar namespace replacement review remains unresolved and requires a real ownership boundary, not another pathname recheck. PR #4805 is not master, and issue #4513 stays OPEN. No merge or change to draft #4776, release #4763, or the independent PR-maintenance workflow. |
Refs #4513. Use Arc::try_unwrap to fence late WeakReaderPool upgrades before acknowledging initialization-reader or attachment shutdown. Restore the same pool and retry within the existing deadline when another owner won. Track worker owners extracted for out-of-lock joins so a concurrent lost file callback cannot report closure and release main/sidecar pins while SQLite connections are still alive. Preserve a failed join on retries. Add deterministic production-path races and independent SQLite subprocess checks for writer exclusion, committed data and integrity. Rust/Bazel and native macOS execution remain pending; local standalone Linux SQLite mechanism probes are not repository test results.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Review repair published: shutdown ownership fences
Refs #4513. Published 89f84bd672c9 on this issue-specific branch. This is not an approval or a claim of default-branch completion.
Source defects repaired
-
A quiescent Arc count did not establish exclusive shutdown ownership. Both initialization-reader close and ordinary attachment drain observed
ReaderPool::is_quiescent, then dropped the pool as though that necessarily joined its SQLite workers. An exact-SQL weak handle can upgrade between the observation and retirement. The surviving strong reference would keep the workers alive after the caller acknowledged reader shutdown.ReaderPool::try_into_shutdownnow usesArc::try_unwrapto atomically claim the actualPoolInnerand fence subsequent weak upgrades. A failed claim restores the same pool and retries within the existing deadline. Successful owners join outside the attachment mutex. -
Worker owners temporarily extracted for an out-of-lock join were invisible to lost-file observations. During that interval, the writer's lost-file callback can see empty normal slots.
lost_file_closedcould then treat empty normal/lost slots as proof of closure and release main/sidecar identity pins while SQLite connections were still live. The newworkers_closingstate covers that interval, prevents premature closure/pin release, and clears only after the joins return.close_and_joinalso requires completed drain. A writer join error is retained so a later drain cannot report success merely because its slots are empty.
The repair does not add any pathname deletion or relax existing WAL/SHM/journal preservation checks.
Regression coverage added
a_late_weak_upgrade_cannot_outlive_an_acknowledged_reader_shutdowninitialization_reader_close_waits_for_a_late_weak_upgradeattachment_drain_waits_for_a_late_weak_upgradelost_notice_during_out_of_lock_join_preserves_pins_and_writer_exclusion(Unix)
The attachment regressions use real SQLite and deterministic one-shot hooks at the production ownership boundary, not a timing-only attempt to win the race. Fresh subprocesses check writer exclusion, committed-row visibility, and integrity_check. The snapshot helper requires successful exit, exactly one executed test, and a completion marker. The lost-notice test deliberately injects the real callback while the files remain in place so an independent process can probe the original locks; it does not claim to simulate a filesystem rename.
Verification and limits
Remote comparison confirms exactly one commit ahead, zero behind the inspected 714474f9dc8ed0362c7088d83cc36c143574795d, changing four SQLite runtime files. The branch was re-read immediately before an expected-old-head, nonforce fast-forward, and the resulting PR head was verified.
Fourteen nonzero standalone Linux SQLite 3.46.1 assertions passed. They demonstrate the raw-descriptor-close lock-loss mechanism for a rollback-mode main file and a WAL SHM file using independent subprocess writers, followed by committed-data and integrity checks. These are not execution of the new Rust regressions, repository Bazel results, a TraceDecay daemon journey, or native macOS qualification. Bazel is unavailable in this runtime; Cargo and direct rustc were not used.
Exact-head hosted CI for 89f84bd is queued. The new source still requires compilation, meaningful nonzero Bazel execution, and native macOS qualification. Previous-head platform results do not qualify this commit.
Keep #4513 open and the existing independent merge hold in place. This lifecycle repair does not resolve the previously documented canonical namespace/sidecar replacement P1 or prove full acceptance on master. No merge, issue closure, protected #4776 change, release #4763 change, CI weakening, or PR-maintenance automation modification was made.
Refs #4513 Apply the exact Rust 2024 rustfmt hunk from hosted repository gates without changing shutdown ownership, SQLite behavior or test assertions.
|
Refs #4513 — published a narrow exact-head CI formatting repair to this existing issue-specific PR: The prior exact-head hosted Bazel repository-gates job failed The GitHub write used an expected-old-head ( Separately, 17 nonzero local standalone Linux SQLite 3.46.1 cross-process assertions passed for WAL writer exclusion, external read-only visibility before/during/after commit, restart integrity, and occupied WAL/SHM/journal preservation. These do not execute TraceDecay Rust/Bazel or native macOS. No acceptance claim or merge: issue #4513 stays OPEN. The documented canonical sidecar namespace replacement P1 and native macOS/current-head regression qualification remain outstanding. |
Refs #4513. Require exact single-entry names before either native no-replace rename or sibling replacement. A dirfd does not confine absolute paths, parent traversal or intermediate symlinks. Reject normalized aliases and NUL without opening any SQLite descriptor or mutating either namespace. Add filesystem preservation regressions for the database and all three sidecars, plus an independent-process intermediate-directory replacement. Keep the separate full-path framed-log API and existing positive tests. Native Linux renameat2 reproduces all three escapes with a committed SQLite row. Repository Bazel and native macOS qualification are pending; this does not resolve the remaining same-UID canonical leaf race.
|
Refs #4513 — published Defect and repair
Both rename operands now require an exact single normal entry before any filesystem mutation. The shared validator also rejects empty/dot names, normalized aliases such as Regression coverage and verification limitsAdded three behavioral Rust regressions plus a child-process helper in An independent disposable Linux/native This runtime has no Bazel installation; Cargo was not used. Required repository verification remains Keep #4513 open and retain the merge hold. Rejecting path escapes does not make replacement of a valid canonical leaf by a noncooperating same-UID process safe. That separate namespace-ownership requirement and default-branch acceptance remain outstanding. No master merge, protected #4776 change, release #4763 change, infrastructure change, or maintenance-task modification was made. |
Refs #4513. Require HAS_MOVED attestation when verifying a live SQLite worker. An unchanged path cannot establish the identity of a VFS handle when file control is unsupported. Exercise the actual SQLite memdb VFS and confirm a normal disk worker still attaches to the unmodified store.
|
Refs #4513 — published Worker identity fail-closed repair: Real-VFS Rust regression added: Verification boundaries: 15 nonzero standalone Linux SQLite 3.46.1 C assertions passed: the ordinary file VFS returned The branch update used the exact previous head |
Refs #4513 Apply the four mechanical Rust 2024 formatting changes reported by hosted Bazel run 42764933700. Preserve all assertions and behavior.
|
Refs #4513 — published an exact-head Bazel formatting repair on the existing issue-specific PR. Commit: The previous head's hosted Repository gates completed with eight Bazel test targets passing but failed two New exact-head hosted run: #42773456440 is queued, not yet passed. Independently, 18 nonzero standalone Linux SQLite 3.46.1 assertions passed across three disposable databases, with subprocess read-only WAL snapshots during a pending write, commit visibility, sidecar presence, reopen and Hold: PR remains unmerged, issue #4513 remains OPEN, native macOS qualification of the latest source remains outstanding, and the canonical sidecar-namespace same-UID replacement boundary is unresolved. Existing draft #4776, release #4763 and ordinary PR-maintenance ownership remain untouched. |
Refs #4513. The owner’s independent review and merge hold remain in effect. Keep the issue open for observations on master and longer-running production use.
On
d8f057b013, all 1,045 tests in the native Linux ARM64 recovery suites passed as an ordinary user on ext4 (456 SQLite runtime, 554 runtime core, 35 private filesystem). An additional Linux sessions test inspected/proc/self/fdand verified the real SQLite reader remains open until its last clone closes. The native macOS suites passed 1,693 tests. The canonical sidecar namespace review is still open.The subsequent master merge
f2d1ab29f3changes only the proximity advisory intracedecay-application. All SQLite recovery, attachment, session and test files match the verifiedd8f057b013tree.Initialization, interrupted work and reset
The registered store initializes SQLite in a private same-filesystem sibling directory under the existing durable-directory lease. Its own SQLite writer stamps the application ID and checks for a complete WAL checkpoint. The runtime then joins its workers and passes the original main and sidecar creation handles to the publisher. The publisher never reopens the staged SQLite database. It validates the full family, syncs owned files, and may clear the disposable SHM index only through its original descriptor. A nonempty WAL or rollback journal prevents publication.
For a fresh family, already-open source and destination directories anchor no-replace moves. Three exclusive empty reservations move to the canonical WAL, SHM and rollback-journal names before the completed main moves to its ordinary SQLite name. Live handles pin the accepted file generations through publication. The published runtime checks the main generation when it attaches. On normal completion the initializer clears its preparation receipt and removes the empty staging directory. The original coordination inode stays in place.
The stable lease temporarily records the four prepared file identities and the SHA-256 of the synchronized main file. Publication verifies that digest before and after the no-replace main move and during reconciliation of a published main. Prepared content review and owner repair Recovery and admission depend on the recorded state.
Authorized offline scoped reset and full-profile wipe coordinate with the existing initialization leases and validate expected file types. Both use the graph owner’s actual verification marker and spill paths. The full wipe checks all three top-level families before its first deletion. Unrelated paths and persistent coordination locks survive.
Review boundaries
The two recent staging findings concerned unlinking unverified leaf names and reopening or publishing a replacement database through a mutable source pathname. Failed cleanup now removes only an empty directory. The publishing code moves checked members through the retained directory capabilities, keeps their file handles live, and verifies the moved generations. Windows pins the directories against delete-sharing while resolving their paths for its native rename. Fresh publication requires same-filesystem no-replace rename. Failed staging cleanup review · Publication path review
SQLite opens its live schema-worker files and later its canonical family through its platform VFS. The store parent, private staging namespace and lease-governed recovery directory must be trusted against noncooperating writers that can replace entries during active SQLite operations or between a leaf identity check and a name-based move. A no-replace move protects its destination and subsequent checks detect mismatches. POSIX does not provide a conditional source rename by an expected inode. In particular there is still an interval from the last canonical sidecar identity check through the first canonical SQLite VFS open, followed by ordinary SQLite use. The original P1 remains open for independent review. The prospective physical factory now retains the original reservation across both writer and reader startup. On failure it returns the raw identity and original file set only after worker references release. The registered initializer moves those verified members back into recovery before returning, for both fresh and restored families. Partial creation and an unsettled or lost generation still require reconciliation. The repository factory and all five direct test callers use this same initialization entry point; the wrapper that discarded returned handles has been removed. Initialization API review Worker startup review The verified-reader POSIX lock change from #4803 is now integrated. It covers SQLite’s main file on immutable reads. Canonical sidecar P1 · Verified-reader PR
The native SQLite open interface accepts the canonical filename and a VFS. Its main and journal file controls expose SQLite’s already-open files. The VFS also manages WAL shared-memory mapping and locks. Keeping the staged connection open while moving the database is unsupported because its journal and WAL names remain tied to the original pathname. Admission already refuses occupied, missing, duplicate or changed family members when their identity can be checked. A successful check cannot establish future control over the directory by another process with the same OS account. Under this threat model a stricter permission check or advisory lease cannot establish the missing boundary. Supporting such a deployment requires a separately protected storage namespace or a different product contract for file access. SQLite open API · File controls · VFS and SHM methods · SQLite on renaming an open database
Existing physical attachments on Darwin and other Unix systems without Linux O_PATH now keep the main inode alive through a shared read-only SQLite VFS connection. Its identity is checked on both sides of SQLite’s HAS_MOVED result. The VFS coordinates the actual descriptor close with other local SQLite connections, including when reader startup fails or a lost route tears down its workers. The private creation descriptors still belong to the unpublished initialization family. Linux retains its O_PATH capability and Windows its existing file-sharing handle. Darwin attachment lifecycle review
Verification
At the previous reset implementation on 945dde8, all 13 scoped reset/restart journeys and all five scoped CLI unit tests passed natively on macOS. An isolated full wipe refused a held initializer and a wrong-kind marker before deleting anything. After those conditions were cleared, it removed both graph families and preserved a foreign directory and both coordination locks. Preceding Linux reset evidence
On the older 06f75fb head, native Windows ran 424 rusqlite and 509 runtime-core tests successfully. The hosted macOS job for 2ff2385 failed DNS resolution before Bazel downloaded. Previous native Windows job · Hosted macOS setup failure
The separate owner hold on the Kev/decision-model draft remains. #4776