Repository navigation
feat(gc): LayerFS 2.0 GC implementation (single PR) [fork mirror of #711] - #71
Open
mengmengmengqiang wants to merge 175 commits into
Open
mengmengmengqiang wants to merge 175 commits into
mengmengmengqiang wants to merge 175 commits into
Conversation
…ab#711 Merge base-snap keeping the PR head tree, so later merges of the fix711 branches use the main export as their three-way base while the Ephemeral-AI-Lab#692/Ephemeral-AI-Lab#693 publish delta is preserved.
Integrates the fix711-contract branch onto the Ephemeral-AI-Lab#692/Ephemeral-AI-Lab#693 publish base as part of PR Ephemeral-AI-Lab#711.
…fixes Combines fix711-contract's holder diagnostics (OpConflict backend detail, scope intents, session/transaction refusal) with fix711-pg's per-key advisory modes, note_backend pooling proof and non-terminating owner settlement. Conflicts: postgresql/compact.rs (standby Replica evaluation, poll_owner_gone over terminate_owner), history/admission.rs (per-key shared mode + session_lock_refusal).
Keeps lifecycle's unfinished-Init/merge guard in postgres compact and its extra resume/phase tests, on top of pg's exclusive Store claim ordering. Shared admission diagnostics keep the per-key advisory mode and scope_kind. Conflicts: history error docs, gc_advisory_holder.sql, postgresql/compact.rs (standby BusyOp, owner settlement, single Store claim), admission.rs (duplicate EMPTY_OP / doc wording), postgresql_compact.rs (backend detail, test ids, lifecycle tests).
Takes fix711-rm's sidecar-before-admission project.delete recovery and the 'retire' marker phase for branch.remove, keeping fix711-life's related Sandbox guard and release_or_mark. Conflicts: project/delete.rs (mark_and_finish recovery body), store-service history/admission.rs (branch removal marker phase).
…ixes Takes fix711-sqlite's compact.rs as authoritative for the SQLite intent lifecycle (intent survives build/install, cleared at cleanup settle; abort takes Option<&op_id>), its keep-through-install crash expectations, and its transient kept_node clear, combined with fix711-contract's retained-roots helpers. Conflicts: backend/sqlite/compact.rs, history/kept.rs, tests/sqlite_compact_crash.rs.
Keeps fix711-rm's sidecar-before-admission delete body and lifecycle's Related.can_remove_project in the tests, plus fix711-sqlite's capped delta-base report imports. Adapts api's new tree_stats visibility filter to fix711-rm's fallible Visibility::for_snap. Conflicts: project/delete.rs, store_transfer_recovery.rs, restore_install.rs, backend/sqlite/compact.rs.
Integrates fix711-perf as part of PR Ephemeral-AI-Lab#711.
Takes fix711-hyg's deliberate revert of the delta cycle-detection change (out of this PR), its AGENTS/docs restructure, and its removal of the 12 implementation notes from core/docs/issues/project-41. Conflicts: layerfs-storage delta/select.rs (cycle-detection revert), modify/delete of the moved implementation notes.
…or import Both fix711-pg and fix711-life added the same standby Replica test helper; the merge kept both. Drop the duplicate and an import fix711-api no longer uses in the SQLite compact test module.
- acquisition_layout: a live acquisition is unfinished work, so compaction refuses (fix711-life §5 guard) and the test releases the row first. - branch_remove: Busy entity.kind reports the resource scope (branch), as fix711-contract's OpConflict now specifies. - sdk_methods golden: ReservedName carries the spec-gap G1 empty detail. - regenerate the SQLite acceptance matrix report (fix711-hyg had removed the generated evidence the matrix suite requires).
…intent test fix711-contract's gc_intent schema CHECK requires a 32-byte Store scope_id; the existing 16-byte placeholder fails registration with Integrity.
…e GC branch Merges the new 2.0 base (Ephemeral-AI-Lab#712: machine/Sandbox logs and files, observed values, doctor.fix, the hardened worker) into gc-2.0-final, keeping the GC implementation (#42-#52, Ephemeral-AI-Lab#692, Ephemeral-AI-Lab#693, Ephemeral-AI-Lab#694). Conflict resolutions keep both sides: - worker_server.rs is Ephemeral-AI-Lab#712's accept loop plus connection/fault/ops modules; ops.rs carries the GC project.delete, project.delete.admit, project.delete.release and project.list operations (project.list added to the read-only set) beside Ephemeral-AI-Lab#712's doctor.fix and inspect. - MCP profiles: the GC admin exceptions (project.delete, project.remove_file, branch.remove, store.compact) plus Ephemeral-AI-Lab#712's ui and none buckets. - The call table is 82 calls in eleven families (GC +3: project.delete, branch.remove, store.compact; Ephemeral-AI-Lab#712 +6: sandbox.files, machine.stats, machine.files, logs.machine, logs.sandbox, doctor.fix). - Docs, Node/Python call-count tests and the SDK pages take 82/11.
added 30 commits
November 17, 2026 08:11
…tion SRC_MMAP_BYTES is a const, so the bare assert! folded to assert!(true) and clippy's assertions_on_constants rejected it under -D warnings; the check now reads the constant through std::hint::black_box.
…refusal owner, seven gc_op columns, receipt vintage caveat
…7, quick-lane target count
…ns and receipt vintage
Move fixr5-43's listed_unknown helper from project/delete.rs (1004 lines, over the product-source boundary) into project/delete/remote.rs, the only module that calls it, and re-export it at project::delete::listed_unknown so the F43R5-2 unit test keeps its path. The product boundary is green again (1526 files); fmt, check and the project_delete test pass.
The merged tree has fixr5-43's extra project_delete test and fixr5-50's extra PostgreSQL compact test, so the runner-derived 'Verified here' counts move project_delete 17 -> 18 and postgresql_compact 29 -> 30. Regenerated with UPDATE_REPORT=1; the matrix test then passes without it.
…tore, and the mark records the Store compaction op
… Unknown answer naming the project
…bly dead intent owner is not in flight
…d report instead of re-deciding
…ctor setup and native unmount reap The upstream base advanced 24 commits past the 5742338 merge base while Ephemeral-AI-Lab#711 was in review. Merged into the R6 integration, keeping both sides: - the new `home` family (`home.init`, `home.get`) joins the table: twelve families, 85 calls (equalling the 83 of the PR head plus the two home calls); the profile buckets are 38 work / 40 admin-only / 6 ui / 1 no tool, so `--profile admin` lists 78 tools, and every generated page and test says 85; - `project.delete`/`project.remove_file` (this PR) replace the base's `project.remove`, and `branch.remove` stays, so the admin exception list and the SDK pages name both; - the base's machine-platforms, doctor host checks, retained-child reaper and native unmount work are untouched by the PR's engine changes.
…der clippy, and count 85 calls - `cargo fmt` on the R6 code (fixr6-41's compact op/install/receipt/run and its `sqlite_compact` arms, fixr6-43's `add_failure`). - fixr6-43's `acquire` grew the run's `op` id for the retained Unknown state (F43R6-1) and crossed clippy's eight-argument bar; it is allowed with the reason, as the other wide recovery paths already are. - the base's non-macOS `daemon_value` stub took `&mut Vec` where a slice does, which `-D warnings` refuses on Linux. - `check_product_boundary.py` named `src/home/` a retired layerfs-api folder while the base re-added the `home` call family there; the family is no longer treated as retired. - the base's `home.init`/`home.get` make the table 85 calls in twelve families: the API, glossary, flow, plugin, skill and 2.0.0 manual pages say so, `calls.md` gains its `home` family section, and the generated `coverage.md` block is regenerated (Node/Python/aio/CLI 85, MCP 78).
The base's `home.init` is the table's first entry and takes no positionals, so the mutation test's `python.table[0].args[0]` was undefined. Pick the first call with an argument and find its Python entry by `family.call` instead. Ran the whole api/node suite with the built addon and command: 63 tests, 58 passed, 5 skipped, 0 failed.
…ds, so a real host block parses; the example gains its two tests
fixr6-41 (F41-1) makes an installed `G + 1` leftover answer the totals the run recorded instead of re-deciding to `none` on the already rebuilt Store. The in-process crash matrix follows that, but `sqlite_compact_process_kill`'s `AfterInstall` and `DuringSettle` arms still asserted `path = none` with `G + 1 -> G + 1`; both now assert `rebuild` with `G -> G + 1` and, for an installed leftover, a non-zero collected `dead_bytes_bound`. Caught by the ext4 `--quick` lane; all 8 arms pass. Also: merge fixr6-benchjson (gc_bench json_host doubled quotes + its two example tests).
…he Store answers NoSuchBranch `scope_references` asserted `InvalidInput` for the two fork references the scope composes into its namespace (`from_commit = "other"`, `from_branch = "other"`); the composed names (`agent.other`) are inside the namespace and name no Branch, so the Store's answer is `NoSuchBranch`. The other five references are still refused by the scope with `InvalidInput: out of scope: branch`, and no Branch appears. Reached at `7702bb21` too: the test is older than the base sync and no evidence run of the earlier rounds included it. The rest of the crate is green (229 passed / 0 failed / 2 ignored across its 77 targets).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Single PR for the LayerFS 2.0 GC implementation, consolidated from the staged
PRs for review. Tracks the GC design in
Ephemeral-AI-Lab/layerfs-project#41and sub-issues #42–#52; the contract is
core/docs/issues/project-41/LayerFS-GC-spec.mdand the open decisions arecore/docs/issues/project-41/gc-42-decisions.md.branch_retired,layer_stack_retired,path_removed),project_epoch,gc_intent,gc_op,gc_dead,kept_node, serials,trace_salt; SQLite + PostgreSQL additive stepsbranch.remove,project.delete(home/worker, sidecar, retry),project.remove_file(visibility-only whiteouts)gc_intent,MountsActive/Busy/Unresolved, PG advisory keys, project-epoch writer fence; a SQLite compaction now refusesBusyon another process's open Store descriptions, and removals order concurrent runs on theproject_epochrowkeptWaste, live-heads digestVACUUM INTO), install fence, crash/kill recovery. PostgreSQL: Phase M/A/B/C,gc_op/gc_dead, scratch, swap, totals, standby/lost replystore.status(SQLite + PG), bounded metrics/SLOmachine.space), with the GC admin exceptions; docs and count tests at 83/11--quick/--full), runbook/rollout/capacity docs; the nine Round-2, seven Round-3, five Round-4 and six Round-5 fix branches and the Ephemeral-AI-Lab#720/Ephemeral-AI-Lab#725/Ephemeral-AI-Lab#726 base sit on the Round-1 headFresh-copy SQLite rebuild (D2)
The SQLite rebuild no longer stages a whole-Store copy and rewrites it. It
creates
<store>.compactwith the sealed Store's schema, copies only the rowsthe mark retained (pack-as-unit), builds secondary indexes and triggers after
the bulk load, verifies the fresh file and renames it.
The final 10 GB measurement on
e824e8a7is three runs per arm on agenerated fixture (9.789 GB live, 83,559 packs):
noneat 8.8% dead,rebuildat 10.0% dead after the dead-object add.rebuild133.076 s median wall (range 132.274–134.601 s, 1.75%relative range): mark 82.050 s, build 23.469 s, verify 6.060 s,
install 21.011 s. Peak temp 9.836 GB (new file 9.832 GB +
4.26 MB visited scratch) and physical shrink 1.093 GB.
noneis 82.683 s median on the same fixture.VACUUM INTObuild(246.371 s median on the earlier 9.987 GB fixture: mark 149.089 s,
build 94.317 s, verify 0.312 s, install 1.916 s, ~2.9× peak). The 23.5 s
fresh build is ~4× shorter than the old 94.3 s; the added time is the page
scan (verify 6.1 s) and the
sync_allof the freshly written 9.8 GB(install 21.0 s), both inside the 133 s total.
overlayfstemp filesystem,warm page cache (
drop_cachesrefused 6/6, so it was not dropped), onedeclared rotational-media host class. These are qualification numbers on
that host, not a cross-host guarantee; no 50 GB arm was run.
Production LOC
Measured with the tree's own
tools/production_loc.pyon the final head.Nonblank, non-comment first-party product lines; tests, docs, benches,
fixtures and generated artifacts excluded.
329a20be57423382f7referenceanddshare 0 on both sides. The Ephemeral-AI-Lab#727 base sync alone moves thehead from 184,030 to 185,983 (+1,953 product lines). The Round-4 branches are
test/doc-only except #43/#49/#50's small product changes; the Round-5
branches add 47 product lines (control +12, engine +35) through #43's
Init/delete naming, #50's PostgreSQL standby/holder/Phase M/status arms and
#49's install/identity fence, while #42/#52 are docs-only. The PR-only delta
is therefore the fresh-copy rebuild and the GC implementation itself. The
fresh-copy rebuild is engine-only (+389 lines over the 11c1896 snapshot:
the
compact/phase modules, the fresh-file build and its verification).Integration
Round-5 fix branches on
b357463b(six branches)b357463b(pr711-head8) is the Round-4 fixed head. Review Round 5 producedsix fix branches, each forked from LayerFS commit
12020f69bea9whose tree isb357463b(only the.lwwdiffers). Each was exported from the LayerFSproject and re-committed with parent
b357463b(.gitand the.lwwdropped), then merged with
--no-ffin this order: 42 → 43 → 49 → 50 → 51 → 52.The merges are
dba16b1f80,cca723ac36,93853740eb,5654607e3d,d05fe297b1,c3632887e6;c07297303bmoves 43'slisted_unknownhelperout of
project/delete.rsafter the added lines took that file over the999-line product boundary.
fixr5-4212cea9b21d7a→f48dd2c24adba16b1f80gc_intentempty and inserts only the run's ownop_idrow, so "never copied" and §5.1's "lives in<store>.compact" agree; the preamble/sign-off say A8–A12 and the heading reads "Contract amendments (11) and one reading". The A9 refusal text namesstore.push(notstore.move) as the non-GC call that can still report the cache'sInUse; the intent-sidecar doc lists the optionalop_id; the two 10 GB receipts in the evidence basis carry the vintage caveat (F52R5-2 wording: both predate #49 round 4's mmap window andquick_checkremoval).fixr5-431266b28b9200→bc4f5c8156cca723ac36initphase:init_unknown_stateis built beforerequest.namemoves and is attached torecord_new_bindings,initialize_layerstack, the Store-name-taken check and the fork (a bareUnknownOutcomereaches the client asUnknownwith{kind: "project", name},phase: "init", no durable op — a definiteBusyOpstaysBusy). F43R5-2 aproject.deleteStore-listing failure that cannot be determined names the project too (remote::listed_stackwraps both the empty-mapUnknownOutcomeand the poisoned-StoreUnknownwithlisted_unknown), so it never stays a bareUnknown {}; the newproject_add_unknownandproject_deletearms pin both.fixr5-49124b565f447c→b2d0757a9793853740eb(dev, ino)identity; a mismatch is nowConflict("the sealed Store identity changed")with no writes, notStaleStore. A Store replaced between seal and install is therefore Conflict;StaleStorestays the §10 session/transaction fence for a handle that opens or transacts after the install. The test is renameda_store_replaced_between_seal_and_install_is_conflict, and the AC-46/AC-59 rows, the regenerated report and the identity/SDK Store docs follow.fixr5-5012da9f6ffe2d→d88d4b32935654607e3dsynchronous_standby_nameslist isFIRST 1, not one synchronous standby per listed name (unit cases for"s1, s2"and"s1"). F50R5-2 the swap55P03holder query reads any granted table lock on the Store base tables, so an ordinaryAccessShareLockreader is named (pid/client_addr/backend_start) with a reader-blocker test. F50R5-3store.statusprojects the registered Phase Mgc_intentwhilegc_opdoes not exist yet (id/name/mark phase,started,updated_ms,busy); a test parks the run inside Phase M ongc_dead. F50R5-4 A8 wording: only the after size is sampled inside the swap transaction; the before size is sampled before it opens.fixr5-51121751691772→1494a9e01cd05fe297b1table.length - 1= 82; the check compares against the profile's own admin count. Test-only, no source change.fixr5-521251cf43c9c4→780421de3dc3632887e6gc-42-decisions.mdsays sevengc_opcolumns (A8 and the PostgreSQL schema already carry seven); 42 carries the identical word change. F52R5-2tools/ci/gc-bench.md§13 records the 10 GB receipts' vintage — both predate #49 round 4's 256 MiBsrc.mmap_sizesource window and itsquick_checkremoval — and marks the ~4 squick_checkcost derived, not measured; the matrix runner's AC-57 note, the report body and the E3 bullet say the same, and the report is regenerated. No 10 GB re-run: the receipts are not replaced. F52R5-3 thestore-ext4.ymlquick-lane comment now says the api suite runs four targets instead of "one or two targets per suite".Conflict resolution. Every branch shares
b357463bas its three-way base,and the six merges are clean. The overlapping files were read semantically:
42+52ongc-42-decisions.md. Both make the identical"six → seven
gc_opcolumns" change, so the merge applies it once;52then has no net change there.
42's A12 reading,store.pushand sidecarlines stay.
42+50on the same file.50's A8 paragraph (before size sampledbefore the swap, after size inside it) is a different region from
42'sA12 insertion and request paragraph; the union keeps both.
49+52on the T12.3a report and the matrix generator.49editsthe AC-46/AC-59 evidence names and notes;
52edits the §13 vintage textand the AC-57 row, and regenerates the report. The union keeps both, and the
merged runner regenerates the merged report (the
sqlite_compact_matrixtest asserts it).
42+43onproject/delete.rs. Different regions (the intent-sidecardoc and the new listing helper); the merge is clean, but the added helper
took the file to 1004 lines over the 999-line boundary, so
c07297303bmoves it to
project/delete/remote.rsand re-exportsproject::delete::listed_unknownfor the F43R5-2 test. The product boundaryis green again (1526 files).
The frozen
LayerFS-GC-spec.mdis still byte-identical toed14ed53b(sha256
84e16978d92d14ee65eb10d38f8d2f5347455b84c2c5b2e49fd28f60282415cf,84,894 bytes) on the merged head.
Round-4 fix branches on
a536f920(five branches)a536f920(pr711-head7) is the Round-3 fixed head. Review Round 4 producedfive fix branches, each forked from LayerFS commit
3230a02565d5whose tree isa536f920(only the.lwwdiffers). Each was exported from the LayerFSproject and re-committed with parent
a536f920(.gitand the.lwwdropped), then merged with
--no-ffin this order: 42 → 43 → 49 → 50 → 52.The merges are
dfdd055503,3370046f3f,bc811e91ea,19679fd104,7b58d4eaa1.fixr4-42122df99487a9→0242ff8ed0dfdd055503gc-42-decisions.mdpreamble counts are consistent: sixteen places (A1–A11 + D1–D5), the note adds A8–A11, and the schema paragraph's "fourgc_opcolumns" becomes six before #50'sphysical_after(the merged tree says seven). F52R4-2 A11's "Where" names the moved unit tests incontrol/crates/layerfs-store/tests/compact_in_process_call.rs, notsrc/compact.rs(which keepsgc_refusal).fixr4-43129ccd5e1e57→355ff7d1833370046f3fnamed_unknownalso fills an empty-mapUnknowndetail (a bareUnknownOutcomerendersSdkValue::map::<0>([]), notNull), soproject.get/project.list(store)and the workerproject.delete.releasepath name the Store/entity and operation instead of returningUnknown {}. F43R4-2project.init's registration/releaseUnknownOutcomereaches the client asUnknownnaming{entity: {kind: "project", name}, phase: "init", op}(both theInitError::Project(ProjectError::History)andInitError::Historyarms), notFailedwith a null detail; a definiteBusyOpstaysBusywith its operation detail.fixr4-491261ccde6d5c→21b401573ebc811e91eaverifyno longer runsPRAGMA quick_check: the pass is schema identity, foreign keys, counts, live-heads digests, the allocator tuple and the recomputed totals, andquick_check/the doctor stay separate commands, so it reads no payload. The T12.3a text and AC-57 record the earlier ~4 s page scan as the then-presentquick_check(removed here) rather than a current verify cost. §7 step 2's bulk copy appliesPRAGMA src.mmap_size = 268,435,456to the attached sealed source (the session default is no mmap) and returns the size SQLite applied.fixr4-5012c9ab78bfd2→3a747da6d919679fd104gc_opgainsphysical_after, filled from the swap transaction's ownpg_total_relation_size(pack, object_location)sample, so a save after the swap (AC-28) cannot inflate a resumedG+1report'sphysicalShrink;Swapped.physical_afternow carries it.store.statusreportsbusyonly while the op'sgc_intentis present and the recorded(pid, backend_start, usename)is alive, andunresolvedonce the intent is released (PostgreSQL's form of SQLite'sreturned) or the owner is gone; thestate != 'idle'test is dropped, so an idle but live owner of a released op is notbusy.effective_rankno longer floors the swap's sync strength atlocal, so twooffsettings stayoff(and the §8 step number is corrected to step 10).fixr4-5212780a4178a8→057e1ffb0e7b58d4eaa1tools/ci/gc-bench.md§13. F52R4-4 theci.yml/format.ymlcomments name the store-ext4 exception (PRs intomainorlayerfs_v2_0_0), andstore-ext4.yml's quick-lane comment counts five cargo groups and the api group's four targets. F52R4-5test_store_ext4.sh --listprints the commands before theLAYERFS_TEST_TMPDIRcheck, so the documented verbatim command exits 0 without the variable.Conflict resolution. Every branch shares
a536f920as its three-way base.Only three files overlapped, and each was read semantically:
42+52ongc-42-decisions.md. The two blobs are identical:42fixes F52R4-1/F52R4-2 and
52carries the same text, so the merge takes42's version and52changes nothing there.42+50on the same file.50addsphysical_afterand the "bothrelation sizes sampled in the swap transaction" sentence;
42had alreadymoved
kept_wasteinto the mark and re-wrapped the paragraph. The unionkeeps both: the paragraph now says seven columns, the mark writes four
(
op_id,mark_fingerprint,mark_visited,kept_waste) and the swapwrites three (
physical_before,physical_after,peak_temp).49+52on the T12.3a report and its generator. Union: the mergedgenerator carries both
49's removed-quick_checkwording and52'ssuperseded-build missing-range deviation, and the report is generated from
it, so AC-57 states both.
43's new unit tests had landed in product source(
src/project/add.rs,src/unknown.rs), which the product-boundary guardforbids:
6efec405bamoves them tolayerfs-store/tests/project_add_unknown.rs(
unknown::named_unknownis#[doc(hidden)] pub), and formats52's matrixgenerator.
983cbb4ca1replaces50's.map(format!).collect::<String>()artifact ids with one hex helper so clippy's
format_collectstays green,and the T12.3a report is regenerated on the merged tree (the extra [upstream issue #365] Benchmark bundle: pjdfstest, fsx, fsstress, smallfile, mdtest, fio on the SDK route (v1.0.0) #43
project_delete_workertest moves its count from 7 to 8).Base sync to
layerfs_v2_0_057423382f7(Ephemeral-AI-Lab#727)The integrated head is merged with the current
layerfs_v2_0_0base57423382f7(Ephemeral-AI-Lab#727, the lp73 integration: the NFS AppleDouble codec, spill andper-inode companion layers, batched READDIRPLUS/LOOKUP facts and macOS mount
options) as commit
b41ebabc93, a plain merge with no rebase and no force.dd4a2cc40fis the merge base and the base moves 44 files. Its only file incommon with this PR is
Cargo.lock(the base addslibctolayerfs-daemon,the PR adds
postgrestolayerfs-persistence), and the merge is clean.The previous base sync already removed the
postgresqlbuild feature (Ephemeral-AI-Lab#726),so the PostgreSQL engine is always compiled; no gate uses a feature flag.
Against the new base
57423382f7the PR is 146 commits, 481 files,+62,710 / −1,426.
Round-3 fix branches on
d5cc150377(seven branches)d5cc150377(pr711-head6) is the Round-2 fixed head. Review Round 3 producedseven fix branches, each forked from LayerFS commit
1280f9b1df18whose treeis
d5cc150377(only the.lwwdiffers). Each was exported from the LayerFSproject and re-committed with parent
d5cc150377(.gitand the.lwwdropped), then merged with
--no-ffin this order: 43 → 44 → 45 → 46 → 49 →51 → 52. The merges are
fee2ebcc49,5ef2ce0153,636469bfb1,87418c8920,30da9172f1,eddf7534ec,02dca44dc1.fixr3-4312e834134847→61212f265e70fee2ebcc49project.listreply whose answer is lost isUnknown, and the caller now names the Store it listed:project.family's worker listing,project.getandproject.listof a named Store attach{entity: {kind: "store", name}, phase, op}throughunknown::named_unknown. A shared intent's failedgc_register/gc_releasekeeps itsSharedIntentscope, project and phase and reportsUnknownthroughlookup::history_inwith the operation id, instead of a bare refusal.fixr3-44122e15d7d904→05bf7e00d4675ef2ce0153branch_any,published_layerandnode_history_serialran their one statement in autocommit with nosearch_pathandqualified()served no{s}body for them, so PostgreSQL answeredIntegrity("persistence"). Added the three{s}-qualified bodies andpostgresql_statement_schema: a guard that discovers everyread_statementbody in the crate, pins the keys it runs, fails when a body or key is added without a{s}body, and runs all of them against a real server (it also reads a brace body's{correctly, not a closure argument's parenthesis).fixr3-45128e61a33007→a9cb4c374852636469bfb1markedis already past the marker: its retry takes no admission, writes no marker, removes the worker's own entry through the newproject.delete.entryworker op and then the caller's home entry, recordingdone. A record or listing that names a different Stack than the sidecar is a re-created project: the stale intent is discarded and the call isNoSuchProject, so the new project's entry is never touched. Listing/read refusals keep their History wire code throughlookup::published(identity,project.family,project.listing) instead offailed's generic code; the 2.0 manual and worker-op docs follow.fixr3-46127e6f7e49e5→671eef261ebc87418c8920remove_unreachabledeleted a Stack's unreachable Layer and Commit rows butstore_totals.logicalis maintained by AFTER INSERT triggers alone, so the deleted rows' logical bytes stayed counted. The prune now retracts exactly this Stack's unreachable logical bytes in the same write transaction, while the rows the statement reads still exist (history_remove_unreachable_totals.sql), withkept_nodeandsqlite_compactregressions.fixr3-49125c76ce3fe7→fe0f790e2b5530da9172f1<store>.compact-oprecord gains thereturnedmarker written before the call replies, so a run that already answered is a leftover the next call judges:store.statusstaysunresolvedfor its record even while the recorded pid lives, andbusyonly for an in-flight live owner. A live in-flight op is refused withBusyOpnaming the record's entity, phase and operation. Install/run/op, the report/status projection, thesqlite_compactreturned-run regressions and the docs follow.fixr3-5112ebf0439213→ad5640140a77eddf7534ecstore.compact's open-cache refusal for a call still running on the Store in this process is the GC vocabulary'sBusynaming the Store with a nullphase/op, not the cache'sInUse(which stays for the non-GCstore.move/store.remove/store.disconnect); thestore.compacterror row and focused tests pin it. F50R3-2 AC-44's evidence and matrix text gain the three Round-2postgresql_compacttests (ended owner without the Store key, live key holderBusy, nopg_signal_backend).fixr3-52122cdc91330d→d3b967de7b5d02dca44dc1LayerFS-GC-spec.mdis byte-identical toed14ed53bagain: D1/D2/A10 stay only ingc-42-decisions.md, and the runbook states the frozen text is unchanged (the D2 peak/journal deviation is recorded there). F52R3-2 the report-only store-ext4 lane is documented for PRs intomainorlayerfs_v2_0_0, not "any base". F52R3-3 / F42R3-1 / F50R3-1 the nonexistentpostgresqlbuild feature is removed from the Store, project, front-end, versioned and doctor docs (always built, selected by the connection string). F42R3-2 thelogicalDiscarded/backendReusablecomments follow A8's semantics.Conflict resolution. Every branch shares
d5cc150377as its three-way base.The overlapping files auto-merged to the stated ownership except the generated
matrix report, and each overlap was then read semantically:
51+52onLayerFS-GC-spec.md.52restores the whole frozen fileto
ed14ed53b;51adds five §10 lines. A plain merge kept both, so thefinal tree takes
52's file byte-for-byte and51's rule moves togc-42-decisions.mdas A11 (A. Contract amendments (11)), with51'sstore.compactdocs row and its unit/matrix tests kept. The spec diff againsted14ed53bis empty.46+49+51on the T12.3a report. Every branch regenerates thereport from its own test sources and each merge conflicted there; the report
is regenerated from the merged test sources (
33sqlite_compact,7project_delete_worker,17project_delete,6compact_report_status), so the derived counts match the merged tree.43+45onproject/family.rs. Union:45'slookup::publishedimport and
live_projectscode with43'snamed_unknownwrapping on theworker reply,
project.getand the named-Storeproject.list.49+52onstore.mdand51+52onstore.mdunion:49'sreturned-run status paragraph and
51'sBusyrow survive52's docscorrections;
52's removedpostgresql-feature sentences win.Integration fixups.
719b5b1651restores the frozen spec, adds A11 and regenerates the T12.3areport on the merged tree.
3dc3453173moves the two Round-3 source unit tests out of product source(the boundary guard forbids
#[cfg(test)]/#[test]there):51'scompact::gc_refusalmapping tests tolayerfs-storetests/compact_in_process_call.rs(gc_refusalis#[doc(hidden)] pub),and
45'spublished()test to the store-servicehistory_wire_codestarget where the other lookup wire-code tests live; it also runs
cargo fmt --allover44's unformatted PostgreSQL statement-schema test.a536f920fa55finishes the A11 references ingc-42-decisions.md(theheader count and the [upstream issue #379] A Commit refused for disk Capacity leaves its packs in the Store, and every later write is refused Capacity #42 sign-off request name A1–A11 / A8–A11).
Round-2 fix branches on
43e72cf5(nine branches)43e72cf5(pr711-head5) is the Round-1 fixed head. Review Round 2 producednine fix branches, each forked from LayerFS commit
126d2796b7ddwhose tree is43e72cf5(only the.lwwdiffers). Each was exported from the LayerFSproject and re-committed with parent
43e72cf5(.gitand the.lwwdropped), then merged with
--no-ffin this order: 43 → 44 → 45 → 46 → 47 →49 → 50 → 51 → 52. The merges are
5868fb3cef,ebcf856041,11f05dc72c,df5d31ac10,837e04a54d,157f62c285,a956794f2a,06621c7869,aaeed8f900; the integration head before the base sync is3b1aa01ab3.fixr2-43120794668f2c→270998cfefc45868fb3cefHistoryError::UnknownOutcomeOpcarriesUnknownOutcomeState {scope_kind, entity, phase, op},with_unknownattaches it at the call boundary, andunknown::named_unknownfills the{entity:{kind,name}, phase, op}detail (branch/project calls,compact,project.deletehome and worker,remote). F43R2-2gc_intent_ops_{branch,project,store}.sqlsee a live Init and merge as well as the removal/compaction subset (never another mount), and the visibility-guard allow-list reason says so.fixr2-44126f9243c75c→e238710a9164ebcf856041BEGINbatch of a transaction (theAccessShareLockit holds blocks a swap),run_observedkeeps the diagnostic observational; retired backup heads —read_headsfiltersbranch_retired/layer_stack_retired, so a backup digest names the visible history; the inline visibility guard gains the statement-set tests.fixr2-4512fbfc773896→8f20720ebc9811f05dc72cproject.deleteresolves the project only through the Stack its Store lists now: a retired project isNoSuchProject(never finished from a stale home record), and a fresh call that meetsRetired/Absentremoves its sidecar and keeps the entry; §10Busydetail names the recorded Sandbox; F45R2-2/4 a stale record without a sidecar isNoSuchProject;inventoryand the 2.0 manual sayproject.delete.fixr2-4612d9639ebc5e→6d36cc25f2cfdf5d31ac10kept.rs,catalog.rs,history_unreachable_count.sql); a cleared mark's prune is a post-compaction no-op;kept_nodetests.fixr2-47125f89fab84e→98e45d499641837e04a54dproject.remove_fileis project-wide — it checks the path's grammar against every visible tree (main's head and every Branch head), so a Branch-only or never-existing path is accepted.fixr2-491232319559eb→1b7aae1ca1fd157f62c285<store>.compact-oprecord whose owner is provably live is not a leftover: the engine asks the caller's process facts (CompactRequest.owner_live), so a run that released the writer lane but has not settled staysBusyinstead of being adopted; the settle removes the record only while it still names the operation being settled (install/run/op/types), with abefore_settle_hooktest window.fixr2-50123a2ff9b23e→5de0eaf88746a956794f2aCOMMITreply is resolved by the nextstore.compacton a second connection: a recorded owner that no longer holds the exclusive advisory key is terminated (pg_terminate_backend), one that still holds it staysBusy; A8'sgc_opcolumns are completed (op_id,mark_fingerprint,mark_visited); Phase C deletes each project's lineage/derived closure in its per-project transaction and records thecleanupphase before the first batch;postgresql_compactgrows the lost-reply, lineage and phase tests.fixr2-5112a9510a10a9→ec6620a36c4f06621c7869api/node/src/index.d.tsand the Python_client.pyi/aio.pyiare regenerated from the 83-call table;docs/architecture/api/calls/coverage.mdis regenerated;store.md'szero_reasonswording is fixed.fixr2-52124cd7bf9388→31faeb43c8eeaaeed8f900deadBytesRefined == deadBytesBoundon every path, the 50 GB receipt is named in the runbook/report, and thestore-ext4lane is wired in CI.Conflict resolution. Every branch shares
43e72cf5as its three-way base,and the overlapping files auto-merged to the stated ownership; each overlap was
then read semantically:
43+45oncontrol/.../project/delete.rs: 45's fresh/retry split andfail-closed
NoSuchProjectkeep 43'slookup::history_innaming on theadmission and marker-error paths (and 43's
remote.rs/admitdetail).43+44ontests/visibility_guard.rs: 44 owns the statement-set tests;43's only change there is the
gc_intent_ops_project.sqlallow-list reason,which matches 43's widened SQL, so the union is the correct text.
43+47onengine/.../methods/project.rs: 47's project-widevisible_treescheck with 43'shistory_inongc_remove_path.43+49oncontrol/.../store/compact.rs: 49'sowner_livefield with43's
history_in(..., Store, store, None, Some(&op_id)).49+50onengine/.../history/admission.rs: 49'spub(crate) current_namespacewith 50'sstore_key_held_by(the lost-COMMITguard).49+52onsqlite/compact/{mod,types}.rs: 49'sOwnerLiveness/owner_live/before_settle_hookwith 52's A10 module and field docs.50+52docs: the A8 six-columngc_opparagraph (50) and A10 (52)are both kept;
gc-runbook.mdcarries 50's automatic lost-COMMITresolution and 52's A10 paragraph and 50 GB/whole-Store receipts; the
matrix report's AC-52/AC-57 rows carry both.
46'skept.rs/catalog.rsand52's matrix/report/receipts do notoverlap; 51's typings/coverage regenerate from the unchanged 83-call table.
Integration fixups.
3b1aa01ab3moves 45's newfreshargument into theexisting
Finishstruct: the added positional argument pushedmark_and_finishto eight arguments and failedclippy::too_many_argumentsunder-D warnings.d5cc150377regenerates theT12.3a report on the merged tree (the derived counts are
sqlite_compact30,sdk_path_hidden_semantics3,sdk_path_removal5 andpostgresql_compact29).
Base sync to
layerfs_v2_0_0dd4a2cc40f(Ephemeral-AI-Lab#720/Ephemeral-AI-Lab#725/Ephemeral-AI-Lab#726)The integrated head is merged with the current
layerfs_v2_0_0basedd4a2cc40fas commitae0a4c4c2f, a plain merge with no rebase and noforce. The new base carries Ephemeral-AI-Lab#720 (a namespaces command cancelled before its
handle was told ends with
SIGKILL, notSIGTERM), Ephemeral-AI-Lab#725 (per-WorkspaceCPU/memory/pids in
sandbox.get; the Sandbox group holds no process) and Ephemeral-AI-Lab#726(the PostgreSQL engine is always compiled beside SQLite, removing the
postgresqlfeature from every crate).The merge conflicts are all the PR's own additions meeting Ephemeral-AI-Lab#726's cfg
removal:
engine/crates/layerfs-persistence/src/lib.rs: the feat(gc): LayerFS 2.0 GC implementation Ephemeral-AI-Lab/layerfs#711 exports(
compact_metrics,internals,report, the sqlite compact andkept/mark re-exports,
OwnerLiveness) are kept without the removedpostgresqlgates.engine/crates/layerfs-persistence/src/diagnostic/mod.rs:pub(crate) mod postgresql_contractis kept without gates.engine/crates/layerfs-store-service/tests/postgresql_open_cost.rs: theURL-or-keyword
conninfo::with_hostproxy fix is kept without the#![cfg(feature = "postgresql")]header.Because Ephemeral-AI-Lab#726 removes the feature, the PR-side code and harness that still
assumed an opt-in build are adapted in the same merge: the 61 remaining
cfg(feature = "postgresql")gates on Ephemeral-AI-Lab#711 code and tests are removed, thethree
not(feature)stubs (storecompact_postgresql, storecompact_status, admission'spostgres()fallback) are deleted or flattened,tools/ci/test_store_ext4.shstops injecting--features postgresql, and thestore-ext4 workflow comment,
testing.md,limitations.md,examples/README.md,parallel-worktrees.mdand the matrix report generatordescribe the always-built engine.
--features layerfs-persistence/postgresqlis consequently no longer a meaningful gate; every check below runs without
it.
The ancestry below this head is unchanged: the nine Round-2 merges above, the
earlier
base-snap(main329a20be) export, the eightfix711-{contract,pg,life,rm,sqlite,api,perf,hyg}merges, thefix711-{spec,bench,pgtest,perf2}merges, the in-place SQLite/PostgreSQLcompaction split (
7223b8d3f2), the eleven Round-1 fix branches and the foure824e8a7fix branches all remain in history.Against the current base
dd4a2cc40fthe PR is 114 commits, 465 files,+60,349 / −1,409; the final head is
d5cc150377.Round-1 final integration (eight branches on
11c1896a01)This head integrates the eight final fix branches onto the Ephemeral-AI-Lab#711 head
11c1896a01(LayerFS commit1201e45276…). Every branch is forked from thathead; each was exported from the LayerFS project and re-committed with the PR
head as its parent (
.git/.lwwdropped), then merged in order:impl-freshcopy12b4547a932f0b41b16d08fix711-none1244bbb8334c687612b39efix711-report2129c4b59c7b61cb3751423fix711-conc12d812591a5df52e932acafix711-remote1287d209cd48ec1c018588fix711-docs12955309f1130f5b1b4f1cfix711-bench312c1bcb00afbb315ac91ecfix711-install12f9403e810dc80de3bc02Conflict resolutions are semantic, since all eight share the PR head as their
three-way base:
engine/.../sqlite/compact/build.rs(the rebuild isthe fresh-copy load);
fix711-report2'sBuilt.free_bytesremoval is keptwith it, because
backendReusableis now the source Store's pre-markfree-page bytes.
fix711-concwins onhistory/admission.rs(open-description admissionand epoch-row removal locks), preserving
fix711-none'sregister_inin-transaction call from the mark.
fix711-report2wins on the PostgreSQL report (canonicallogicalDiscarded, sourcebackendReusable, resumed-run report), withfix711-remote'smarkBytesmetric kept.fix711-nonewins onsqlite/compact/run.rs(thenonepath's mark andintent are one rolled-back transaction), with
fix711-conc'sBusyprobeand fresh-copy's build integration around it.
fix711-remoteandfix711-docswin on their docs (control/architecturedocs and the call manual respectively).
gc-42-decisions.mdis a union: A8 (fix711-docs), D5 (fix711-remote),G7 (
fix711-install), the fresh-copy D1/D2 rewrite (impl-freshcopy), therollback-only D3 (
fix711-none) and the fixture-basis note(
fix711-bench3); the count is now thirteen places = eight A + five D.gc-runbook.md, the T12.3a matrix report andtools/ci/gc-bench.mdcarrythe same union (D5 row, fresh-copy and old-build capacity numbers, fixture
shape/cold-cache caveat, scratch sizing).
rustfmt on the fresh-copy build and matrix text; the six report/receipt
unit checks moved out of product source into
tests/{sqlite,postgresql}_compact_units.rs(the product-boundary guard).Base sync to
layerfs_v2_0_04a8efe63b3(Ephemeral-AI-Lab#718/Ephemeral-AI-Lab#719/Ephemeral-AI-Lab#724)The head is then merged with the current
layerfs_v2_0_0base4a8efe63b3(Ephemeral-AI-Lab#718 a forced native stop cancels running commands first, Ephemeral-AI-Lab#719 each exec's
helper and command inside their cgroups, Ephemeral-AI-Lab#724
machine.space) as commit5e73795d98, a plain merge with no rebase and no force. The resolution keepsthe GC implementation, the fresh-copy rebuild and the remote/docs work, and
takes the new machine, Sandbox and monitoring code with its docs, CI and
tools/check_product_boundary.pyaudit-note updates. The only files bothsides changed are the call-count files: the merged table is 83 calls in
eleven families (76 GC + six Ephemeral-AI-Lab#712 monitoring +
machine.space). Thework/adminbuckets stay 38/38 (the MCP server lists 76) and the UI-onlyset becomes six;
calls_table, the generated MCP profile table, the coveragematrix and the 2.0.0 manual are at 83. Follow-ups
6daca0bc39and89fdbde155keep themachine.spacemanual rows and the 83-call counts indocs/versioned/2.0.0/calls.md, the calls page, the SDK page inventory andthe calls module. Against base
4a8efe63b3the PR is 68 commits, 430 files,+54,843 / −1,302.
The ancestry below this head is unchanged: the earlier
base-snap(main329a20be) export, the eightfix711-{contract,pg,life,rm,sqlite,api,perf,hyg}merges, the
fix711-{spec,bench,pgtest,perf2}merges, and the in-placeSQLite/PostgreSQL compaction split (
7223b8d3f2) all remain in history.Final fix branches on
e824e8a7(docs4 → clippy4 → opencost4)e824e8a7is a test-only fix on top of89fdbde155: the PG concurrent-removefence now counts live Branches through the
branch_retiredmarker insteadof raw
count(*) FROM branch, so the long-standingpostgresql_gc_fenceexpectation matches the soft-removal design (§4.4).
The last three branches fork from
pr711-head4(LayerFS12d276e2, whosetree is byte-identical to
e824e8a7apart from the LayerFS.lww). Each wasexported and re-committed with parent
e824e8a7(.gitand.lwwdropped), then merged in order; no conflicts — docs vs code vs one test:
fix711-docs45be3f41c07cf6e107804fix711-clippy48b2cf7748edcacc2ddacfix711-opencost4a9b1042da19ab2869cf2fix711-docs4brings the SDK/observe skills and the versioned referencesto 83 calls in eleven families (the new
doctorandlogspages,branch.commits/get_commit/remove,file.revision,sandbox.files,the
machineandstore/backuptables) plus thedocs/pluginsanddocs/versioned/2.0.0README/migration/SDK pages.fix711-clippy4clears the workspace clippy warnings under-D warningsacross 10 files: test assertions, needless
format!/return, theadvisory_holdertype alias, thewrite!owner token, doc-commentformatting.
fix711-opencost4makespostgresql_open_costform-agnostic: it takes theupstream
host:portfrompostgres::Configand rewrites the endpoint withconninfo::with_host, so the proxy accepts a URL or a libpq keywordconnection string.
Round-1 fix branches on
e824e8a7(11 branches)Final counts after this integration: 93 commits (25 on top of
9ab2869c),450 files, +57,881 / −1,358 against the base
4a8efe63b3; final head43e72cf538.Review Round 1 produced eleven fix branches. Each was exported from its LayerFS
branch (
.gitand.lwwdropped) and re-committed with parente824e8a7, then merged with--no-ffin this order: 42 → 43 → 44 → 45 → 47→ 48 → 49 → 50 → 51 → 52 → pginit.
fixr1-425411baa2cab24792Capacity(53100/53200/53400/54000); 53300 isBusy, while 54001/54011/54023 keep a statement verdict. F42-2StorageError::CapacityExceededis a §10 capacity refusal with an explicitPersistenceError::Refusedarm. F42-3 PG Phase C and thenonesettle clear the transient §6.1kept_nodeclosure. F42-4 the SQLite rebuild copiesstore_policyby explicit column list.fixr1-43b798a383da8e19dabranch.removeregisters its retire-phase intent inside the marker transaction, so the intent is visible to the holder check. F43-3 a retried op replaces its predecessor's owner evidence.fixr1-446a29ecbbbf7fb3d1branch/layer_stackstatement, with a documented, proved allow-list; the §4.4 name comment; A9 records that the project-delete retry reads the marker plus the filtered by-id lookup instead of the unshippedlayerstack_stack_by_id_retired.fixr1-450ec97429f2a5b876project.deleteadmission detail carries{kind:project,name}(the name flows throughEffects::admitandDeleteAdmitFrame). F45-2 the same universal guard from the other branch; the merge keeps one test. F45-3risk:deletenamesproject/delete.rs, and the deadproject/remove.rspaths and architecture links are gone.fixr1-47b4d67c91f2730663project.remove_fileresolves against the Store tree and refuses a non-directory; the read path refuses a non-directory component; symlink and hidden-leaf path families are covered through the SDK.fixr1-483a62729ae9ac1984{value, cap}shape reaches the wire. F48-2 the PG rebuild decision uses the mark tuple's stored total (takes_rebuild_path), so a post-mark publication cannot flipnoneto rebuild. F48-3/F48-4 signature-stamp and digest-root cleanup (F50's batched Phase C subsumes these; see below). F48-5 a retained Raw group whose inventory no longer explains its extent is read exactly after a rebuild.fixr1-49ff438425b1534fcffixr1-506d05e2f3c75c2fcffile_digestand exact(slot, stamp)content signatures; the Phase B ownership precheck; thegc_optable fingerprint with the C/I and T/F swap contract; batched Phase C over the persistedgc_deadset; per-test bindings; the catalog-derived scratch name andgc_op.op_idstatus decoding; 1 GiB pack-id batches;ANALYZEon the visited temp tables; scratch+WALpeakTempBytes.fixr1-51bb360f4c0f03b428machine.spaceis named in the machine manual rows; the call tables are rebuilt;store.statusphases and errors are documented; theaiostub is named.fixr1-524317fa89e2c1a99csigkill_at!arms + 8 CLI SIGKILL arms), AC-38 and AC-47 (whole artifact set) are named; the 10 GBcapacity_measureandgc_benchreceipts are committed; thestore-ext4PR base set is stated in AGENTS and the review policy.fixr1-pginitf5720d8ab42ab420init_operation,init_entryorinit_host_filerows (acquisition release owns that cleanup); the regression parks Phase C with a statement trigger and proves the Init survives and completes.Conflict resolution.
fixr1-44andfixr1-45both addengine/crates/layerfs-persistence/tests/visibility_guard.rs. The mergekeeps fixr1-45's version (the superset allow-list wording and the
clippy-clean test), not 44's copy.
fixr1-48vsfixr1-50: F50's batched Phase C (compact/{op,cleanup}.rs,the persisted
gc_deadset, exact(slot, stamp)signatures and root-keyeddigests) subsumes F48-3/F48-4, so the merge keeps F50's
op/cleanupanddoes not re-apply 48's signature/digest fixes. F48-1, F48-2 and F48-5 are
kept, and the tests are unioned.
fixr1-pginitkeeps Phase C's three Init tables untouched on top of F50'sbatching: the settle batch drops only its dead set,
binding_hint_blockandthe transient
kept_node.gc-42-decisions.md; the T12.3a report/matrix carries the regenerated8/30/16/6 counts, 44's AC-27
visibility_guardevidence and 52's committed10 GB receipts.
9ab2869cdocs4/clippy4/opencost4 work is preserved: the twoproject.deleteclippy hunks, theadvisory_holdertype alias, thestandby_satisfiedassert and the URL-or-keywordpostgresql_open_costall remain.
cargo fmt --allreflow (06e965ad), ande629a522splitshistory/objects.rs's inherent impls intohistory/objects/impls.rs(the merged file was 1046 lines, over the999-line product boundary) and points pginit's test at F50's catalog-derived
scratch name.
Docs in this PR
core/docs/issues/project-41/gc-42-decisions.mdgains A8 (§11 reportsemantics:
logicalDiscarded= canonical bytes of the collected objects,shared once, orphans included;
backendReusable= source-Store pre-markfree-page bytes; zero vocabulary adds
no-free-pagesandno-shrink; thepre-release
gc_opcolumns with no additive step), A9 (§4.2: theproject-delete retry resolves the caller's recorded Stack with the
layerstack_deleted_msmarker plus the filteredlayerstack_stack_by_id, sono unfiltered
layerstack_stack_by_id_retiredis shipped), D5 (store.compactis local-only; a sqlite Store pushed to another machine is
Unsupportedbefore any effect and is compacted after
store.pushback tonative) andG7 (the install-only lane stays deferred; the prototype and its 10 GiB
measurements live in
tools/bench/installlane/and change no product code).D1, D3 and D4 remain deviations; D2 is now the settled fresh-copy build
decision; D5 is the new local-only deviation.
deadBytesRefinedequalsdeadBytesBoundon everypath, not only on
none: the mark resolves the whole retained set and §7step 4 runs no second payload sweep, so no refinement exists to report; the
field and its snake_case wire key stay, and a report never names a
different refined figure on either backend).
docs/versioned/2.0.0/calls.mdanddocs/architecture/api/calls/coverage.mdare reconciled to the table:calls 83 in eleven families (main
329a20be76/10 → 83/11; thenew base
4a8efe63b380/11 → 83/11); the manual's family counts and thenine missing call rows are added, with the feat(monitoring): machine and Sandbox logs, files and observed values on one bounded read route (layerfs-project#179) Ephemeral-AI-Lab/layerfs#712/feat(monitoring): machine and Sandbox metrics — processor counters, per-Sandbox rows, memory and pids controllers (layerfs-project#182) Ephemeral-AI-Lab/layerfs#717/feat(monitoring): machine.space — where a machine's data disk goes, and what is not LayerFS's Ephemeral-AI-Lab/layerfs#724 monitoring
sections (processor counters, per-Sandbox
usagerows andmachine.space). The coverage matrix was re-derived from the API calltable and its generated block now has 83 rows in table order.
runbook's deviation table and disk-headroom section carry the fresh-copy
rebuild (old Store + about one new Store; no 2.9× peak).
.agents/skills/layerfs-js-sdkandlayerfs-sdk-observeguides arereconciled to the same 83 calls / eleven families table, with the
doctorandlogsfamily pages added and thebranch,file,machine,sandboxandworkspacepages extended to their current arguments.Test evidence
Final head
7702bb2114(baselayerfs_v2_0_057423382f7, still current; theRound-2 head
d5cc150377, the seven Round-3 mergesfee2ebcc49…02dca44dc1, the five Round-4 mergesdfdd055503…7b58d4eaa1, the sixRound-5 merges
dba16b1f80…c3632887e6, the base syncb41ebabc93and theintegration fixups
6efec405ba,983cbb4ca1,66ff210e83,b357463b6c,c07297303band7702bb2114) was gated on the final tree:cargo check --workspaceexit 0;cargo fmt --all -- --checkexit 0;cargo clippy --workspace --all-targets --no-deps -- -D warningsexit 0;python3 tools/check_product_boundary.py --skip-sdkPASS (1526 productionRust/SQL/JavaScript files). The
postgresqlbuild feature no longer exists(build(store): always compile the PostgreSQL engine beside SQLite Ephemeral-AI-Lab/layerfs#726), so no check passes a feature flag.
layerfs-persistence,layerfs-store-service,layerfs-store): 791 passed / 0 failed / 18ignored (Round-4 head 786). The Round-5 branches add [upstream issue #365] Benchmark bundle: pjdfstest, fsx, fsstress, smallfile, mdtest, fio on the SDK route (v1.0.0) #43's named-Unknown
arms (
project_add_unknown,project_delete) and [upstream pr #504] Use VecDeque for Sandbox log tails #50's PostgreSQLstandby/holder/Phase M/status cases; [upstream pr #505] Area P batch 1: P2, P14, P1, P7, P8 and commit cost accounting fix #49 renames one test and [upstream pr #498] feat(sandbox): 统一 Sandbox provider 接口,收编 Docker / Linux 原生 / macOS 原生(B1/B2/B3,叠在 #427 上) #52
regenerates the report.
sqlite_compact33 +sqlite_compact_crash24 +sqlite_compact_fixtures1 +
sqlite_compact_matrix1 +sqlite_compact_units7 = 66 passed / 0failed / 0 ignored. The T12.3a report is regenerated on the merged tree
(
7702bb2114): its "Verified here" list namesproject_delete(18) andpostgresql_compact(30), the added F43R5-2 and F50R5-2 cases, and itsderived dispositions are executed 8 / covered 30 / partial 16 / deferred 6
over the 60 rows.
calls_table21 +gc_calls8 +gc_trace5 = 34 passed;publish_expected_base9 passed;compact_report_statusis inside thedefault
layerfs-storerun.127.0.0.1:55433(Docker, trust auth), keywordconninfo
host=127.0.0.1 port=55433 user=postgres dbname=postgres sslmode=disable,--test-threads=1: persistence, store, store-service andproject suites = 815 passed, 0 failed, 18
ignored. The new
postgresql_statusPhase M/reader-holder arms, thelegacy
synchronous_standby_namesFIRST-1 units and the swapphysical_beforesampling wording all run.tools/ci/test_store_ext4.sh --quickwithLAYERFS_TEST_TMPDIRon ext4:5/5 suites (83 passed, 0 failed, 1 ignored).
--full(PostgreSQL cases NOT_RUN, as in the earlier evidence): 6/6suites (879 passed, 0 failed, 19 ignored).
--listruns with noLAYERFS_TEST_TMPDIRand exits 0(F52R4-5).
node api/scripts/gen-node-types.mjs --checkexit 0,python3 api/python/tools/gen_stubs.py --checkexit 0, andapi/python/test/test_stubs.py8 tests, OK (6 passed / 2 skipped; mypyabsent).
npm test(api/mcp, Node 24.22.2): 17 passed / 0 failed; theadmin profile lists 76 tools, not 82 (F51R5-1), every generated tool
validates, and the stdout purity and native-addon paths pass.
gc_benchreceipts are committed inengine/crates/layerfs-persistence/tests/fixtures/(fresh-copy 118.440 s vswhole-Store 246.371 s on the same fixture = 2.08x). Both receipts'
pre-head8 build/verify vintage is recorded in
gc-bench.md§13 and AC-57(F52R5-2); the 50 GB arm was not run and no 50 GB Store was created on
this head.
store ext4 (--quick)success; thestore ext4 (--full, PostgreSQL 17)lane is skipped by the workflow. Thefull
layerfs-controlcrate was not rerun in Round 5(its only known failure is the unprivileged FUSE
mount fusectlcase; thetouched
layerfs-storecrate is in the default gate).Known limitations
Store was created on this head. The owner-final 50 GB receipt in the
runbook is the earlier owner measurement; the 30% arm remains
[E].generated fixture on overlayfs, with a warm page cache
(
drop_cachesrefused 6/6) on one declared rotational-media host class.It is a qualification number for that host, not a cross-host guarantee;
the fresh-copy path is 2.08× faster than the old whole-Store
VACUUM INTOon the same fixture (118.440 s vs 246.371 s); the earlier 1.85× mixed two
different generated fixtures and is withdrawn.
(
127.0.0.1:55433, Docker, trust auth); the CIstore-ext4full lane isthe gate. Both
postgresql_open_costconninfo forms pass, andpostgresql_gc_fence::two_concurrent_branch_removes_in_one_project_are_orderedcounts live Branches through the marker, so the two earlier PG failures
are closed here.
partial 16 / deferred 6 over the 60 rows, with no
pendingrow and noSQLite gap. The 6 deferred rows are the PostgreSQL halves (AC-28, AC-35,
AC-41, AC-42, AC-43, AC-60). Partial rows are named in
core/docs/issues/project-41/gc-t12.3a-sqlite-matrix-report.md.a proposal for owner sign-off; the frozen spec is byte-identical. [upstream pr #518] Report mtime and nlink from fs.stat, and mode, mtime and nlink from fs.list #46 (pruning) is deferred
entirely; [upstream pr #498] feat(sandbox): 统一 Sandbox provider 接口,收编 Docker / Linux 原生 / macOS 原生(B1/B2/B3,叠在 #427 上) #52 stays open until the deferred PG rows and the 30%/50 GB arms
run.
Related
branch.publishstale-base guardbranch.publishaccepts a stale base and overwritesmain(lost update) Ephemeral-AI-Lab/layerfs#691, SQLite compaction peak_temp_bytes under-reports the rollback journal Ephemeral-AI-Lab/layerfs#695Ephemeral-AI-Lab/layerfs-project#41–#52core/docs/issues/project-41/gc-42-decisions.md(A1–A11, theA12 reading, D1–D5, G7)
Commits
f4659a13schema, markers and logical removals — layerfs-project#42, [upstream issue #362] Benchmark baseline: 39 Sandbox cost cases at issue359-impl 2d84fe1bb (2026-10-18) #44,[upstream pr #519] Open a Store read-only from StoreService and the SDK #45, [upstream pr #515] docs(512): propose a VM Sandbox backend (Sandboxes as namespaces in one VM per Store) #47
6620be31admission and reachability/closure — layerfs-project#43, [upstream pr #507] #506: Store backends (SQLite + PostgreSQL) and remote Store files (MinIO first) #487c99e7d4SQLite and PostgreSQL compaction — layerfs-project#49, [upstream pr #504] Use VecDeque for Sandbox log tails #50c551cddbreport/status, API/SDK/CLI, qualification — layerfs-project#51,[upstream pr #498] feat(sandbox): 统一 Sandbox provider 接口,收编 Docker / Linux 原生 / macOS 原生(B1/B2/B3,叠在 #427 上) #52
d0ee8fc8,d03c31b6,d26ae548rebase onto upstream fix(publish): refuse stale-base publishes with expected-base CAS (#691) Ephemeral-AI-Lab/layerfs#692/fix(publish): keep UpToDate/NoChanges answers behind the stale-base guard (#691) Ephemeral-AI-Lab/layerfs#693 andpreserve UpToDate/NoChanges
6f2f4c19aanchorbase-snap(main329a20be) behind the PR head8b923165a,d58005fbc,bca8597d3,7d31384be,2788f8fa4,5f4629377,efa91bd6e,ed886a7f7merge the eight fix711 branches47a1e0071,3523db464,2386c350aintegration/test fixups564217c9b3,b0e62a0dab,c8b70d62b2,261f1241bcmerge fix711-spec,fix711-bench, fix711-pgtest and fix711-perf2
7223b8d3f2split the integrated SQLite/PostgreSQL compaction into phasemodules (fix711-split layout; the branch itself is not merged)
12d7be63cf1d,12faa18dac51,12393988624c,125398805245,12f7f3fdf513,12955309f113docs: A8 §11 report amendment, A8 wordingand the 82-call manual/coverage reconciliation
11c1896a01PR feat(gc): LayerFS 2.0 GC implementation Ephemeral-AI-Lab/layerfs#711 head: LAYERFS_TEST_TMPDIR process-kill fix, visited atseal, and the SQLite/PostgreSQL phase-module split
4f9b3f0d73,8cba10f893,f2ed5780f2,14beddb7fa,ab6079d8f2,09f750aba7,8e94b4d76b,0028527c3dexported trees of the eight finalfix branches (each parented on
11c1896a01)0b41b16d08,687612b39e,1cb3751423,f52e932aca,ec1c018588,0f5b1b4f1c,b315ac91ec,c80de3bc02merge them in order:freshcopy → none → report2 → conc → remote → docs → bench3 → install
2c84702132,db5f7ac06d,15a872a3d9fixups: runbook D5 wording,rustfmt, and the product-boundary move of the six report/receipt unit
checks
5e73795d98merge thelayerfs_v2_0_0base4a8efe63b3(fix(machine): a forced native stop cancels running commands first Ephemeral-AI-Lab/layerfs#718/perf(sandbox): create each exec's helper and command inside their cgroups Ephemeral-AI-Lab/layerfs#719/feat(monitoring): machine.space — where a machine's data disk goes, and what is not LayerFS's Ephemeral-AI-Lab/layerfs#724:forced-stop cancellation, per-exec cgroups,
machine.space);6daca0bc39keeps the
machine.spacemanual rows and the 83-call counts;89fdbde155names 83 in the SDK page inventory and the calls modulee824e8a747test(pg): count live Branches after soft removal in theconcurrent-remove fence
5be3f41c07,8b2cf7748e,a9b1042da1exported trees of fix711-docs4,fix711-clippy4 and fix711-opencost4 (each parented on
e824e8a747)cf6e107804,dcacc2ddac,9ab2869cf2merge them in orderdocs4 → clippy4 → opencost4
Round 1 (each exported tree parented on
e824e8a7):5411baa2fixr1-42,b798a38343,6a29ecbb44,0ec9742945,b4d67c9147,3a62729a48,ff43842549,6d05e2f350,bb360f4c51,4317fa8952,f5720d8apginitcab24792,da8e19da,bf7fb3d1,f2a5b876,f2730663,e9ac1984,b1534fcf,c75c2fcf,0f03b428,e2c1a99c,b42ab420merge them inorder 42 → 43 → 44 → 45 → 47 → 48 → 49 → 50 → 51 → 52 → pginit
06e965adfmt reflow after the merges;e629a522splitshistory/objects.rsunder the 999-line boundary and points the pginit testat F50's catalog-derived scratch name;
43e72cf5parks that test on thebatched signature delete
Round 2 (each exported tree parented on
43e72cf5):270998cfefc4fixr2-43,e238710a916444,8f20720ebc9845,6d36cc25f2cf46,98e45d49964147,1b7aae1ca1fd49,5de0eaf8874650,ec6620a36c4f51,31faeb43c8ee525868fb3cef,ebcf856041,11f05dc72c,df5d31ac10,837e04a54d,157f62c285,a956794f2a,06621c7869,aaeed8f900merge them in order43 → 44 → 45 → 46 → 47 → 49 → 50 → 51 → 52;
3b1aa01ab3carries thefreshflag inFinishsomark_and_finishkeeps seven arguments underclippy
ae0a4c4c2fmerges thelayerfs_v2_0_0basedd4a2cc40f(fix(sandbox): a namespaces command cancelled before its handle was told ends with SIGKILL (137), not SIGTERM (layerfs-project#113) Ephemeral-AI-Lab/layerfs#720/feat(sandbox): per-Workspace CPU, memory and pids in sandbox.get — the Sandbox group holds no process Ephemeral-AI-Lab/layerfs#725/build(store): always compile the PostgreSQL engine beside SQLite Ephemeral-AI-Lab/layerfs#726)and adapts the removed
postgresqlfeature;d5cc150377regenerates theT12.3a report on the merged tree
Round 3 (each exported tree parented on
d5cc150377):61212f265e70fixr3-43,05bf7e00d46744,a9cb4c37485245,671eef261ebc46,fe0f790e2b5549,ad5640140a7751,d3b967de7b5d52fee2ebcc49,5ef2ce0153,636469bfb1,87418c8920,30da9172f1,eddf7534ec,02dca44dc1merge them in order 43 → 44 → 45 → 46 → 49 →51 → 52;
719b5b1651restores the frozen spec toed14ed53b, records A11and regenerates the T12.3a report;
3dc3453173moves the Round-3 sourceunit tests into test targets (boundary) and formats 44's test;
a536f920fa55finishes the A11 references
Round 4 (each exported tree parented on
a536f920):0242ff8ed0fixr4-42,355ff7d18343,21b401573e49,3a747da6d950,057e1ffb0e52dfdd055503,3370046f3f,bc811e91ea,19679fd104,7b58d4eaa1merge them in order 42 → 43 → 49 → 50 → 52;
b41ebabc93merges thelayerfs_v2_0_0base57423382f7(nfs: land #637, #713, #714, #722 with review fixes (lp73) Ephemeral-AI-Lab/layerfs#727);6efec405bamoves the Round-4source unit tests into test targets (boundary);
983cbb4ca1fixes clippy'sformat_collectinpostgresql_status;66ff210e83regenerates theT12.3a report;
b357463b6ckeeps the mmap-window check a runtime assertionRound 5 (each exported tree parented on
b357463b):f48dd2c24afixr5-42,bc4f5c815643,b2d0757a9749,d88d4b329350,1494a9e01c51,780421de3d52dba16b1f80,cca723ac36,93853740eb,5654607e3d,d05fe297b1,c3632887e6merge them in order 42 → 43 → 49 → 50 → 51 → 52;c07297303bmoves [upstream issue #365] Benchmark bundle: pjdfstest, fsx, fsstress, smallfile, mdtest, fio on the SDK route (v1.0.0) #43's
listed_unknownhelper under the 999-line product boundary and7702bb2114regenerates the T12.3a report on the merged treeSuggested final commit message: