Repository navigation
Description
Phase 2.1: shared construction, Workspace staging, and efficient initialization
Status: proposed implementation specification, 2026-09-05. Tracked by issue #40, the prerequisite for #38 (scaling) and #39 (runtime-suppressed cases), under roadmap #21. Creating this specification does not claim that its targets have been achieved or launch measurements.
Outcome and scope
Demonstrate the existing namespace-v2 100,000-file / 500,000,000-byte initialization at no more than 2.2 seconds median, using at most eight CPU cores, with lower total CPU work and bounded memory. Extract the useful construction/admission components from namespace initialization and connect only the minimal Workspace staging lifecycle needed to prove the boundary. Add exactly one three-column staging table and short conditional Branch publication.
The shared components must reduce work, not merely rename functions or increase parallelism. They must support the namespace initializer across ordinary input layouts and expose narrow reuse boundaries for later work without benchmark-specific dispatch. Phase 2.1 does not optimize or qualify the #38/#39 families.
This prerequisite owns namespace optimization/refactoring plus the staging foundation. It does not implement, execute, qualify or close #38/#39. After #40 reaches a terminal outcome, review its measured results with the user and separately plan #38 and #39. Neither child is closed automatically when this prerequisite closes.
Source and evidence boundary
- Read the v0.1.1 architecture record and namespace optimization specification.
- Preserve the final Phase 1 observations, suppression policy, and verification withdrawal. The withdrawn verification campaign must not be restarted by this specification.
- Reference experiment: task
01a06b53-f424-7080-98a6-c294549114e0, checkout/Users/yifanxu/Ephemeral-AI-Lab/layerfs-workspace-commit-engine, particularlyinvestigations/workspace-commit/{report,source-ledger,streaming-admission-review}.md. - Revised experimental semantics:
/Users/yifanxu/Ephemeral-AI-Lab/layerfs-bulk-create-feasibility/investigations/bulk-create/no-rollback-streaming-amendment.md. - Pin the actual implementation base and evidence-producing revisions before editing. The inspected experiment had committed HEAD
69bca3950, basea40b17e0, and an uncommitted streaming successor; the final Phase 1 report was published at4c9b14a6b. Reinspect current state, preserve newer functional repairs, and do not copy older complete files over the current implementation. - Retained namespace initialization 2.766279583 s is a selected median; 12.968067459 CPU-s is a resource maximum, not its paired CPU median. The historical initializer used eight host producers and uncontrolled source cache. Its favorable directory layout and decimal-MB fixture differ from the 500-MiB Workspace fixture.
- The existing direct initializer already eliminates its approximately 647-MB canonical payload write and reread. Do not claim that saving a second time. The streaming Workspace successor targets a different approximately 566-MB write/reread boundary and has no qualified streaming performance result in the inspected evidence. Determine remaining test failures from exact source-bound receipts rather than stale summaries.
Acceptance targets
All numerical goals below are prospective Phase 2.1 targets, not measured promises.
| Measure | Acceptance |
|---|---|
Namespace namespace-100000 initialization |
Three prescribed fresh-process samples; median <=2.2 s, including final tree/root/LayerStack publication |
| Namespace initialization CPU | Median total product CPU <=11 CPU-s and <=90% of the matched baseline median; candidate maximum <=matched baseline maximum |
| CPU accounting | User+system CPU across participating product processes; do not double-count overlapping process/child/thread counters |
| CPU capacity | Aggregate allocation <=8 CPU cores across all participating owner processes, including host-side construction/admission; baseline and candidate use the same fixed allocation and placement |
| Workers | One bounded policy across direct/fallback/staging paths; <=8 content producers across the measured owner workload, not eight per concurrent Workspace; use fewer when effective |
| Namespace explicit buffers | <=10 MiB aggregate named construction/admission ownership; report whole-process RSS separately |
| Namespace high-water limits | Initialization incremental native HWM <=128 MiB; complete lifecycle <=256 MiB; preserve the existing 32-MiB SQLite cache target |
| Other runtime/resource limits | Preserve existing applicable Workspace/candidate/index/spool budgets; no swap or OOM; account simultaneous ownership across workers and stages |
| Workload sizes | Each file <=500 MiB and aggregate logical files <1 GiB at every workload state; do not reinterpret namespace-v2's exact decimal bytes |
| Small-case regressions | Disclose every observed latency/CPU regression in affected smaller controls; resolve reproducible regressions or mark qualification inconclusive. No deliberate slowing, automatic tolerance, or extra significance-testing campaign |
| Timing integrity | Workspace performance includes stage + publication + required installation/cleanup. Report these phases separately; never compare publication-only against old complete Commit |
Reuse a truly compatible baseline. If unavailable, collect one controlled baseline once under the selected profile. Do not use the experiment's imbalanced 13–19-second fallback initializer as the baseline for the optimized namespace-v2 path. An eight-core result does not establish a one-/two-core target. No full CPU-count sweep is required.
Destination file and folder structure
Keep the existing crate boundaries. layerfs-content owns canonical algorithms, layerfs-layerstack-store owns persistence and publication, layerfs-workspace owns mutable Workspace state, and fs-bench-pro owns workloads and measurements. Do not add a new crate or a generic backend/plugin layer.
The expected destination is below. new means the file does not exist in the current checkout. adapt experiment means port the relevant implementation onto the pinned current base; never copy a complete older file over later functional repairs.
layerfs/
├── crates/
│ ├── layerfs-content/
│ │ └── src/
│ │ ├── filesystem/
│ │ │ └── change.rs # modify: exact metadata-result helper
│ │ ├── file/rope/
│ │ │ └── build.rs # reuse: CDC/full-content builders
│ │ └── tree/
│ │ ├── batch.rs # new; adapt experiment: sorted final updates
│ │ ├── directory/ # reuse existing codecs and roots
│ │ └── inode/ # reuse existing codecs and roots
│ │
│ ├── layerfs-layerstack-store/
│ │ ├── src/
│ │ │ ├── objects.rs # modify: neutral slabs and checked insertion
│ │ │ ├── construction.rs # new; adapt bounded experiment pool
│ │ │ ├── staging.rs # new: three-column stage lifecycle
│ │ │ ├── layerstack.rs # modify: native discovery/scheduling adapter
│ │ │ ├── workspace.rs # modify: publish and retire staged root
│ │ │ ├── records.rs # modify: minimal internal stage record
│ │ │ ├── schema.rs # modify: v5 create/migrate/verify
│ │ │ ├── statements.rs # modify: register v5/stage SQL
│ │ │ ├── telemetry.rs # modify only for required work/timing fields
│ │ │ └── lib.rs # modify only for necessary internal exports
│ │ ├── sql/
│ │ │ ├── schema/
│ │ │ │ ├── v4.sql # keep unchanged
│ │ │ │ ├── v5.sql # new: fresh six-table Store
│ │ │ │ └── migrate_v4_to_v5.sql # new: additive migration
│ │ │ └── workspace/
│ │ │ ├── insert_stage.sql # new
│ │ │ ├── get_stage.sql # new
│ │ │ ├── delete_stage.sql # new
│ │ │ ├── insert_commit.sql # reuse
│ │ │ └── advance_branch.sql # reuse conditional update
│ │ └── tests/
│ │ ├── v4.rs # retain old-schema interpretation checks
│ │ └── v5.rs # new: one focused migration/staging check
│ │
│ └── layerfs-workspace/
│ └── src/
│ ├── changes.rs # modify: feed final deltas to shared builders
│ ├── commit_file.rs # new; adapt frozen-file experiment adapter
│ ├── lifecycle.rs # modify: stage → publish → continue/close
│ ├── registry.rs # modify: Workspace-local ownership/lease use
│ └── limits.rs # modify only if aggregate accounting needs it
│
├── benchmark/fs-bench-pro/
│ ├── families/ # keep scenario definitions unchanged
│ ├── src/main.rs # reuse namespace measurement; add counters only
│ ├── src/workspace_bench.rs # keep family performance paths unchanged in #40
│ ├── src/workspace_verify.rs # reuse existing bounded/sampled checks
│ ├── src/dedup_verify.rs # reuse existing CAS/CDC transcript checks
│ ├── verify-selected.py # new: one-target, <60-second companion
│ ├── run-namespace.sh # reuse
│ └── run-*.sh # retain; no #38/#39 execution in #40
│
└── docs/roadmap/0.1/0.1.3/
├── phase-2.1-shared-construction-staging-spec.md
├── phase-2.1-handoff-prompt.md # implementation loop and terminal rules
└── phase-2.1-results.md # create only when results exist
Do not create empty scaffolding. A listed new file appears only when its first real caller lands. Keep an experimental support module only when the selected implementation uses it. Adapt the experiment's streaming.rs behavior into staging.rs; do not retain two competing staging/streaming implementations.
Delete superseded duplicate construction, metadata-cache, payload-replay and admission code only after every supported caller has transferred. Keep the canonical fallback until its input domain is covered. Do not delete the test-only append helper for a nonexistent runtime speedup; remove it only if its reference/equivalence role has a replacement.
Optimization mental model
The v0.1.1 gains came from deleting stages and repeated work: construct final state instead of replaying point mutations, avoid duplicate canonical metadata, move owned slabs, carry bounded transactions, and overlap construction with one admission owner. The current 2.766-second namespace path already has metadata reuse, bounded owned slabs and direct admission; it has no large canonical-object spool pass to remove again.
The retained selected-median sample decomposes approximately as follows:
flowchart LR
A["Source scan + canonical construction<br/>overlapped with SQLite admission<br/>2.612 s"] --> B["Final inode/root<br/>0.133 s"]
B --> C["Other initialization<br/>0.022 s"]
C --> D["Visible LayerStack<br/>2.766 s total"]
The 2.2-second target needs about 0.566 seconds /20.5% less complete wall time. If the final tail remains approximately 0.154 seconds, the overlapped pipeline must fall from about 2.612 to about 2.046 seconds—roughly 22%.
flowchart LR
subgraph CURRENT["Retained direct path"]
C1["Pipeline<br/>2.612 s"] --> C2["Tail<br/>≈0.154 s"]
end
subgraph TARGET["Prospective budget"]
T1["Pipeline<br/>≈2.046 s"] --> T2["Tail<br/>≈0.154 s"]
end
CURRENT -->|"remove ≈0.566 s of real work/wait"| TARGET
The sample records about 1.269 seconds of admission-consumer waiting, 0.886 seconds of SQLite row stepping and 0.409 seconds of bounded transaction commits inside the overlapping pipeline. These are diagnostic components, not additive phases. The first investigation asks why the consumer waits and whether grouped construction, fewer intermediate nodes/copies, or better bounded task distribution can feed it earlier while reducing CPU. Do not increase queues/workers merely to hide waiting.
The shared end state is:
flowchart TD
N["Native source discovery"] --> C
W["Frozen Workspace changes"] --> C
C["Existing content builders<br/>+ exact metadata-result reuse"] --> T
U["Authenticated unchanged subtree IDs"] --> T
T["One sorted affected-page tree updater"] --> O
O["Selected owned canonical objects"] --> A
A["Bounded checked admission batches"] --> P
P["Operation-specific publication"] --> L["LayerStack root"]
P --> S["Workspace stage → Commit/Branch → retire"]
This model improves CPU only when counters show less construction, hashing, copying, decoding, transaction or traversal work. Concurrency can reduce wall time without reducing CPU; report both. The staging table and shorter locks improve preservation/concurrency but are not credited as an isolated-operation speedup unless complete stage+publish+finalization measurements improve.
Two required approaches after the feasibility check
Start with one matched baseline and at most two bounded, counter-driven namespace experiments. Estimate whether the required ≈0.566-second wall reduction and matched CPU reduction can come from removable work under the existing canonical/Store contract. Do not run a broad parameter sweep.
The executor must choose and record one of these paths:
flowchart TD
F["Matched baseline + bounded diagnostics"] --> Q{"Is ≤2.2 s with lower CPU<br/>plausible under current contracts?"}
Q -->|"Yes; mechanism and counters support it"| A["Approach A: optimize initializer first"]
A --> A1["Meet/approach target with smallest useful changes"]
A1 --> R["Extract proven shared components"]
Q -->|"No; required work/limits dominate<br/>or bounded hypotheses fail"| B["Approach B: stop specialized tuning"]
B --> B1["Record target MISS and measured constraint"]
B1 --> R
R --> X["Publish reusable boundaries and stop #40"]
Approach A — optimization is plausible
- Optimize the existing namespace path before abstracting it, changing one evidenced mechanism at a time.
- Require the predicted work counter and total CPU to fall; lower wall from added cores alone is insufficient.
- After the mechanism is stable, extract only the implementation that produced the gain into the shared pieces below.
- Remeasure the initializer through its thin adapter, then exercise namespace input-shape controls and the minimal Workspace staging lifecycle.
- Claim 2.2 seconds only after the three prescribed matched samples pass all latency/CPU/resource targets.
Approach B — optimization is not plausible
- Stop spending iterations on the specialized 2.2-second target after the bounded feasibility work; do not force a benchmark-specific shortcut or alter the fixture.
- Publish the target as MISS with the measured lower bound/constraint and attempted hypotheses.
- Refactor the already-proven v0.1.1 mechanisms into the same shared components while preserving the retained initializer performance and CPU profile.
- Publish the narrow reusable boundaries and expected applicability for later [v0.1.3][Phase 2] Fix worse-than-linear scaling in cases below the runtime cutoff #38/[v0.1.3][Phase 2] Optimize and restore the 15 runtime-suppressed benchmark cases #39 planning; do not integrate those family callers in [v0.1.3][Phase 2.1] Share construction and Workspace staging; reach 2.2s initialization with lower CPU #40.
- Do not claim 2.2 seconds or close any family performance gate from the refactor alone.
Both approaches must deliver the namespace refactor, the three-column staging workflow, namespace/input-shape evidence and a deferred applicability note for #38/#39. Approach B prevents the 2.2-second stretch target from blocking useful reusable work indefinitely. It is an honest completed feasibility outcome, not a performance PASS.
Shared implementation: reuse six existing pieces
| Piece | Existing source / extraction | Required change and useful evidence |
|---|---|---|
| Checked insertion | Experimental layerfs-layerstack-store/src/objects.rs::insert_checked_object_batch |
Both callers reuse exact INSERT/conflict checks. Caller retains transaction/publication policy. Outcomes remain provisional until transaction commit |
| Exact metadata results | NativeImport::portable_metadata and experimental Workspace metadata cache |
One small operation/construction-scope helper beside build_portable_metadata; eight exact entries, existing normalization, no global cache. Cache entries may not outlive discarded output |
| Owned slabs and carried batches | Existing object slab/writer and admission code | Share neutral internal types. Reuse <=256-KiB/512-object slabs, bounded queue, <4-MiB/8,191-object admission batches. Do not flush per file/task/slab |
| Final tree updates | Experimental layerfs-content/src/tree/batch.rs directory/inode sorted APIs |
One affected-page algorithm; initialization supplies empty root + all entries, Workspace supplies existing root + sorted final changes. Reuse untouched children |
| Content construction | Existing rope::build, build_bytes, incremental edit machinery and FrozenFile::compile_selected adapter |
Share codecs/builders, not an artificial common mutable file representation. Preserve borrowed extents; do not rechunk an entire file for a small edit |
| Bounded worker ownership | Existing construction pool try_submit, pump, finish |
Reuse workers within an operation, account stacks/buffers/blocked sends, release DB guards before waiting. Consolidate scheduling only where callers actually share behavior |
Keep native source discovery, mutable Workspace state, reconciliation selection, and publication with their owners. No new crate, universal executor, backend plugin framework, public bulk API, persistent preloader, or operation-family selector is required.
Broaden import coverage without moving fixture files:
- Root-level regular files and directories can feed the same bounded construction path. Preserve supported symlink and hard-link semantics.
- A single large directory can supply multiple bounded file tasks; top-level directory count must not determine available content parallelism.
- Replace shape-driven structural fallback with bounded/spilled construction where proven. Keep existing fallback until its supported domain transfers correctly.
- The current >1,000-task preflight already exists; it is an eligibility limit, not an unresolved missing-guard bug.
- Production direct import caps producers at eight; production parallel fallback currently caps at sixteen. Harmonize the live policy. The ten-producer append-only helper is test-only and deleting it is not a runtime optimization.
Before replacing the initializer's tree construction, compare exact same-seed roots and canonical bytes around existing split/balance boundaries. Logical equality alone is insufficient where exact canonical equivalence is required. Never rebuild a whole namespace for a sparse Commit. Retain compact identity/pair spill streams unless measured material; do not replace them with full in-memory manifests.
Exactly one new table
CREATE TABLE workspace_stages (
workspace_id BLOB PRIMARY KEY CHECK (length(workspace_id) = 16),
branch_id BLOB NOT NULL CHECK (length(branch_id) = 17)
REFERENCES branches(branch_id),
root_id BLOB NOT NULL CHECK (length(root_id) = 32)
REFERENCES objects(object_id)
) STRICT, WITHOUT ROWID;Reuse objects, commits, branches, layers, and layer_stacks. No extra indexes, stage/generation IDs, statuses, timestamps, conflict fields, receipt table, lock table, or per-file staging rows.
One row means one complete unpublished candidate. No row is inserted for partial construction. At most one pending stage exists per Workspace: an identical stage can be reused; a different root must not overwrite a retained stage. Freeze/block incompatible mutation while that pending stage awaits publication or explicit discard.
Implement a strict versioned v4→v5 additive migration, accounting for preflight_connect, exact schema verification and the prepared statement manifest. Validate supported v4 before mutation; acquire normal Store ownership; transactionally create the table and update user_version; verify v5 and foreign keys. Preserve all existing identities/rows. Reject malformed or unsupported versions without speculative repair. Existing database journaling/synchronization settings remain unchanged; staging adds no power-loss durability claim.
Stage, publish, then retire
- Under Workspace-local coordination, freeze the submitted state and keep its existing expected head/base/root context in the live operation.
- Reuse unchanged objects; construct new final content with the existing CDC/hash algorithms; admit selected objects through bounded checked CAS batches.
- Flush admission and validate complete selected-root closure. Presence of one root object alone is insufficient, especially with retained failed-attempt CAS rows.
- Persist
workspace_stages(workspace_id, branch_id, root_id)in a committed transaction before attempting Branch publication. - In a short transaction, validate row/owner/destination, use the live expected Branch context, insert/reuse the deterministic Commit, conditionally advance the Branch, and delete the stage. All publication metadata changes commit together.
- Record the exact published result, perform required runtime finalization, then return the result with any finalization error/status. An internal publication function may return to lifecycle code first, but the public completion boundary must not hide remaining required work. No content rechunking, full membership pass, or tree rebuilding belongs inside final publication.
No-op Commit also follows the user's always-stage rule: stage the existing root, check the expected Branch state, delete the stage, and return UpToDate without creating a new history entry. Measure the extra stage transaction cost; do not bypass it to manufacture a faster clean-Commit score.
Publication failure or unexpected head movement retains the complete stage; this attempt makes no Branch change and preserves whichever head is current. It must never undo another publisher. Report an error; do not implement automatic conflict resolution. A failed construction may leave admitted CAS objects without a stage. No object undo journal, GC implementation, or interrupted-build/crash-recovery project belongs here. Preserve exact output, preexisting historical roots, collisions, ordinary errors, and cleanup of owned private resources.
Three-column limitations are intentional: the stage saves a complete candidate, but does not save enough base context for self-contained restart-time reconciliation. There is no durable exactly-once publication receipt after stage deletion. A known live result can be returned again; missing stage after an uncertain outcome is not proof of success or permission to restage against a newer head. Report uncertainty/not-found rather than inventing a result.
Closing versus continuing
Successful publication always retires the staging row. It does not physically remove shared CAS objects.
- Existing SDK Commit-and-continue remains usable: install the exact returned committed snapshot, then accept the next edit/Commit. History and endurance cases must retain their original session/Commit schedules.
- Explicit commit-and-close ends the runtime without constructing a continuing view. Reuse existing end/cleanup primitives; skip installation only when closure is known before that work. Do not silently reinterpret an existing Commit+End benchmark as a different API workload or omit End from timing.
- Closing a live Workspace still releases handles, projections, workers, leases and private spool resources. Stage-row deletion alone is not proof that shutdown is constant time.
- A cleanup failure after successful publication must report the published result plus cleanup failure; it must not report publication rollback or create the Commit twice.
Short ownership boundaries
Use Workspace-local coordination for its own state and short conditional publication for its Branch. A separate Branch mutex map is optional: the existing conditional database transaction may supply the required publication serialization without another locking framework.
Remove the streaming session's lifetime Store permit; acquire only the ownership required for bounded admission and publication. Audit every operation previously protected by the broad gate, including candidate cleanup, before narrowing it. Never wait on producers, private file I/O, or full construction while holding the SQLite connection. Preserve fairness between batches. SQLite writes and the current shared connection remain serialized; this is not a lock-free database claim.
Remove lifetime Branch exclusivity only together with correct Workspace isolation. Fix the postpublication race: current continuing-Workspace installation repins the latest Branch and expects it still to be the caller's Commit. With concurrent Workspaces, pin/install the caller's returned immutable Commit/root instead. Do not let another successful publisher invalidate installation of an already-published result.
An aggregate owner budget bounds all participating workers/buffers. Per-Workspace locks do not authorize one eight-worker pool and full memory allowance for every simultaneous Workspace without accounting.
Deferred issues #38 and #39
Phase 2.1 stops at the reusable namespace components and minimal staging boundary. It does not run the #38 scaling matrix, restore a #39 suppressed case, modify their workload definitions, or claim a family speedup. The prior baseline observations remain useful only for describing likely applicability.
After #40 reaches an Approach A or B terminal outcome, review its actual latency, CPU, memory, construction, admission and input-shape results with the user. Then separately decide which extracted pieces to apply to #38 and #39, what additional live-write/read/Git work they require, and what measurements should close them. Do not pre-commit those implementation plans in #40.
The #40 results document records a short deferred map only:
| Later issue | What #40 hands off | What remains undecided |
|---|---|---|
| #38 | Shared initializer APIs, canonical compatibility, root-file/single-directory behavior, namespace performance/CPU evidence | Which CAS/CDC/payload/Workspace callers change and their complete scaling acceptance |
| #39 | Shared construction/admission APIs and staging semantics | Which live create/delete/read/Git/history causes to change and how to recover all fifteen cases |
#38 and #39 remain blocked by #40 so they do not build competing infrastructure. Their detailed plans begin only after this handoff discussion.
Implementation and measurement sequence
- Pin current baseline, experimental dependencies, workload/profile identities and existing proofs. Inventory exact touched callers. Preserve unrelated edits and current repair commits.
- Run the bounded feasibility check and record Approach A or B under the decision contract above.
- Under Approach A, optimize the existing namespace path and qualify the mechanism before extraction. Under Approach B, record the target MISS and proceed without further specialized tuning.
- Extract checked insertion and exact metadata reuse. Keep wrappers working; establish one small equality/error check, then measure the predicted work-counter change.
- Integrate owned output and bounded final tree updates. Resolve actual source-qualified resource failures and compare canonical split boundaries. No full scans for sparse input.
- Broaden native discovery using namespace-focused root-file-plus-directory and single-large-directory controls. Keep path, bytes, metadata and canonical identities fixed. Do not run CAS/CDC family cases in [v0.1.3][Phase 2.1] Share construction and Workspace staging; reach 2.2s initialization with lower CPU #40.
- Implement schema/staging/publication and short ownership. Preserve continuing sessions and add the explicit closing path only where intended. Reuse the generic engine rather than creating a second implementation.
- Remeasure namespace under the chosen path. Demonstrate the selected namespace input-shape controls and minimal staging lifecycle. Use counter attribution to choose the next change; do not conduct blind worker/cache/SQL parameter sweeps.
- Publish the namespace/staging results and short deferred applicability note. Stop [v0.1.3][Phase 2.1] Share construction and Workspace staging; reach 2.2s initialization with lower CPU #40 and return to the user before planning or executing [v0.1.3][Phase 2] Fix worse-than-linear scaling in cases below the runtime cutoff #38/[v0.1.3][Phase 2] Optimize and restore the 15 runtime-suppressed benchmark cases #39.
Development uses one representative case/seed per hypothesis; proceed to larger work only when the mechanism evidence warrants it. Final affected curves retain their prescribed three seeds, and inherited capped definitions retain their own sample count. Reuse valid baseline and green candidate evidence until a change actually invalidates it. Do not repeat a slow suppressed workload solely to reconfirm its old failure. Phase 1 suppressions remain immutable; any authorized Phase 2 probe has a separate identity and bounded deadline.
Initial useful probes are namespace-10000 before namespace-100000, plus small native root-file/directory and single-large-directory controls. Select them incrementally by the changed path, not as a mandatory suite. Use one focused staging lifecycle fixture for the three-column boundary. Do not run #38/#39 cases.
Selective automatic verification companion
Add one executable benchmark/fs-bench-pro/verify-selected.py beside the family performance scripts. “Parallel to the family script” means a separate companion entry point with the same selectors and evidence identities. It does not run concurrently with performance: simultaneous verification would consume CPU/I/O and contaminate timing.
The companion accepts exactly one explicit target:
verify-selected.py
--family FAMILY
--case CASE
--seed SEED
--source-arm baseline|candidate
--assets PATH
--output PATH
[--verification-certificate PATH | --independent-current]
It rejects --all, ranges, implicit family expansion, missing source/input identity, and more than one target. It resolves membership through the existing registry, dispatches to the existing namespace or Workspace verification implementation, and writes one immutable verification.json receipt. Do not copy family oracles into the wrapper or add one verification script per family.
The receipt records source/harness/product/environment/input identities; family/case/seed; exact checks and sampled paths/ranges; reused proof/certificate identities; omitted coverage; start/end monotonic wall; cleanup; status; and retained evidence path. Expected values must come from the existing independent fixture/oracle, never the candidate output. PASS, FAIL, TIMEOUT, and INCOMPLETE remain distinct.
Every invocation has a hard 59-second end-to-end wall limit, strictly below one minute. The clock begins before setup/authentication and ends after checking, cleanup and receipt publication. Budget work so checking stops early enough to leave cleanup time. Crossing the limit cannot PASS; it records TIMEOUT or INCOMPLETE, preserves useful evidence, performs bounded cleanup, and returns nonzero. A retry requires a new source or a concrete corrected cause and never overwrites the prior receipt.
Use the existing bounded representative verification model:
- authenticate the selected benchmark route, fixture identity, operation counts, phase boundaries and result;
- check the complete staged/canonical root through existing structural receipts where available;
- verify affected content with deterministic sampled files/ranges and the independent expected generator;
- retain exact link/rename/absence/metadata checks only when the selected route changes them;
- run exact CAS/CDC transcript checks for a bounded selected cohort when dedup construction changes;
- confirm resource/cleanup receipts; and
- reuse source-compatible focused product proofs rather than replaying them.
During #40, the handoff loop uses this companion only for namespace targets and a selected Phase 2.1 staging-lifecycle check; the generic selector is retained for later discussion. It invokes the companion automatically only when verification is required by an invalidated route:
- after the first successful selected benchmark on a changed construction/staging/publication path;
- after a correction to an actual verification failure; and
- once on the final stable candidate for each materially distinct affected route selected for qualification.
An unchanged (source, harness, environment, input, case, seed, verification profile) PASS is reused. Do not rerun it after repeated performance samples or unrelated changes. A performance miss with a correct new path may retain its one verification; a functional failure is fixed before broader performance work. Verification failure or timeout triggers diagnosis and a smaller/corrected check—it does not launch a larger suite.
Small, bounded verification
Reuse existing infrastructure and proofs. Performance collection and verification are separate. No per-sample full-file or per-history-Commit FUSE verification, new verification framework, or automatic replay of the withdrawn Phase 1 suite.
The final stable-candidate qualification batch is bounded to <=60 seconds per family and <=600 seconds total, measured through its setup, checking, retries, cleanup and tool-generated reporting (excluding human/agent review idle time). Both limits apply; a retry or relabeling the same attempt does not reset the budget. Development uses separate small, targeted checks with recorded costs; this is not permission to run a full qualification batch at every revision. The supported claim is bounded representative benchmark qualification, not exhaustive filesystem proof. If the necessary check cannot finish, report incomplete evidence and the exact remaining risk; do not silently expand the campaign or call a timeout PASS.
Because the final family budget is at most 60 seconds and each invocation is independently capped at 59 seconds, select at most one final companion invocation per family. Combine checks inside the existing family verifier where necessary; do not multiply setup by spawning more wrappers.
Reuse or add the smallest relevant checks:
- Same-seed canonical tree parity near split boundaries; sparse locality at two namespace sizes.
- Exact/all-unique metadata, same-content duplicates, genuine collision rejection, and nonempty Store behavior.
- Mixed root-file/directory and hard-link semantics; bounded structure on formerly ineligible layouts.
- One staging lifecycle fixture: readable complete stage before publication; success atomically advances/deletes; an injected SQL failure after the stage DELETE rolls back that attempt’s Commit/head/DELETE together; unexpected head movement retains the stage and current head; different-root replacement refused; no-op unchanged history.
- Continuing Workspace installs its own returned Commit even after another Branch publication; second edit/Commit works. Explicit close skips continuing-view work and cleans owned runtime state.
- One bounded concurrency interleaving test: no lifetime Branch exclusion or Store permit across worker waits; measure small-operation wait separately from its execution, without inventing a negligible-lock-time threshold.
- One v4→v5 migration fixture with preserved old roots/history and unchanged data on unsupported/malformed input rejection.
Large performance outputs can be sampled after measurement using retained independent expected data. Report sampled paths/ranges, actual checked scope, reused proof identities, omissions, resource/cleanup outcomes and actual qualification wall. Broader investigation is justified only by a concrete failure, not by automatic matrix multiplication.
Done means
- The bounded feasibility decision names Approach A or B and retains its matched evidence. Approach A meets <=2.2s median, CPU targets and resource bounds under the fixed <=8CPU profile. Approach B records an explicit target MISS and measured constraint without claiming a performance pass.
- Shared pieces are exercised through real native import and Workspace staging, with original workload/SDK/FUSE semantics and sparse locality intact.
- Exactly the three-column staging table is supported in new and migrated Stores; stage→publish→retire and short ownership work as specified.
- Focused evidence supports successful output, historical data, canonical identity, error propagation and cleanup under the intentionally limited recovery contract.
-
verify-selected.pyaccepts one explicit target, reuses existing oracles, rejects bulk selection, produces an immutable receipt and cannot report PASS at or beyond 59 seconds. Automatic handoff use reuses identity-matched PASS results and never overlaps performance timing. - Required namespace/input-shape results and the deferred [v0.1.3][Phase 2] Fix worse-than-linear scaling in cases below the runtime cutoff #38/[v0.1.3][Phase 2] Optimize and restore the 15 runtime-suppressed benchmark cases #39 applicability note are published; no [v0.1.3][Phase 2] Fix worse-than-linear scaling in cases below the runtime cutoff #38/[v0.1.3][Phase 2] Optimize and restore the 15 runtime-suppressed benchmark cases #39 case is marked fixed or qualified.
- Superseded duplicate code is removed only after its supported input domain transfers; no additional framework or benchmark-specialized route remains.
- All selected samples, phase timings, CPU, memory, work counters, scope changes and misses are visible. [v0.1.3][Phase 2] Fix worse-than-linear scaling in cases below the runtime cutoff #38/[v0.1.3][Phase 2] Optimize and restore the 15 runtime-suppressed benchmark cases #39 and roadmap [v0.1.3] Benchmark-first Workspace and CAS/CDC roadmap #21 retain their own completion decisions.
Out of scope: #38/#39 implementation, execution or detailed planning; conflict resolution, persistent generation history/receipts, GC, crash/power-loss recovery, CAS object rollback, alternative DB/FUSE/overlay implementation, Store-object format changes, arbitrary new workers, and release publication. Existing instruction to preserve successful semantics takes precedence over performance targets; later explicit user scope changes supersede this draft.
Implementation handoff prompt
Phase 2.1 implementation handoff prompt
Implement and drive LayerFS Phase 2.1 issue #40 to one of its two defined terminal outcomes. Continue through implementation failures, performance misses and verification failures by diagnosing the measured cause and replanning. Do not stop at a plan, partial refactor, compile success, one favorable timing, or the first failed experiment.
AUTHORITATIVE SCOPE
- GitHub issue: https://github.com/Ephemeral-AI-Lab/layerfs/issues/40
- Local specification: docs/roadmap/0.1/0.1.3/phase-2.1-shared-construction-staging-spec.md
- Dependencies: #40 is a prerequisite of #38 and #39 and a child of #21.
- Read the complete specification before editing. Its latest explicit user scope overrides conflicting older planning language.
- Read docs/roadmap/0.1/0.1.1/architecture_shift.md and namespace-optimization-spec.md for the proved v0.1.1 mechanisms and rejected experiments.
- Read docs/roadmap/0.1/0.1.3/phase-1-verification-withdrawal.md before designing verification. Its failure record is a constraint: do not recreate the withdrawn exhaustive suite.
- The user's 2026-09-05 instruction explicitly authorizes the selective Phase 2.1 verification companion below. It does not restart Phase 1 verification; it supersedes the old “no further verification run” boundary only for these new, bounded Phase 2.1 checks.
- Inspect task 01a06b53-f424-7080-98a6-c294549114e0 and the isolated experiment checkout /Users/yifanxu/Ephemeral-AI-Lab/layerfs-workspace-commit-engine. Reuse selected code only after pinning its source and reconciling it with the current functional repairs.
- Read /Users/yifanxu/Ephemeral-AI-Lab/layerfs-bulk-create-feasibility/investigations/bulk-create/no-rollback-streaming-amendment.md for the authorized recovery limits.
WORKTREE AND OWNERSHIP
Create or use one dedicated Codex worktree on a codex/ branch from the actual current v0.1.3 implementation state. Record the base, experiment revision and uncommitted prototype dependencies before editing. Preserve all unrelated work; never reset, clean, restore or overwrite another agent's changes. Local implementation commits are authorized. Do not merge, push, open a pull request or publish a release unless the user separately authorizes it.
Use subagents for bounded, nonoverlapping review/implementation responsibilities inside #40 only: canonical tree/content construction, Store schema/admission/staging, Workspace lifecycle/concurrency, and namespace performance/resource audit. Assign file ownership and remind every worker that others share the codebase. Serialize builds and performance measurements; do not run competing resource-sensitive commands. The primary agent owns integration, measurements, issue updates and terminal truth.
REQUIRED OUTCOME
Deliver the namespace construction/admission refactor, exactly the three-column workspace_stages foundation, short publication ownership, selective automatic verification, and namespace/input-shape results. Do not implement, run, qualify or plan the #38/#39 family fixes in this task. Use at most eight CPU cores in aggregate across participating owner processes and reduce matched total CPU work rather than buying wall time with more cores.
The namespace target is 100,000 files / 500,000,000 logical bytes in no more than 2.2 seconds median, with median product CPU no more than 11 CPU-seconds and no more than 90% of the matched baseline median. Preserve the other resource, workload and timing boundaries in #40.
FIRST DECISION: APPROACH A OR B
Pin one matched baseline and run at most two bounded, counter-driven namespace experiments. Test whether approximately 0.566 seconds of complete wall and the required CPU reduction plausibly come from removable work under the current canonical and Store contracts. No broad worker, queue, cache or SQL parameter sweep.
Approach A — optimization is plausible:
1. Optimize the existing namespace initializer first, one evidenced mechanism at a time.
2. Require the predicted work counter and total CPU to fall.
3. Stabilize the useful mechanism, then extract it into the shared components.
4. Re-run the thin initializer adapter and exercise namespace input-shape controls plus the minimal Workspace-staging lifecycle. Do not run CAS/CDC benchmark families.
5. Claim the target only after all three prescribed matched samples pass.
Approach B — optimization is not plausible:
1. Stop specialized namespace tuning after the bounded evidence shows required work/limits dominate or the two concrete hypotheses fail.
2. Record the 2.2-second target as MISS with the measured constraint and retained attempts.
3. Still extract the proved v0.1.1 mechanisms and implement the minimal staging foundation. Publish narrow reuse boundaries without integrating #38/#39 callers.
4. Preserve the retained initializer performance/CPU profile; do not invent a benchmark-specific route or begin later family work.
5. Never relabel the refactor as a 2.2-second performance PASS.
Record the chosen approach and evidence in issue #40. A failed Approach A experiment may revise the hypothesis or move to Approach B; it is not terminal until the common deliverables below pass.
IMPLEMENTATION CHECKPOINTS
1. Inventory every caller before moving a shared function. Keep existing wrappers compiling while each caller transfers.
2. Reuse/extract checked object insertion. Preserve exact conflict authentication; caller owns transaction and publication policy.
3. Consolidate the exact eight-entry, operation-scoped metadata-result cache beside the canonical builder. No persistent/global cache.
4. Reuse owned bounded slabs and carried admission batches. Never flush merely because a file, directory, task or worker ends.
5. Integrate the sorted affected-page directory/inode updater. Preserve unchanged child identities and sparse-edit locality. Establish same-seed canonical parity at split/balance boundaries before replacing initializer construction.
6. Broaden native discovery using namespace-focused root-file-plus-directory and single-large-directory controls so they reach the efficient path without changing fixtures. Harmonize live production workers at no more than eight; the test-only ten-worker helper is not a runtime optimization.
7. Add schema v5 and only workspace_stages(workspace_id, branch_id, root_id). Add no status, generation, timestamp, conflict, receipt, lock or per-file tables.
8. Implement stage -> conditional Commit/Branch publication -> stage deletion. Insert a stage only after complete selected-root validation. No-op also stages and retires without creating history.
9. Remove lifetime Store ownership around construction/worker waits. Preserve bounded transactional serialization and fairness. Remove lifetime Branch exclusivity only with Workspace isolation; continuing sessions install the exact returned Commit/root rather than rereading the latest Branch head.
10. Preserve Commit-and-continue for history/endurance. Skip continuing-view installation only on an explicit commit-and-close path. Report publication success plus later cleanup failure accurately.
11. Delete duplicated or superseded code only after its supported input domain transfers. Keep fallback behavior until then. Do not add a new crate, universal executor, public bulk API, backend plugin system or operation-family dispatch.
SELECTIVE AUTOMATIC VERIFICATION
Create exactly one executable companion: benchmark/fs-bench-pro/verify-selected.py. It lives beside the existing family performance scripts and reuses workspace-runner.py, run-namespace.sh, workspace_verify.rs, dedup_verify.rs and existing independent family oracles. Do not create one verification script per family or copy oracle logic into the wrapper.
The script accepts exactly one explicit family/case/seed/source/input selection. It rejects --all, ranges, implicit expansion and missing identities. It produces one immutable verification.json with source/harness/product/environment/input identities, checks, sampled paths/ranges, reused proof identities, omissions, cleanup, monotonic wall, evidence path and distinct PASS/FAIL/TIMEOUT/INCOMPLETE status.
Every verification invocation must finish in strictly less than one minute. Enforce a hard 59-second end-to-end limit beginning before setup/authentication and ending after cleanup and receipt publication. Reserve time for cleanup. At or beyond 59 seconds the invocation cannot PASS; retain TIMEOUT/INCOMPLETE evidence and return nonzero. Do not automatically retry unchanged work.
“Parallel to the benchmark family script” means a separate companion entry point, not simultaneous execution. Never run verification concurrently with performance because it would contaminate CPU/I/O measurements.
Invoke the companion automatically:
- after the first successful selected benchmark on a changed construction/staging/publication route;
- after correcting an actual verification failure; and
- once per materially distinct selected route on the final stable candidate.
Reuse an exact identity-matched verification PASS. Do not rerun it after repeated performance samples or unrelated changes. Verification failure triggers diagnosis and the smallest corrected check, not a larger suite. Fix functional failures before scaling out performance work. Use bounded representative content/range checks and existing structural/CAS/CDC receipts; do not perform per-sample full-file verification or replay every history Commit.
FAST ITERATION
For each hypothesis:
1. State the predicted counter and affected phase.
2. Make the smallest change in the shared cause.
3. Run the smallest relevant build/check.
4. Run one representative benchmark sample only when needed.
5. Run verify-selected.py only when its invalidation rules require it.
6. Record the result and continue, revise or select Approach B.
Never rerun repeatedly passing tests, fixtures, performance samples or verification. Reuse prepared inputs and source-compatible evidence. Keep preparation, verification and reporting outside product timing while still reporting their real wall. Do not run a full workspace test suite during ordinary iteration. Broaden checks once on the stable affected path or after a concrete failure requires it.
Final stable-candidate verification remains at most 60 seconds per family and 600 seconds total. Select at most one final companion invocation per family. A timeout or omitted check remains explicit and cannot PASS.
ISSUE #40 UPDATES
Update issue #40 only at meaningful checkpoints:
- pinned implementation/experiment/evidence identities;
- Approach A/B decision and its evidence;
- each shared component integrated through its real callers;
- staging/schema/ownership path working;
- first source-bound namespace and transferred-family measurements;
- terminal outcome and short deferred applicability note for later user discussion.
Do not post routine command logs. Every update distinguishes measured fact, hypothesis, failure and deferred work. Link immutable evidence paths or commits. Preserve all failed and superseded attempts under their actual identities.
TERMINAL OUTCOMES
APPROACH_A_TERMINAL_PASS requires:
- the complete shared refactor and staging/ownership/verification deliverables;
- three matched namespace-100000 samples with median <=2.2 seconds;
- CPU median <=11 CPU-s and <=90% of matched baseline median, candidate maximum <=matched baseline maximum;
- resource, canonical, staging, publication, cleanup and bounded verification checks passing;
- namespace input-shape controls and the minimal staging lifecycle showing the intended path changes; and
- a short deferred applicability note, without executing or planning #38/#39.
APPROACH_B_REFACTOR_TERMINAL_PASS requires:
- a source-bound report explaining why the 2.2-second/lower-CPU target is not plausible under the bounded experiments and current contracts;
- the complete shared refactor and staging/ownership/verification deliverables;
- retained namespace performance/CPU without an unexplained reproducible regression;
- namespace input-shape controls and the minimal staging lifecycle exercising the shared path with truthful results; and
- a short deferred applicability note, without executing or planning #38/#39. The namespace target remains MISS and no family is called passed from code reuse alone.
Do not stop until one terminal definition is actually satisfied. Compilation alone, one sample, an unresolved correctness/resource failure, a missing selective verifier, or a plan for later is not terminal. If one implementation attempt fails, preserve it, diagnose it and replan. Ask the user only for an actual external decision or unavailable resource that cannot be resolved from the repository and existing authorization.
At terminal, update issue #40 with the chosen outcome, every measured namespace sample, CPU/memory/work counters, verification receipts, source identities, remaining risks and narrow reusable interfaces. Stop and return to the user. Do not execute, plan or close #38/#39; they will be discussed separately after #40. Do not claim release readiness.
Activity
yifanxuaaa commentedon Sep 4, 2026
Phase 2.1 implementation identity checkpoint (2026-09-05)
Measured/pinned facts
/Users/yifanxu/Ephemeral-AI-Lab/layerfs-phase21; branch:codex/phase21-shared-construction.4c9b14a6b489eb6de08d4bfd0d4a723745013ab4(codex/v013-phase1). The source checkout has unrelated uncommitted roadmap/release/diagram work; none was copied or modified.954f2b5c453f1ea0bde052313716cc25230449fbb6a24c067ec73dfd3e7130b2; handoff SHA-256:52c6856ab4475ff36ea0a2bcb49f1eafe5e75c14cce846d795423a91fae34b5e.69bca3950a02f465f36aafb3d93bdf0e9eda991f, base:a40b17e05486e5b747b689e7710475d739556a69. Its uncommitted tracked prototype diff SHA-256 isbde89ea9e9061d360bc5415025acab5c9182be366da8054c9baf49c64f2fb853; untracked Store prototype modules are pinned individually in the local source ledger before reuse.7d78ee7e68373bfcf6ec562f13e07d67b2572372; local file SHA-25668789626c6be122d266e45f09a07fd1a6e34fab4a7bd9b06d00a3eb6475ab7be.Hypothesis, not yet a result
The first matched namespace baseline will determine whether the retained ~1.269 s consumer idle contains removable construction/scheduling work sufficient to save ~0.566 s while reducing total CPU. At most two concrete hypotheses will be tested; no worker/cache/SQL sweep.
Deferred/out of scope
No #38/#39 family is being implemented, run, qualified, or planned here. No merge, push, PR, release, or release-readiness claim is authorized.
yifanxuaaa commentedon Sep 4, 2026
Phase 2.1 feasibility decision: Approach B selected
Measured baseline
Pinned base
4c9b14a6b489eb6de08d4bfd0d4a723745013ab4, unchangednamespace-100000/ 500,000,000-byte fixture, eight-producer direct path. Three-sample native diagnostic evidence:/Users/yifanxu/Ephemeral-AI-Lab/layerfs-phase21-evidence/init_namespace/phase21-baseline-4c9b14a6-matched3. Wall samples: 3.386390708 / 3.422449958 / 4.131761792 s (median 3.422449958). Product CPU: 12.410827625 / 11.407668666 / 11.706084209 CPU-s (median 11.706084209; max 12.410827625). This feasibility evidence is nonterminal and not yet the final fixed-8-core comparison.Experiment 1 — per-worker exact metadata results
Evidence:
/Users/yifanxu/Ephemeral-AI-Lab/layerfs-phase21-evidence/init_namespace/phase21-h1-worker-metadata-100k. Predicted counters were met: misses 2,000→16, canonical frames 439,056→425,168, SQL submissions 423,819→422,433. Result: 3.331898584 s / 12.078214293 CPU-s. Fact: real work fell. Failure: the result missed the explicit ≤2.75 s / ≤11 CPU-s Approach-A continuation threshold.Experiment 2 — preserve authenticated IDs across structural handoff
Evidence:
/Users/yifanxu/Ephemeral-AI-Lab/layerfs-phase21-evidence/init_namespace/phase21-h2-prehashed-structural-100k. Predicted counter was met exactly: canonical hashes 528,165→425,165 while emitted frames remained 425,165. Result: 3.287184166 s / 11.848280792 CPU-s. Fact: 103,000 redundant hashes were removed. Failure: wall and CPU again missed the continuation threshold and remain far from 2.2 s / ≤90% matched CPU.Decision and constraint
The permitted duplicate-work mechanisms are useful and retained, but they cover only small structural hashes and 13,888 duplicate metadata frames, not the unavoidable 500 MB source read, 100,000 opens, 101,001 metadata observations, ~422k unique canonical objects, or corresponding SQLite B-tree writes. The pipeline remains producer/source-bound. The conditional fixed-128 SQL experiment is not authorized by its own prerequisite because complete wall never reached 2.5–2.75 s; it was not run. No SQL/cache/worker sweep will follow. The 2.2-second target is therefore a measured MISS, never a refactor PASS.
Approach B common deliverables continue: shared canonical/admission boundaries, schema v5 with exactly the three-column
workspace_stages, short stage→publish→retire ownership, Workspace exact-result continuation, selective verifier, native input-shape controls, focused staging checks, and final retained-performance/resource evidence. #38/#39 remain unexecuted and unplanned.yifanxuaaa commentedon Sep 4, 2026
Phase 2.1 shared-component checkpoint (measured fact)
Final implementation source is
95578a5e24ac15f38a07535dfdf1fcc9fee80065oncodex/phase21-shared-construction(base4c9b14a6b489eb6de08d4bfd0d4a723745013ab4). Integrated commits:e11abde3f: exact operation-scoped eight-entry metadata-result cache and sorted affected-page directory/inode construction; canonical split/balance and sparse-identity checks passed.a849a1308: checked insertion reuse plus schema v5 with exactlyworkspace_stages(workspace_id, branch_id, root_id)and stage → conditional publish → retire semantics; v4 migration, no-op retirement, stale-head retention and conflict authentication checks passed.30b54c44c: Workspace-local construction ownership, no lifetime Branch/Store gate across worker waits, exact returned Commit/root continuation, retained-stage recovery, and real caller transfer.14b00ca68,9929c27dc,95578a5e2: the single boundedverify-selected.pycompanion, affected-path lint repairs, and explicit proof-only reliability dispatch.Native discovery controls passed for root files plus directories, a 4,000-file single directory on the direct path with canonical parity, and hard-link fallback before publication. Workspace checks passed for owner isolation, publish-success/presentation-failure recovery without recommit, exact returned-snapshot continuation after later publication, and stale-stage freeze/discard.
Final source-bound verifier receipts have been recovered after an external stale cleanup deleted the first evidence paths:
/Users/yifanxu/Ephemeral-AI-Lab/layerfs-p21-terminal-evidence/verification/namespace-100-95578a5e/verification.json, PASS, 10.768865292 s;/Users/yifanxu/Ephemeral-AI-Lab/layerfs-p21-terminal-evidence/verification/workspace-published-presentation-95578a5e/verification.json, PASS, 9.398156292 s.Both invocations are below the hard 59-second end-to-end limit and report cleanup pass. This does not run or qualify any #38/#39 family.
yifanxuaaa commentedon Sep 4, 2026
Terminal outcome:
APPROACH_B_REFACTOR_TERMINAL_PASSMeasured fact: the complete Phase 2.1 refactor, exact staging foundation, short publication ownership, namespace input-shape transfer, resource checks, cleanup, and bounded selective verification are complete. The namespace performance target remains an explicit MISS. This is not a release-readiness claim.
Final product/harness source is
95578a5e24ac15f38a07535dfdf1fcc9fee80065; the source-bound result ledger isd2df5a9d6b0f02e9b88c2a1e0dacf26ac50605a4oncodex/phase21-shared-construction. Base:4c9b14a6b489eb6de08d4bfd0d4a723745013ab4. Final seals: sourced4dbf7f0d3f7171f832438bfaf735806b7f12c46b004c5c69442295e879953ef, product06094354e417d373a4f391104fb3b5f2c2b5c414d9d83385b89d083af6ff6c02, harness28baa5b3b8bbbcd34afb555f83a3ea679c7dc51e1e9bef3cc2e348d3e3b0d2d8.Every 100,000-file / 500,000,000-byte namespace measurement
Wall / product CPU seconds:
3.386390708 / 12.410827625;3.422449958 / 11.407668666;4.131761792 / 11.706084209;3.331898584 / 12.078214293;3.287184166 / 11.848280792;2.526632960 / 6.32;4.114179960 / 7.78;3.593851502 / 6.99;2.668513626 / 7.26;4.974437044 / 6.96;3.660614169 / 7.74;2.573101210 / 6.15;2.546558459 / 6.45;2.502368667 / 6.22;2.249678626 / 5.90;2.536243835 / 6.31;2.442280001 / 6.11;2.527612334 / 6.27;2.635165918 / 6.30;2.539132709 / 6.19;2.564084876 / 6.18;2.592729709 / 6.21;2.497588043 / 6.11.The terminal matched medians are 2.539132709 s / 6.27 CPU-s baseline and 2.564084876 s / 6.18 CPU-s candidate. Candidate wall is +0.98%; CPU is lower by 1.44% but is 98.56% of baseline, not at most 90%. Candidate maximum CPU is 6.21 CPU-s versus baseline 6.30, so the maximum constraint passes. Median wall exceeds 2.2 s by 16.55%. Approach A is therefore not claimable.
The final matched candidate retained the intended work reductions: metadata misses
2,000 → 16; frames439,052–439,073 → 425,159–425,195; hashes542,052–542,073 → exactly the candidate frame count; SQL submissions423,852–423,879 → 422,445–422,479. All samples used eight workers and the direct path, ended with zero active producers, and performed zero object-segment writes/rereads and zero parent merge. Candidate consumer idle remained 231.621–235.465 ms versus baseline 196.324–212.550 ms, explaining why reduced work did not produce the required wall reduction.Resource facts: candidate explicit ownership maximum 9,975,250 B (<10 MiB); initialization incremental HWM maximum 77,033,472 B (<128 MiB); process peak RSS maximum 83,529,728 B; SQLite cache target 32 MiB; admission maximum 6,682 objects / 4,194,297 B; swap/OOM/OOM-kill all zero. Candidate performance is retained without an unexplained reproducible regression.
Functional and selected verification
Integrated real callers now use the exact eight-entry operation cache, sorted affected-page directory/inode updater, checked insertion, three-column v5 stage, conditional publication, and exact returned-snapshot Workspace recovery. Schema v5 contains only
workspace_stages(workspace_id, branch_id, root_id). No-op stages and retires without history; stale publication retains the stage; presentation failure after publication recovers without a second Commit.Input-shape controls passed for root files plus directories, one 4,000-file directory with exact canonical parity, and hard-link fallback before publication. Focused checks passed for all seven sorted batch boundary/locality cases, v4→v5 migration, owner isolation, stale-stage freeze/discard, returned-snapshot continuation after later publication, and publish-success/presentation-failure recovery.
Exactly one companion exists:
benchmark/fs-bench-pro/verify-selected.py./Users/yifanxu/Ephemeral-AI-Lab/layerfs-p21-terminal-evidence/verification/namespace-100-95578a5e/verification.json, SHA-25684f808224b81ae6f37a8d35310454d540a6fab4814ab025e96e2b7be8621a10a, PASS in 10.768865292 s with independent digest, fresh reopen, resource, and cleanup checks;/Users/yifanxu/Ephemeral-AI-Lab/layerfs-p21-terminal-evidence/verification/workspace-published-presentation-95578a5e/verification.json, SHA-256f794bfca072300f4af90a65b9cf5bb44d0095d87b45f26721e0f06e1cd993c2e, PASS in 9.398156292 s with the existing independent Workspace oracle, resource receipt, and cleanup;/Users/yifanxu/Ephemeral-AI-Lab/layerfs-p21-terminal-evidence/evidence.sha256, SHA-256264f1857f102b86039dd5eb35513d6234d8d34f05fcbfb3c25d8e2c7c1e97934.Both invocations are strictly below 59 seconds and explicitly omit exhaustive Phase 1 replay, per-sample full-file verification, and history replay. The complete report, evidence-loss record, source ledger, checks, and narrow interface list are in
docs/roadmap/0.1/0.1.3/phase-2.1-shared-construction-staging-results.mdatd2df5a9d6.Remaining risk is normal source-cache/host/SQLite scheduling variation; it cannot justify relabeling 2.564 s / 98.56% CPU as the required 2.2 s / 90%. The reusable boundaries are limited to the metadata cache, sorted updater, checked insertion, stage API, conditional publication result, and exact-snapshot recovery. Applicability to #38/#39 is deferred for later user discussion. No #38/#39 implementation, execution, qualification, closure, or detailed plan occurred.