Repository navigation
Description
Latest Phase 1 scope amendment (2026-09-04)
The user authorized permanent Phase 1 scheduling suppressions under the committed 15-second policy. This supersedes the original full-inventory execution wording below. Keep every case implementation and historical outcome for Phase 2 testing/optimization; suppression is not deletion or a passing result.
The original 130 new timed definitions remain intact. The initial 14 suppressed scenario IDs remove 42 performance samples and their 42 associated verifiers from required Phase 1 scheduling, leaving 116 active new cases / 348 performance samples, plus 25 capped inherited samples. All 29 standalone proof recipes remain required. Any additional single performance sample exceeding 15 seconds of cumulative declared product time permanently suppresses that exact scenario across all seeds/repetitions and future Phase 1 sources. Preparation, standalone proofs and external cleanup are exempt. Still-active correctness, resource, custody and cleanup gates remain unchanged; #21 stays open.
Phase 1 completion amendment (2026-09-04)
The user requires functional failures to be repaired in Phase 1. The committed amendment supersedes earlier baseline-only closure and deferral wording below. Required functional, correctness, integrity, resource, safety-deadline and cleanup gates must pass before PHASE1_TERMINAL_PASS. Optional latency/storage optimization of correct operations remains Phase 2.
Preserve all original baseline failures and producing identities; record corrected candidate evidence separately. Fix shared causes and recollect only genuinely invalidated coverage. Do not shrink fixtures, omit cases, weaken oracles, raise budgets/timeouts or add benchmark-only bypasses. Keep affected family issues and #35 open until corrected execution and independent verification pass; #21 stays open for later phases/release.
Parent: #21
Depends on: #22 (shared infrastructure and committed specification).
Problem statement
LayerFS needs an authentic initial baseline for populated subtree relocation and deletion before deciding product improvements.
Goal and family scope
Move one populated subtree and delete another against a fixed unrelated tree; vary affected subtree size, not the number of top-level mutations.
Inventory: 4 new timed cases, 12 prescribed initial performance sample slots, and 0 standalone proof recipes. The canonical planning file is docs/roadmap/0.1/0.1.3/namespace-mutation.md. The snapshot below makes the current proposal reviewable until its committed revision is published.
Stage 1 work
- Freeze this family and its exact fixture/schedule/oracle identities under the shared contract; implement its complete membership in the existing harness with one canonical definition and thin adapter.
- Qualify route, call counts, size/resource bounds, independent oracles and selected-case self-checks. Resolve harness defects before collecting baseline evidence.
- Run one selected case/seed to establish a valid execution path, then collect every prescribed initial sample and separate proof variant on the pinned unoptimized product. Required extended cases are explicit but remain required.
- Retain raw failures/timeouts as classified product findings. Keep setup, performance, verification, cleanup and external command wall distinct.
- Produce a source-bound family report with all outcomes and measured questions for later optimization; do not tune product algorithms within this issue.
Acceptance criteria
- Complete family implementation, unique IDs, exact counts, deterministic fixtures/schedules and independent oracle checks.
- Committed specification and product/harness/build/runtime/input identities linked in the evidence manifest.
- Every required case/subcase executed with a retained initial outcome; no missing mandatory counter, unimplemented member or silent skipped slot.
- All prescribed valid sample repetitions retained, with separate independent verification of distinct input/schedule variants. Failed/invalid outcomes remain explicit and ineligible for performance claims.
- Preparation reuse, per-sample isolation, byte limits and cleanup checked; actual first-use/cache-hit command wall reported.
- Family baseline summary delivered to the consolidated report issue; known product failures remain failing release gates and are recorded for Stage 2.
Files to read
docs/roadmap/0.1/0.1.3/namespace-mutation.md, docs/roadmap/0.1/0.1.3/testing-rules.md, docs/general/benchmark_rules.md, and the existing source/test references in the specification below.
Stage 1 boundary
Build the benchmark and retain its initial outcomes. Product algorithm, storage-format and performance optimization are deferred to Stage 2. Improve benchmark preparation/reuse and correct harness defects now; do not silently change the measured product to make results pass. Pin the unoptimized product identity and distinguish harness changes from product changes.
A slow result, product failure or timeout is an initial finding, not permission to lower a gate, drop a case or start tuning. Keep outcome, harness validity, verification, resources, cleanup and coverage completeness separate. Failed/invalid cases cannot support performance claims. Unimplemented cases, missing mandatory observations and unexecuted slots remain incomplete.
Shared execution rules
- Reuse
fs-bench-pro, existing generators, runner helpers, custody, qualified pristine inputs, independent writable samples and reports. Add only needed family adapters. - Use 1/10/100/500 of the declared unit; no Cartesian size/count matrix. Each workload file is at most 500 MiB and aggregate logical content is strictly below 1 GiB at every intermediate state, including temporary files and Git internals. Physical Store/spool/cache budgets are separately reported.
- Resolve the selected case/seed before preparation. Reuse compatible artifacts; prepare only required inputs. Measured creation/import/history must still execute all work.
- Iterate one selected case and seed; ordinary warm-prepared 1–5 second command wall is a target to qualify, not a blanket guarantee. Required extended cases are explicitly selected and cannot be omitted from complete baseline coverage.
- Separate performance and verification. Added benchmark hashes, full manifests, reopen and fault injection stay outside performance timing; intrinsic CAS/Git hashing remains measured.
- Collect several metrics from one execution and regenerate reports from raw evidence without rerunning successful workloads. Retain all valid results and failures, with exact source, fixture, oracle, runtime and cache identities.
Planning specification snapshot — namespace-mutation.md
Source file: docs/roadmap/0.1/0.1.3/namespace-mutation.md. Local draft SHA-256: 43f905f8546b9e2beb5e59a960f8a6d3e27d0fe0b469d1512e728880cff3cb08. This snapshot is reviewable now; it is not a committed admission contract. The infrastructure issue must publish/freeze the source revision before implementation and collection.
Populated subtree mutation
Status: v0.1.3 implementation plan; no measurements or passing claim.
Family ID:namespace_mutation. Four timed cases; no separate proofs.
Shared testing rules (docs/roadmap/0.1/0.1.3/testing-rules.md; planning file awaiting publication) own infrastructure, seeds, custody,
timing, verification, size limits, and campaign admission.
Question and scope
What does moving or deleting a large populated subtree cost when a substantial
untouched Workspace must survive exactly? Scale affected subtree size while
holding the background fixed. A directory rename remains one namespace syscall
even when it relocates 100,000 files; do not call that 100,000 rename operations.
This replaces the earlier four small independent mutation mixtures. Creation
belongs to the tiny-file and directory families; single-file moves belong to
Workspace change locality (docs/roadmap/0.1/0.1.3/workspace-change-locality.md; planning file awaiting publication). Link lifetime and
invalid mutations belong to Workspace reliability (docs/roadmap/0.1/0.1.3/workspace-reliability.md; planning file awaiting publication).
Fixture and exact membership
Use a fixed untouched background/ of 100,000 independently generated files,
each 2,500 bytes: exactly 250,000,000 logical bytes. Each of source/tree-a/
and source/tree-b/ contains N shards, each with 200 independently generated
1 KiB files. Prepare empty destination/ before timing. Freeze directory names,
placement, metadata, and whole-tree manifests; N uses nested shard prefixes.
| Scenario ID | N | Files in each affected tree | Initial regular files | Peak logical file bytes |
|---|---|---|---|---|
namespace-subtree-relocate-delete-1 |
1 | 200 | 100,400 | 250,409,600 |
namespace-subtree-relocate-delete-10 |
10 | 2,000 | 104,000 | 254,096,000 |
namespace-subtree-relocate-delete-100 |
100 | 20,000 | 140,000 | 290,960,000 |
namespace-subtree-relocate-delete-500 |
500 | 100,000 | 300,000 | 454,800,000 |
The primary unit is affected subtree shards, not operation count. These are
the only four members. All files are at most 2,500 bytes; the initial state is
the peak, strictly below 1 GiB. The bound includes both affected trees, including
the one subsequently deleted. No temporary copied tree is permitted.
Measured operation
One real-FUSE Workspace and one fresh managed workload execution perform:
- Rename
source/tree-atodestination/moved-a, changing parent and basename
with one native directory rename. Do not copy or traverse A in the workload. - Remove all files and directories under
source/tree-busing a frozen
deterministic postorder traversal and ordinary unlink/rmdir calls. - Complete the declared synchronization and exit before one LayerFS Commit.
Record move and delete phase walls separately, then Commit and End separately.
Use one Branch and one final unpromoted Commit. The expected outcome is
Created; fresh reconnect and full verification are outside performance.
Any metadata normalization needed for the oracle is a declared operation in
the workload phase; it may touch only the declared affected paths.
Independent verification
Derive the final manifest by renaming every A path, removing every B path, and
retaining every other path from the frozen input manifest. Compare the full
reopened Workspace: paths/types, bytes, lengths, modes, normalized timestamps,
and directory structure. Check the moved tree immediately after rename in
verification mode. Verify absence of both old tree bindings and all deleted
descendants, and exact equality of every background file.
Final regular-file count is 100000 + 200*N; final logical bytes are
250000000 + 204800*N. Derive directory counts from the frozen manifests.
Check published head and canonical root against independently qualified
expectations. Report payload object reuse and unchanged subtree identities;
zero inserted payload is the expected result of namespace-only work, not a
claim of zero metadata work. Require complete runtime cleanup.
Measurements, budgets, and grounding
Report actual rename/unlink/rmdir counts, affected/untouched paths, payload
reads/writes, metadata/object reads, candidate and inserted/reused objects/bytes,
transaction maxima, RSS, spool/Store growth, and all phase walls. Moving A and
deleting B have different scaling behavior and remain separately attributed.
The selected-case 1–5 second goal after cached preparation is provisional.
Large deletion and full verification can exceed it; qualify and freeze each
phase's target/hard wall before admission. The complete family contains
12 performance samples, three per case, plus separate verification.
changes.rs:
try_build_localized_candidate,base_manifest, andfinal_manifest
expose the risk of whole-tree work for directory mutations.cow_tree.rs:
rename,unlink, andreplace_path_prefixown live namespace semantics.- Existing mutation tests:
group_3_rename_parent_replace_unlink_and_final_alias_reclamation_are_inode_exact
retains basic correctness without another timed micro-family.
Completion requires all four members, exact full-tree proofs and transient
bounds, unchanged background, cleanup, complete source/fixture/oracle identities,
and qualified pre-admission budgets.
Activity
yifanxuaaa commentedon Sep 4, 2026
Performance-results checkpoint for #27: 9 independently validated active VM8 performance samples. The report validates 0 retained full proof(s) for this family at this checkpoint; further full checks are running. This comment does not claim global Phase1 terminal pass.
Published report, exact rows, source identities and verification status, phase/counter evidence.
VM8 performance source:
7948df2de269e5ffd47a232ffd8091ff83f8869f; fixed 8-CPU Docker VM, unchanged 2-CPU/2-GiB benchmark-container caps. No VM4/VM8 timing pooling. Every listed performance row passed the independent authenticity, custody, observation, resource and cleanup validation; workload admission still requires its separate full proof.Values below are medians across the prescribed three seeds. Product time is the declared cumulative public-operation time; preparation and complete sample wall are separate. Store growth is the observed physical Store-file delta, not a deduplication ratio or verified canonical byte count.
namespace-subtree-relocate-delete-1namespace-subtree-relocate-delete-10namespace-subtree-relocate-delete-100Explicit Phase1 suppressions (not passing coverage):
namespace-subtree-relocate-delete-500. Definitions, historical failures and incomplete attempts remain retained for Phase2.All prior actual failures, source incompatibilities, invalidations and preparation interruptions remain in the linked report and raw evidence. The fast development verifier contributes zero full-verification credit. Completed family tickets will close only after their remaining mandatory full checks pass; #21 remains open for later phases.
yifanxuaaa commentedon Sep 4, 2026
The user has amended Phase 1 acceptance: qualified fast verification is sufficient for routine benchmark verification. The published amendment supersedes earlier comments requiring every routine exhaustive proof before closure. The reference qualification contract preserves independent expected states, affected-content/metadata/alias/absence checks, authenticated references and deterministic witnesses.
The posted performance results remain valid at their original source/environment identities. Completed full proofs remain accepted and are reused. At the scope checkpoint,48 routine slots already had full proofs; the305 unrun routine exhaustive counterparts are now deferred to Phase2 while their qualified fast checks are implemented and collected. Fast evidence keeps its actual assurance and explicit skipped coverage; it will not be labelled
fully_verified.Required targeted reliability checks, the CDC boundary proof, exact expected-error oracles, resource and cleanup gates still apply. Observed failures still require focused repair confirmation. The already-qualified600-second proof is retained rather than repeated. This amendment does not manufacture missing evidence or itself establish terminal pass.
Remaining routine families will close after their accepted fast/full coverage and applicable gates pass. #21 remains open for later phases and release.
yifanxuaaa commentedon Sep 4, 2026
Rename-cache repair performance update for #27: 9 corrected performance samples independently qualified PASS, with zero validation issues or resource violations. These are performance results, not a declaration that every remaining routine verification or the global terminal gate is complete.
Actual runtime source:
e0922904e2bb607a138157755dab9613b441d5b9. All 9 active namespace performance samples now use the corrected source.namespace-subtree-relocate-delete-500remains suppressed; the active tiers are 1, 10 and 100. Ordinary Exec/FUSE performs the frozen subtree relocation and recursive deletion.The repair corrects optimistic Rename cache handling after the hard-link alias proof exposed a stale cached link count. The targeted repaired hard-link proof is independently qualified at this source; its original failed attempt remains preserved. This change required 33 replacement timings across #27/#28/#29. All 33 passed qualification once; the campaign keeps 337 other qualified timings at their original producing sources, for 370 active performance samples. There are still 15 suppressed case IDs, with no new suppression from this collection.
Medians below use the three seeds of each exact corrected case. Different tiers, earlier source revisions and retained samples are not pooled. Pure-call time is the sum of the declared public product phases; preparation is reported separately. Store delta is the physical SQLite Store file-length delta, not a canonical-byte or logical-data measurement.
namespace-subtree-relocate-delete-1e0922904namespace-subtree-relocate-delete-10e0922904namespace-subtree-relocate-delete-100e0922904Evidence: 33-sample qualification checkpoint, exact corrected rows and attempt seals, per-case median data, and sealed repaired hard-link full proof.
Retained evidence: source selections and compatibility predicates, prior qualified VM8 report, and the earlier read-only Pin replacement checkpoint. The older affected 33 timings remain historical observations and are excluded from the corrected medians; no speedup or old/new cost-equivalence claim is made.
The fast-verification amendment permits qualified
fast_iteration_verifiedassurance for routine Phase 1 cases while keeping skipped work explicit. It does not turn fast results intofully_verified. The 29 required targeted proofs are qualified through separate sealed receipts. Remaining routine acceptance, resource, cleanup and source joins still belong to the final report. Suppressed and failed outcomes retain their actual status. Central issue #21 stays open.yifanxuaaa commentedon Sep 4, 2026
Phase 1 is closed under the user's explicit withdrawal of all remaining verification tests. This is a scope withdrawal, NOT a claim that the original verification terminal criteria passed.
The assistant designed an excessively expensive verification suite and spent too long expanding it after the user requested a much faster finish. That design is withdrawn and must be reassessed in Phase 2. No more verification work is scheduled.
Final performance results:370 independently qualified active observations, with actual source identities and unoptimized timings. All15 suppressed case definitions, original failures and incomplete attempts remain preserved. Completed29 targeted proofs,48 routine full proofs and qualified fast evidence remain historical evidence; unrun verification is withdrawn/deferred, never relabeled passing.
Explicit withdrawal and accountability record. Separate Phase 2 backlog. Central#21 stays open for Phase 2/release. No release tag was created.