Repository navigation
Description
Latest Phase 1 scope amendment (2026-09-04)
The user authorized permanent Phase 1 scheduling suppressions under the committed 15-second policy. This supersedes the original full-inventory execution wording below. Keep every case implementation and historical outcome for Phase 2 testing/optimization; suppression is not deletion or a passing result.
The original 130 new timed definitions remain intact. The initial 14 suppressed scenario IDs remove 42 performance samples and their 42 associated verifiers from required Phase 1 scheduling, leaving 116 active new cases / 348 performance samples, plus 25 capped inherited samples. All 29 standalone proof recipes remain required. Any additional single performance sample exceeding 15 seconds of cumulative declared product time permanently suppresses that exact scenario across all seeds/repetitions and future Phase 1 sources. Preparation, standalone proofs and external cleanup are exempt. Still-active correctness, resource, custody and cleanup gates remain unchanged; #21 stays open.
Phase 1 completion amendment (2026-09-04)
The user requires functional failures to be repaired in Phase 1. The committed amendment supersedes earlier baseline-only closure and deferral wording below. Required functional, correctness, integrity, resource, safety-deadline and cleanup gates must pass before PHASE1_TERMINAL_PASS. Optional latency/storage optimization of correct operations remains Phase 2.
Preserve all original baseline failures and producing identities; record corrected candidate evidence separately. Fix shared causes and recollect only genuinely invalidated coverage. Do not shrink fixtures, omit cases, weaken oracles, raise budgets/timeouts or add benchmark-only bypasses. Keep affected family issues and #35 open until corrected execution and independent verification pass; #21 stays open for later phases/release.
Problem statement
v0.1.2 is released. v0.1.3 needs a reproducible whole-Workspace and CAS/CDC baseline before choosing product optimizations. One central roadmap issue tracks the release; scenarios are grouped into one sub-issue per family rather than 130 individual tickets.
Goal and stages
Stage 1 — build and establish initial results: publish/freeze the drafted specifications; extend existing benchmark infrastructure; implement all twelve families; collect the complete initial performance and independent correctness/resource outcomes on a pinned unoptimized product. Review preparation/command wall as well as product timings.
Stage 2 — evidence-driven improvements: review the consolidated baseline, prioritize measured shared root causes, then create focused correctness/performance/storage issues. Preserve the baseline and remeasure identical definitions. Release qualification and publication follow. Do not start Stage 2 tuning while establishing Stage 1.
Keep this central release issue open after Stage 1. Closing a baseline-build child means its benchmark and initial evidence are delivered, not that release gates pass.
Inventory
- 11 performance families: 130 unique new timed cases and three prescribed samples per case, giving 390 initial sample slots.
- CDC boundary proof: one standalone recipe with separate cohorts.
- Workspace reliability: 12 recipes / 28 isolated subcases, including the separately selected 600-second active endurance case.
- Five inherited v0.1.2 families contain 63 definitions/controls. Keep their evidence separate; five oversized growth rows need explicitly versioned capped-result replacements before any capped rerun. Do not relabel old IDs or revive obsolete POSIX/temp-copy rows as SDK evidence.
- One Branch per measured trajectory; bounded retained-Commit history is included. Multi-Branch sharing/fan-out/Add/conflicts stay in v0.1.4. No unsupported crash/power-loss durability claim.
Sub-issues and order
The attached fourteen sub-issues are: shared infrastructure first, twelve independently owned family builds/baselines, then one consolidated baseline review. Family implementation may proceed independently after common contracts are ready; timing measurements remain resource-isolated. All fourteen children are attached using GitHub sub-issue relationships and linked below.
Stage 1 acceptance
- Draft specifications and shared rules are committed/published and linked by exact revision before benchmark implementation or sample collection.
- Shared infrastructure reuses the existing binary/runners/custody; preparation is lazy and repeat runs reuse qualified inputs.
- Every family is implemented and every required case/subcase has an executed, reproducible initial outcome; no missing observations or silent skips.
- Successful cases have the prescribed independent verification; failures, timeouts and capability mismatches remain explicit findings with retained raw evidence.
- Initial reports distinguish harness validity, coverage, product correctness, resource/cleanup status and latency; a failed gate remains failed.
- The consolidated report provides a prioritized root-cause backlog and proposes Stage 2 targets before optimization.
Files to read
docs/roadmap/0.1/0.1.3/README.md— family index and delivery stages.docs/roadmap/0.1/0.1.3/testing-rules.md— Stage 1 and fast execution/preparation rules.docs/general/benchmark_rules.md— authentic routes, frozen identities and evidence policy.
These specifications are currently drafted locally; family sub-issues contain reviewable snapshots. The infrastructure task publishes their committed revisions before admission. Current repository HEAD when drafting: 1e81e9b8cf871324341c221a51b0a0239c580da9.
Stage 1 boundary
Build the benchmark and retain its initial outcomes. Product algorithm, storage-format and performance optimization are deferred to Stage 2. Improve benchmark preparation/reuse and correct harness defects now; do not silently change the measured product to make results pass. Pin the unoptimized product identity and distinguish harness changes from product changes.
A slow result, product failure or timeout is an initial finding, not permission to lower a gate, drop a case or start tuning. Keep outcome, harness validity, verification, resources, cleanup and coverage completeness separate. Failed/invalid cases cannot support performance claims. Unimplemented cases, missing mandatory observations and unexecuted slots remain incomplete.
Shared execution rules
- Reuse
fs-bench-pro, existing generators, runner helpers, custody, qualified pristine inputs, independent writable samples and reports. Add only needed family adapters. - Use 1/10/100/500 of the declared unit; no Cartesian size/count matrix. Each workload file is at most 500 MiB and aggregate logical content is strictly below 1 GiB at every intermediate state, including temporary files and Git internals. Physical Store/spool/cache budgets are separately reported.
- Resolve the selected case/seed before preparation. Reuse compatible artifacts; prepare only required inputs. Measured creation/import/history must still execute all work.
- Iterate one selected case and seed; ordinary warm-prepared 1–5 second command wall is a target to qualify, not a blanket guarantee. Required extended cases are explicitly selected and cannot be omitted from complete baseline coverage.
- Separate performance and verification. Added benchmark hashes, full manifests, reopen and fault injection stay outside performance timing; intrinsic CAS/Git hashing remains measured.
- Collect several metrics from one execution and regenerate reports from raw evidence without rerunning successful workloads. Retain all valid results and failures, with exact source, fixture, oracle, runtime and cache identities.
Stage 1 ownership
| Work | Issue |
|---|---|
| Shared infrastructure and specification freeze | #22 |
| payload creation and random reads | #23 |
| tiny-file operations and bulk churn | #24 |
| directory construction and whole-workspace reads | #25 |
| git workflow in a populated workspace | #26 |
| populated subtree relocation and deletion | #27 |
| workspace change locality | #28 |
| complete agent work episodes | #29 |
| cross-file cas deduplication | #30 |
| cdc locality and resynchronization | #31 |
| incremental workspace content reuse | #32 |
| single-branch history storage growth | #33 |
| workspace reliability and session endurance | #34 |
| Consolidated initial-baseline review | #35 |
Activity
yifanxuaaa commentedon Sep 4, 2026
Phase 1 checkpoint: all 373 active VM8 performance samples are independently qualified, with no invalid rows, product failures, or global report issues. See the published report. Per-case performance and cost updates are posted on every family issue #23–#34; #35 also has the capped inherited results.
Closed with scoped evidence: #22 shared infrastructure, #23 complete payload coverage (24 timings and 24 full proofs), and #26 explicit Git suppression disposition. Other children remain open for required full checks. The latest source binding retains 41 independently qualified full proofs and schedules 341 remaining checks. That campaign is running, and new successes are preserved without repeats. The separate fast profile receives zero exhaustive-verification credit.
The 8-CPU VM retains the 2-CPU/2-GiB per-container caps. Definitions and actual failures remain preserved; the 14 authorized exact-case suppressions remain explicit and are not passing coverage. Full proofs now reuse validated immutable metadata while retaining every path's expected-metadata binding, all file-body reads, and the typed census. See the qualified source binding.
#21 remains open for later phases and release. No global Phase 1 terminal pass is claimed while full checks remain.
yifanxuaaa commentedon Sep 4, 2026
The user has amended Phase 1 acceptance: qualified fast verification is sufficient for routine benchmark verification. The published amendment supersedes earlier comments requiring every routine exhaustive proof before closure. The reference qualification contract preserves independent expected states, affected-content/metadata/alias/absence checks, authenticated references and deterministic witnesses.
The posted performance results remain valid at their original source/environment identities. Completed full proofs remain accepted and are reused. At the scope checkpoint,48 routine slots already had full proofs; the305 unrun routine exhaustive counterparts are now deferred to Phase2 while their qualified fast checks are implemented and collected. Fast evidence keeps its actual assurance and explicit skipped coverage; it will not be labelled
fully_verified.Required targeted reliability checks, the CDC boundary proof, exact expected-error oracles, resource and cleanup gates still apply. Observed failures still require focused repair confirmation. The already-qualified600-second proof is retained rather than repeated. This amendment does not manufacture missing evidence or itself establish terminal pass.
Remaining routine families will close after their accepted fast/full coverage and applicable gates pass. #21 remains open for later phases and release.
yifanxuaaa commentedon Sep 4, 2026
Phase1children#22–#35 are closed following the user’s explicit withdrawal of remaining verification. Central#21 stays OPEN for Phase2/release.
Phase 1 is closed under the user's explicit withdrawal of all remaining verification tests. This is a scope withdrawal, NOT a claim that the original verification terminal criteria passed.
The assistant designed an excessively expensive verification suite and spent too long expanding it after the user requested a much faster finish. That design is withdrawn and must be reassessed in Phase 2. No more verification work is scheduled.
Final performance results:370 independently qualified active observations, with actual source identities and unoptimized timings. All15 suppressed case definitions, original failures and incomplete attempts remain preserved. Completed29 targeted proofs,48 routine full proofs and qualified fast evidence remain historical evidence; unrun verification is withdrawn/deferred, never relabeled passing.
Explicit withdrawal and accountability record. Separate Phase 2 backlog. Central#21 stays open for Phase 2/release. No release tag was created.
yifanxuaaa commentedon Sep 6, 2026
New v0.1.3 infra child: #65 — 20× fewer file writes for
dedup-history-unrelated-{100,500}via versioned 10-file / 1 MiB mixed snapshots (history-unrelated-mixed-v2). Commit depth stays 100/500. Other history kinds unchanged.This is not product Exec/Commit work and does not close #39 / #61 leftover proofs. Mixed-v4 (#62) is already closed.
yifanxuaaa commentedon Sep 6, 2026
#65 landed on origin/main as PR #66 (
2b12825d8):dedup-history-unrelated-{100,500}-mixed-v2collected COMPLETE with sampled-proof PASS. That is benchmark-infra load reduction, not v0.1.3 release qualification. This umbrella stays open. #39 / #46 / #47 / #48 / #50 / #51 stay open. Leftover proofs: #67.yifanxuaaa commentedon Sep 6, 2026
Closing at the maintainer's request to retire the remaining v0.1.3 roadmap scope. Only #50 (Exec overhead) and #51 (upstream fuser) remain active for v0.1.3. Delivered evidence is retained in #53/#54/#63/#65/#66. This administrative closure does not certify every original release gate or publish a release; historical failures, sampled-proof limits and unmet qualification requirements remain recorded.