From ac8a2baa54f9ff4c6b41266b6c04e71bd8323257 Mon Sep 17 00:00:00 2001 From: YifanXu1999 Date: Sun, 8 Nov 2026 07:21:48 +0800 Subject: [PATCH 1/3] fs-bench-pro: new suite with the export family (E01, E02) A new fs-bench-pro (the earlier one stays archived and frozen). Its first family times `layerfs project export` from a SQLite Store on native, Docker, Lima and Apple Container into a fresh macOS host folder, one sample per command, from an enforced cold Store (guest posix_fadvise via a static tool; native disk-image detach), with an untimed manifest check against pinned fixtures. Receipts stay local under fs-bench-pro/results/. Production LOC: not applicable (benchmark repository; no LayerFS product code). Co-Authored-By: Claude Opus 5.5 --- fs-bench-pro/.gitignore | 3 + fs-bench-pro/AGENTS.md | 104 ++++++++++ fs-bench-pro/README.md | 123 ++++++++++++ fs-bench-pro/environments.example.json | 57 ++++++ fs-bench-pro/src/cases.py | 39 ++++ fs-bench-pro/src/cold.py | 115 ++++++++++++ fs-bench-pro/src/fixture.py | 98 ++++++++++ fs-bench-pro/src/report.py | 61 ++++++ fs-bench-pro/src/run.py | 230 +++++++++++++++++++++++ fs-bench-pro/tests/test_export_family.py | 111 +++++++++++ fs-bench-pro/tools/resident.c | 33 ++++ 11 files changed, 974 insertions(+) create mode 100644 fs-bench-pro/.gitignore create mode 100644 fs-bench-pro/AGENTS.md create mode 100644 fs-bench-pro/README.md create mode 100644 fs-bench-pro/environments.example.json create mode 100644 fs-bench-pro/src/cases.py create mode 100644 fs-bench-pro/src/cold.py create mode 100644 fs-bench-pro/src/fixture.py create mode 100644 fs-bench-pro/src/report.py create mode 100644 fs-bench-pro/src/run.py create mode 100644 fs-bench-pro/tests/test_export_family.py create mode 100644 fs-bench-pro/tools/resident.c diff --git a/fs-bench-pro/.gitignore b/fs-bench-pro/.gitignore new file mode 100644 index 0000000..474ef11 --- /dev/null +++ b/fs-bench-pro/.gitignore @@ -0,0 +1,3 @@ +/results/ +/environments.local.json +/tools/resident-linux-* diff --git a/fs-bench-pro/AGENTS.md b/fs-bench-pro/AGENTS.md new file mode 100644 index 0000000..895d483 --- /dev/null +++ b/fs-bench-pro/AGENTS.md @@ -0,0 +1,104 @@ +# fs-bench-pro agent instructions + +> Status: Current general guide. + +These instructions apply to `fs-bench-pro/` and extend the repository-root +[CLI and cache rules](../AGENTS.md). This file is the suite's current contract; +[cases.py](src/cases.py) pins the cases and expected trees. Reading it does not +start a campaign or authorize repeating completed samples. + +This is a new suite, created on 2026-11-08 at the owner's instruction. The +earlier fs-bench-pro in `.archive/` stays frozen: do not edit, build or run it. + +## Families and cases + +Each family is a case group in [cases.py](src/cases.py) with its own workload +version. Only `export` exists: + +| Case | Work | Fixture | +| --- | --- | --- | +| E01 | `project export` of 10,000 small files, 50 symlinks, 132 folders (46,486,561 bytes) | `small` | +| E02 | `project export` of one 256 MiB file and a README (268,435,480 bytes) | `large` | + +A case exports branch `main` of one project from a SQLite Store on the selected +machine (native, Docker, Lima, Apple Container) into a fresh folder on the +macOS host. Workload version `export-v1` pins the fixtures' generator and +manifests. Changing a fixture, the command or the verifier needs a new +workload version. Add a family by adding its cases, fixtures and verifier +here, not a second runner. + +## Select the product + +- Record the full source commit, branch and whether it is pushed; build the + CLI and the Linux bundle from that one snapshot and record their hashes. +- Use the absolute CLI path from the config file, never `layerfs` on `PATH`. +- Use a private `LAYERFS_HOME` and `XDG_CACHE_HOME`, and + `LAYERFS_CONSTRUCTION_WORKERS=1`. Store profile: Disposable only. + +## Prepare (untimed, through the CLI) + +```text +layerfs store create +layerfs project init /small /small +layerfs project init /large /large +layerfs store push # guests only +``` + +`run.py setup` does this for every machine. Init runs on native from the same +fixture bytes; a guest Store is then moved, not rebuilt. Prepared Stores are +reused across samples because export only reads them; the exported folder is +the sample's output and is removed after verification. + +## Cold control + +A cold claim needs enforcement and evidence on owned resources only. Never +purge the host's or Docker Desktop's shared cache. + +- Guests: `tools/resident.c`, built static and copied to `/tmp/resident` in the + guest, runs `--evict` (fdatasync + `posix_fadvise(DONTNEED)`) on the Store's + own files, then counts their resident pages with mincore. +- Native: the home sits on an owned APFS disk image; detach and re-attach it, + then count the Store's resident pages with mincore. +- Any resident page or a failed control: INELIGIBLE. The control's cost is + overhead, recorded apart from case time. + +Limit to state in every report: the guest control empties the guest's page +cache only. The host may still cache the VM's disk file, which belongs to the +shared VM and is not purged, so a guest's cold state is weaker than native's. + +## Run one sample per command + +```text +python3 src/run.py run --env environments.local.json --case --machine +``` + +- One sample per case per machine; no warm-up, rerun or best-of. A failure + keeps its receipt and is reported. +- Order: E01 then E02; within a case native, Docker, Lima, Apple Container. +- Record the 1-minute load (the runner does) and any build or test another + session runs at the same time. +- The timed export has a 15 s budget, verification 10 s, and every command a + wall limit of at most 120 s. +- A labeled diagnostic (for example a warm export to separate cold read cost) + may follow, outside the receipts, and is reported as a diagnostic. + +## Verdicts + +QUALIFIED needs execution PASS (exit 0 and counts equal to the fixture), +verification PASS (manifest equal to the pinned hash), a shown cold Store and +the export within budget. Otherwise OVER_BUDGET, INELIGIBLE or FAIL. +Receipts record the harness version; a harness change never relabels an +earlier receipt. + +## Report + +Publish `result//fs-bench-pro//report.md` per the root +rules: bold outcome sentence, configuration and identities, one table per case +(native first) from `src/report.py`, observations, limits, failures and +cleanup. Then run `python3 shared/src/readme_index.py`. Receipts stay local. + +## Cleanup + +`run.py cleanup` removes the family's Stores. Remove machines with the CLI, +detach the disk image and delete fixtures and outputs you created; report +anything kept. diff --git a/fs-bench-pro/README.md b/fs-bench-pro/README.md new file mode 100644 index 0000000..ad971dd --- /dev/null +++ b/fs-bench-pro/README.md @@ -0,0 +1,123 @@ +# fs-bench-pro — cold project export from each machine into a host folder +> group: filesystem · measures: speed, correctness + +## Basic info + +| | | +| --- | --- | +| Tool | In-house: Python standard library in [src/](src/), plus a small static C tool for cold-cache control in [tools/](tools/); every LayerFS step uses the public CLI | +| Product | LayerFS 2.0 with `project.export` ([layerfs-project#53](https://github.com/Ephemeral-AI-Lab/layerfs-project/issues/53)), pinned per campaign | +| Machines | macOS host: native, Docker, Lima, Apple Container | +| Store backends | SQLite (Disposable) | +| Families | `export` (E01, E02); more families are added as their own case groups | +| Status | Current general guide. New suite (2026-11-08); the earlier fs-bench-pro is archived in `.archive/` and stays frozen | +| Frozen rules | [AGENTS.md](AGENTS.md) (cases, cold control, verdicts, cleanup); root [CLI and cache rules](../AGENTS.md) | + +Agents must read the root [CLI and cache rules](../AGENTS.md) and the +[suite instructions](AGENTS.md) before preparing or running selections. + +The suite root holds documentation and the configuration example. Python +implementation lives in `src/`, focused checks in `tests/`, and the guest +cache tool's source in `tools/`. [cases.py](src/cases.py) pins the cases and +the expected trees. + +## Overview + +fs-bench-pro measures one product operation per family from a declared, +enforced cold state, with an independent content check. The `export` family +times `layerfs project export / --progress`: +the Store lives on one machine and the exported tree must land, complete and +exact, in a fresh folder on the macOS host. + +| Case | Work | Fixture | +| --- | --- | --- | +| E01 | Export 10,000 small files (46.5 MB), 50 symlinks, 132 folders | `small` | +| E02 | Export one 256 MiB file and a README | `large` | + +Fixtures come from [fixture.py](src/fixture.py): deterministic, incompressible +and distinct per file, so chunk dedupe saves nothing. Their manifest hashes +(path, kind, mode, size, content hash or link target, mtime) are pinned in +[cases.py](src/cases.py). + +One `run` is one sample of one case on one machine: + +1. **Cold control (untimed).** Guests: `resident --evict` syncs and drops the + Store's own files from the guest page cache, then `resident` counts the + pages still resident (mincore). Native: the home lives on an owned APFS disk + image, which is detached and re-attached; mincore then counts the Store's + resident pages. Any resident page, or a failed control, makes the row + INELIGIBLE. +2. **Timed export.** One CLI process; each progress line is stamped as it + arrives. The export wall time is the case time. +3. **Verification (untimed).** The CLI's counts must match the fixture, and the + manifest of the exported folder must equal the pinned hash. No staging + folder may be left behind. +4. **Release.** The exported folder is removed; the receipt is written to + `results//receipt.json` (ignored, local). + +Verdicts: **QUALIFIED** (execution and verification PASS, cold shown, export +within budget), **OVER_BUDGET**, **INELIGIBLE** (cold not shown) or **FAIL**. + +## Setup + +### Prerequisites + +- A LayerFS CLI and Linux bundle built from one clean source snapshot, with + `project export` (` help project export`). +- Ready machines for the guests you select, created through the CLI + (`layerfs machine …`), each with its own SQLite Store. +- An owned APFS sparse disk image mounted where the private `LAYERFS_HOME` + lives, so native cold control never touches a shared volume: + + ```bash + hdiutil create -size 20g -type SPARSE -fs APFS -volname lfs-bench native.sparseimage + hdiutil attach -nobrowse -mountpoint vol native.sparseimage + ``` + +- The guest tool, built once for the guests' architecture: + + ```bash + docker run --rm -v "$PWD/tools:/w" -w /w gcc gcc -O2 -static -o resident-linux-aarch64 resident.c + ``` + +- The fixtures: `python3 src/fixture.py /small small` and + `python3 src/fixture.py /large large`; each prints its manifest, + which must equal the pinned one. + +### Config file + +Copy [environments.example.json](environments.example.json) to +`environments.local.json` (ignored) and fill in absolute paths, the guest exec +prefixes and the campaign's identities. `XDG_CACHE_HOME` points at a private +cache so the bundle never mixes with another session's. + +### One selection + +```bash +python3 src/run.py setup --env environments.local.json +python3 src/run.py run --env environments.local.json --case E01 --machine lima +python3 src/report.py ... +python3 src/run.py cleanup --env environments.local.json +``` + +`setup` creates one Store per machine with both fixtures, pushes guest Stores +and installs the guest tool. `cleanup` removes those Stores; machines, the disk +image and fixtures are released by the caller. + +### Time limits + +The timed export has the 15 s default budget; verification has 10 s. No case +has a declared exception. The cold control's cost (up to 12 s for a disk-image +detach) is enforcement overhead, recorded as `cold.control_s` and never case +time. + +### Focused checks + +```bash +python3 -m unittest discover -s tests -v +``` + +## Past runs + + + diff --git a/fs-bench-pro/environments.example.json b/fs-bench-pro/environments.example.json new file mode 100644 index 0000000..557c624 --- /dev/null +++ b/fs-bench-pro/environments.example.json @@ -0,0 +1,57 @@ +{ + "cli": "/ABSOLUTE/layerfs-checkout/target/release/layerfs", + "home": "/ABSOLUTE/bench-root/vol/home", + "xdg_cache": "/ABSOLUTE/bench-root/xdg-cache", + "fixtures": "/ABSOLUTE/bench-root/fixtures", + "out": "/ABSOLUTE/bench-root/out", + "guest_tool": "/ABSOLUTE/bench-root/tools/resident-linux-aarch64", + "native": { + "store": "s-native", + "image": "/ABSOLUTE/bench-root/native.sparseimage", + "mount": "/ABSOLUTE/bench-root/vol" + }, + "machines": { + "docker": { + "machine": "docker", + "store": "s-docker", + "guest": [ + "docker", + "exec", + "-i", + "lfs--docker" + ] + }, + "lima": { + "machine": "lm1", + "store": "s-lima", + "guest": [ + "limactl", + "shell", + "lfs--lm1", + "sudo" + ] + }, + "apple-container": { + "machine": "ac1", + "store": "s-ac", + "guest": [ + "container", + "exec", + "-i", + "lfs--ac1" + ] + } + }, + "identity": { + "source": "", + "cli_sha256": "", + "bundle_json_sha256": "", + "linux_layerfs_sha256": "", + "profile": "", + "host": "", + "docker": "", + "lima": "", + "apple_container": "", + "store": "" + } +} diff --git a/fs-bench-pro/src/cases.py b/fs-bench-pro/src/cases.py new file mode 100644 index 0000000..b38fd66 --- /dev/null +++ b/fs-bench-pro/src/cases.py @@ -0,0 +1,39 @@ +"""The export family: pinned cases, fixtures and expected results. + +A case exports one committed project (`main`) from a SQLite Store on one +machine to a fresh folder on the macOS host with the public +`layerfs project export / --progress`. +""" + +FAMILY = "export" +WORKLOAD_VERSION = "export-v1" +MACHINES = ("native", "docker", "lima", "apple-container") + +# fixture kind -> expected tree, from fixture.py at WORKLOAD_VERSION +FIXTURES = { + "small": { + "manifest": "8638d2973cfce62d1169e771f9649a35c5d2d78f5f553695c6342b8544e50c1e", + "files": 10000, + "directories": 132, + "symlinks": 50, + "bytes": 46486561, + }, + "large": { + "manifest": "a4735ecdaf8a95a18953f474c5ee616b47eca599769843ac56ccd148b8b5e205", + "files": 2, + "directories": 0, + "symlinks": 0, + "bytes": 268435480, + }, +} + +CASES = { + "E01": {"fixture": "small", "work": "export 10,000 small files (46.5 MB) with 50 symlinks and 132 folders"}, + "E02": {"fixture": "large", "work": "export one 256 MiB file and a README"}, +} + +# complete command (cold control + timed export + verification + cleanup) +BUDGET_S = 15.0 +# declared exceptions, each up to 25 s (none yet) +DECLARED_EXCEPTIONS: dict = {} +VERIFY_BUDGET_S = 10.0 diff --git a/fs-bench-pro/src/cold.py b/fs-bench-pro/src/cold.py new file mode 100644 index 0000000..24ab0d2 --- /dev/null +++ b/fs-bench-pro/src/cold.py @@ -0,0 +1,115 @@ +"""Cold Store pages before a timed export, enforced and shown on owned resources only. + +Guests (docker, lima, apple-container): `resident --evict` fdatasyncs each of the +Store's own files and drops their clean pages with posix_fadvise(DONTNEED), then +`resident` counts the pages still resident with mincore. Nothing shared is +purged: other files, other containers and the Docker Desktop VM keep their cache. +native (macOS): the home sits on an owned APFS disk image; detaching and +re-attaching it drops that volume's cached pages, then mincore counts the Store +files' resident pages. A row whose Store shows any resident page is INELIGIBLE. +""" + +import ctypes +import glob +import mmap +import os +import subprocess +import time +from pathlib import Path + +GUEST_TOOL = "/tmp/resident" +STORE_GLOB = "/layerfs/data/store/{store}/store.sqlite*" + + +def guest_prefix(env: dict, machine: str) -> list: + return env["machines"][machine]["guest"] + + +def guest(env: dict, machine: str, script: str) -> str: + out = subprocess.run( + guest_prefix(env, machine) + ["sh", "-c", script], + capture_output=True, text=True, timeout=30, check=True, + ) + return out.stdout + + +def install_tool(env: dict, machine: str, tool: Path) -> None: + """Copies the static tool into the guest's /tmp (owned, outside every Store).""" + with open(tool, "rb") as data: + subprocess.run( + guest_prefix(env, machine) + ["sh", "-c", f"cat > {GUEST_TOOL} && chmod 755 {GUEST_TOOL}"], + stdin=data, capture_output=True, timeout=30, check=True, + ) + + +def parse(lines: str) -> list: + rows = [] + for line in lines.strip().splitlines(): + resident, total, path = line.split(" ", 2) + rows.append({"path": path, "resident_pages": int(resident), "pages": int(total)}) + return rows + + +def evict_guest(env: dict, machine: str, store: str) -> dict: + files = STORE_GLOB.format(store=store) + script = f"{GUEST_TOOL} --evict {files} >/dev/null && {GUEST_TOOL} {files}" + started = time.monotonic() + rows = parse(guest(env, machine, script)) + return { + "method": "posix_fadvise(DONTNEED) on the Store's own files, then mincore", + "files": rows, + "control_s": round(time.monotonic() - started, 3), + } + + +def _mincore(path: str) -> dict: + """Resident pages of `path` on macOS: libc mmap (read-only, shared) + mincore.""" + size = os.path.getsize(path) + page = mmap.PAGESIZE + pages = (size + page - 1) // page + if pages == 0: + return {"path": path, "resident_pages": 0, "pages": 0} + libc = ctypes.CDLL(None, use_errno=True) + libc.mmap.restype = ctypes.c_void_p + libc.mmap.argtypes = [ctypes.c_void_p, ctypes.c_size_t, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_long] + libc.munmap.argtypes = [ctypes.c_void_p, ctypes.c_size_t] + libc.mincore.argtypes = [ctypes.c_void_p, ctypes.c_size_t, ctypes.c_char_p] + fd = os.open(path, os.O_RDONLY) + try: + addr = libc.mmap(None, size, mmap.PROT_READ, mmap.MAP_SHARED, fd, 0) + if addr in (None, ctypes.c_void_p(-1).value): + raise OSError(ctypes.get_errno(), "mmap") + try: + vec = ctypes.create_string_buffer(pages) + if libc.mincore(addr, size, vec) != 0: + raise OSError(ctypes.get_errno(), "mincore") + resident = sum(1 for b in vec.raw[:pages] if b & 1) + finally: + libc.munmap(addr, size) + finally: + os.close(fd) + return {"path": path, "resident_pages": resident, "pages": pages} + + +def evict_native(env: dict, store: str) -> dict: + image, mount = env["native"]["image"], env["native"]["mount"] + started = time.monotonic() + subprocess.run(["hdiutil", "detach", mount], capture_output=True, timeout=60, check=True) + subprocess.run( + ["hdiutil", "attach", "-nobrowse", "-mountpoint", mount, image], + capture_output=True, timeout=60, check=True, + ) + files = sorted(glob.glob(os.path.join(env["home"], "store", store, "store.sqlite*"))) + if not files: + raise FileNotFoundError(f"no Store files for {store} under {env['home']}") + return { + "method": "detach and re-attach the owned APFS disk image holding the home, then mincore", + "files": [_mincore(f) for f in files], + "control_s": round(time.monotonic() - started, 3), + } + + +def evict(env: dict, machine: str, store: str) -> dict: + evidence = evict_native(env, store) if machine == "native" else evict_guest(env, machine, store) + evidence["cold"] = bool(evidence["files"]) and all(f["resident_pages"] == 0 for f in evidence["files"]) + return evidence diff --git a/fs-bench-pro/src/fixture.py b/fs-bench-pro/src/fixture.py new file mode 100644 index 0000000..9cd4e04 --- /dev/null +++ b/fs-bench-pro/src/fixture.py @@ -0,0 +1,98 @@ +"""Deterministic synthetic export fixtures (no user data). + + python3 fixture.py small|large writes the fixture, prints its manifest hash + python3 fixture.py --manifest prints the manifest hash of a folder + +small: 10,000 files of random bytes (512 B to 16 KiB, log-uniform) in 10 x 10 +folders, 50 symlinks (relative, absolute, dangling), 20 empty folders, every +50th file 0755. large: one 256 MiB random file and a README. Content is +incompressible and distinct per file, so chunk dedupe saves nothing. +""" + +import hashlib +import json +import math +import os +import random +import stat +import sys +from pathlib import Path + +MTIME = 1_700_000_000_123_456_789 + + +def small(root: Path) -> None: + rng = random.Random(53) + n = 0 + for a in range(10): + for b in range(10): + folder = root / f"pkg{a:02d}" / f"mod{b:02d}" + folder.mkdir(parents=True) + for _ in range(100): + size = int(math.exp(rng.uniform(math.log(512), math.log(16384)))) + path = folder / f"f{n:05d}.bin" + path.write_bytes(rng.randbytes(size)) + if n % 50 == 0: + path.chmod(0o755) + n += 1 + for i in range(20): + (root / "empty" / f"e{i:02d}").mkdir(parents=True) + links = root / "links" + links.mkdir() + for i in range(50): + target = ( + f"../pkg{i % 10:02d}/mod00/f{i * 100:05d}.bin" if i % 3 == 0 + else "/etc/hosts" if i % 3 == 1 + else f"missing-{i}" + ) + os.symlink(target, links / f"l{i:02d}") + + +def large(root: Path) -> None: + rng = random.Random(530) + root.mkdir(parents=True, exist_ok=True) + (root / "README").write_text("one 256 MiB random file\n") + with open(root / "blob.bin", "wb") as out: + for _ in range(256): + out.write(rng.randbytes(1 << 20)) + + +def manifest(root: Path) -> str: + """sha256 over sorted (path, kind, mode, size, content sha256 or link target, mtime_ns).""" + rows = [] + for dirpath, dirnames, filenames in os.walk(root, followlinks=False): + for name in sorted(dirnames + filenames): + p = Path(dirpath) / name + st = p.lstat() + rel = str(p.relative_to(root)) + if stat.S_ISLNK(st.st_mode): + rows.append([rel, "l", 0, 0, os.readlink(p), 0]) + elif stat.S_ISDIR(st.st_mode): + rows.append([rel, "d", st.st_mode & 0o7777, 0, "", 0]) + else: + h = hashlib.sha256(p.read_bytes()).hexdigest() + rows.append([rel, "f", st.st_mode & 0o7777, st.st_size, h, st.st_mtime_ns]) + rows.sort() + return hashlib.sha256(json.dumps(rows).encode()).hexdigest() + + +def settle(root: Path) -> None: + """Fixed mtimes on files, then folders bottom-up, so manifests compare.""" + for dirpath, dirnames, filenames in os.walk(root, topdown=False): + for name in filenames: + p = Path(dirpath) / name + if not p.is_symlink(): + os.utime(p, ns=(MTIME, MTIME)) + + +if __name__ == "__main__": + if sys.argv[1] == "--manifest": + print(manifest(Path(sys.argv[2]))) + sys.exit(0) + dest, kind = Path(sys.argv[1]), sys.argv[2] + if dest.exists(): + sys.exit(f"{dest} exists") + dest.mkdir(parents=True) + {"small": small, "large": large}[kind](dest) + settle(dest) + print(manifest(dest)) diff --git a/fs-bench-pro/src/report.py b/fs-bench-pro/src/report.py new file mode 100644 index 0000000..06144c4 --- /dev/null +++ b/fs-bench-pro/src/report.py @@ -0,0 +1,61 @@ +#!/usr/bin/env python3 +"""Markdown tables from export receipts, one table per case, native first. + + report.py RUN_ID... (receipts under results//receipt.json) + +Prints the tables only; the report's outcome, configuration, observations, +limits and cleanup are written by the agent who ran the campaign. +""" + +import json +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) +import cases # noqa: E402 + +RESULTS = HERE.parent / "results" +COLUMNS = ( + "Machine", "Execution", "Verification", "Performance", "Cold pages", + "Export ms", "MB/s", "Files/s", "First event ms", "Cold control s", "Verify s", "Load 1m", +) + + +def load(run_ids: list) -> list: + return [json.loads((RESULTS / run_id / "receipt.json").read_text()) for run_id in run_ids] + + +def cell(value) -> str: + return "—" if value is None else str(value) + + +def row(r: dict) -> list: + files = r["cold"].get("files", []) + resident = f"{sum(f['resident_pages'] for f in files)}/{sum(f['pages'] for f in files)}" if files else "unknown" + m = r["metrics"] + return [ + r["machine"], r["execution"], r["verification"]["status"], r["performance"], resident, + cell(m["wall_ms"]), cell(m["mb_per_s"]), cell(m["files_per_s"]), cell(m["prepare_ms"]), + cell(r["cold"].get("control_s")), cell(r["verification"]["wall_s"]), f"{r['load_1m_before']:.1f}", + ] + + +def tables(receipts: list) -> str: + out = [] + for case_id, case in cases.CASES.items(): + rows = sorted( + (r for r in receipts if r["case"] == case_id), key=lambda r: cases.MACHINES.index(r["machine"]) + ) + if not rows: + continue + out.append(f"## {case_id} — {case['work']}\n") + out.append("| " + " | ".join(COLUMNS) + " |") + out.append("| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |") + out.extend("| " + " | ".join(row(r)) + " |" for r in rows) + out.append("") + return "\n".join(out) + + +if __name__ == "__main__": + print(tables(load(sys.argv[1:]))) diff --git a/fs-bench-pro/src/run.py b/fs-bench-pro/src/run.py new file mode 100644 index 0000000..27ad491 --- /dev/null +++ b/fs-bench-pro/src/run.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +"""fs-bench-pro export family: setup, one sample per invocation, receipts. + + run.py setup --env environments.local.json + run.py run --env environments.local.json --case E01 --machine lima + run.py cleanup --env environments.local.json + +Every LayerFS step is the public CLI (absolute path from the env file). One +`run` is one sample of one case on one machine: cold control (outside the +timer), the timed `project export --progress`, independent verification +(outside the timer), output removal, and an append-only receipt under +results//. Nothing is retried; a failure keeps its receipt. +""" + +import argparse +import json +import os +import shutil +import subprocess +import sys +import threading +import time +import uuid +from pathlib import Path + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) +import cases # noqa: E402 +import cold # noqa: E402 +import fixture # noqa: E402 + +ROOT = HERE.parent +# 2: the budget judges the timed export alone; cold control and verification +# are reported beside it (receipt 20261107T231724Z-4dd18c ran under 1). +# 3: stderr lines that are not progress events are kept verbatim (under 2 a +# final refusal line was parsed and dropped: 20261107T231818Z-c7a8fa). +HARNESS = 3 + + +def load_env(path: str) -> dict: + env = json.loads(Path(path).read_text()) + for key in ("cli", "home", "xdg_cache", "fixtures", "out", "machines", "native"): + if key not in env: + sys.exit(f"{path}: missing `{key}`") + return env + + +def process_env(env: dict) -> dict: + e = dict(os.environ) + e.update( + LAYERFS_HOME=env["home"], + XDG_CACHE_HOME=env["xdg_cache"], + LAYERFS_CONSTRUCTION_WORKERS="1", + ) + return e + + +def cli(env: dict, *args: str, timeout: float = 120) -> dict: + out = subprocess.run( + [env["cli"], *args], capture_output=True, text=True, timeout=timeout, env=process_env(env) + ) + if out.returncode != 0: + raise RuntimeError(f"layerfs {' '.join(args)}: rc={out.returncode} {out.stderr.strip()[:400]}") + return json.loads(out.stdout) if out.stdout.strip() else {} + + +def store_of(env: dict, machine: str) -> str: + return env["machines"][machine]["store"] if machine != "native" else env["native"]["store"] + + +def setup(env: dict) -> None: + """Untimed: one Store per machine holding both fixtures, inited on native, then pushed.""" + for machine in cases.MACHINES: + store = store_of(env, machine) + cli(env, "store", "create", store) + for kind in cases.FIXTURES: + cli(env, "project", "init", f"{store}/{kind}", str(Path(env["fixtures"]) / kind)) + if machine != "native": + cli(env, "store", "push", store, env["machines"][machine]["machine"], timeout=300) + cold.install_tool(env, machine, Path(env["guest_tool"])) + print(json.dumps(cli(env, "store", "list"), indent=1)) + + +def timed_export(env: dict, project: str, dest: Path) -> dict: + """The timed phase: one CLI process; progress lines stamped as they arrive.""" + events, stamps = [], [] + started = time.monotonic_ns() + proc = subprocess.Popen( + [env["cli"], "project", "export", project, str(dest), "--progress"], + stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, env=process_env(env), + ) + + def read_progress() -> None: + for line in proc.stderr: + stamps.append(time.monotonic_ns() - started) + events.append(line.rstrip("\n")) + + reader = threading.Thread(target=read_progress, daemon=True) + reader.start() + try: + stdout, _ = proc.communicate(timeout=cases.BUDGET_S * 4) + except subprocess.TimeoutExpired: + proc.kill() + stdout, _ = proc.communicate() + wall_ns = time.monotonic_ns() - started + reader.join(timeout=5) + progress = [] + for stamp, line in zip(stamps, events): + try: + value = json.loads(line)["progress"] + progress.append({"t_ms": round(stamp / 1e6, 2), "phase": value["phase"], "bytes": value["bytes"]}) + except (ValueError, KeyError, TypeError): + progress.append({"t_ms": round(stamp / 1e6, 2), "line": line[:2000]}) + result = None + try: + result = json.loads(stdout) if stdout.strip() else None + except ValueError: + pass + return { + "rc": proc.returncode, "wall_ms": round(wall_ns / 1e6, 2), "result": result, + "stdout": None if result is not None else stdout[:2000], "progress": progress, + } + + +def phase_ms(progress: list, phase: str, last: bool = False): + hits = [p["t_ms"] for p in progress if p.get("phase") == phase] + return (hits[-1] if last else hits[0]) if hits else None + + +def run(env: dict, case_id: str, machine: str) -> dict: + case = cases.CASES[case_id] + kind = case["fixture"] + expect = cases.FIXTURES[kind] + store = store_of(env, machine) + run_id = time.strftime("%Y%m%dT%H%M%SZ", time.gmtime()) + "-" + uuid.uuid4().hex[:6] + out_dir = Path(env["out"]) / run_id + receipts = ROOT / "results" / run_id + receipts.mkdir(parents=True) + out_dir.mkdir(parents=True) + dest = out_dir / "exported" + started = time.monotonic() + receipt = { + "family": cases.FAMILY, "harness": HARNESS, "workload": cases.WORKLOAD_VERSION, "case": case_id, "machine": machine, + "store": store, "project": f"{store}/{kind}", "fixture": kind, "run_id": run_id, + "identity": env.get("identity", {}), "load_1m_before": os.getloadavg()[0], + } + try: + receipt["cold"] = cold.evict(env, machine, store) + except Exception as error: # the control failed: the row cannot be cold + receipt["cold"] = {"cold": False, "error": str(error)[:400]} + receipt["export"] = timed_export(env, f"{store}/{kind}", dest) + receipt["load_1m_after"] = os.getloadavg()[0] + res = receipt["export"]["result"] or {} + counts_ok = all(res.get(k) == expect[k] for k in ("files", "directories", "symlinks", "bytes")) + receipt["execution"] = "PASS" if receipt["export"]["rc"] == 0 and counts_ok else "FAIL" + v0 = time.monotonic() + if dest.is_dir(): + got = fixture.manifest(dest) + receipt["verification"] = {"status": "PASS" if got == expect["manifest"] else "FAIL", "manifest": got} + else: + receipt["verification"] = {"status": "FAIL", "reason": "no exported folder"} + receipt["verification"]["wall_s"] = round(time.monotonic() - v0, 3) + left = [p.name for p in out_dir.iterdir() if p.name.startswith(".layerfs-export-")] + receipt["staging_left"] = left + shutil.rmtree(out_dir, ignore_errors=True) + receipt["complete_s"] = round(time.monotonic() - started, 3) + receipt["performance"] = verdict(receipt) + wall_s = receipt["export"]["wall_ms"] / 1000 + receipt["metrics"] = { + "wall_ms": receipt["export"]["wall_ms"], + "prepare_ms": phase_ms(receipt["export"]["progress"], "prepare"), + "finalize_ms": phase_ms(receipt["export"]["progress"], "finalize", last=True), + "mb_per_s": round(expect["bytes"] / 1e6 / wall_s, 1) if receipt["execution"] == "PASS" else None, + "files_per_s": round(expect["files"] / wall_s, 1) if receipt["execution"] == "PASS" else None, + } + (receipts / "receipt.json").write_text(json.dumps(receipt, indent=1)) + return receipt + + +def verdict(receipt: dict) -> str: + """QUALIFIED only with execution and verification PASS, a shown cold Store and the budget met. + + The budget judges the timed export command; the cold control's cost is + enforcement overhead, reported as `cold.control_s`, never case time. + """ + if receipt["execution"] != "PASS" or receipt["verification"]["status"] != "PASS": + return "FAIL" + if not receipt["cold"].get("cold"): + return "INELIGIBLE" + budget = cases.DECLARED_EXCEPTIONS.get(f"{receipt['case']}/{receipt['machine']}", cases.BUDGET_S) + if receipt["export"]["wall_ms"] / 1000 > budget or receipt["verification"]["wall_s"] > cases.VERIFY_BUDGET_S: + return "OVER_BUDGET" + return "QUALIFIED" + + +def cleanup(env: dict) -> None: + """Removes the Stores this family made; machines, images and fixtures are the caller's.""" + for machine in cases.MACHINES: + try: + cli(env, "store", "remove", store_of(env, machine), timeout=300) + print("removed", store_of(env, machine)) + except RuntimeError as error: + print("KEPT", store_of(env, machine), error) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("command", choices=("setup", "run", "cleanup")) + parser.add_argument("--env", required=True) + parser.add_argument("--case", choices=sorted(cases.CASES)) + parser.add_argument("--machine", choices=cases.MACHINES) + args = parser.parse_args() + env = load_env(args.env) + if args.command == "setup": + setup(env) + elif args.command == "cleanup": + cleanup(env) + else: + if not (args.case and args.machine): + parser.error("run needs --case and --machine") + receipt = run(env, args.case, args.machine) + m = receipt["metrics"] + print(f"{receipt['run_id']} {args.case} {args.machine}: execution {receipt['execution']}, " + f"verification {receipt['verification']['status']}, performance {receipt['performance']}, " + f"wall {m['wall_ms']} ms, {m['mb_per_s']} MB/s, {m['files_per_s']} files/s, " + f"cold={receipt['cold'].get('cold')}, complete {receipt['complete_s']} s") + + +if __name__ == "__main__": + main() diff --git a/fs-bench-pro/tests/test_export_family.py b/fs-bench-pro/tests/test_export_family.py new file mode 100644 index 0000000..4680ffc --- /dev/null +++ b/fs-bench-pro/tests/test_export_family.py @@ -0,0 +1,111 @@ +"""Focused local checks of the export family's harness; these are not samples.""" + +import importlib.util +import os +import sys +import tempfile +import unittest +from pathlib import Path +from unittest import mock + +SRC = Path(__file__).resolve().parents[1] / "src" +sys.path.insert(0, str(SRC)) + + +def load(name): + spec = importlib.util.spec_from_file_location(f"pro_{name}", SRC / f"{name}.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +fixture, cold, run = load("fixture"), load("cold"), load("run") + + +class ManifestTests(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.root = Path(self.tmp.name) / "t" + self.root.mkdir() + (self.root / "a").write_bytes(b"one") + (self.root / "d").mkdir() + os.symlink("../a", self.root / "d" / "l") + fixture.settle(self.root) + + def test_manifest_is_stable_and_sees_content_mode_and_links(self): + base = fixture.manifest(self.root) + self.assertEqual(base, fixture.manifest(self.root)) + (self.root / "a").write_bytes(b"two") + fixture.settle(self.root) + changed = fixture.manifest(self.root) + self.assertNotEqual(base, changed) + os.chmod(self.root / "a", 0o755) + self.assertNotEqual(changed, fixture.manifest(self.root)) + os.remove(self.root / "d" / "l") + os.symlink("a", self.root / "d" / "l") + self.assertNotEqual(changed, fixture.manifest(self.root)) + + +class ColdTests(unittest.TestCase): + def test_any_resident_page_or_no_file_is_not_cold(self): + lines = "0 10 /s/store.sqlite\n3 4 /s/store.sqlite-wal\n" + with mock.patch.object(cold, "evict_guest", return_value={"files": cold.parse(lines)}): + self.assertFalse(cold.evict({}, "docker", "s")["cold"]) + with mock.patch.object(cold, "evict_guest", return_value={"files": cold.parse("0 10 /s/store.sqlite\n")}): + self.assertTrue(cold.evict({}, "docker", "s")["cold"]) + with mock.patch.object(cold, "evict_guest", return_value={"files": []}): + self.assertFalse(cold.evict({}, "docker", "s")["cold"]) + + def test_mincore_counts_pages_of_a_file_just_read(self): + with tempfile.NamedTemporaryFile() as f: + f.write(os.urandom(1 << 20)) + f.flush() + Path(f.name).read_bytes() + got = cold._mincore(f.name) + self.assertGreater(got["pages"], 0) + self.assertGreater(got["resident_pages"], 0) + + +def receipt(**over): + r = { + "case": "E01", "machine": "docker", "execution": "PASS", + "verification": {"status": "PASS", "wall_s": 0.5}, + "cold": {"cold": True, "control_s": 30.0}, "export": {"wall_ms": 1000.0}, + } + r.update(over) + return r + + +class VerdictTests(unittest.TestCase): + def test_verdicts(self): + self.assertEqual(run.verdict(receipt()), "QUALIFIED") # a slow control is not case time + self.assertEqual(run.verdict(receipt(execution="FAIL")), "FAIL") + self.assertEqual(run.verdict(receipt(verification={"status": "FAIL", "wall_s": 0})), "FAIL") + self.assertEqual(run.verdict(receipt(cold={"cold": False})), "INELIGIBLE") + self.assertEqual(run.verdict(receipt(export={"wall_ms": 15001.0})), "OVER_BUDGET") + self.assertEqual(run.verdict(receipt(verification={"status": "PASS", "wall_s": 10.5})), "OVER_BUDGET") + + +class TimedExportTests(unittest.TestCase): + def test_progress_is_stamped_and_a_refusal_line_is_kept(self): + with tempfile.TemporaryDirectory() as tmp: + cli = Path(tmp) / "layerfs" + cli.write_text( + "#!/bin/sh\n" + "echo '{\"progress\":{\"phase\":\"prepare\",\"bytes\":0}}' >&2\n" + "echo '{\"progress\":{\"phase\":\"export\",\"bytes\":7}}' >&2\n" + "echo '{\"refusal\":{\"kind\":\"Failed\"}}' >&2\n" + "exit 125\n" + ) + cli.chmod(0o755) + env = {"cli": str(cli), "home": tmp, "xdg_cache": tmp} + got = run.timed_export(env, "s/p", Path(tmp) / "out") + self.assertEqual(got["rc"], 125) + self.assertEqual([p.get("phase") for p in got["progress"]], ["prepare", "export", None]) + self.assertIn("refusal", got["progress"][-1]["line"]) + self.assertTrue(all(p["t_ms"] >= 0 for p in got["progress"])) + + +if __name__ == "__main__": + unittest.main() diff --git a/fs-bench-pro/tools/resident.c b/fs-bench-pro/tools/resident.c new file mode 100644 index 0000000..af788a3 --- /dev/null +++ b/fs-bench-pro/tools/resident.c @@ -0,0 +1,33 @@ +/* resident [--evict] FILE...: per file, optionally fdatasync + posix_fadvise(DONTNEED) + * (drops only that file's clean pages; no privilege, nothing shared), then prints + * " " from mincore. Exit 1 on any error. */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +int main(int argc, char **argv) { + int evict = argc > 1 && strcmp(argv[1], "--evict") == 0, rc = 0; + long page = sysconf(_SC_PAGESIZE); + for (int i = 1 + evict; i < argc; i++) { + int fd = open(argv[i], O_RDONLY); + struct stat st; + if (fd < 0 || fstat(fd, &st) < 0) { perror(argv[i]); rc = 1; continue; } + if (evict && (fdatasync(fd) < 0 || posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED) != 0)) { perror(argv[i]); rc = 1; } + size_t pages = (st.st_size + page - 1) / page, res = 0; + if (pages) { + void *m = mmap(NULL, st.st_size, PROT_READ, MAP_SHARED, fd, 0); + unsigned char *v = malloc(pages); + if (m == MAP_FAILED || !v || mincore(m, st.st_size, v) < 0) { perror(argv[i]); rc = 1; } + else for (size_t p = 0; p < pages; p++) res += v[p] & 1; + if (m != MAP_FAILED) munmap(m, st.st_size); + free(v); + } + printf("%zu %zu %s\n", res, pages, argv[i]); + close(fd); + } + return rc; +} From af67c0cb97623443c912239f525ca9f632886f52 Mon Sep 17 00:00:00 2001 From: YifanXu1999 Date: Sun, 8 Nov 2026 07:22:33 +0800 Subject: [PATCH 2/3] fs-bench-pro: first export report at layerfs 4cd63f16dd Eight samples, one per case per machine: seven exports complete with exact content (six QUALIFIED; native E01 OVER_BUDGET under harness 1, export 2.04 s); Apple Container E01 failed mid-stream (exit 125). Native E02's 8.2 s is cold-volume read cost (1.67 s warm, diagnostic). Indexes regenerated with shared/src/readme_index.py. Production LOC: not applicable (benchmark repository; no LayerFS product code). Co-Authored-By: Claude Opus 5.5 --- README.md | 1 + fs-bench-pro/README.md | 3 + .../report.md | 91 +++++++++++++++++++ 3 files changed, 95 insertions(+) create mode 100644 result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md diff --git a/README.md b/README.md index cc8dc09..1947603 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ The LayerFS engine: Workspaces, file I/O, metadata and Commit. | --- | --- | --- | --- | | [fio](fio/) | speed | buffered sequential and random file I/O | 2026-11-06 · `76a32a9a` (pgr2-six) · Diagnostic terminal: INCOMPLETE. 36 registered rows; execution {'PASS': 36}. Final infrastructure cleanup: INCOMPLETE · [report](result/2026-11-06/fio/76a32a9a10e7b7b1379699ece52d45e233da5a50/pgr2-six/report.md) | | [fs-bench-cloudflare-computer](fs-bench-cloudflare-computer/) | speed | Cloudflare's fs-bench.sh on LayerFS | 2026-11-07 · `85a0865c` · 8 of 8 rows ran all twelve scenarios (8 PASS); final cleanup COMPLETE · [report](result/2026-11-07/fs-bench-cloudflare-computer/85a0865cde92f1d85a4e8bf402c597c935ff8db6/report.md) | +| [fs-bench-pro](fs-bench-pro/) | speed, correctness | cold project export from each machine into a host folder | 2026-11-08 · `4cd63f16` · Export works from all four machines into a host folder: 7 of 8 samples completed with exact content (6 QUALIFIED, native E01 OVER_BUDGET under the first harness rule); Apple Container E01 FAILED mid-stream · [report](result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md) | | [fs-bench-quick](fs-bench-quick/) | speed | CLI case times for eighteen small file workloads | 2026-11-06 · `76a32a9a` (pgr2-final) · Diagnostic terminal: INCOMPLETE. 162 registered rows; execution {'PASS': 162}. Final infrastructure cleanup: INCOMPLETE · [report](result/2026-11-06/fs-bench-quick/76a32a9a10e7b7b1379699ece52d45e233da5a50/pgr2-final/report.md) | | [fs-storage-dsh](fs-storage-dsh/) | storage | Store bytes for a real repository history replayed through Workspace Commits | 2026-11-06 · `7691ac20` · 4 of 4 rows PASS (applied, committed, measured and verified at state 157); net Store bytes D10 47,927,296 B SQLite / 53,272,576 B PostgreSQL, D03 61,014,016 B / 68,419,584 B, against a Git pack + idx of 41,805,317 / 48,166,223 B; final cleanup COMPLETE · [report](result/2026-11-06/fs-storage-dsh/7691ac20b0a8c8b741dd2b701e7aa77b63aaa8a3/report.md) | | [fs-storage-history](fs-storage-history/) | storage | Store bytes by history shape: long history, branches, forks and projects | 2026-11-06 · `7691ac20` · All 14 rows (7 cases × Docker × SQLite, PostgreSQL) PASS with every Branch head verified on final integration 7691ac20b0 (= 8b701f42f0 + #92 + #91 + #90, PRs pending owner merge); against the Phase 0 reference b4fa173e40, net Store bytes fell in every comparable row (SQLite −2.93 % … −17.93 %, PostgreSQL −18.03 % … −53.26 %), Σ new_metadata fell 19–41 % with payload unchanged, and PostgreSQL ÷ SQLite is now 1.07–1.10× in every case (Phase 0: 1.11–2.22×). Final cleanup COMPLETE · [report](result/2026-11-06/fs-storage-history/7691ac20b0a8c8b741dd2b701e7aa77b63aaa8a3/report.md) | diff --git a/fs-bench-pro/README.md b/fs-bench-pro/README.md index ad971dd..610444d 100644 --- a/fs-bench-pro/README.md +++ b/fs-bench-pro/README.md @@ -120,4 +120,7 @@ python3 -m unittest discover -s tests -v ## Past runs +| Date | LayerFS commit | Outcome | Report | +| --- | --- | --- | --- | +| 2026-11-08 | `4cd63f16dd` | Export works from all four machines into a host folder: 7 of 8 samples completed with exact content (6 QUALIFIED, native E01 OVER_BUDGET under the first harness rule); Apple Container E01 FAILED mid-stream | [report](../result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md) | diff --git a/result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md b/result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md new file mode 100644 index 0000000..fab057f --- /dev/null +++ b/result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md @@ -0,0 +1,91 @@ +# fs-bench-pro — 2026-11-08 + +> Status: Research; informative and not a product contract. + +**Export works from all four machines into a host folder: 7 of 8 samples completed with exact content (6 QUALIFIED, native E01 OVER_BUDGET under the first harness rule); Apple Container E01 FAILED mid-stream.** + +LayerFS: `4cd63f16ddbea5791c88fe34be369de27bcfff81`, local branch `pr600-export-port` +(`layerfs_v2_0_0` at `98ebfdfb96` plus PR #600 and its review fixes), **not pushed**. +CLI release build (`cargo +1.85.1`) sha256 `5bed87e829a378be6c02f20dbf95d085008edbbd24639d15e8f37e646497f769`; +Linux bundle `bundle.json` sha256 `90671e7df78b9e900230bd44a7b93474dcb4f00e268d5b7a5d5f538aae079acc`, +bundled `layerfs` `fa3fdb038ee992e74e4c6d122c20f5de378e759d270953fba9d7339d1c726c01`. Harness: `ac8a2baa54` (receipts carry harness 1–3, below). +Workload `export-v1`. Guest tool `resident-linux-aarch64` sha256 `cd5210676f24ec0ba2803ba402417644cfd6ef7b3415be5d24d19b8560034780`. + +Host: macOS 26.4.1, Apple M3 Max, 36 GiB. Docker Desktop engine 29.5.2, Lima 2.2.1, +Apple Container CLI 1.6.1. SQLite 3.53.2, Disposable. One Store per machine (`s-native`, +`s-docker`, `s-lima` on `lm1`, `s-ac` on `ac1`), each 320 MB holding both fixtures, inited on +native and pushed to the guest before the campaign. Destination: a fresh folder per sample on the +host's internal APFS volume. `LAYERFS_CONSTRUCTION_WORKERS=1`, private `LAYERFS_HOME` and `XDG_CACHE_HOME`. + +## Main observations + +- Every machine exports into a host folder. All seven completed exports matched the pinned + manifests exactly (content, modes, symlinks, mtimes) and left no staging folder. +- Small files (E01) run at about 5,000–6,300 files/s on every machine (1.6–2.0 s for 10,000 files). + Guests and native differ by under 0.5 s. +- One large file (E02): guests take 1.2–1.5 s (175–220 MB/s); native takes 8.2 s (33 MB/s). + A labeled diagnostic after the campaign separated the cause: native E02 took 1.67 s warm and + 7.95 s right after re-attaching the disk image, while a plain sequential read of the 320 MB Store + file from a freshly attached image ran at 1.8 GB/s. The native cost is the export's cold read + pattern on a cold volume, not the writer. The guests' cold control is weaker (limits below), so + this gap is not a native-vs-guest product difference. +- Apple Container E01 failed after 44.2 of 46.5 MB with exit code 125 and published nothing (no + folder, no staging left). Under harness 2 the final error line was parsed and dropped, so its + message is lost. The functional pre-campaign export of the same project succeeded once. This + matches the open Apple Container issue recorded with the port: its port forward truncates large + worker answers (PR #600 author: about 36% of 9.4 MB exports; `store.pull` too). Not rerun. +- Native E01's receipt is OVER_BUDGET because harness 1 counted the 12.0 s disk-image detach in + the 15 s budget; its export took 2.04 s. The receipt is kept as recorded. + +## Reading the tables + +One row is one sample. **Export ms** is the wall time of the single timed CLI process +(`project export … --progress`). **MB/s** and **Files/s** divide the fixture's bytes and files by it. +**First event ms** is when the `prepare` progress line arrived. **Cold pages** are the Store files' +resident pages after the control (native counts 16 KiB pages, guests 4 KiB). +**Cold control s** and **Verify s** are untimed overhead, outside Export ms. **Load 1m** is the host's +1-minute load before the sample. Order: E01 then E02; native, Docker, Lima, Apple Container. + +## E01 — export 10,000 small files (46.5 MB) with 50 symlinks and 132 folders + +| Machine | Execution | Verification | Performance | Cold pages | Export ms | MB/s | Files/s | First event ms | Cold control s | Verify s | Load 1m | +| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| native | PASS | PASS | OVER_BUDGET | 0/19549 | 2039.17 | 22.8 | 4904.0 | 73.88 | 11.967 | 0.588 | 4.9 | +| docker | PASS | PASS | QUALIFIED | 0/78200 | 1685.83 | 27.6 | 5931.8 | 33.62 | 0.103 | 0.544 | 4.3 | +| lima | PASS | PASS | QUALIFIED | 0/78199 | 1581.86 | 29.4 | 6321.7 | 22.88 | 0.123 | 0.569 | 5.0 | +| apple-container | FAIL | FAIL | FAIL | 0/78193 | 1980.52 | — | — | 24.77 | 0.173 | 0.0 | 7.2 | + +## E02 — export one 256 MiB file and a README + +| Machine | Execution | Verification | Performance | Cold pages | Export ms | MB/s | Files/s | First event ms | Cold control s | Verify s | Load 1m | +| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| native | PASS | PASS | QUALIFIED | 0/19549 | 8229.23 | 32.6 | 0.2 | 21.1 | 0.775 | 0.362 | 7.3 | +| docker | PASS | PASS | QUALIFIED | 0/78200 | 1534.93 | 174.9 | 1.3 | 22.41 | 0.095 | 0.189 | 6.8 | +| lima | PASS | PASS | QUALIFIED | 0/78199 | 1220.72 | 219.9 | 1.6 | 27.22 | 0.099 | 0.191 | 6.8 | +| apple-container | PASS | PASS | QUALIFIED | 0/78193 | 1371.34 | 195.7 | 1.5 | 19.3 | 0.084 | 0.177 | 6.8 | +## Harness versions + +The runner was fixed twice during the campaign; no receipt was relabeled or rerun. + +| Harness | Receipts | Change | +| --- | --- | --- | +| 1 | E01 native | budget counted the cold control (12.0 s detach) plus verification | +| 2 | E01 Docker, Lima, Apple Container | budget judges the timed export only, as fs-bench-quick reports control overhead apart | +| 3 (`ac8a2baa54`) | E02 all | stderr lines that are not progress events are kept verbatim | + +## Limits + +- Guest cold control drops the Store's pages from the guest page cache only. The host may still + cache the VM's disk file (Docker Desktop VM, Lima VM, Apple Container VM), which is shared and was + not purged. Native detaches the whole owned volume, which is the stronger state. +- One sample per row; no variance claim. Host load was 4.3–7.3. A build in another worktree (the + Microsandbox/SSH removal) ran 3–4 `rustc` processes during E02 native, Lima and Apple Container. +- Product source is an unpushed branch; SQLite only (no PostgreSQL arm); macOS host only; Durable: + `NOT_RUN — disabled by owner until explicit reauthorization`. +- Prepared Stores were reused across samples; export only reads them. The cold control ran before + every sample. + +## Cleanup + +Every exported folder was removed after verification. Machines, Stores, the disk image and fixtures +were kept for the follow-up on the Apple Container failure; they are released separately. From 1a27c67f74104f5fd09376f1f92c620c6b5938c0 Mon Sep 17 00:00:00 2001 From: YifanXu1999 Date: Sun, 8 Nov 2026 07:40:55 +0800 Subject: [PATCH 3/3] fs-bench-pro: export report at layerfs 97fac452bd (worker close-order fix) All eight samples QUALIFIED. 97fac452bd adds one worker fix to 4cd63f16dd: the worker waits for the client's close, so Apple Container's published port no longer drops an answer's tail. Apple Container E01, FAIL in the earlier report, passes. Indexes regenerated. Production LOC: not applicable (benchmark repository; no LayerFS product code). Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- fs-bench-pro/README.md | 1 + .../report.md | 73 +++++++++++++++++++ 3 files changed, 75 insertions(+), 1 deletion(-) create mode 100644 result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md diff --git a/README.md b/README.md index 1947603..ae9c910 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ The LayerFS engine: Workspaces, file I/O, metadata and Commit. | --- | --- | --- | --- | | [fio](fio/) | speed | buffered sequential and random file I/O | 2026-11-06 · `76a32a9a` (pgr2-six) · Diagnostic terminal: INCOMPLETE. 36 registered rows; execution {'PASS': 36}. Final infrastructure cleanup: INCOMPLETE · [report](result/2026-11-06/fio/76a32a9a10e7b7b1379699ece52d45e233da5a50/pgr2-six/report.md) | | [fs-bench-cloudflare-computer](fs-bench-cloudflare-computer/) | speed | Cloudflare's fs-bench.sh on LayerFS | 2026-11-07 · `85a0865c` · 8 of 8 rows ran all twelve scenarios (8 PASS); final cleanup COMPLETE · [report](result/2026-11-07/fs-bench-cloudflare-computer/85a0865cde92f1d85a4e8bf402c597c935ff8db6/report.md) | -| [fs-bench-pro](fs-bench-pro/) | speed, correctness | cold project export from each machine into a host folder | 2026-11-08 · `4cd63f16` · Export works from all four machines into a host folder: 7 of 8 samples completed with exact content (6 QUALIFIED, native E01 OVER_BUDGET under the first harness rule); Apple Container E01 FAILED mid-stream · [report](result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md) | +| [fs-bench-pro](fs-bench-pro/) | speed, correctness | cold project export from each machine into a host folder | 2026-11-08 · `97fac452` · All 8 export samples QUALIFIED: every machine exported both fixtures into a host folder with exact content, from a cold Store, within budget · [report](result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md) | | [fs-bench-quick](fs-bench-quick/) | speed | CLI case times for eighteen small file workloads | 2026-11-06 · `76a32a9a` (pgr2-final) · Diagnostic terminal: INCOMPLETE. 162 registered rows; execution {'PASS': 162}. Final infrastructure cleanup: INCOMPLETE · [report](result/2026-11-06/fs-bench-quick/76a32a9a10e7b7b1379699ece52d45e233da5a50/pgr2-final/report.md) | | [fs-storage-dsh](fs-storage-dsh/) | storage | Store bytes for a real repository history replayed through Workspace Commits | 2026-11-06 · `7691ac20` · 4 of 4 rows PASS (applied, committed, measured and verified at state 157); net Store bytes D10 47,927,296 B SQLite / 53,272,576 B PostgreSQL, D03 61,014,016 B / 68,419,584 B, against a Git pack + idx of 41,805,317 / 48,166,223 B; final cleanup COMPLETE · [report](result/2026-11-06/fs-storage-dsh/7691ac20b0a8c8b741dd2b701e7aa77b63aaa8a3/report.md) | | [fs-storage-history](fs-storage-history/) | storage | Store bytes by history shape: long history, branches, forks and projects | 2026-11-06 · `7691ac20` · All 14 rows (7 cases × Docker × SQLite, PostgreSQL) PASS with every Branch head verified on final integration 7691ac20b0 (= 8b701f42f0 + #92 + #91 + #90, PRs pending owner merge); against the Phase 0 reference b4fa173e40, net Store bytes fell in every comparable row (SQLite −2.93 % … −17.93 %, PostgreSQL −18.03 % … −53.26 %), Σ new_metadata fell 19–41 % with payload unchanged, and PostgreSQL ÷ SQLite is now 1.07–1.10× in every case (Phase 0: 1.11–2.22×). Final cleanup COMPLETE · [report](result/2026-11-06/fs-storage-history/7691ac20b0a8c8b741dd2b701e7aa77b63aaa8a3/report.md) | diff --git a/fs-bench-pro/README.md b/fs-bench-pro/README.md index 610444d..c0f045c 100644 --- a/fs-bench-pro/README.md +++ b/fs-bench-pro/README.md @@ -122,5 +122,6 @@ python3 -m unittest discover -s tests -v | Date | LayerFS commit | Outcome | Report | | --- | --- | --- | --- | +| 2026-11-08 | `97fac452bd` | All 8 export samples QUALIFIED: every machine exported both fixtures into a host folder with exact content, from a cold Store, within budget | [report](../result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md) | | 2026-11-08 | `4cd63f16dd` | Export works from all four machines into a host folder: 7 of 8 samples completed with exact content (6 QUALIFIED, native E01 OVER_BUDGET under the first harness rule); Apple Container E01 FAILED mid-stream | [report](../result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md) | diff --git a/result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md b/result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md new file mode 100644 index 0000000..bdd2426 --- /dev/null +++ b/result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md @@ -0,0 +1,73 @@ +# fs-bench-pro — 2026-11-08 (worker close-order fix) + +> Status: Research; informative and not a product contract. + +**All 8 export samples QUALIFIED: every machine exported both fixtures into a host folder with exact content, from a cold Store, within budget.** + +LayerFS: `97fac452bd4c1f834bdc2df005cf01caf8b415d0`, local branch `pr600-export-port`, **not pushed**: +the [earlier run](../4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md)'s commit plus one fix, the worker +waits for the client to close after its answer. CLI release build (`cargo +1.85.1`) sha256 +`acf1e63b0bfe888c10d29f2f9eb10631e22610c0b5ffb8b4a632c7dfadf61b43`; sealed Linux bundle hash +`238efd1913e0910b66d6af08e1b0b436addeb763c74212b90fddaaad23e5f843`, installed on all three guests by +`machine stop` + `machine start`. Harness: `ac8a2baa54` (harness 3 in every receipt). Workload `export-v1`. + +Host, machines, Stores, fixtures and cold control are as in the earlier run: macOS 26.4.1, Apple M3 Max, +36 GiB; Docker Desktop 29.5.2, Lima 2.2.1, Apple Container 1.6.1; SQLite 3.53.2, Disposable; one 320 MB +Store per machine holding both fixtures; a fresh destination folder per sample on the host's internal +APFS volume. `LAYERFS_CONSTRUCTION_WORKERS=1`, private `LAYERFS_HOME` and `XDG_CACHE_HOME`. + +## Main observations + +- Apple Container E01, which failed in the earlier run, now passes. The cause: Apple Container's + published port drops what it still holds for the host when the guest side closes first, and the + worker closed right after its last frame. A plain 50 MB stream through a throwaway Apple Container + lost 78–570 KB at the tail in 4 of 30 slow reads when the guest closed first, and 0 of 16 when the + host closed first. After the fix, 12 of 12 small and 4 of 4 large Apple Container exports + (untimed checks before this campaign) matched the manifests. +- Small files (E01): 1.3–1.9 s on every machine, 5,400–7,500 files/s. +- One large file (E02): Apple Container 1.18 s (227 MB/s); Lima 3.58 s, Docker 4.00 s; native 7.57 s. + Docker and Lima took 2.6x and 2.9x their earlier-run times. That run had a lower host load (6.8 vs + 10.5–10.9) and its guests had not just been restarted; one sample each does not separate these. + Native E02 remains cold-read bound (1.67 s warm in the earlier run's diagnostic). + +## Reading the tables + +One row is one sample. **Export ms** is the wall time of the single timed CLI process +(`project export … --progress`). **MB/s** and **Files/s** divide the fixture's bytes and files by it. +**First event ms** is when the `prepare` progress line arrived. **Cold pages** are the Store files' +resident pages after the control (native counts 16 KiB pages, guests 4 KiB). +**Cold control s** and **Verify s** are untimed overhead, outside Export ms. **Load 1m** is the host's +1-minute load before the sample. Order: E01 then E02; native, Docker, Lima, Apple Container. + +## E01 — export 10,000 small files (46.5 MB) with 50 symlinks and 132 folders + +| Machine | Execution | Verification | Performance | Cold pages | Export ms | MB/s | Files/s | First event ms | Cold control s | Verify s | Load 1m | +| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| native | PASS | PASS | QUALIFIED | 0/19549 | 1862.49 | 25.0 | 5369.2 | 24.55 | 1.11 | 0.601 | 10.7 | +| docker | PASS | PASS | QUALIFIED | 0/78200 | 1573.13 | 29.6 | 6356.8 | 31.32 | 0.088 | 0.602 | 10.9 | +| lima | PASS | PASS | QUALIFIED | 0/78199 | 1377.3 | 33.8 | 7260.6 | 25.69 | 0.1 | 0.577 | 10.9 | +| apple-container | PASS | PASS | QUALIFIED | 0/78193 | 1326.26 | 35.1 | 7540.0 | 21.56 | 0.108 | 0.615 | 11.7 | + +## E02 — export one 256 MiB file and a README + +| Machine | Execution | Verification | Performance | Cold pages | Export ms | MB/s | Files/s | First event ms | Cold control s | Verify s | Load 1m | +| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| native | PASS | PASS | QUALIFIED | 0/19549 | 7566.83 | 35.5 | 0.3 | 21.06 | 4.106 | 0.196 | 12.1 | +| docker | PASS | PASS | QUALIFIED | 0/78200 | 3996.32 | 67.2 | 0.5 | 19.75 | 0.086 | 0.18 | 10.9 | +| lima | PASS | PASS | QUALIFIED | 0/78199 | 3577.84 | 75.0 | 0.6 | 17.68 | 0.097 | 0.222 | 10.5 | +| apple-container | PASS | PASS | QUALIFIED | 0/78193 | 1180.54 | 227.4 | 1.7 | 23.43 | 0.1 | 0.201 | 9.8 | + +## Limits + +- Guest cold control drops the Store's pages from the guest page cache only. The host may still + cache the VM's disk file, which is shared and was not purged; native detaches the whole owned volume. +- One sample per row; no variance claim. Host load was 9.8–12.1; two `rustc` processes of another + session ran through the campaign (gone before they could be attributed). +- Product source is an unpushed branch; SQLite only; macOS host only; Durable: + `NOT_RUN — disabled by owner until explicit reauthorization`. +- Prepared Stores were reused across samples; export only reads them. + +## Cleanup + +Every exported folder was removed after verification and no staging folder was left. Machines, +Stores, the disk image and fixtures are released after this report.