diff --git a/README.md b/README.md index cc8dc09..ae9c910 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ The LayerFS engine: Workspaces, file I/O, metadata and Commit. | --- | --- | --- | --- | | [fio](fio/) | speed | buffered sequential and random file I/O | 2026-11-06 · `76a32a9a` (pgr2-six) · Diagnostic terminal: INCOMPLETE. 36 registered rows; execution {'PASS': 36}. Final infrastructure cleanup: INCOMPLETE · [report](result/2026-11-06/fio/76a32a9a10e7b7b1379699ece52d45e233da5a50/pgr2-six/report.md) | | [fs-bench-cloudflare-computer](fs-bench-cloudflare-computer/) | speed | Cloudflare's fs-bench.sh on LayerFS | 2026-11-07 · `85a0865c` · 8 of 8 rows ran all twelve scenarios (8 PASS); final cleanup COMPLETE · [report](result/2026-11-07/fs-bench-cloudflare-computer/85a0865cde92f1d85a4e8bf402c597c935ff8db6/report.md) | +| [fs-bench-pro](fs-bench-pro/) | speed, correctness | cold project export from each machine into a host folder | 2026-11-08 · `97fac452` · All 8 export samples QUALIFIED: every machine exported both fixtures into a host folder with exact content, from a cold Store, within budget · [report](result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md) | | [fs-bench-quick](fs-bench-quick/) | speed | CLI case times for eighteen small file workloads | 2026-11-06 · `76a32a9a` (pgr2-final) · Diagnostic terminal: INCOMPLETE. 162 registered rows; execution {'PASS': 162}. Final infrastructure cleanup: INCOMPLETE · [report](result/2026-11-06/fs-bench-quick/76a32a9a10e7b7b1379699ece52d45e233da5a50/pgr2-final/report.md) | | [fs-storage-dsh](fs-storage-dsh/) | storage | Store bytes for a real repository history replayed through Workspace Commits | 2026-11-06 · `7691ac20` · 4 of 4 rows PASS (applied, committed, measured and verified at state 157); net Store bytes D10 47,927,296 B SQLite / 53,272,576 B PostgreSQL, D03 61,014,016 B / 68,419,584 B, against a Git pack + idx of 41,805,317 / 48,166,223 B; final cleanup COMPLETE · [report](result/2026-11-06/fs-storage-dsh/7691ac20b0a8c8b741dd2b701e7aa77b63aaa8a3/report.md) | | [fs-storage-history](fs-storage-history/) | storage | Store bytes by history shape: long history, branches, forks and projects | 2026-11-06 · `7691ac20` · All 14 rows (7 cases × Docker × SQLite, PostgreSQL) PASS with every Branch head verified on final integration 7691ac20b0 (= 8b701f42f0 + #92 + #91 + #90, PRs pending owner merge); against the Phase 0 reference b4fa173e40, net Store bytes fell in every comparable row (SQLite −2.93 % … −17.93 %, PostgreSQL −18.03 % … −53.26 %), Σ new_metadata fell 19–41 % with payload unchanged, and PostgreSQL ÷ SQLite is now 1.07–1.10× in every case (Phase 0: 1.11–2.22×). Final cleanup COMPLETE · [report](result/2026-11-06/fs-storage-history/7691ac20b0a8c8b741dd2b701e7aa77b63aaa8a3/report.md) | diff --git a/fs-bench-pro/.gitignore b/fs-bench-pro/.gitignore new file mode 100644 index 0000000..474ef11 --- /dev/null +++ b/fs-bench-pro/.gitignore @@ -0,0 +1,3 @@ +/results/ +/environments.local.json +/tools/resident-linux-* diff --git a/fs-bench-pro/AGENTS.md b/fs-bench-pro/AGENTS.md new file mode 100644 index 0000000..895d483 --- /dev/null +++ b/fs-bench-pro/AGENTS.md @@ -0,0 +1,104 @@ +# fs-bench-pro agent instructions + +> Status: Current general guide. + +These instructions apply to `fs-bench-pro/` and extend the repository-root +[CLI and cache rules](../AGENTS.md). This file is the suite's current contract; +[cases.py](src/cases.py) pins the cases and expected trees. Reading it does not +start a campaign or authorize repeating completed samples. + +This is a new suite, created on 2026-11-08 at the owner's instruction. The +earlier fs-bench-pro in `.archive/` stays frozen: do not edit, build or run it. + +## Families and cases + +Each family is a case group in [cases.py](src/cases.py) with its own workload +version. Only `export` exists: + +| Case | Work | Fixture | +| --- | --- | --- | +| E01 | `project export` of 10,000 small files, 50 symlinks, 132 folders (46,486,561 bytes) | `small` | +| E02 | `project export` of one 256 MiB file and a README (268,435,480 bytes) | `large` | + +A case exports branch `main` of one project from a SQLite Store on the selected +machine (native, Docker, Lima, Apple Container) into a fresh folder on the +macOS host. Workload version `export-v1` pins the fixtures' generator and +manifests. Changing a fixture, the command or the verifier needs a new +workload version. Add a family by adding its cases, fixtures and verifier +here, not a second runner. + +## Select the product + +- Record the full source commit, branch and whether it is pushed; build the + CLI and the Linux bundle from that one snapshot and record their hashes. +- Use the absolute CLI path from the config file, never `layerfs` on `PATH`. +- Use a private `LAYERFS_HOME` and `XDG_CACHE_HOME`, and + `LAYERFS_CONSTRUCTION_WORKERS=1`. Store profile: Disposable only. + +## Prepare (untimed, through the CLI) + +```text +layerfs store create +layerfs project init /small /small +layerfs project init /large /large +layerfs store push # guests only +``` + +`run.py setup` does this for every machine. Init runs on native from the same +fixture bytes; a guest Store is then moved, not rebuilt. Prepared Stores are +reused across samples because export only reads them; the exported folder is +the sample's output and is removed after verification. + +## Cold control + +A cold claim needs enforcement and evidence on owned resources only. Never +purge the host's or Docker Desktop's shared cache. + +- Guests: `tools/resident.c`, built static and copied to `/tmp/resident` in the + guest, runs `--evict` (fdatasync + `posix_fadvise(DONTNEED)`) on the Store's + own files, then counts their resident pages with mincore. +- Native: the home sits on an owned APFS disk image; detach and re-attach it, + then count the Store's resident pages with mincore. +- Any resident page or a failed control: INELIGIBLE. The control's cost is + overhead, recorded apart from case time. + +Limit to state in every report: the guest control empties the guest's page +cache only. The host may still cache the VM's disk file, which belongs to the +shared VM and is not purged, so a guest's cold state is weaker than native's. + +## Run one sample per command + +```text +python3 src/run.py run --env environments.local.json --case --machine +``` + +- One sample per case per machine; no warm-up, rerun or best-of. A failure + keeps its receipt and is reported. +- Order: E01 then E02; within a case native, Docker, Lima, Apple Container. +- Record the 1-minute load (the runner does) and any build or test another + session runs at the same time. +- The timed export has a 15 s budget, verification 10 s, and every command a + wall limit of at most 120 s. +- A labeled diagnostic (for example a warm export to separate cold read cost) + may follow, outside the receipts, and is reported as a diagnostic. + +## Verdicts + +QUALIFIED needs execution PASS (exit 0 and counts equal to the fixture), +verification PASS (manifest equal to the pinned hash), a shown cold Store and +the export within budget. Otherwise OVER_BUDGET, INELIGIBLE or FAIL. +Receipts record the harness version; a harness change never relabels an +earlier receipt. + +## Report + +Publish `result//fs-bench-pro//report.md` per the root +rules: bold outcome sentence, configuration and identities, one table per case +(native first) from `src/report.py`, observations, limits, failures and +cleanup. Then run `python3 shared/src/readme_index.py`. Receipts stay local. + +## Cleanup + +`run.py cleanup` removes the family's Stores. Remove machines with the CLI, +detach the disk image and delete fixtures and outputs you created; report +anything kept. diff --git a/fs-bench-pro/README.md b/fs-bench-pro/README.md new file mode 100644 index 0000000..c0f045c --- /dev/null +++ b/fs-bench-pro/README.md @@ -0,0 +1,127 @@ +# fs-bench-pro — cold project export from each machine into a host folder +> group: filesystem · measures: speed, correctness + +## Basic info + +| | | +| --- | --- | +| Tool | In-house: Python standard library in [src/](src/), plus a small static C tool for cold-cache control in [tools/](tools/); every LayerFS step uses the public CLI | +| Product | LayerFS 2.0 with `project.export` ([layerfs-project#53](https://github.com/Ephemeral-AI-Lab/layerfs-project/issues/53)), pinned per campaign | +| Machines | macOS host: native, Docker, Lima, Apple Container | +| Store backends | SQLite (Disposable) | +| Families | `export` (E01, E02); more families are added as their own case groups | +| Status | Current general guide. New suite (2026-11-08); the earlier fs-bench-pro is archived in `.archive/` and stays frozen | +| Frozen rules | [AGENTS.md](AGENTS.md) (cases, cold control, verdicts, cleanup); root [CLI and cache rules](../AGENTS.md) | + +Agents must read the root [CLI and cache rules](../AGENTS.md) and the +[suite instructions](AGENTS.md) before preparing or running selections. + +The suite root holds documentation and the configuration example. Python +implementation lives in `src/`, focused checks in `tests/`, and the guest +cache tool's source in `tools/`. [cases.py](src/cases.py) pins the cases and +the expected trees. + +## Overview + +fs-bench-pro measures one product operation per family from a declared, +enforced cold state, with an independent content check. The `export` family +times `layerfs project export / --progress`: +the Store lives on one machine and the exported tree must land, complete and +exact, in a fresh folder on the macOS host. + +| Case | Work | Fixture | +| --- | --- | --- | +| E01 | Export 10,000 small files (46.5 MB), 50 symlinks, 132 folders | `small` | +| E02 | Export one 256 MiB file and a README | `large` | + +Fixtures come from [fixture.py](src/fixture.py): deterministic, incompressible +and distinct per file, so chunk dedupe saves nothing. Their manifest hashes +(path, kind, mode, size, content hash or link target, mtime) are pinned in +[cases.py](src/cases.py). + +One `run` is one sample of one case on one machine: + +1. **Cold control (untimed).** Guests: `resident --evict` syncs and drops the + Store's own files from the guest page cache, then `resident` counts the + pages still resident (mincore). Native: the home lives on an owned APFS disk + image, which is detached and re-attached; mincore then counts the Store's + resident pages. Any resident page, or a failed control, makes the row + INELIGIBLE. +2. **Timed export.** One CLI process; each progress line is stamped as it + arrives. The export wall time is the case time. +3. **Verification (untimed).** The CLI's counts must match the fixture, and the + manifest of the exported folder must equal the pinned hash. No staging + folder may be left behind. +4. **Release.** The exported folder is removed; the receipt is written to + `results//receipt.json` (ignored, local). + +Verdicts: **QUALIFIED** (execution and verification PASS, cold shown, export +within budget), **OVER_BUDGET**, **INELIGIBLE** (cold not shown) or **FAIL**. + +## Setup + +### Prerequisites + +- A LayerFS CLI and Linux bundle built from one clean source snapshot, with + `project export` (` help project export`). +- Ready machines for the guests you select, created through the CLI + (`layerfs machine …`), each with its own SQLite Store. +- An owned APFS sparse disk image mounted where the private `LAYERFS_HOME` + lives, so native cold control never touches a shared volume: + + ```bash + hdiutil create -size 20g -type SPARSE -fs APFS -volname lfs-bench native.sparseimage + hdiutil attach -nobrowse -mountpoint vol native.sparseimage + ``` + +- The guest tool, built once for the guests' architecture: + + ```bash + docker run --rm -v "$PWD/tools:/w" -w /w gcc gcc -O2 -static -o resident-linux-aarch64 resident.c + ``` + +- The fixtures: `python3 src/fixture.py /small small` and + `python3 src/fixture.py /large large`; each prints its manifest, + which must equal the pinned one. + +### Config file + +Copy [environments.example.json](environments.example.json) to +`environments.local.json` (ignored) and fill in absolute paths, the guest exec +prefixes and the campaign's identities. `XDG_CACHE_HOME` points at a private +cache so the bundle never mixes with another session's. + +### One selection + +```bash +python3 src/run.py setup --env environments.local.json +python3 src/run.py run --env environments.local.json --case E01 --machine lima +python3 src/report.py ... +python3 src/run.py cleanup --env environments.local.json +``` + +`setup` creates one Store per machine with both fixtures, pushes guest Stores +and installs the guest tool. `cleanup` removes those Stores; machines, the disk +image and fixtures are released by the caller. + +### Time limits + +The timed export has the 15 s default budget; verification has 10 s. No case +has a declared exception. The cold control's cost (up to 12 s for a disk-image +detach) is enforcement overhead, recorded as `cold.control_s` and never case +time. + +### Focused checks + +```bash +python3 -m unittest discover -s tests -v +``` + +## Past runs + + +| Date | LayerFS commit | Outcome | Report | +| --- | --- | --- | --- | +| 2026-11-08 | `97fac452bd` | All 8 export samples QUALIFIED: every machine exported both fixtures into a host folder with exact content, from a cold Store, within budget | [report](../result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md) | +| 2026-11-08 | `4cd63f16dd` | Export works from all four machines into a host folder: 7 of 8 samples completed with exact content (6 QUALIFIED, native E01 OVER_BUDGET under the first harness rule); Apple Container E01 FAILED mid-stream | [report](../result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md) | + diff --git a/fs-bench-pro/environments.example.json b/fs-bench-pro/environments.example.json new file mode 100644 index 0000000..557c624 --- /dev/null +++ b/fs-bench-pro/environments.example.json @@ -0,0 +1,57 @@ +{ + "cli": "/ABSOLUTE/layerfs-checkout/target/release/layerfs", + "home": "/ABSOLUTE/bench-root/vol/home", + "xdg_cache": "/ABSOLUTE/bench-root/xdg-cache", + "fixtures": "/ABSOLUTE/bench-root/fixtures", + "out": "/ABSOLUTE/bench-root/out", + "guest_tool": "/ABSOLUTE/bench-root/tools/resident-linux-aarch64", + "native": { + "store": "s-native", + "image": "/ABSOLUTE/bench-root/native.sparseimage", + "mount": "/ABSOLUTE/bench-root/vol" + }, + "machines": { + "docker": { + "machine": "docker", + "store": "s-docker", + "guest": [ + "docker", + "exec", + "-i", + "lfs--docker" + ] + }, + "lima": { + "machine": "lm1", + "store": "s-lima", + "guest": [ + "limactl", + "shell", + "lfs--lm1", + "sudo" + ] + }, + "apple-container": { + "machine": "ac1", + "store": "s-ac", + "guest": [ + "container", + "exec", + "-i", + "lfs--ac1" + ] + } + }, + "identity": { + "source": "", + "cli_sha256": "", + "bundle_json_sha256": "", + "linux_layerfs_sha256": "", + "profile": "", + "host": "", + "docker": "", + "lima": "", + "apple_container": "", + "store": "" + } +} diff --git a/fs-bench-pro/src/cases.py b/fs-bench-pro/src/cases.py new file mode 100644 index 0000000..b38fd66 --- /dev/null +++ b/fs-bench-pro/src/cases.py @@ -0,0 +1,39 @@ +"""The export family: pinned cases, fixtures and expected results. + +A case exports one committed project (`main`) from a SQLite Store on one +machine to a fresh folder on the macOS host with the public +`layerfs project export / --progress`. +""" + +FAMILY = "export" +WORKLOAD_VERSION = "export-v1" +MACHINES = ("native", "docker", "lima", "apple-container") + +# fixture kind -> expected tree, from fixture.py at WORKLOAD_VERSION +FIXTURES = { + "small": { + "manifest": "8638d2973cfce62d1169e771f9649a35c5d2d78f5f553695c6342b8544e50c1e", + "files": 10000, + "directories": 132, + "symlinks": 50, + "bytes": 46486561, + }, + "large": { + "manifest": "a4735ecdaf8a95a18953f474c5ee616b47eca599769843ac56ccd148b8b5e205", + "files": 2, + "directories": 0, + "symlinks": 0, + "bytes": 268435480, + }, +} + +CASES = { + "E01": {"fixture": "small", "work": "export 10,000 small files (46.5 MB) with 50 symlinks and 132 folders"}, + "E02": {"fixture": "large", "work": "export one 256 MiB file and a README"}, +} + +# complete command (cold control + timed export + verification + cleanup) +BUDGET_S = 15.0 +# declared exceptions, each up to 25 s (none yet) +DECLARED_EXCEPTIONS: dict = {} +VERIFY_BUDGET_S = 10.0 diff --git a/fs-bench-pro/src/cold.py b/fs-bench-pro/src/cold.py new file mode 100644 index 0000000..24ab0d2 --- /dev/null +++ b/fs-bench-pro/src/cold.py @@ -0,0 +1,115 @@ +"""Cold Store pages before a timed export, enforced and shown on owned resources only. + +Guests (docker, lima, apple-container): `resident --evict` fdatasyncs each of the +Store's own files and drops their clean pages with posix_fadvise(DONTNEED), then +`resident` counts the pages still resident with mincore. Nothing shared is +purged: other files, other containers and the Docker Desktop VM keep their cache. +native (macOS): the home sits on an owned APFS disk image; detaching and +re-attaching it drops that volume's cached pages, then mincore counts the Store +files' resident pages. A row whose Store shows any resident page is INELIGIBLE. +""" + +import ctypes +import glob +import mmap +import os +import subprocess +import time +from pathlib import Path + +GUEST_TOOL = "/tmp/resident" +STORE_GLOB = "/layerfs/data/store/{store}/store.sqlite*" + + +def guest_prefix(env: dict, machine: str) -> list: + return env["machines"][machine]["guest"] + + +def guest(env: dict, machine: str, script: str) -> str: + out = subprocess.run( + guest_prefix(env, machine) + ["sh", "-c", script], + capture_output=True, text=True, timeout=30, check=True, + ) + return out.stdout + + +def install_tool(env: dict, machine: str, tool: Path) -> None: + """Copies the static tool into the guest's /tmp (owned, outside every Store).""" + with open(tool, "rb") as data: + subprocess.run( + guest_prefix(env, machine) + ["sh", "-c", f"cat > {GUEST_TOOL} && chmod 755 {GUEST_TOOL}"], + stdin=data, capture_output=True, timeout=30, check=True, + ) + + +def parse(lines: str) -> list: + rows = [] + for line in lines.strip().splitlines(): + resident, total, path = line.split(" ", 2) + rows.append({"path": path, "resident_pages": int(resident), "pages": int(total)}) + return rows + + +def evict_guest(env: dict, machine: str, store: str) -> dict: + files = STORE_GLOB.format(store=store) + script = f"{GUEST_TOOL} --evict {files} >/dev/null && {GUEST_TOOL} {files}" + started = time.monotonic() + rows = parse(guest(env, machine, script)) + return { + "method": "posix_fadvise(DONTNEED) on the Store's own files, then mincore", + "files": rows, + "control_s": round(time.monotonic() - started, 3), + } + + +def _mincore(path: str) -> dict: + """Resident pages of `path` on macOS: libc mmap (read-only, shared) + mincore.""" + size = os.path.getsize(path) + page = mmap.PAGESIZE + pages = (size + page - 1) // page + if pages == 0: + return {"path": path, "resident_pages": 0, "pages": 0} + libc = ctypes.CDLL(None, use_errno=True) + libc.mmap.restype = ctypes.c_void_p + libc.mmap.argtypes = [ctypes.c_void_p, ctypes.c_size_t, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_long] + libc.munmap.argtypes = [ctypes.c_void_p, ctypes.c_size_t] + libc.mincore.argtypes = [ctypes.c_void_p, ctypes.c_size_t, ctypes.c_char_p] + fd = os.open(path, os.O_RDONLY) + try: + addr = libc.mmap(None, size, mmap.PROT_READ, mmap.MAP_SHARED, fd, 0) + if addr in (None, ctypes.c_void_p(-1).value): + raise OSError(ctypes.get_errno(), "mmap") + try: + vec = ctypes.create_string_buffer(pages) + if libc.mincore(addr, size, vec) != 0: + raise OSError(ctypes.get_errno(), "mincore") + resident = sum(1 for b in vec.raw[:pages] if b & 1) + finally: + libc.munmap(addr, size) + finally: + os.close(fd) + return {"path": path, "resident_pages": resident, "pages": pages} + + +def evict_native(env: dict, store: str) -> dict: + image, mount = env["native"]["image"], env["native"]["mount"] + started = time.monotonic() + subprocess.run(["hdiutil", "detach", mount], capture_output=True, timeout=60, check=True) + subprocess.run( + ["hdiutil", "attach", "-nobrowse", "-mountpoint", mount, image], + capture_output=True, timeout=60, check=True, + ) + files = sorted(glob.glob(os.path.join(env["home"], "store", store, "store.sqlite*"))) + if not files: + raise FileNotFoundError(f"no Store files for {store} under {env['home']}") + return { + "method": "detach and re-attach the owned APFS disk image holding the home, then mincore", + "files": [_mincore(f) for f in files], + "control_s": round(time.monotonic() - started, 3), + } + + +def evict(env: dict, machine: str, store: str) -> dict: + evidence = evict_native(env, store) if machine == "native" else evict_guest(env, machine, store) + evidence["cold"] = bool(evidence["files"]) and all(f["resident_pages"] == 0 for f in evidence["files"]) + return evidence diff --git a/fs-bench-pro/src/fixture.py b/fs-bench-pro/src/fixture.py new file mode 100644 index 0000000..9cd4e04 --- /dev/null +++ b/fs-bench-pro/src/fixture.py @@ -0,0 +1,98 @@ +"""Deterministic synthetic export fixtures (no user data). + + python3 fixture.py small|large writes the fixture, prints its manifest hash + python3 fixture.py --manifest prints the manifest hash of a folder + +small: 10,000 files of random bytes (512 B to 16 KiB, log-uniform) in 10 x 10 +folders, 50 symlinks (relative, absolute, dangling), 20 empty folders, every +50th file 0755. large: one 256 MiB random file and a README. Content is +incompressible and distinct per file, so chunk dedupe saves nothing. +""" + +import hashlib +import json +import math +import os +import random +import stat +import sys +from pathlib import Path + +MTIME = 1_700_000_000_123_456_789 + + +def small(root: Path) -> None: + rng = random.Random(53) + n = 0 + for a in range(10): + for b in range(10): + folder = root / f"pkg{a:02d}" / f"mod{b:02d}" + folder.mkdir(parents=True) + for _ in range(100): + size = int(math.exp(rng.uniform(math.log(512), math.log(16384)))) + path = folder / f"f{n:05d}.bin" + path.write_bytes(rng.randbytes(size)) + if n % 50 == 0: + path.chmod(0o755) + n += 1 + for i in range(20): + (root / "empty" / f"e{i:02d}").mkdir(parents=True) + links = root / "links" + links.mkdir() + for i in range(50): + target = ( + f"../pkg{i % 10:02d}/mod00/f{i * 100:05d}.bin" if i % 3 == 0 + else "/etc/hosts" if i % 3 == 1 + else f"missing-{i}" + ) + os.symlink(target, links / f"l{i:02d}") + + +def large(root: Path) -> None: + rng = random.Random(530) + root.mkdir(parents=True, exist_ok=True) + (root / "README").write_text("one 256 MiB random file\n") + with open(root / "blob.bin", "wb") as out: + for _ in range(256): + out.write(rng.randbytes(1 << 20)) + + +def manifest(root: Path) -> str: + """sha256 over sorted (path, kind, mode, size, content sha256 or link target, mtime_ns).""" + rows = [] + for dirpath, dirnames, filenames in os.walk(root, followlinks=False): + for name in sorted(dirnames + filenames): + p = Path(dirpath) / name + st = p.lstat() + rel = str(p.relative_to(root)) + if stat.S_ISLNK(st.st_mode): + rows.append([rel, "l", 0, 0, os.readlink(p), 0]) + elif stat.S_ISDIR(st.st_mode): + rows.append([rel, "d", st.st_mode & 0o7777, 0, "", 0]) + else: + h = hashlib.sha256(p.read_bytes()).hexdigest() + rows.append([rel, "f", st.st_mode & 0o7777, st.st_size, h, st.st_mtime_ns]) + rows.sort() + return hashlib.sha256(json.dumps(rows).encode()).hexdigest() + + +def settle(root: Path) -> None: + """Fixed mtimes on files, then folders bottom-up, so manifests compare.""" + for dirpath, dirnames, filenames in os.walk(root, topdown=False): + for name in filenames: + p = Path(dirpath) / name + if not p.is_symlink(): + os.utime(p, ns=(MTIME, MTIME)) + + +if __name__ == "__main__": + if sys.argv[1] == "--manifest": + print(manifest(Path(sys.argv[2]))) + sys.exit(0) + dest, kind = Path(sys.argv[1]), sys.argv[2] + if dest.exists(): + sys.exit(f"{dest} exists") + dest.mkdir(parents=True) + {"small": small, "large": large}[kind](dest) + settle(dest) + print(manifest(dest)) diff --git a/fs-bench-pro/src/report.py b/fs-bench-pro/src/report.py new file mode 100644 index 0000000..06144c4 --- /dev/null +++ b/fs-bench-pro/src/report.py @@ -0,0 +1,61 @@ +#!/usr/bin/env python3 +"""Markdown tables from export receipts, one table per case, native first. + + report.py RUN_ID... (receipts under results//receipt.json) + +Prints the tables only; the report's outcome, configuration, observations, +limits and cleanup are written by the agent who ran the campaign. +""" + +import json +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) +import cases # noqa: E402 + +RESULTS = HERE.parent / "results" +COLUMNS = ( + "Machine", "Execution", "Verification", "Performance", "Cold pages", + "Export ms", "MB/s", "Files/s", "First event ms", "Cold control s", "Verify s", "Load 1m", +) + + +def load(run_ids: list) -> list: + return [json.loads((RESULTS / run_id / "receipt.json").read_text()) for run_id in run_ids] + + +def cell(value) -> str: + return "—" if value is None else str(value) + + +def row(r: dict) -> list: + files = r["cold"].get("files", []) + resident = f"{sum(f['resident_pages'] for f in files)}/{sum(f['pages'] for f in files)}" if files else "unknown" + m = r["metrics"] + return [ + r["machine"], r["execution"], r["verification"]["status"], r["performance"], resident, + cell(m["wall_ms"]), cell(m["mb_per_s"]), cell(m["files_per_s"]), cell(m["prepare_ms"]), + cell(r["cold"].get("control_s")), cell(r["verification"]["wall_s"]), f"{r['load_1m_before']:.1f}", + ] + + +def tables(receipts: list) -> str: + out = [] + for case_id, case in cases.CASES.items(): + rows = sorted( + (r for r in receipts if r["case"] == case_id), key=lambda r: cases.MACHINES.index(r["machine"]) + ) + if not rows: + continue + out.append(f"## {case_id} — {case['work']}\n") + out.append("| " + " | ".join(COLUMNS) + " |") + out.append("| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |") + out.extend("| " + " | ".join(row(r)) + " |" for r in rows) + out.append("") + return "\n".join(out) + + +if __name__ == "__main__": + print(tables(load(sys.argv[1:]))) diff --git a/fs-bench-pro/src/run.py b/fs-bench-pro/src/run.py new file mode 100644 index 0000000..27ad491 --- /dev/null +++ b/fs-bench-pro/src/run.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +"""fs-bench-pro export family: setup, one sample per invocation, receipts. + + run.py setup --env environments.local.json + run.py run --env environments.local.json --case E01 --machine lima + run.py cleanup --env environments.local.json + +Every LayerFS step is the public CLI (absolute path from the env file). One +`run` is one sample of one case on one machine: cold control (outside the +timer), the timed `project export --progress`, independent verification +(outside the timer), output removal, and an append-only receipt under +results//. Nothing is retried; a failure keeps its receipt. +""" + +import argparse +import json +import os +import shutil +import subprocess +import sys +import threading +import time +import uuid +from pathlib import Path + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) +import cases # noqa: E402 +import cold # noqa: E402 +import fixture # noqa: E402 + +ROOT = HERE.parent +# 2: the budget judges the timed export alone; cold control and verification +# are reported beside it (receipt 20261107T231724Z-4dd18c ran under 1). +# 3: stderr lines that are not progress events are kept verbatim (under 2 a +# final refusal line was parsed and dropped: 20261107T231818Z-c7a8fa). +HARNESS = 3 + + +def load_env(path: str) -> dict: + env = json.loads(Path(path).read_text()) + for key in ("cli", "home", "xdg_cache", "fixtures", "out", "machines", "native"): + if key not in env: + sys.exit(f"{path}: missing `{key}`") + return env + + +def process_env(env: dict) -> dict: + e = dict(os.environ) + e.update( + LAYERFS_HOME=env["home"], + XDG_CACHE_HOME=env["xdg_cache"], + LAYERFS_CONSTRUCTION_WORKERS="1", + ) + return e + + +def cli(env: dict, *args: str, timeout: float = 120) -> dict: + out = subprocess.run( + [env["cli"], *args], capture_output=True, text=True, timeout=timeout, env=process_env(env) + ) + if out.returncode != 0: + raise RuntimeError(f"layerfs {' '.join(args)}: rc={out.returncode} {out.stderr.strip()[:400]}") + return json.loads(out.stdout) if out.stdout.strip() else {} + + +def store_of(env: dict, machine: str) -> str: + return env["machines"][machine]["store"] if machine != "native" else env["native"]["store"] + + +def setup(env: dict) -> None: + """Untimed: one Store per machine holding both fixtures, inited on native, then pushed.""" + for machine in cases.MACHINES: + store = store_of(env, machine) + cli(env, "store", "create", store) + for kind in cases.FIXTURES: + cli(env, "project", "init", f"{store}/{kind}", str(Path(env["fixtures"]) / kind)) + if machine != "native": + cli(env, "store", "push", store, env["machines"][machine]["machine"], timeout=300) + cold.install_tool(env, machine, Path(env["guest_tool"])) + print(json.dumps(cli(env, "store", "list"), indent=1)) + + +def timed_export(env: dict, project: str, dest: Path) -> dict: + """The timed phase: one CLI process; progress lines stamped as they arrive.""" + events, stamps = [], [] + started = time.monotonic_ns() + proc = subprocess.Popen( + [env["cli"], "project", "export", project, str(dest), "--progress"], + stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, env=process_env(env), + ) + + def read_progress() -> None: + for line in proc.stderr: + stamps.append(time.monotonic_ns() - started) + events.append(line.rstrip("\n")) + + reader = threading.Thread(target=read_progress, daemon=True) + reader.start() + try: + stdout, _ = proc.communicate(timeout=cases.BUDGET_S * 4) + except subprocess.TimeoutExpired: + proc.kill() + stdout, _ = proc.communicate() + wall_ns = time.monotonic_ns() - started + reader.join(timeout=5) + progress = [] + for stamp, line in zip(stamps, events): + try: + value = json.loads(line)["progress"] + progress.append({"t_ms": round(stamp / 1e6, 2), "phase": value["phase"], "bytes": value["bytes"]}) + except (ValueError, KeyError, TypeError): + progress.append({"t_ms": round(stamp / 1e6, 2), "line": line[:2000]}) + result = None + try: + result = json.loads(stdout) if stdout.strip() else None + except ValueError: + pass + return { + "rc": proc.returncode, "wall_ms": round(wall_ns / 1e6, 2), "result": result, + "stdout": None if result is not None else stdout[:2000], "progress": progress, + } + + +def phase_ms(progress: list, phase: str, last: bool = False): + hits = [p["t_ms"] for p in progress if p.get("phase") == phase] + return (hits[-1] if last else hits[0]) if hits else None + + +def run(env: dict, case_id: str, machine: str) -> dict: + case = cases.CASES[case_id] + kind = case["fixture"] + expect = cases.FIXTURES[kind] + store = store_of(env, machine) + run_id = time.strftime("%Y%m%dT%H%M%SZ", time.gmtime()) + "-" + uuid.uuid4().hex[:6] + out_dir = Path(env["out"]) / run_id + receipts = ROOT / "results" / run_id + receipts.mkdir(parents=True) + out_dir.mkdir(parents=True) + dest = out_dir / "exported" + started = time.monotonic() + receipt = { + "family": cases.FAMILY, "harness": HARNESS, "workload": cases.WORKLOAD_VERSION, "case": case_id, "machine": machine, + "store": store, "project": f"{store}/{kind}", "fixture": kind, "run_id": run_id, + "identity": env.get("identity", {}), "load_1m_before": os.getloadavg()[0], + } + try: + receipt["cold"] = cold.evict(env, machine, store) + except Exception as error: # the control failed: the row cannot be cold + receipt["cold"] = {"cold": False, "error": str(error)[:400]} + receipt["export"] = timed_export(env, f"{store}/{kind}", dest) + receipt["load_1m_after"] = os.getloadavg()[0] + res = receipt["export"]["result"] or {} + counts_ok = all(res.get(k) == expect[k] for k in ("files", "directories", "symlinks", "bytes")) + receipt["execution"] = "PASS" if receipt["export"]["rc"] == 0 and counts_ok else "FAIL" + v0 = time.monotonic() + if dest.is_dir(): + got = fixture.manifest(dest) + receipt["verification"] = {"status": "PASS" if got == expect["manifest"] else "FAIL", "manifest": got} + else: + receipt["verification"] = {"status": "FAIL", "reason": "no exported folder"} + receipt["verification"]["wall_s"] = round(time.monotonic() - v0, 3) + left = [p.name for p in out_dir.iterdir() if p.name.startswith(".layerfs-export-")] + receipt["staging_left"] = left + shutil.rmtree(out_dir, ignore_errors=True) + receipt["complete_s"] = round(time.monotonic() - started, 3) + receipt["performance"] = verdict(receipt) + wall_s = receipt["export"]["wall_ms"] / 1000 + receipt["metrics"] = { + "wall_ms": receipt["export"]["wall_ms"], + "prepare_ms": phase_ms(receipt["export"]["progress"], "prepare"), + "finalize_ms": phase_ms(receipt["export"]["progress"], "finalize", last=True), + "mb_per_s": round(expect["bytes"] / 1e6 / wall_s, 1) if receipt["execution"] == "PASS" else None, + "files_per_s": round(expect["files"] / wall_s, 1) if receipt["execution"] == "PASS" else None, + } + (receipts / "receipt.json").write_text(json.dumps(receipt, indent=1)) + return receipt + + +def verdict(receipt: dict) -> str: + """QUALIFIED only with execution and verification PASS, a shown cold Store and the budget met. + + The budget judges the timed export command; the cold control's cost is + enforcement overhead, reported as `cold.control_s`, never case time. + """ + if receipt["execution"] != "PASS" or receipt["verification"]["status"] != "PASS": + return "FAIL" + if not receipt["cold"].get("cold"): + return "INELIGIBLE" + budget = cases.DECLARED_EXCEPTIONS.get(f"{receipt['case']}/{receipt['machine']}", cases.BUDGET_S) + if receipt["export"]["wall_ms"] / 1000 > budget or receipt["verification"]["wall_s"] > cases.VERIFY_BUDGET_S: + return "OVER_BUDGET" + return "QUALIFIED" + + +def cleanup(env: dict) -> None: + """Removes the Stores this family made; machines, images and fixtures are the caller's.""" + for machine in cases.MACHINES: + try: + cli(env, "store", "remove", store_of(env, machine), timeout=300) + print("removed", store_of(env, machine)) + except RuntimeError as error: + print("KEPT", store_of(env, machine), error) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("command", choices=("setup", "run", "cleanup")) + parser.add_argument("--env", required=True) + parser.add_argument("--case", choices=sorted(cases.CASES)) + parser.add_argument("--machine", choices=cases.MACHINES) + args = parser.parse_args() + env = load_env(args.env) + if args.command == "setup": + setup(env) + elif args.command == "cleanup": + cleanup(env) + else: + if not (args.case and args.machine): + parser.error("run needs --case and --machine") + receipt = run(env, args.case, args.machine) + m = receipt["metrics"] + print(f"{receipt['run_id']} {args.case} {args.machine}: execution {receipt['execution']}, " + f"verification {receipt['verification']['status']}, performance {receipt['performance']}, " + f"wall {m['wall_ms']} ms, {m['mb_per_s']} MB/s, {m['files_per_s']} files/s, " + f"cold={receipt['cold'].get('cold')}, complete {receipt['complete_s']} s") + + +if __name__ == "__main__": + main() diff --git a/fs-bench-pro/tests/test_export_family.py b/fs-bench-pro/tests/test_export_family.py new file mode 100644 index 0000000..4680ffc --- /dev/null +++ b/fs-bench-pro/tests/test_export_family.py @@ -0,0 +1,111 @@ +"""Focused local checks of the export family's harness; these are not samples.""" + +import importlib.util +import os +import sys +import tempfile +import unittest +from pathlib import Path +from unittest import mock + +SRC = Path(__file__).resolve().parents[1] / "src" +sys.path.insert(0, str(SRC)) + + +def load(name): + spec = importlib.util.spec_from_file_location(f"pro_{name}", SRC / f"{name}.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +fixture, cold, run = load("fixture"), load("cold"), load("run") + + +class ManifestTests(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.root = Path(self.tmp.name) / "t" + self.root.mkdir() + (self.root / "a").write_bytes(b"one") + (self.root / "d").mkdir() + os.symlink("../a", self.root / "d" / "l") + fixture.settle(self.root) + + def test_manifest_is_stable_and_sees_content_mode_and_links(self): + base = fixture.manifest(self.root) + self.assertEqual(base, fixture.manifest(self.root)) + (self.root / "a").write_bytes(b"two") + fixture.settle(self.root) + changed = fixture.manifest(self.root) + self.assertNotEqual(base, changed) + os.chmod(self.root / "a", 0o755) + self.assertNotEqual(changed, fixture.manifest(self.root)) + os.remove(self.root / "d" / "l") + os.symlink("a", self.root / "d" / "l") + self.assertNotEqual(changed, fixture.manifest(self.root)) + + +class ColdTests(unittest.TestCase): + def test_any_resident_page_or_no_file_is_not_cold(self): + lines = "0 10 /s/store.sqlite\n3 4 /s/store.sqlite-wal\n" + with mock.patch.object(cold, "evict_guest", return_value={"files": cold.parse(lines)}): + self.assertFalse(cold.evict({}, "docker", "s")["cold"]) + with mock.patch.object(cold, "evict_guest", return_value={"files": cold.parse("0 10 /s/store.sqlite\n")}): + self.assertTrue(cold.evict({}, "docker", "s")["cold"]) + with mock.patch.object(cold, "evict_guest", return_value={"files": []}): + self.assertFalse(cold.evict({}, "docker", "s")["cold"]) + + def test_mincore_counts_pages_of_a_file_just_read(self): + with tempfile.NamedTemporaryFile() as f: + f.write(os.urandom(1 << 20)) + f.flush() + Path(f.name).read_bytes() + got = cold._mincore(f.name) + self.assertGreater(got["pages"], 0) + self.assertGreater(got["resident_pages"], 0) + + +def receipt(**over): + r = { + "case": "E01", "machine": "docker", "execution": "PASS", + "verification": {"status": "PASS", "wall_s": 0.5}, + "cold": {"cold": True, "control_s": 30.0}, "export": {"wall_ms": 1000.0}, + } + r.update(over) + return r + + +class VerdictTests(unittest.TestCase): + def test_verdicts(self): + self.assertEqual(run.verdict(receipt()), "QUALIFIED") # a slow control is not case time + self.assertEqual(run.verdict(receipt(execution="FAIL")), "FAIL") + self.assertEqual(run.verdict(receipt(verification={"status": "FAIL", "wall_s": 0})), "FAIL") + self.assertEqual(run.verdict(receipt(cold={"cold": False})), "INELIGIBLE") + self.assertEqual(run.verdict(receipt(export={"wall_ms": 15001.0})), "OVER_BUDGET") + self.assertEqual(run.verdict(receipt(verification={"status": "PASS", "wall_s": 10.5})), "OVER_BUDGET") + + +class TimedExportTests(unittest.TestCase): + def test_progress_is_stamped_and_a_refusal_line_is_kept(self): + with tempfile.TemporaryDirectory() as tmp: + cli = Path(tmp) / "layerfs" + cli.write_text( + "#!/bin/sh\n" + "echo '{\"progress\":{\"phase\":\"prepare\",\"bytes\":0}}' >&2\n" + "echo '{\"progress\":{\"phase\":\"export\",\"bytes\":7}}' >&2\n" + "echo '{\"refusal\":{\"kind\":\"Failed\"}}' >&2\n" + "exit 125\n" + ) + cli.chmod(0o755) + env = {"cli": str(cli), "home": tmp, "xdg_cache": tmp} + got = run.timed_export(env, "s/p", Path(tmp) / "out") + self.assertEqual(got["rc"], 125) + self.assertEqual([p.get("phase") for p in got["progress"]], ["prepare", "export", None]) + self.assertIn("refusal", got["progress"][-1]["line"]) + self.assertTrue(all(p["t_ms"] >= 0 for p in got["progress"])) + + +if __name__ == "__main__": + unittest.main() diff --git a/fs-bench-pro/tools/resident.c b/fs-bench-pro/tools/resident.c new file mode 100644 index 0000000..af788a3 --- /dev/null +++ b/fs-bench-pro/tools/resident.c @@ -0,0 +1,33 @@ +/* resident [--evict] FILE...: per file, optionally fdatasync + posix_fadvise(DONTNEED) + * (drops only that file's clean pages; no privilege, nothing shared), then prints + * " " from mincore. Exit 1 on any error. */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +int main(int argc, char **argv) { + int evict = argc > 1 && strcmp(argv[1], "--evict") == 0, rc = 0; + long page = sysconf(_SC_PAGESIZE); + for (int i = 1 + evict; i < argc; i++) { + int fd = open(argv[i], O_RDONLY); + struct stat st; + if (fd < 0 || fstat(fd, &st) < 0) { perror(argv[i]); rc = 1; continue; } + if (evict && (fdatasync(fd) < 0 || posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED) != 0)) { perror(argv[i]); rc = 1; } + size_t pages = (st.st_size + page - 1) / page, res = 0; + if (pages) { + void *m = mmap(NULL, st.st_size, PROT_READ, MAP_SHARED, fd, 0); + unsigned char *v = malloc(pages); + if (m == MAP_FAILED || !v || mincore(m, st.st_size, v) < 0) { perror(argv[i]); rc = 1; } + else for (size_t p = 0; p < pages; p++) res += v[p] & 1; + if (m != MAP_FAILED) munmap(m, st.st_size); + free(v); + } + printf("%zu %zu %s\n", res, pages, argv[i]); + close(fd); + } + return rc; +} diff --git a/result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md b/result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md new file mode 100644 index 0000000..fab057f --- /dev/null +++ b/result/2026-11-08/fs-bench-pro/4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md @@ -0,0 +1,91 @@ +# fs-bench-pro — 2026-11-08 + +> Status: Research; informative and not a product contract. + +**Export works from all four machines into a host folder: 7 of 8 samples completed with exact content (6 QUALIFIED, native E01 OVER_BUDGET under the first harness rule); Apple Container E01 FAILED mid-stream.** + +LayerFS: `4cd63f16ddbea5791c88fe34be369de27bcfff81`, local branch `pr600-export-port` +(`layerfs_v2_0_0` at `98ebfdfb96` plus PR #600 and its review fixes), **not pushed**. +CLI release build (`cargo +1.85.1`) sha256 `5bed87e829a378be6c02f20dbf95d085008edbbd24639d15e8f37e646497f769`; +Linux bundle `bundle.json` sha256 `90671e7df78b9e900230bd44a7b93474dcb4f00e268d5b7a5d5f538aae079acc`, +bundled `layerfs` `fa3fdb038ee992e74e4c6d122c20f5de378e759d270953fba9d7339d1c726c01`. Harness: `ac8a2baa54` (receipts carry harness 1–3, below). +Workload `export-v1`. Guest tool `resident-linux-aarch64` sha256 `cd5210676f24ec0ba2803ba402417644cfd6ef7b3415be5d24d19b8560034780`. + +Host: macOS 26.4.1, Apple M3 Max, 36 GiB. Docker Desktop engine 29.5.2, Lima 2.2.1, +Apple Container CLI 1.6.1. SQLite 3.53.2, Disposable. One Store per machine (`s-native`, +`s-docker`, `s-lima` on `lm1`, `s-ac` on `ac1`), each 320 MB holding both fixtures, inited on +native and pushed to the guest before the campaign. Destination: a fresh folder per sample on the +host's internal APFS volume. `LAYERFS_CONSTRUCTION_WORKERS=1`, private `LAYERFS_HOME` and `XDG_CACHE_HOME`. + +## Main observations + +- Every machine exports into a host folder. All seven completed exports matched the pinned + manifests exactly (content, modes, symlinks, mtimes) and left no staging folder. +- Small files (E01) run at about 5,000–6,300 files/s on every machine (1.6–2.0 s for 10,000 files). + Guests and native differ by under 0.5 s. +- One large file (E02): guests take 1.2–1.5 s (175–220 MB/s); native takes 8.2 s (33 MB/s). + A labeled diagnostic after the campaign separated the cause: native E02 took 1.67 s warm and + 7.95 s right after re-attaching the disk image, while a plain sequential read of the 320 MB Store + file from a freshly attached image ran at 1.8 GB/s. The native cost is the export's cold read + pattern on a cold volume, not the writer. The guests' cold control is weaker (limits below), so + this gap is not a native-vs-guest product difference. +- Apple Container E01 failed after 44.2 of 46.5 MB with exit code 125 and published nothing (no + folder, no staging left). Under harness 2 the final error line was parsed and dropped, so its + message is lost. The functional pre-campaign export of the same project succeeded once. This + matches the open Apple Container issue recorded with the port: its port forward truncates large + worker answers (PR #600 author: about 36% of 9.4 MB exports; `store.pull` too). Not rerun. +- Native E01's receipt is OVER_BUDGET because harness 1 counted the 12.0 s disk-image detach in + the 15 s budget; its export took 2.04 s. The receipt is kept as recorded. + +## Reading the tables + +One row is one sample. **Export ms** is the wall time of the single timed CLI process +(`project export … --progress`). **MB/s** and **Files/s** divide the fixture's bytes and files by it. +**First event ms** is when the `prepare` progress line arrived. **Cold pages** are the Store files' +resident pages after the control (native counts 16 KiB pages, guests 4 KiB). +**Cold control s** and **Verify s** are untimed overhead, outside Export ms. **Load 1m** is the host's +1-minute load before the sample. Order: E01 then E02; native, Docker, Lima, Apple Container. + +## E01 — export 10,000 small files (46.5 MB) with 50 symlinks and 132 folders + +| Machine | Execution | Verification | Performance | Cold pages | Export ms | MB/s | Files/s | First event ms | Cold control s | Verify s | Load 1m | +| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| native | PASS | PASS | OVER_BUDGET | 0/19549 | 2039.17 | 22.8 | 4904.0 | 73.88 | 11.967 | 0.588 | 4.9 | +| docker | PASS | PASS | QUALIFIED | 0/78200 | 1685.83 | 27.6 | 5931.8 | 33.62 | 0.103 | 0.544 | 4.3 | +| lima | PASS | PASS | QUALIFIED | 0/78199 | 1581.86 | 29.4 | 6321.7 | 22.88 | 0.123 | 0.569 | 5.0 | +| apple-container | FAIL | FAIL | FAIL | 0/78193 | 1980.52 | — | — | 24.77 | 0.173 | 0.0 | 7.2 | + +## E02 — export one 256 MiB file and a README + +| Machine | Execution | Verification | Performance | Cold pages | Export ms | MB/s | Files/s | First event ms | Cold control s | Verify s | Load 1m | +| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| native | PASS | PASS | QUALIFIED | 0/19549 | 8229.23 | 32.6 | 0.2 | 21.1 | 0.775 | 0.362 | 7.3 | +| docker | PASS | PASS | QUALIFIED | 0/78200 | 1534.93 | 174.9 | 1.3 | 22.41 | 0.095 | 0.189 | 6.8 | +| lima | PASS | PASS | QUALIFIED | 0/78199 | 1220.72 | 219.9 | 1.6 | 27.22 | 0.099 | 0.191 | 6.8 | +| apple-container | PASS | PASS | QUALIFIED | 0/78193 | 1371.34 | 195.7 | 1.5 | 19.3 | 0.084 | 0.177 | 6.8 | +## Harness versions + +The runner was fixed twice during the campaign; no receipt was relabeled or rerun. + +| Harness | Receipts | Change | +| --- | --- | --- | +| 1 | E01 native | budget counted the cold control (12.0 s detach) plus verification | +| 2 | E01 Docker, Lima, Apple Container | budget judges the timed export only, as fs-bench-quick reports control overhead apart | +| 3 (`ac8a2baa54`) | E02 all | stderr lines that are not progress events are kept verbatim | + +## Limits + +- Guest cold control drops the Store's pages from the guest page cache only. The host may still + cache the VM's disk file (Docker Desktop VM, Lima VM, Apple Container VM), which is shared and was + not purged. Native detaches the whole owned volume, which is the stronger state. +- One sample per row; no variance claim. Host load was 4.3–7.3. A build in another worktree (the + Microsandbox/SSH removal) ran 3–4 `rustc` processes during E02 native, Lima and Apple Container. +- Product source is an unpushed branch; SQLite only (no PostgreSQL arm); macOS host only; Durable: + `NOT_RUN — disabled by owner until explicit reauthorization`. +- Prepared Stores were reused across samples; export only reads them. The cold control ran before + every sample. + +## Cleanup + +Every exported folder was removed after verification. Machines, Stores, the disk image and fixtures +were kept for the follow-up on the Apple Container failure; they are released separately. diff --git a/result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md b/result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md new file mode 100644 index 0000000..bdd2426 --- /dev/null +++ b/result/2026-11-08/fs-bench-pro/97fac452bd4c1f834bdc2df005cf01caf8b415d0/report.md @@ -0,0 +1,73 @@ +# fs-bench-pro — 2026-11-08 (worker close-order fix) + +> Status: Research; informative and not a product contract. + +**All 8 export samples QUALIFIED: every machine exported both fixtures into a host folder with exact content, from a cold Store, within budget.** + +LayerFS: `97fac452bd4c1f834bdc2df005cf01caf8b415d0`, local branch `pr600-export-port`, **not pushed**: +the [earlier run](../4cd63f16ddbea5791c88fe34be369de27bcfff81/report.md)'s commit plus one fix, the worker +waits for the client to close after its answer. CLI release build (`cargo +1.85.1`) sha256 +`acf1e63b0bfe888c10d29f2f9eb10631e22610c0b5ffb8b4a632c7dfadf61b43`; sealed Linux bundle hash +`238efd1913e0910b66d6af08e1b0b436addeb763c74212b90fddaaad23e5f843`, installed on all three guests by +`machine stop` + `machine start`. Harness: `ac8a2baa54` (harness 3 in every receipt). Workload `export-v1`. + +Host, machines, Stores, fixtures and cold control are as in the earlier run: macOS 26.4.1, Apple M3 Max, +36 GiB; Docker Desktop 29.5.2, Lima 2.2.1, Apple Container 1.6.1; SQLite 3.53.2, Disposable; one 320 MB +Store per machine holding both fixtures; a fresh destination folder per sample on the host's internal +APFS volume. `LAYERFS_CONSTRUCTION_WORKERS=1`, private `LAYERFS_HOME` and `XDG_CACHE_HOME`. + +## Main observations + +- Apple Container E01, which failed in the earlier run, now passes. The cause: Apple Container's + published port drops what it still holds for the host when the guest side closes first, and the + worker closed right after its last frame. A plain 50 MB stream through a throwaway Apple Container + lost 78–570 KB at the tail in 4 of 30 slow reads when the guest closed first, and 0 of 16 when the + host closed first. After the fix, 12 of 12 small and 4 of 4 large Apple Container exports + (untimed checks before this campaign) matched the manifests. +- Small files (E01): 1.3–1.9 s on every machine, 5,400–7,500 files/s. +- One large file (E02): Apple Container 1.18 s (227 MB/s); Lima 3.58 s, Docker 4.00 s; native 7.57 s. + Docker and Lima took 2.6x and 2.9x their earlier-run times. That run had a lower host load (6.8 vs + 10.5–10.9) and its guests had not just been restarted; one sample each does not separate these. + Native E02 remains cold-read bound (1.67 s warm in the earlier run's diagnostic). + +## Reading the tables + +One row is one sample. **Export ms** is the wall time of the single timed CLI process +(`project export … --progress`). **MB/s** and **Files/s** divide the fixture's bytes and files by it. +**First event ms** is when the `prepare` progress line arrived. **Cold pages** are the Store files' +resident pages after the control (native counts 16 KiB pages, guests 4 KiB). +**Cold control s** and **Verify s** are untimed overhead, outside Export ms. **Load 1m** is the host's +1-minute load before the sample. Order: E01 then E02; native, Docker, Lima, Apple Container. + +## E01 — export 10,000 small files (46.5 MB) with 50 symlinks and 132 folders + +| Machine | Execution | Verification | Performance | Cold pages | Export ms | MB/s | Files/s | First event ms | Cold control s | Verify s | Load 1m | +| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| native | PASS | PASS | QUALIFIED | 0/19549 | 1862.49 | 25.0 | 5369.2 | 24.55 | 1.11 | 0.601 | 10.7 | +| docker | PASS | PASS | QUALIFIED | 0/78200 | 1573.13 | 29.6 | 6356.8 | 31.32 | 0.088 | 0.602 | 10.9 | +| lima | PASS | PASS | QUALIFIED | 0/78199 | 1377.3 | 33.8 | 7260.6 | 25.69 | 0.1 | 0.577 | 10.9 | +| apple-container | PASS | PASS | QUALIFIED | 0/78193 | 1326.26 | 35.1 | 7540.0 | 21.56 | 0.108 | 0.615 | 11.7 | + +## E02 — export one 256 MiB file and a README + +| Machine | Execution | Verification | Performance | Cold pages | Export ms | MB/s | Files/s | First event ms | Cold control s | Verify s | Load 1m | +| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| native | PASS | PASS | QUALIFIED | 0/19549 | 7566.83 | 35.5 | 0.3 | 21.06 | 4.106 | 0.196 | 12.1 | +| docker | PASS | PASS | QUALIFIED | 0/78200 | 3996.32 | 67.2 | 0.5 | 19.75 | 0.086 | 0.18 | 10.9 | +| lima | PASS | PASS | QUALIFIED | 0/78199 | 3577.84 | 75.0 | 0.6 | 17.68 | 0.097 | 0.222 | 10.5 | +| apple-container | PASS | PASS | QUALIFIED | 0/78193 | 1180.54 | 227.4 | 1.7 | 23.43 | 0.1 | 0.201 | 9.8 | + +## Limits + +- Guest cold control drops the Store's pages from the guest page cache only. The host may still + cache the VM's disk file, which is shared and was not purged; native detaches the whole owned volume. +- One sample per row; no variance claim. Host load was 9.8–12.1; two `rustc` processes of another + session ran through the campaign (gone before they could be attributed). +- Product source is an unpushed branch; SQLite only; macOS host only; Durable: + `NOT_RUN — disabled by owner until explicit reauthorization`. +- Prepared Stores were reused across samples; export only reads them. + +## Cleanup + +Every exported folder was removed after verification and no staging folder was left. Machines, +Stores, the disk image and fixtures are released after this report.