diff --git a/README.md b/README.md index ae9c910..a2e2a4f 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,7 @@ Connect: moving Stores between a home and a bucket. | Benchmark | Measures | What it checks | Latest run | | --- | --- | --- | --- | | [backup-verify](backup-verify/) | correctness, recovery | backup and restore on MinIO and R2 under failures | 2026-11-06 · `7691ac20` · Outcome: PASS on MinIO, 51 of 51 checks: schema v2 Stores round-trip, and a schema v1 backup is refused at open until migrated · [report](result/2026-11-06/backup-verify/7691ac20b0a8c8b741dd2b701e7aa77b63aaa8a3/report.md) | -| [store-transfer](store-transfer/) | correctness, recovery, speed | moving a SQLite Store between a home and a machine | 2026-11-08 · `d8dddd50` · Outcome: PASS on every Store-transfer route, refusal and recovery path of layerfs-project #8–#13 at the fixed build · [report](result/2026-11-08/store-transfer/d8dddd50d04c92e4656131e71273fbe0bac0daae/report.md) | +| [store-transfer](store-transfer/) | correctness, recovery, speed | moving a SQLite Store between a home and a machine | 2026-11-08 · `ab4c1a47` · Outcome: PASS on every Store-transfer route, refusal, recovery path and the guide at layerfs `ab4c1a4712` (Ephemeral-AI-Lab/layerfs#617, merged into `layerfs_v2_0_0` as `716197c499` with the same tree), and every Apple Container pull now succeeds on its first attempt · [report](result/2026-11-08/store-transfer/ab4c1a4712ec0e4d032a2ac341638053a02e2c91/report.md) | ## Layout diff --git a/result/2026-11-08/store-transfer/ab4c1a4712ec0e4d032a2ac341638053a02e2c91/report.md b/result/2026-11-08/store-transfer/ab4c1a4712ec0e4d032a2ac341638053a02e2c91/report.md new file mode 100644 index 0000000..3f79643 --- /dev/null +++ b/result/2026-11-08/store-transfer/ab4c1a4712ec0e4d032a2ac341638053a02e2c91/report.md @@ -0,0 +1,207 @@ +# store-transfer — a SQLite Store between a home and a machine + +> Status: verification outcomes; timings are diagnostic and performance-ineligible. + +**Outcome: PASS on every Store-transfer route, refusal, recovery path and the +guide at layerfs `ab4c1a4712` (Ephemeral-AI-Lab/layerfs#617, merged into +`layerfs_v2_0_0` as `716197c499` with the same tree), and every Apple Container +pull now succeeds on its first attempt.** + +This re-runs the [first report](../d8dddd50d04c92e4656131e71273fbe0bac0daae/report.md) +on the code that landed. Three things changed in the product since then: + +- Apple Container's cut answers are fixed by the worker waiting for the client + to close first (`97fac452bd`), not by the worker closing its write half; +- `send` reads each piece once: the 3× read retry is gone, so a lost answer is + `Unknown` at `send` like any other; +- Firecracker is removed, as Microsandbox and SSH were by layerfs#616: `machine create` + refuses all three with ``platform `` was removed in LayerFS 2.0``, and + `machine connect` no longer exists. + +## Configuration + +| Item | Identity | +| --- | --- | +| LayerFS | `ab4c1a4712ec0e4d032a2ac341638053a02e2c91`, tree `0e15263bec`, merged as `716197c499` (same tree) | +| CLI | release, macOS arm64, `CARGO_PROFILE_RELEASE_STRIP=none`, sha256 `044960d5b366e022…` | +| Daemon | release, sha256 `a25891a8ae16cb1a…` | +| Node addon | `layerfs-node` release, sha256 `0a736c2b833533f5…` | +| Linux bundle | release, `linux-aarch64`, hash `723629fa0dbfa43c…`, `source.head ab4c1a4712`; guest `layerfs` sha256 `af9e26ee2ef115d5…`; Sandbox image `afa1ef9c2729` (2.7 GB). Installed in a private folder through `LAYERFS_BUNDLE`, not the shared cache | +| Host | macOS 26.4.1, Apple silicon, 36 GiB; Docker Desktop 29.5.2 (8 CPUs, 7.75 GiB); `limactl` 2.2.1; `container` 1.6.1 | +| Guests | `docker` = Docker Desktop's engine (not colima); Lima and Apple Container 2 CPU / 2 GiB / 8 GiB disk | +| Suite | this folder at `e79289d` plus the changes in this report's PR (two commits; see "Suite fixes and re-run") | +| Stores | #9/#12 fixture 21.4 MB; #11 `cap256` 271.7 MB, `cap512` 543.1 MB, `quiet-s` 17.2 MB; #13 64 MiB fixture | +| Duration | 2026-11-08 09:03–09:36 CST; every driver step was its own command, at most 290 s | + +Changes to the suite, all in the commit that adds this report: + +- `matrix.py`, `lp08s2.py`, `crash.py`: the docker socket comes from + `LP08_DOCKER_SOCK` (default unchanged, colima's). +- `matrix.py` refusals: the removal text replaces the #7 text; SSH is checked + with `machine create … ssh` (`REF-SSH-CREATE`); the teardown is + `machine remove` for both records. +- `lp08s2.py`: `LP08_DISK` sets the Lima/Apple Container disk; the pull + ladder expects every size to pull on its first attempt (it used to expect + failures above 32 MiB). +- `GUIDE-zh.md` and `guide_repro.py`: removed platforms are not listed by + `machine platforms`; the docker socket can be Docker Desktop's + (`DOCKER_SOCK`); step 3 checks `docker info` instead of running + `colima start`; block 0 keeps a preset `LAYERFS_HOME` and `FIXTURE`. + +## Results + +### #9 — routes, the two-hop and refusals + +Checks passed / total. `matrix.py`. + +| Case | docker | lima | apple-container | +| --- | ---: | ---: | ---: | +| `R9-*-RT` round trip, nothing changed | 43 / 43 | 43 / 43 | 43 / 43 | +| `R9-*-MOD` edit on the target, pull back | 53 / 53 | 53 / 53 | 53 / 53 | +| `R9-X-RETIRE-DIR-*` | 5 / 5 | 5 / 5 | 5 / 5 | +| `R12-PROGRESS` | 64 / 64 | 64 / 64 | 64 / 64 | +| `R9-A-BLOCKER` size sweep, 3–21.7 MiB | — | — | 10 / 10, every pull first attempt | +| `R9-H-HOP` docker → native → lima | 76 / 76 | | | +| `R12-RAW` | 2 / 2 | | | + +The docker route ran twice: once as its own row, once to stage `mod-d` for +the hop (the README's order). Both passed. + +Refusals: 13 of 16 PASS — `REF-BADNAME`, `REF-BUSY`, `REF-EXISTS`, +`REF-FC-CREATE`, `REF-SSH-CREATE`, `REF-INUSE-SANDBOX`, `REF-LEASE`, +`REF-NOMACHINE`, `REF-NOSTORE`, `REF-PULL-NATIVE`, `REF-PUSH-NATIVE`, +`REF-STOPPED`, `REF-WRONGDIR-PUSH`. The other three: + +| Case | Checks | The check that did not pass | +| --- | ---: | --- | +| `REF-FC-RECORD` | 29 / 30 | `machine status fc1` is `Unsupported` with the removal text. The suite expects it answered, as for a *disabled* platform; for a *removed* platform only `machine list` and `machine get` answer (layerfs#616's contract, now Firecracker's too). Every new-work refusal, every kept read and the forgetting `machine remove` passed | +| `REF-SSH-RECORD` | 29 / 30 | the same, for `machine status h1` | +| `REF-PG` | 8 / 9 | `store connect` needs the `lp08-pg` server's password; there is no such server here, so the call failed `Unavailable` before reaching the engine check: NOT_RUN. Both `store push`/`store pull` "does not move" refusals passed | + +### #11 — 256 MiB and 512 MiB, capacity and mid-`send` ENOSPC + +`lp08s2.py` and `docker-cap.py`. Seconds are the CLI call's wall time (warm +host cache, diagnostic); MB/s is bytes ÷ seconds; caller RSS is the CLI's +`maxrss`; worker peak is the guest worker's `VmHWM` after the call (5.1–5.9 +MB before). + +| Platform | Transfer | Bytes | Seconds | MiB/s | Caller RSS (MB) | Worker peak (MB) | +| --- | --- | ---: | ---: | ---: | ---: | ---: | +| docker | 256 push | 271,679,488 | 2.512 | 103.1 | 49.1 | 31.4 | +| docker | 256 pull | 271,679,488 | 2.374 | 109.1 | 29.0 | 32.5 | +| docker | 512 push | 543,162,368 | 5.063 | 102.3 | 39.3 | 29.5 | +| docker | 512 pull | 543,162,368 | 4.441 | 116.6 | 22.5 | 32.5 | +| lima | 256 push | 271,683,584 | 1.278 | 202.7 | 49.1 | 56.7 | +| lima | 256 pull | 271,683,584 | 1.303 | 198.9 | 29.0 | 32.3 | +| lima | 512 push | 543,084,544 | 2.598 | 199.4 | 39.3 | 54.5 | +| lima | 512 pull | 543,084,544 | 2.442 | 212.1 | 22.5 | 32.3 | +| apple-container | 256 push | 271,675,392 | 0.958 | 270.5 | 49.1 | 30.5 | +| apple-container | 256 pull | 271,675,392 | 0.930 | 278.6 | 29.0 | 33.2 | +| apple-container | 512 push | 543,145,984 | 1.881 | 275.4 | 39.2 | 30.4 | +| apple-container | 512 pull | 543,145,984 | 1.718 | 301.5 | 22.5 | 33.1 | + +All `R11-*-256-push` (13/13), `-256-pull` (14/14), `-512` (21/21), +`-TARGET-EDIT` (5/5), `-BOOT` (2/2) and `R12-*-QUIET` (6/6) rows pass on all +three platforms. + +| Injection | docker | lima | apple-container | +| --- | ---: | ---: | ---: | +| `R11-CAP-PRE` (refused `Capacity` at `check`, nothing sent) | 8 / 8 (`docker-cap.py`) | 6 / 6 | 6 / 6 | +| `R11-ENOSPC-MID` (disk full during `send`, same command recovers) | 11 / 11 (`docker-cap.py`) | 8 / 8 | 8 / 8 | +| `R11-ENOSPC-MID-SYNC` (`nodelalloc`) | — | 9 / 9 | NOT_RUN: `remount,nodelalloc` refused | + +Apple Container pulls, each the first attempt of its row: + +| Run | Sizes | Pulls | First-attempt successes | +| --- | --- | ---: | ---: | +| `R11-A-PULL-SWEEP` | 8.7, 17.2, 34.1, 68.1, 136.0 MB | 5 | 5 | +| `R11-A-PULL-ROUNDS` | 256 and 512 MiB Stores, 3 rounds each | 6 | 6 (bytes = snapshot every time) | + +The first report's build needed up to 25 attempts for a 256 MiB pull and +expected failures above 32 MiB. + +Lima's first route ran on the driver's 4 GiB disk: its 256 MiB rows passed, +then the 512 MiB push was refused `Capacity` (1,097,236,480 bytes free; +1,616,863,232 needed, the Store plus 1 GiB headroom) because the 2.7 GB +Sandbox image sits on the same disk, and the ENOSPC case aborted on that +refusal. The refusal is the product's documented behaviour, not a defect. The +machine was removed and the whole route re-run on an 8 GiB disk (the first +report's Lima size); the 4 GiB receipts are kept locally. + +### #12 — progress + +CLI progress: `R12-PROGRESS` 64/64 on each route and `R12-RAW` 2/2. Quiet +path: `R12-*-QUIET` 6/6 on each platform. Node SDK `on_progress` +(`R12-D-SDK-ONPROGRESS`): 14/14. + +The three rows above are the first refusal run. See "Suite fixes and re-run" +below: with the expectations corrected, all 16 refusals pass. + +### #13 — interruption, ownership and cleanup + +`crash.py only …`, one case per command; `R13-A-AC-PULL` was run on its own +through the module because `only` has no entry for it. + +| Platform | Cases | Checks | +| --- | ---: | ---: | +| docker (full set: 7 phase kills, 2 pull kills, flip, truncate, record, retire, read-only, reverse, concurrent, remove, lost answer, lost retire) | 22 | all pass | +| lima (sampled set) | 11 | all pass | +| apple-container (sampled set) | 9 | all pass | +| **total** | **42** | **630 / 630** | + +`R13-D-LOST-RETIRE` records 4/4, but its note says the retire finished before +the worker died: the window was missed again, so it counts as NOT_RUN, as in +the first report. `R13-A-AC-PULL` saw no lost answer: 0 of 1 pulls failed. + +### #10 — the guide in a fresh home + +`guide_repro.py`: 21 blocks, **74 / 79** checks, 85.2 s, no leftover Store +records. The five misses are blocks 11 and 12, which teach recovering from an +Apple Container lost answer and assume one occurs within ten pulls: all ten +succeeded (`attempt N: pull ok`), so there was no unresolved pull to show or to +refuse. Block 13 and every other block passed. See "Suite fixes and re-run" +for the rewritten section and its 76 / 76 replay. + +## Suite fixes and re-run + +The first run's non-passing checks tested the suite, not the product, so the +suite was corrected and only the affected drivers re-run on the same binaries +and bundle (1-minute load 5.5). The first runs' receipts are kept locally. + +| Fix | Re-run | +| --- | --- | +| `REF-FC-RECORD` / `REF-SSH-RECORD`: `machine status` on a removed-platform record moves from "stays available" to the refusal battery (`Unsupported` with the removal text); `machine list`/`get` stay available | 31 / 31 each | +| `REF-PG`: with no `lp08-pg` server the driver now passes a placeholder password, so `store connect` reaches the engine check | 9 / 9: `Unsupported`, "this LayerFS build has no PostgreSQL engine" | +| Refusal matrix as a whole (second re-run, after the `REF-PG` fix) | **16 / 16 PASS** | +| `GUIDE-zh.md` §9 rewritten: block 11 is ten pull + push round trips that must all succeed, block 12 the push to another machine refused `InvalidInput` while the Store is on `ac`, block 13 the pull back; a lost answer's text and the same-command recovery stay as prose pointing at §10's deterministic demo | `guide_repro.py` in a new home: 21 blocks, **76 / 76**, 86.2 s, no leftover records | + +## Main observations + +- Apple Container is no longer special. With `97fac452bd`'s close order and + no read retry, every Apple Container pull this run did not interrupt on + purpose (routes, sweeps, rounds, #13, the guide's ten) succeeded on its first + attempt, 3.4 MB to 543 MB. +- Removed platforms behave as layerfs#616 documents: creation is refused with the + exact text, an old record refuses every call except `machine list`/`get`, + and `machine remove` forgets it. +- Caller and worker memory stays flat from 256 to 512 MiB on every platform. + Lima's push worker peak was 55–57 MB here and 32 MB on the same binaries on + the earlier 4 GiB Lima machine; it follows the guest, not the Store size. + +## Limitations + +- The refusal matrix ran three times and the guide twice: the first runs + exposed outdated suite expectations, and the re-runs changed only those + expectations, not the product, binaries or workload. +- NOT_RUN: `R11-ENOSPC-MID-SYNC-A`, `R13-D-LOST-RETIRE`. +- Timings are diagnostic: warm host cache, one sample, other agents' Docker + containers running (1-minute load 7.7–21.3 during the run). +- Docker Desktop replaced colima, so docker numbers are not comparable with + the first report's. + +## Cleanup + +All test machines were stopped at the end; their homes, VMs and the run folder +(4.6 GB) remain on the host, owned by this run, for the owner to release. +Nothing outside this run's homes, its bundle folder and its worktree +registration was touched. diff --git a/store-transfer/GUIDE-zh.md b/store-transfer/GUIDE-zh.md index 28c60fc..4e14ecc 100644 --- a/store-transfer/GUIDE-zh.md +++ b/store-transfer/GUIDE-zh.md @@ -19,8 +19,8 @@ - 做:`store remove` 删 Store(两侧副本、两块记录)。 - 不做:PostgreSQL Store 不搬(`InvalidInput: … is a PostgreSQL database; it does not move`)。 - 不做:两台非 native 机器之间没有直连。要 `docker → lima`,先 `pull` 回 native 再 `push`。 -- 不做:`firecracker`、`ssh` 平台从产品层永久禁用(`layerfs-project#7`), - 任何 `machine create/connect` 都会 `Unsupported`,不需要试。 +- 不做:`firecracker`、`ssh`、`microsandbox` 平台已在 LayerFS 2.0 移除(owner 2026-11-08), + `machine create` 会 `Unsupported`(``platform `` was removed in LayerFS 2.0``),不需要试。 一次 push / pull 固定七阶段,顺序不变: @@ -56,8 +56,8 @@ check → snapshot → send → verify → install → record → retire **按顺序、逐字**执行一次。所以请不要把多行拆开、不要改顺序。 ```bash -export LAYERFS_HOME="$HOME/LayerFS/tmp/lp08-s3/guide-home" -export FIXTURE="$HOME/LayerFS/tmp/lp08-s3/fixture/p1" +export LAYERFS_HOME="${LAYERFS_HOME:-$HOME/LayerFS/tmp/lp08-s3/guide-home}" +export FIXTURE="${FIXTURE:-$HOME/LayerFS/tmp/lp08-s3/fixture/p1}" export PATH="${LAYERFS_BIN_DIR:-$HOME/.local/bin}:$PATH" mkdir -p "$LAYERFS_HOME" chmod 700 "$LAYERFS_HOME" @@ -73,15 +73,15 @@ layerfs machine platforms ``` **预期**:版本行含 **`layerfs 2.0.0`**;`machine platforms` 是 JSON, -`native`、`docker`、`apple-container` 的 `ready` 为 **true**, -`firecracker`、`ssh` 的 `ready` 为 **false** 且带 -**`permanently disabled by product policy`** 文案(这是产品策略,不是环境问题)。 +`native`、`docker`、`apple-container` 的 `ready` 为 **true**; +已移除的 `firecracker`、`ssh`、`microsandbox` 不在列表里。 ```bash # 步骤 2:登记内置机器与 native daemon CLI_DIR="$(cd "$(dirname "$(command -v layerfs)")" && pwd)" layerfs machine update native --daemon "$CLI_DIR/layerfs-daemon" -layerfs machine update docker --docker "$HOME/.colima/default/docker.sock" +# colima: $HOME/.colima/default/docker.sock;Docker Desktop: $HOME/.docker/run/docker.sock +layerfs machine update docker --docker "${DOCKER_SOCK:-$HOME/.colima/default/docker.sock}" ``` **预期**:两条都 exit 0,各打印一条机器记录,`native` 与 `docker` 的 `state` @@ -89,8 +89,8 @@ layerfs machine update docker --docker "$HOME/.colima/default/docker.sock" `machine status`,此时 docker 还没起 worker)。 ```bash -# 步骤 3:启动 Docker 机器 -colima start +# 步骤 3:启动 Docker 机器(先 `colima start` 或打开 Docker Desktop) +docker info >/dev/null layerfs machine start docker layerfs machine status docker ``` @@ -219,12 +219,12 @@ ext4 数据盘(8 GiB 盘,本次 **7.7 GB 量级**)。push/pull 都 exit 0 --- -## 9. Apple Container:1.6.1 的丢应答、诊断与恢复 +## 9. Apple Container:稳定往返,以及丢应答时怎么办 -Apple Container 1.6.1 的端口转发在“机器 → 宿主的大块应答”上会丢连接: -`store pull` 在 `send` 阶段报 `Unknown`。**它不是数据损坏,也不是 bundle 旧**; -正确反应是**同一条命令再跑一次**(先删不可信的那半份,再重来),不要手工删文件、 -不要改用 `push`。这一段同时演示 `Unresolved` 状态的诊断、异向拒绝和恢复。 +早先的构建在 Apple Container 1.6.1 上 `store pull` 会在 `send` 阶段丢大块应答。 +LayerFS `97fac452bd` 起 worker 写完应答后等调用方先关连接,转发层不再截掉应答尾部; +`send` 的每个分片只读一次、不重试(`ab4c1a4712` 实测:所有 pull 第一次就成功)。 +这一段演示往返稳定、Store 在机器上时的方向规则,以及正常拉回。 ```bash # 步骤 10:建 Apple Container 机器,把另一个 Store 推上去 @@ -235,71 +235,52 @@ layerfs project init acshop/web "$FIXTURE" layerfs store push acshop ac --progress ``` -**预期**:`machine create` 返回 `"state":"running"`;**push 方向稳定**(LP08-S1 的 -7/7 与 LP08-S3 全部 push 都成功),exit 0。 +**预期**:`machine create` 返回 `"state":"running"`;push exit 0。 ```bash -# 步骤 11:反复 pull,直到遇到一次丢应答(本机的 1.6.1 转发经常一次就丢) +# 步骤 11:连续 10 次 pull + push 往返,每一次都应成功 for attempt in 1 2 3 4 5 6 7 8 9 10; do - if layerfs store pull acshop; then - echo "attempt $attempt: pull ok, pushing again to retry" - layerfs store push acshop ac >/dev/null || break + if layerfs store pull acshop >/dev/null && layerfs store push acshop ac >/dev/null; then + echo "round trip $attempt ok" else - echo "attempt $attempt: lost the worker's answer" + echo "round trip $attempt failed" break fi done layerfs store status acshop -layerfs doctor ``` -**预期**:某一次 `store pull` exit **125**,stderr 是 - -```text -{"code":"Unknown","message":"store pull, send: store.file: the worker's answer was lost (worker connection: failed to fill whole buffer); it may have run, and was not resent","detail":{"phase":"send"}} -``` +**预期**:`round trip 1 ok` 到 `round trip 10 ok`,没有 `failed`;`store status` 是 +**`"state":"idle"`**、`machine` 为 **`ac`**、`op:null`。 -随后 `store status` 是 **`"state":"unresolved"`**、`op` 为 -**`{"name":"store.pull","phase":"send","uncertain":true}`**、`machine` 仍是 -**`ac`**;`doctor` 是 **error**: -`store acshop has an unresolved store.pull (…, phase send): its outcome is not known`, -fix 为 **`` `layerfs store status acshop` shows it; `layerfs store pull acshop` clears it ``**。 -机器上的副本完好(`store status` 的 `file.bytes` 仍是 68,046,848),native 侧没有权威副本。 - -> 如果 10 次都没丢(转发这次健康),这一段就演示不出 `Unresolved`:可以把项目目录 -> 做大一点再试,或直接跳到步骤 15(那里演示“确定失败”的中断)。**不要**为了制造失败 -> 去手工改 `LAYERFS_HOME` 里的记录。 -> -> 反过来,如果连续多次都丢(本机 1.6.1 在负载高时会这样),只做一件环境动作: -> `layerfs machine stop ac` + `layerfs machine start ac`,转发重新建立后再跑同一条 -> `store pull`(步骤 13 的循环就是这么写的)。**不要**手工删 `.incoming`、 -> 不要改 `store push`:数据面没问题,坏的是转发层。 +> **如果某次 pull 丢了应答**(网络或转发层的问题仍可能发生),它 exit **125**,stderr 是 +> `{"code":"Unknown","message":"store pull, send: … the worker's answer was lost …; it may have run, and was not resent","detail":{"phase":"send"}}`。 +> 这时 `store status` 是 **`"state":"unresolved"`**、`op` 为 +> `{"name":"store.pull","phase":"send","uncertain":true}`、`machine` 仍是 `ac`; +> `doctor` 报 error 并给出 **`layerfs store pull acshop`** 作为清除命令;异向命令 +> (如 `store push acshop docker`)以 `Unresolved` 拒绝且不删任何副本。 +> 正确做法只有一个:**同一条 `store pull` 再跑一次**;连续丢就 +> `layerfs machine stop ac` + `layerfs machine start ac` 后再跑同一条命令。 +> **不要**手工删 `.incoming` 或改记录。可复现的同类演示见第 10 节(确定失败的中断)。 ```bash -# 步骤 12:异向命令必须拒绝,且不删任何副本 +# 步骤 12:Store 在 ac 上时,推往另一台机器必须拒绝(先拉回 native) layerfs store push acshop docker ``` **预期**:exit **125**, -`{"code":"Unresolved","message":"store push, check: Store acshop has an unresolved store.pull (op-…, phase send): its outcome is not known; see `layerfs store status acshop`, then clear it with `layerfs store pull acshop`","detail":{"entity":{"kind":"store","name":"acshop"},"phase":"check","op":"op-…"}}`。 -**两侧副本与 home 逐字节不变**(LP08-S3 `REVERSE` 实测:home 树、目标目录、文件 sha256 全同)。 +`{"code":"InvalidInput","message":"store push, check: Store acshop is on machine ac; `layerfs store pull acshop` brings it to native first","detail":{"phase":"check"}}`; +不发送任何字节,`store status` 仍是 `idle`、`machine` 为 `ac`。 ```bash -# 步骤 13:同一条命令恢复;连续丢应答就把机器重启一次(转发层重新建立) -for attempt in 1 2 3; do - if layerfs store pull acshop; then echo "recovered on attempt $attempt"; break; fi - layerfs store status acshop - layerfs machine stop ac - layerfs machine start ac -done +# 步骤 13:拉回 native +layerfs store pull acshop layerfs store status acshop ``` -**预期**:`recovered on attempt `,pull exit 0, -`{"store":"acshop","from":"ac","to":"native",…}`; +**预期**:pull exit 0,`{"store":"acshop","from":"ac","to":"native",…}`; `store status` 是 **`"state":"idle"`**、**`"machine":"native"`**、**`op:null`**; -`last_transfer.retired` 为 **true**;native 的 `store.sqlite` 的 sha256 等于 -`last_transfer.sha256`;机器上的副本已删(`file` 为 unavailable 或不存在)。 +`last_transfer.retired` 为 **true**;机器上的副本已删。 ```bash # 步骤 14:删掉这个 Store(两侧副本 + 记录) @@ -473,7 +454,7 @@ exit 0(`lm`/`ac` 的 VM 与数据盘一起删除); | `Busy: … runs in process ` | 同一 Store 有操作在跑 | 等它结束 | | `Unresolved` + fix 指向同方向命令 | 上次结果未知 | 跑 fix 里的那条命令 | | `Unknown: … the worker's answer was lost … was not resent` | 应答丢了(Apple Container 常见) | 同一命令重跑(幂等) | -| `Unsupported: firecracker/ssh is permanently disabled by product policy (layerfs-project#7)` | 平台被产品禁用 | 换 Docker/Lima/Apple Container | +| ``Unsupported: platform `firecracker` was removed in LayerFS 2.0``(`ssh`、`microsandbox` 同) | 平台已移除 | 换 Docker/Lima/Apple Container;旧记录用 `machine remove` 忘掉 | --- diff --git a/store-transfer/README.md b/store-transfer/README.md index 2da142c..69b32f4 100644 --- a/store-transfer/README.md +++ b/store-transfer/README.md @@ -82,9 +82,12 @@ prose evidence they support is in the report. second, empty home. 4. `native` needs its daemon registered before a Workspace can mount on it: `layerfs machine update native --daemon /layerfs-daemon`. -5. Bring up the routing layer: `colima start` for docker, `limactl` on - `PATH` for lima (the first `machine create` may fetch the Ubuntu image), - `container system status` for apple-container. +5. Bring up the routing layer: `colima start` (or Docker Desktop, with + `LP08_DOCKER_SOCK=$HOME/.docker/run/docker.sock` for the drivers and + `DOCKER_SOCK` for the guide) for docker, `limactl` on `PATH` for lima (the + first `machine create` may fetch the Ubuntu image), `container system + status` for apple-container. `lp08s2.py` needs `LP08_DISK=8G`: the Sandbox + image in the bundle leaves too little of a 4 GiB disk for the 512 MiB Store. 6. Fixtures: `python3 make_fixture.py /p1`, `python3 make_fixture-s2.py ` and `python3 make_fixture-s3.py /p1` — the same seeds every time, so a @@ -146,15 +149,20 @@ stayed within 0.2 %. - `install`'s ENOSPC and a `.snapshot` temporary file are **N/A**: `snapshot` and `verify` hash in place, and `install` is one `rename`. The only temporary file in the transfer is the target's `store.sqlite.incoming`. -- apple-container 1.6.1 answer loss is documented in the report: it is a - transport property of that platform, visible as extra read retries in - `LAYERFS_TRACE`'s `worker.connection` events, not in the transfer's - verdict. Do not raise a retry count to force a PASS. +- apple-container 1.6.1 used to lose the tail of large answers. Since + layerfs `97fac452bd` the worker waits for the client to close first and + the product reads each piece once (no retry); every pull in the + `ab4c1a4712` run succeeded first time. The guide's §9 shows stable round + trips; a lost answer's recovery is prose there and demonstrated by §10. +- A record of a removed platform (firecracker, ssh, microsandbox) answers + only `machine list`/`get`; `REF-*-RECORD` checks that every other call, + `machine status` included, is refused with the removal text. - `store connect` needs a quoted password (`env:NAME` or `file:/path`); the drivers read it from the server's environment and never write it down. | Date | LayerFS commit | Outcome | Report | | --- | --- | --- | --- | +| 2026-11-08 | `ab4c1a4712` | Outcome: PASS on every Store-transfer route, refusal, recovery path and the guide at layerfs `ab4c1a4712` (Ephemeral-AI-Lab/layerfs#617, merged into `layerfs_v2_0_0` as `716197c499` with the same tree), and every Apple Container pull now succeeds on its first attempt | [report](../result/2026-11-08/store-transfer/ab4c1a4712ec0e4d032a2ac341638053a02e2c91/report.md) | | 2026-11-08 | `d8dddd50d0` | Outcome: PASS on every Store-transfer route, refusal and recovery path of layerfs-project #8–#13 at the fixed build | [report](../result/2026-11-08/store-transfer/d8dddd50d04c92e4656131e71273fbe0bac0daae/report.md) | diff --git a/store-transfer/crash.py b/store-transfer/crash.py index 2ca7011..8370616 100644 --- a/store-transfer/crash.py +++ b/store-transfer/crash.py @@ -1343,7 +1343,7 @@ def after(): def setup(): HOME.mkdir(mode=0o700, parents=True, exist_ok=True) (BASE / "receipts").mkdir(parents=True, exist_ok=True) - steps = [(["machine", "update", "docker", "--docker", str(Path.home() / ".colima/default/docker.sock")], 300), + steps = [(["machine", "update", "docker", "--docker", os.environ.get("LP08_DOCKER_SOCK", str(Path.home() / ".colima/default/docker.sock"))], 300), (["machine", "update", "native", "--daemon", DAEMON], 300)] for argv, timeout in steps: result = call(argv, timeout=timeout) diff --git a/store-transfer/guide_repro.py b/store-transfer/guide_repro.py index b444b28..7006f70 100644 --- a/store-transfer/guide_repro.py +++ b/store-transfer/guide_repro.py @@ -97,8 +97,8 @@ def main(): out, err = read_block(i) check(i, "layerfs is 2.0.0", "layerfs 2.0.0" in out, out[:300]) check(i, "native/docker/apple-container are ready", out.count('"ready":true') >= 3, out[:200]) - check(i, "firecracker and ssh are refused by product policy", - "permanently disabled by product policy" in out, "policy text present") + check(i, "removed platforms (firecracker, ssh, microsandbox) are not listed", + not any('"%s"' % p in out for p in ("firecracker", "ssh", "microsandbox")), out[:400]) check(i, "the block exits 0", exits[i] == 0, exits[i]) i = 2 @@ -191,29 +191,23 @@ def main(): i = 11 out, err = read_block(i) - check(i, "one pull lost the worker's answer", "lost the worker's answer" in out, out[-600:]) - check(i, "the loss is Unknown at send with the documented text", - '"code":"Unknown"' in err and '"phase":"send"' in err and "answer was lost" in err, err[-600:]) - check(i, "status reads unresolved with an uncertain pull at send", - '"state":"unresolved"' in out and '"name":"store.pull"' in out - and '"phase":"send"' in out and '"uncertain":true' in out, out[-1200:]) - check(i, "the store still points at apple-container and keeps its copy", - '"machine":"ac"' in out and '"bytes":68046848' in out.replace(" ", ""), out[-900:]) - check(i, "doctor reports it as an error and names the same pull as the fix", - "unresolved store.pull" in out and "store pull acshop" in out, out[-1400:]) + check(i, "ten round trips to apple-container succeed", + all(f"round trip {n} ok" in out for n in range(1, 11)) and "failed" not in out, out[-600:]) + check(i, "status is idle on ac with no op", + '"state":"idle"' in out and '"machine":"ac"' in out and '"op":null' in out.replace(" ", ""), out[-900:]) check(i, "the block exits 0", exits[i] == 0, exits[i]) i = 12 out, err = read_block(i) refusal = last_json(err) or {} - check(i, "the opposite direction is refused with exit 125", exits[i] == 125, exits[i]) - check(i, "the refusal is Unresolved and names the same pull as the clearing call", - refusal.get("code") == "Unresolved" and "store pull acshop" in (refusal.get("message") or ""), - json.dumps(refusal)[:600]) + check(i, "the push to another machine is refused with exit 125", exits[i] == 125, exits[i]) + check(i, "the refusal is InvalidInput at check and names the pull to native first", + refusal.get("code") == "InvalidInput" and "store pull acshop" in (refusal.get("message") or "") + and (refusal.get("detail") or {}).get("phase") == "check", json.dumps(refusal)[:600]) i = 13 out, err = read_block(i) - check(i, "the same pull recovers", "recovered on attempt" in out, out[-400:]) + check(i, "the pull comes back from ac", '"from":"ac"' in out and '"to":"native"' in out, out[-400:]) check(i, "status is idle on native", '"machine":"native"' in out and '"state":"idle"' in out, out[-500:]) check(i, "the block exits 0", exits[i] == 0, exits[i]) diff --git a/store-transfer/lp08s2.py b/store-transfer/lp08s2.py index 1f04aa5..72ef11d 100644 --- a/store-transfer/lp08s2.py +++ b/store-transfer/lp08s2.py @@ -43,7 +43,7 @@ MIB = 1 << 20 MACHINE = {"docker": "docker", "lima": "lm-s2", "ac": "ac-s2"} PLATFORM = {"docker": "docker", "lima": "lima", "ac": "apple-container"} -DISK = {"lima": "4G", "ac": "4G"} +DISK = {"lima": os.environ.get("LP08_DISK", "4G"), "ac": os.environ.get("LP08_DISK", "4G")} STORE = {"256": "cap256", "512": "cap512", "quiet": "quiet-s"} CASES = {} @@ -934,11 +934,14 @@ def ac_pull_sweep(cid="R11-A-PULL-SWEEP"): large = [row for row in rows if row["bytes"] > 40 * MIB] check(cid, "every push to the Apple Container succeeded", all(row["push_exit"] == 0 for row in rows), json.dumps(rows)) + # PR #617: v2's worker waits for the client to close (97fac452bd) and the + # product reads each piece once, so every size must pull on its first attempt check(cid, "Stores up to 32 MiB pull on their first attempt", bool(small) and all(row["pull_exit"] == 0 and row["pull_attempts"] == 1 for row in small), json.dumps(small)) - check(cid, "above 32 MiB a pull needs retries or never succeeds", - all(row["pull_attempts"] > 1 for row in large), json.dumps(large)) + check(cid, "Stores above 32 MiB pull on their first attempt too", + bool(large) and all(row["pull_exit"] == 0 and row["pull_attempts"] == 1 for row in large), + json.dumps(large)) check(cid, "a failed pull never reports success and never damages the source", all(row["pull_exit"] in (0, 125) for row in rows), json.dumps(rows)) return rows @@ -1085,7 +1088,7 @@ def setup(): closed = close_sandboxes() print("closed sandboxes:", closed) steps = [ - (["machine", "update", "docker", "--docker", str(Path.home() / ".colima/default/docker.sock")], 600), + (["machine", "update", "docker", "--docker", os.environ.get("LP08_DOCKER_SOCK", str(Path.home() / ".colima/default/docker.sock"))], 600), (["machine", "update", "native", "--daemon", DAEMON], 600), (["machine", "start", "docker"], 1800), ] diff --git a/store-transfer/matrix.py b/store-transfer/matrix.py index f29acce..86a41ac 100644 --- a/store-transfer/matrix.py +++ b/store-transfer/matrix.py @@ -610,7 +610,7 @@ def cleanup(): def setup(): HOME.mkdir(mode=0o700, parents=True, exist_ok=True) steps = [ - (["machine", "update", "docker", "--docker", str(Path.home() / ".colima/default/docker.sock")], 300), + (["machine", "update", "docker", "--docker", os.environ.get("LP08_DOCKER_SOCK", str(Path.home() / ".colima/default/docker.sock"))], 300), (["machine", "update", "native", "--daemon", DAEMON], 300), ] for argv, timeout in steps: @@ -787,7 +787,9 @@ def refusals(): password = subprocess.run(["docker", "exec", "lp08-pg", "printenv", "POSTGRES_PASSWORD"], capture_output=True, text=True, timeout=60).stdout.strip() url = os.environ.get("LP08_PG_URL") or "postgres://postgres@127.0.0.1:55432/postgres" - os.environ["LP08_PG_PASSWORD"] = password + # without an lp08-pg server a placeholder still reaches the engine check: + # a build without the PostgreSQL engine refuses before it connects + os.environ["LP08_PG_PASSWORD"] = password or "no-server-placeholder" connected = call(["store", "connect", "ref-pg", url, "--password", "env:LP08_PG_PASSWORD"], timeout=300) ran(cid, connected) check(cid, "store connect is refused in this build without the PostgreSQL engine", @@ -969,13 +971,14 @@ def filling(): started = machine_start("lm") check(cid, "machine start lm brings it back", started["exit"] == 0, started["stderr"][:200]) - # ---- REF-FC-CREATE / REF-SSH-CONNECT: the #7 product refusal - fc_text = "firecracker is permanently disabled by product policy (layerfs-project#7)" - ssh_text = "ssh is permanently disabled by product policy (layerfs-project#7)" + # ---- REF-FC-CREATE / REF-SSH-CREATE: removed platforms (owner 2026-11-08, + # superseding the #7 "disabled" policy); `machine connect` no longer exists + fc_text = "platform `firecracker` was removed in LayerFS 2.0" + ssh_text = "platform `ssh` was removed in LayerFS 2.0" result, bad = refusal_item("REF-FC-CREATE", ["machine", "create", "fctmp", "firecracker"], "Unsupported", None, None) check("REF-FC-CREATE", "the authoritative firecracker text is used", bad.get("message") == fc_text, bad.get("message")) - result, bad = refusal_item("REF-SSH-CONNECT", ["machine", "connect", "sshtmp", "10.0.0.5"], "Unsupported", None, None) - check("REF-SSH-CONNECT", "the authoritative ssh text is used", bad.get("message") == ssh_text, bad.get("message")) + result, bad = refusal_item("REF-SSH-CREATE", ["machine", "create", "sshtmp", "ssh"], "Unsupported", None, None) + check("REF-SSH-CREATE", "the authoritative ssh text is used", bad.get("message") == ssh_text, bad.get("message")) # ---- REF-FC-RECORD / REF-SSH-RECORD: a hand-written record cannot be used for cid, machine, platform, text in (("REF-FC-RECORD", "fc1", "firecracker", fc_text), @@ -996,6 +999,8 @@ def filling(): battery = [ (["machine", "start", machine], "Unsupported"), (["machine", "update", machine, "--cpus", "2"], "Unsupported"), + # a removed platform's record answers only machine list and get + (["machine", "status", machine], "Unsupported"), (["store", "push", native, machine], "Unsupported"), (["store", "pull", store], "Unsupported"), (["sandbox", "create", f"{store}/p1", f"newsb-{machine}", machine], "Unsupported"), @@ -1016,14 +1021,14 @@ def filling(): check(cid, f"layerfs {' '.join(argv)} carries the authoritative text", bad.get("message") == text, str(bad.get("message"))[:200]) kept = [ - ["machine", "list"], ["machine", "get", machine], ["machine", "status", machine], + ["machine", "list"], ["machine", "get", machine], ["store", "list"], ["store", "get", store], ["store", "status", store], ["doctor"], ] for argv in kept: outcome = call(argv, timeout=300) ran(cid, outcome) check(cid, f"layerfs {' '.join(argv)} stays available", outcome["exit"] == 0, outcome["stderr"][:200]) - teardown_argv = ["machine", "remove", machine] if platform == "firecracker" else ["machine", "disconnect", machine] + teardown_argv = ["machine", "remove", machine] blocked = call(teardown_argv, timeout=300) ran(cid, blocked) note(cid, f"{' '.join(teardown_argv)} while records still name the machine: exit {blocked['exit']} " @@ -1037,7 +1042,7 @@ def filling(): removed = call(teardown_argv, timeout=300) ran(cid, removed) removed_bad = refusal(removed) or {} - check(cid, f"layerfs {' '.join(teardown_argv)} is not refused by the #7 policy", + check(cid, f"layerfs {' '.join(teardown_argv)} is not refused by the removal policy", removed_bad.get("code") != "Unsupported", f"exit {removed['exit']} {json.dumps(removed_bad)[:240]}") note(cid, f"{' '.join(teardown_argv)} on the hand-written record: exit {removed['exit']} "