Skip to content

verify: make the harness suite runnable off this one machine (dd bs=1M + disk.img builder) - #125

Merged
douglasmun merged 2 commits into
mainfrom
fix/harness-portability-and-diskimg
Aug 23, 2026
Merged

verify: make the harness suite runnable off this one machine (dd bs=1M + disk.img builder)#125
douglasmun merged 2 commits into
mainfrom
fix/harness-portability-and-diskimg

Conversation

@douglasmun

Copy link
Copy Markdown
Owner

Why

The first full nightly harness run scored 53 FAIL / 5 INCONCLUSIVE / 5 PASS. None of it was a kernel defect. Two fixtures that existed only on the machine the harnesses were written on account for all of it — and both fail in a way that reads as a kernel regression rather than a missing prerequisite.

1. dd bs=1m — 14 harnesses

bs=1m is a BSD-ism; the GNU suffix is case-sensitive. Confirmed both directions rather than assumed:

$ docker run --rm alpine dd if=/dev/zero of=/tmp/a bs=1m count=1
dd: invalid number '1m'
$ docker run --rm alpine dd if=/dev/zero of=/tmp/b bs=1M count=1
1048576 bytes (1.0MB) copied

BSD dd accepts 1M identically (verified locally: 2097152 bytes for bs=1M count=2). So bs=1M is correct on both — a strict portability fix, no behaviour change on macOS.

2. disk.img — 51 harnesses reference it

51 harnesses copy disk.img to a scratch path and exit 2 without it, and nothing in the repo built it. It was an untracked, gitignored 128 MB binary that had only ever been made by hand. Invisible locally — the file is just there — and fatal anywhere else.

tools/make-disk-img.sh now builds it reproducibly via mtools, and the nightly workflow calls it before the suite. It stays gitignored: a 128 MB file, ~99.99% zeroes, to carry one 14 KB fixture that is already tracked.

Two properties are load-bearing, and the script asserts them itself rather than letting them surface 800 s later inside a harness:

  • It must be a real FAT32 volume. verify-ring3-fatls.sh checks the FAT32 signature at offset 0x52 and refuses a zeroed image — against one, C: would not mount and ls C:/ would fail for a reason unrelated to what it tests, a vacuous run scoring as a kernel bug. Verified a zeroed image is correctly rejected by that check.
  • C:/HELLO.ELF must be exactly 14144 bytes. userspace/fileio.c opens it O_RDONLY and lseeks SEEK_END; verify-fsyscalls.sh asserts exactly 14144. The fixture is userspace/info.elf.signed, which is tracked and is that size — not hello.elf.signed, which is 14388 and would fail. The hand-made image held a stale 14144-byte HELLO.ELF from an older signing run, which is why the name never matched its content.

Verification

Both disk paths were run end to end against the generated image:

Harness Exercises Result
verify-ring3-fatls.sh FAT32 signature check + ls C:/ PASS
verify-fsyscalls.sh the hardcoded 14144 SEEK_END assertion PASSfileio: fat32 seek end=14144

Guest-side evidence that the generated geometry is what the driver reads:

[IDE] Disk capacity: 128 MB (262144 sectors)
[FAT32] Sectors per cluster: 2
[FAT32] Root directory cluster: 2
[FAT32] Mount successful [OK]

Geometry matches the hand-made original exactly (512 B sectors, 2 sec/cluster, 32 reserved, 2 FATs, 63 sec/track, 16 heads, 262144 sectors); only the volume serial and free-cluster count differ.

The workflow YAML is schema-checked (YAML.safe_load) — this caught a genuine error in my own first draft, where a step named ... mount as C: made YAML parse the trailing colon as a nested mapping and would have broken the entire nightly job.

Not in scope

  • CLAUDE.md is deliberately untouched. This branch is off main, which does not yet carry the CLAUDE.md trim in audit: state that the log is volatile, and trim CLAUDE.md #124; editing it here would collide. The doc note belongs in audit: state that the log is volatile, and trim CLAUDE.md #124's version once that merges.
  • Two unrelated things surfaced while reading logs, both left alone: src/fat32.c:753 prints [FAT32] Volume label: %.11s literally (TinyOS's formatter has no precision support — same class as the earlier %zu bug), and verify/firstexec-trial.sh is gitignored so CI never checks it out (harmless, it is on the exclusion list anyway).

This does not claim the suite now passes — it removes the two environmental causes so the remaining failures can be read as what they actually are.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CEkhAhgTxbE5TgifyYf8v4

douglasmun and others added 2 commits August 23, 2026 18:32
The first full nightly run scored 53 FAIL / 5 INCONCLUSIVE / 5 PASS. None of
it was a kernel defect. Two fixtures that existed only on the machine the
harnesses were written on accounted for all of it, and both failed in a way
that reads as a kernel regression rather than a missing prerequisite.

1. `dd bs=1m` (14 harnesses)

   `bs=1m` is a BSD-ism: the GNU suffix is case-sensitive. Confirmed both
   directions rather than assumed -- under GNU coreutils `bs=1m` fails with
   "dd: invalid number '1m'" while `bs=1M` writes 1048576 bytes, and BSD dd
   accepts `1M` identically. So every one of these harnesses died creating its
   scratch disk on a Linux runner, and this is a strict portability fix with
   no behaviour change on macOS.

2. disk.img (51 harnesses reference it)

   51 harnesses copy disk.img to a scratch path and exit 2 without it, and
   nothing in the repo built it -- it was an untracked, gitignored 128 MB binary that had only ever
   been made by hand. On a runner it simply did not exist.

   tools/make-disk-img.sh now builds it reproducibly via mtools, and the
   nightly workflow calls it before the suite. It stays gitignored: a 128 MB
   file, ~99.99% zeroes, to carry one 14 KB fixture that is already tracked.

   Two properties are load-bearing and are asserted by the script itself
   rather than left to be discovered 800 s later inside a harness:

     - It must be a REAL FAT32 volume. verify-ring3-fatls.sh checks the FAT32
       signature at offset 0x52 and refuses a zeroed image, because against
       one C: would not mount and `ls C:/` would fail for a reason unrelated
       to what it tests -- a vacuous run scoring as a kernel bug. (Verified:
       a zeroed image is correctly rejected by that check.)

     - C:/HELLO.ELF must be exactly 14144 bytes. userspace/fileio.c opens it
       O_RDONLY and lseeks SEEK_END; verify-fsyscalls.sh asserts the result is
       exactly 14144. The fixture used is userspace/info.elf.signed, which is
       tracked and is that size. NOT hello.elf.signed, which is 14388 and
       would fail -- the hand-made image held a stale 14144-byte HELLO.ELF
       from an older signing run, which is why the name never matched the
       content.

Verified locally against the generated image: the guest's IDE driver reports
262144 sectors and FAT32 mounts with the expected 2 sectors/cluster and root
cluster 2; verify-ring3-fatls.sh RESULT: PASS.

Geometry matches the hand-made original exactly (512 B sectors, 2 sec/cluster,
32 reserved, 2 FATs, 63 sec/track, 16 heads, 262144 sectors); only the volume
serial and free-cluster count differ.

Verified: verify-ring3-fatls.sh RESULT: PASS and verify-fsyscalls.sh
RESULT: PASS ("fileio: fat32 seek end=14144") against the generated image.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CEkhAhgTxbE5TgifyYf8v4
The hand-made image held two files; the generated one held only HELLO.ELF.
Nothing opens INFO.ELF -- the only two mentions in the tree are a usage example
in qemu_typist.py and a comment in verify-exec-frame-leak.sh warning NOT to
exec it (larger image, would move that harness's expected page count) -- so
dropping it was safe.

Including it anyway costs one mcopy and makes the generated image a faithful
replacement rather than a subset, so "it worked before" cannot quietly mean
something different. Free space now matches the original exactly
(133,131,264 bytes).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CEkhAhgTxbE5TgifyYf8v4
@douglasmun

Copy link
Copy Markdown
Owner Author

Classified the first nightly run's 58 non-passing rows against this branch's fixes, and validated the generated image on a third harness.

Coverage of this PR: 48 of 58 are the two causes fixed here — 35 × disk.img not found, 13 × dd: invalid number '1m'. Several of the rest are downstream of the same two rather than independent:

  • verify-secure-boot-scope.sh FAILs on "exec of a signed binary no longer works" — its leg 5 delegates to auto-verify-exec.sh, which died on dd bs=1m. Chain confirmed from the logs, not inferred.
  • verify-ids-spray.sh (dd), verify-dns-noprivinsn.sh and verify-tcp-rx-counters.sh (disk.img) likewise.

Third validation of the generated image: verify-fat32-write.shRESULT: PASS — FAT32 writes persist across reboot. That one writes to the volume, reboots, and reads it back from the platter, so it exercises writability and dirent update, not just mounting. Also re-ran verify-ring3-fatls.sh after adding INFO.ELF back (983b78b): still PASS.

But the suite will not go green on this PR alone, and the reason is worth stating plainly: filed as #126.

Every CI boot that got far enough to start a guest panicked identically — EIP=0x00000000, error code 0x11 (present | instruction fetch), i.e. a call through a null function pointer, right after the scheduler brings all six tasks up. Four boots, same signature, deterministic. It does not reproduce under the local i686-elf cross.

The sharpest form of it: all 5 of the run's PASSes are static source checks that never boot a guest, and no CI boot has ever reached the login prompt. So this PR unblocks ~48 harnesses into a kernel that, under the CI toolchain, does not boot. Both fixes are needed; neither substitutes for the other.

@douglasmun
douglasmun merged commit e83f2eb into main Aug 23, 2026
2 checks passed
@douglasmun
douglasmun deleted the fix/harness-portability-and-diskimg branch August 23, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant