platform: combine remaining POSIX, runtime, and SQLite fixes by brandonpayton · Pull Request #934 · Automattic/kandelo

brandonpayton · 2026-07-13T20:08:23Z

Why

This pull request batches already reviewed fixes so one complete continuous-integration (CI) run can test how they work together before merge. It preserves one commit per purpose, which keeps the history understandable and bisectable.

#907 already landed PHP 8.3.15, ABI 39, and the shared process, VFS,
networking, host-runtime, package, browser-image, Dinit, and Vitest fixes found
during that port. This batch does not replay those changes.

This PR brings the remaining reviewed follow-ups onto one current-main branch
so we can validate and land them together while keeping one commit per purpose.
The focused source PRs remain the review record; their exact heads are listed
below. They were closed only after this batch contained their work and the
combined CI run was green.

What changed

The batch carries the focused changes listed below onto current main. ABI means the application binary interface between programs, the kernel, and the host; VFS means the virtual file system; libc is the C standard library used by guest programs.

Unique work that remains

Browser regressions and wiring: stale clearImmediate handles, nonzero-exit
stderr, and configured CORS proxy routing.
VFS and process contracts: positioned SharedFS I/O, canonical rootfs overlay
assembly, zero-byte copy-back, initial credentials, mremap metadata,
ENOLCK, and directory fsync.
libc and ABI-adjacent tests: zero unreported stat fields, align wasm32
pthread entry stacks, deterministic fork instrumentation, and SjLj/noexcept
coverage.
Package/build cleanup: shared MariaDB writable-directory setup.
SQLite contract work: supported recursion limits, truthful all-mode outcome
evidence, UTF-16 STAT4 accounting, and TestRecover command lifetime.

Exact source heads

PR	Exact reviewed head	What this batch keeps
#782	`3f133f92bd971cc83fecefcd64709ed7b9f13ade`	`clearImmediate` cancellation cleanup
#784	`fc25245984f346a5620998f02f93ad9b85dae80b`	wasm32 pthread stack alignment and direct regression
#786	`905dbc0091a40ad9dce04f6ebc3e36e113b40129`	positioned SharedFS reads/writes
#875	`545048b123d9efabc1474d257329e4e6d154a0f6`	Chromium nonzero-exit stderr regression
#880	`79dd37dbffa2d373988585f954eb45fb162f7c4c`	CORS proxy runner wiring, assertion, docs, and integration test
#888	`ce70dc0061ce379d2d50872b48ee0b82908eb8bc`	fresh-process fork output determinism
#896	`224aeeb056829c190680e78bc1ac9a4d68398b54`	LAMP use of the shared MariaDB helper and dead helper removal
#923	`6ad38869d71274e773530a9d0860680292235dd1`	shared rootfs overlay helper and strict preservation tests
#924	`432857531a6969e9ae7d397c2c59406bfb21214d`	zero-byte syscall output-buffer preservation
#925	`8943e3155d495f9908f8c9ecc2d0e1e06f438805`	SjLj/noexcept controls, SIGCHLD fixture, and SDK boundary docs
#926	`471dc2335abe77c2c696f7bfc1d0cce529710ad7`	requested initial user/group credentials
#927	`2a8bfbcd6ae7f7ebb5799a8e2db3c05bb491cf02`	metadata-consistent `mremap` growth
#929	`38af72dcaa3c1550f572a63eaa2b7c79c5016218`	shared lock-table exhaustion as `ENOLCK`
#930	`c9f9a1a649c5551f02a24ab5487becccabb6068b`	directory `fsync` across kernel, host, and browser
#931	`c6f4977f3ea2c6bd909e4daaf3f6fbb01c7ca716`	zero initialization for unreported `stat` fields
#780	`4b04ff3de6b022549250358f6fe44f02b4475f20`	supported SQLite expression/JSON/compound limits
#781	`9e3fd3f6d8ae1693076906f6e462ad6e1f8113a3`	supported SQLite trigger recursion limit
#783	`f758ecb30b92b9c914e950094706f05169bb205e`	truthful SQLite all-mode runner and durable evidence
#932	`52ba9a72036d2bfc631157c737f91ed61093713d`	UTF-16 STAT4 allocation range regression
#933	`a0bc004f569898244f03989558086663d2ba1d78`	TestRecover Tcl command/handle lifetime

The batch preserves the 20 reviewed purpose changes in their original order. The current branch is a linear rebase onto main at ab952e6863c89561ed659f27ee6dacaa317b4e2b: all 20 range-diff entries are =, no merge commits remain, and no manual conflict-resolution bytes were introduced. The resulting repository tree is db3d4d21560a55ffacab337510ecc40094f0cf44.

The #780 and #781 changes remain one SQLite recursion/stack-limit policy commit, and the two #926 commits remain one credentials commit. #783 remains represented by one all-mode runner commit and one truthful-outcome-evidence commit. #931 keeps the reviewed code and test bytes; its docs/posix-status.md context follows the preceding #929 and #930 updates. #891 remains excluded because #936 owns that Homebrew workflow rename.

What was deliberately removed

mmap/munmap work, CORS runtime plumbing, open-unlinked inode lifetime,
fork-instrument implementation, MariaDB helper implementation, serialized
image coverage, Dinit production fixes, and other work already on main
through Batch PHP support and bundle ABI updates with their runtime changes #907;
stale Make teardown reclamation CI wait for a blocked worker #889 ancestry and the old composite Fix mmap accounting under partial munmap churn #769 ancestry;
the rejected fork: size continuation save buffers from live frames #862 ABI-17 cursor/frame-counter design;
SQLite no_mutex workarounds, fabricated outcome/database evidence, fake
host metadata, synthetic stat block counts, capacity bumps, and broad
pager-fault test omissions;
package revision bumps where declared output bytes do not change.

ABI and package-artifact impact

ABI stays at 39. The ABI snapshot and generated C/TypeScript bindings remain in
sync. The wasm32/wasm64 libc artifacts change for stat; wasm32 libc also
changes for pthread stack alignment. SQLite revisions 3 and 4 intentionally
cover the published recursion-limit byte changes. #896, #932, and #933 add no
extra revision because their declared package outputs do not change.

Combined staging must rebuild the affected toolchain and package closure. This
PR makes no performance claim; performance was not measured.

Validation

Exact current head: 3e3070b5ead7a42ad0e3821c7e96cc3131d9d513
Exact current base: ab952e6863c89561ed659f27ee6dacaa317b4e2b
Exact current tree: db3d4d21560a55ffacab337510ecc40094f0cf44

The branch contains exactly 20 linear purpose commits and zero merge commits. git range-diff maps all 20 reviewed commits to the current rebased commits with =.
Exact-head staging run 29511275381 is running on the corrected linear history.
Exact-head browser smoke run 29511275326 passed on the corrected linear history.
Staging run 29293767304 passed the same reviewed purpose tree with 84 successful jobs, including 14 library rebuilds, 57 package/image rebuilds, materialization, browser assets, cargo/kernel, fork instrumentation, Vitest, browser, libc, POSIX, Sortix, and the aggregate gate.
Browser smoke run 29293767232 passed Chromium, Firefox, and WebKit coverage on that reviewed purpose tree.
Prepare Merge run 29458645742, attempt 2 passed all 85 substantive package and runtime jobs. Its old finalizer alone failed because it loaded lifecycle code from the PR head; merged ci: seal merge candidates with trusted base helpers #964 fixes that exact control-plane defect and is part of the current base.
ABI snapshot validation, host declaration generation, focused SharedFS/VFS tests, wasm32/wasm64 musl builds, 1,154 cargo/kernel tests, and 187 fork-instrument tests passed on the reviewed purpose series. Each focused source PR records its contract-specific evidence.

Validation not run

A manual ./run.sh browser pass and the full SQLite upstream matrix were not
rerun locally. Performance was not measured, and this PR makes no performance
claim.

Merge and cleanup

Keep the 20 purpose commits in order and use Rebase and merge only; do not squash them or introduce merge commits. Apply ready-to-ship only after the exact current-head staging and browser runs are green, then require Prepare Merge and merge-gate=success against an unchanged main. After merge, require candidate activation to finish successfully before treating the ABI 39 package ledger as published.

The absorbed focused PRs and their branches were already closed and deleted with exact-SHA leases. This PR intentionally adds no batch manifest or batch-management files.

github-actions · 2026-07-13T22:30:53Z

Phase B-1 matrix build status — `pr-934-staging`

ABI v39. 71 built, 0 failed, 71 total.

Package	Arch	Status	Sha
icu	wasm32	built	`fb59d3c6`
libcurl	wasm32	built	`f9f653e6`
libcxx	wasm32	built	`07d40bfc`
libcxx	wasm64	built	`5081cd82`
libiconv	wasm32	built	`94d10414`
libpng	wasm32	built	`be4f14cc`
libxml2	wasm32	built	`7b5e9ca8`
libzip	wasm32	built	`eca038cc`
openssl	wasm32	built	`96904ce5`
openssl	wasm64	built	`587dd133`
sqlite	wasm32	built	`071fb186`
sqlite	wasm64	built	`86557407`
zlib	wasm32	built	`58606bc9`
zlib	wasm64	built	`a21688b9`
bc	wasm32	built	`decbdd49`
bzip2	wasm32	built	`23082b55`
coreutils	wasm32	built	`d96858c0`
curl	wasm32	built	`5d84b276`
dash	wasm32	built	`b1f5ee2e`
diffutils	wasm32	built	`740be0bd`
dinit	wasm32	built	`a5496db6`
fbdoom	wasm32	built	`54a5e3c0`
file	wasm32	built	`a8e59999`
findutils	wasm32	built	`35f644aa`
gawk	wasm32	built	`f5141872`
git	wasm32	built	`82eda20c`
grep	wasm32	built	`4bc4ba3b`
gzip	wasm32	built	`a8aacc0b`
hello	wasm32	built	`d0dddbb8`
kandelo-sdk	wasm32	built	`0081fbf4`
kernel	wasm32	built	`13dd15de`
less	wasm32	built	`59b31fe3`
lsof	wasm32	built	`847dbb12`
m4	wasm32	built	`a4991861`
make	wasm32	built	`a342cba2`
mariadb	wasm32	built	`1a8f6840`
mariadb	wasm64	built	`9488dbce`
modeset	wasm32	built	`fab20aee`
msmtpd	wasm32	built	`fa522960`
nano	wasm32	built	`4f8c8890`
ncurses	wasm32	built	`5408d40a`
netcat	wasm32	built	`e999d3ab`
nginx	wasm32	built	`a1235a4a`
php	wasm32	built	`5f576c56`
posix-utils-lite	wasm32	built	`00e10a95`
ruby	wasm32	built	`95add9d9`
sed	wasm32	built	`003700fe`
spidermonkey	wasm32	built	`22353182`
tar	wasm32	built	`64c2405a`
tcl	wasm32	built	`cbe8b5ee`
unzip	wasm32	built	`6ca32f45`
userspace	wasm32	built	`4833b618`
vim	wasm32	built	`1801b00c`
wget	wasm32	built	`9a0257b4`
xz	wasm32	built	`5416789e`
zip	wasm32	built	`14f828d3`
zstd	wasm32	built	`0f0ef84b`
bash	wasm32	built	`e79710a1`
mariadb-test	wasm32	built	`2d6fc95b`
mariadb-vfs	wasm32	built	`04a057b5`
mariadb-vfs	wasm64	built	`a32db09d`
nethack	wasm32	built	`7baf00e5`
node	wasm32	built	`381e45c1`
spidermonkey-node	wasm32	built	`58b3731d`
vim-browser-bundle	wasm32	built	`4b623e65`
nethack-browser-bundle	wasm32	built	`1f1035b8`
rootfs	wasm32	built	`bbc67a72`
shell	wasm32	built	`d2a14d0d`
lamp	wasm32	built	`ce836bad`
node-vfs	wasm32	built	`3ea36f3f`
wordpress	wasm32	built	`4b20422b`

_{Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.}

brandonpayton · 2026-07-14T05:44:53Z

Merge-order note for the Homebrew migration: keep this PR queued behind #913. #913 has already completed its durable package promotion against the current main tree. Landing #934 first would change that tree and require #913 to be rebased and its package evidence rebuilt before merge. After #913 lands, rebase this exact reviewed batch onto the new main and rerun the required gate; its current 86/86 successful jobs remain review evidence, but are not a substitute for the post-#913 synthetic merge.

brandonpayton · 2026-07-16T03:41:20Z

Root cause of the final gate failure is fixed by #964. All 85 substantive jobs in Prepare run 29458645742 passed. merge-gate-post alone failed because it checked lifecycle helpers out from this older PR head, where mark-merge-candidate-ready.sh does not exist, instead of from the tested base that owns the write-authorized helper. Rerunning the same run preserves the old workflow and fails again. After #964 merges, reapply ready-to-ship here to start a fresh Prepare run against the corrected workflow.

## Why `Prepare merge` seals the exact package candidate that continuous integration tested, then posts the `merge-gate` status that allows a maintainer to merge the pull request. That final job can write release assets, so the helper code it executes must come from Kandelo's trusted base branch. The workflow currently checks out lifecycle helpers from the pull request head instead. This breaks older pull requests and crosses the trust boundary in the wrong direction. PR #934 demonstrates the concrete failure: all 85 substantive jobs passed, but its older head does not contain `.github/scripts/mark-merge-candidate-ready.sh`. The write-authorized `merge-gate-post` job checked out that old head and failed with exit 127 before it could seal the tested candidate. Rerunning the same workflow repeats the failure. ## What changed - Check out candidate lifecycle helpers from the exact base SHA that `synthesize-merge` used to build and test the candidate. - Keep the existing base-drift check, so the job fails before sealing if `main` advanced during the run. - Add workflow-contract assertions that require the prepared-base ref and reject a pull-request-head ref for write-authorized helpers. - Document why pull request bytes are candidate data, not merge-gate authority. This changes no runtime, kernel ABI, package recipe, archive, virtual filesystem image, or browser/Node behavior. ## Validation Exact commit: `564a9a874ae55373d54e3c0dbcc863d176f16677` Exact tree: `280de4b0ede54a9393dc90cc39a673a8af2e3d2f` - `./scripts/dev-shell.sh bash .github/scripts/test-merge-candidate-workflows.sh` passed. - `./scripts/dev-shell.sh bash -n .github/scripts/test-merge-candidate-workflows.sh` passed. - `git diff --check` passed. - The incident shape was reproduced directly: the helper is absent from #934 head `822f35ea...` and present on prepared base `44258756...`. - Independent adversarial review approved this exact change with no findings and confirmed the workflow with `actionlint`. - [Exact-head staging run 29469545391](https://github.com/Automattic/kandelo/actions/runs/29469545391) passed change-scope, preflight, publisher trust, package-publication flow, binary materialization, kernel/test workspace preparation, cargo/kernel, fork instrumentation, full Vitest, browser, libc, POSIX, Sortix, and the aggregate test gate. Package staging correctly skipped because no package inputs changed. Runtime, browser, libc, POSIX, and Sortix suites were not rerun locally. The exact-head hosted staging run above exercised all of them. ## Recovery order Merge this focused workflow fix first. Do not rerun the existing failed #934 workflow, because GitHub reruns retain its original workflow revision. Reapply `ready-to-ship` to #934 after this change is on `main`; the fresh event will use the corrected workflow and validate against the new base.

Configure the browser test runner with its same-origin Vite proxy and keep a Chromium Wget regression for pages without a service-worker controller. Assert that BrowserKernel forwards the existing option and document the browser transport boundary.

The deterministic region ordering already landed through #907, but its in-process regression does not vary Rust HashMap random state. Run the CLI in twelve fresh processes against an alternating-type nested-region fixture and compare the emitted bytes. Document the cross-process byte-reproducibility contract and update the fork-instrumentation reference to the current ABI 39. This adds no ABI or package-artifact change. Validation: the full fork-instrument suite passed 187 tests; the focused fresh-process test passed; and the ABI snapshot check passed through scripts/dev-shell.sh.

Move the recursive /etc merge landed with #907 out of the browser app and into the shared host VFS layer. Preserve caller-owned entries and source metadata, and keep missing canonical state, short reads, and capacity failures visible during image assembly.

Add raw wasm32/wasm64 and fork-instrumented controls for issue #918, plus a positive SIGCHLD child-reaping fixture. Cover the behavior in Node and Chromium and document the pinned LLVM 21 limitation without replaying Dinit production or package changes from #911.

Delegate fsync on directory descriptors to the active filesystem backend so Node can issue its native durability barrier. Keep memory-backed and OPFS behavior truthful at their host boundaries, and cover the kernel, host backends, Chromium OPFS, and POSIX regression path.

Apply one supported recursion policy to both the shipped library and official testfixture: compound-select, expression, JSON, and trigger recursion stay within current browser and Node WebAssembly stacks. Keep the test patches aligned with those compiled limits and preserve the existing package revision recorded by the reviewed batch tree.

Keep analyze9's memory-accounting assertion, but use the encoding-correct allocation delta for SQLite's utf16 permutation.

Align the stack passed to kernel_clone to the 16-byte boundary required by Wasm code generation. This keeps stack-based variadic arguments intact when a new pthread begins. Add a local Sortix regression that formats the SQLite temporary-file pattern from fresh pthreads. The kernel_clone interface and ABI layout are unchanged.

brandonpayton · 2026-07-16T15:30:02Z

Corrected the branch history before Prepare Merge. The prior head db6436e288171af3e2c7650fcf1249872e96b1e6 contained the 20 reviewed purpose commits plus two merge commits that pulled #964 and #963 into the branch. Prepare would have recorded 22 PR commits, while the required rebase merge would land 20; candidate activation would then reject the result for a commit-count mismatch.

The branch is now force-rebased with an exact lease to 3e3070b5ead7a42ad0e3821c7e96cc3131d9d513: 20 linear commits, zero merge commits, base ab952e6863c89561ed659f27ee6dacaa317b4e2b, tree db3d4d21560a55ffacab337510ecc40094f0cf44. All 20 range-diff entries are = and the tree is identical to the automatically composed prior head, so this is a history correction with no source-byte change.

Fresh exact-head staging and browser runs are now in progress. Do not apply ready-to-ship until they are green.

github-actions · 2026-07-16T19:09:15Z

prepare-merge: test-gate passed against the synthetic PR merge and sealed merge-candidate-abi-v39-pr-934-run-29524090642-attempt-1. The canonical ABI index is unchanged. merge-gate=success was posted on PR HEAD. This PR is ready for a reviewer to rebase merge; default-branch reconciliation will verify the exact merged tree before activation.

brandonpayton marked this pull request as ready for review July 13, 2026 23:23

brandonpayton marked this pull request as draft July 13, 2026 23:25

brandonpayton force-pushed the integration/post-907-focused-batch branch 2 times, most recently from 7dce296 to 260d75d Compare July 13, 2026 23:41

brandonpayton marked this pull request as ready for review July 14, 2026 02:40

brandonpayton force-pushed the integration/post-907-focused-batch branch from 260d75d to 822f35e Compare July 14, 2026 02:50

This was referenced Jul 14, 2026

bash, pcre2: require reproducible ABI-valid Wasm artifacts Automattic/kandelo-homebrew#75

Closed

sqlite: ship the real command-line shell Automattic/kandelo-homebrew#76

Merged

brandonpayton mentioned this pull request Jul 16, 2026

CI: restore fast artifact reuse and shard conformance suites #963

Merged

brandonpayton added 20 commits July 16, 2026 11:23

Browser: stop retaining stale immediate cancellations

38e00db

test(browser): preserve guest stderr on nonzero exit

321aa7b

MariaDB: share writable-directory setup across VFS images

b26b750

VFS: keep positioned I/O off shared file offsets

19cb17d

host: preserve output buffers on zero-byte syscalls

1092a47

Runner: start examples with requested user and group IDs

a757932

mremap: reject growth without matching mapping metadata

4d9c74f

Locks: report shared lock-table exhaustion as ENOLCK

06551eb

libc: zero stat fields the kernel does not report

75d9d3c

SQLite tests: run all-mode child jobs on Kandelo

0ba71dd

SQLite tests: preserve truthful outcome evidence

c7eb124

SQLite tests: account for UTF-16 STAT4 sample bytes

590f48d

Keep analyze9's memory-accounting assertion, but use the encoding-correct allocation delta for SQLite's utf16 permutation.

SQLite: clean up TestRecover Tcl commands

224faa7

brandonpayton force-pushed the integration/post-907-focused-batch branch from db6436e to 3e3070b Compare July 16, 2026 15:28

brandonpayton added the ready-to-ship Maintainer attests the exact tested head and requests merge preparation. label Jul 16, 2026

brandonpayton merged commit 62089b1 into main Jul 16, 2026
199 checks passed

brandonpayton deleted the integration/post-907-focused-batch branch July 16, 2026 19:09

brandonpayton mentioned this pull request Jul 16, 2026

platform: make Homebrew packages build, compose, and run on Kandelo #967

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

platform: combine remaining POSIX, runtime, and SQLite fixes#934

platform: combine remaining POSIX, runtime, and SQLite fixes#934
brandonpayton merged 20 commits into
mainfrom
integration/post-907-focused-batch

brandonpayton commented Jul 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 13, 2026 •

edited

Loading

Uh oh!

brandonpayton commented Jul 14, 2026

Uh oh!

brandonpayton commented Jul 16, 2026

Uh oh!

brandonpayton commented Jul 16, 2026

Uh oh!

github-actions Bot commented Jul 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

brandonpayton commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

Unique work that remains

Exact source heads

What was deliberately removed

ABI and package-artifact impact

Validation

Validation not run

Merge and cleanup

Uh oh!

github-actions Bot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Phase B-1 matrix build status — pr-934-staging

Uh oh!

brandonpayton commented Jul 14, 2026

Uh oh!

brandonpayton commented Jul 16, 2026

Uh oh!

brandonpayton commented Jul 16, 2026

Uh oh!

github-actions Bot commented Jul 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

brandonpayton commented Jul 13, 2026 •

edited

Loading

github-actions Bot commented Jul 13, 2026 •

edited

Loading

Phase B-1 matrix build status — `pr-934-staging`