Upgrade kernel to ShenOSKernel 42.2 (shen-scheme 0.50 parity)

[pyrex41]

Summary

Adopts the community ShenOSKernel 42.2 kernel wholesale (option (a) in #79) in place of Tarver's S42.0 kernel. shen-lua now runs the same kernel as shen-scheme 0.50, byte for byte. (version) returns "42.2".

Two behaviour changes beyond the kernel bump, both matching shen-scheme 0.50:

• programmable-pattern-matching is now booted (it was opt-in);
• <-address past the end of a vector now raises instead of returning nil. This also fixes (vector 3) printing stray funex… entries, a bug that is present on main too.

Changes

• Vendored kernel: all 23 klambda/*.kl files and the tests/ suite are byte-identical to ShenOSKernel-42.2.tar.gz (SHA-256 1d02273003654d34ec020ae67cb214ecec410dee3350419ff807d82b652678d8; shen-sources tag shen-42.2, commit 73507e069fe147aefb728d632c4ce40bc6a7b314). backend.kl is removed; dict.kl, init.kl, stlib.kl and the namespaces and type-annotations extensions are added.
• Boot (boot.lua):
  • 42.2 files contain only function definitions. After loading, boot calls (shen.initialise), the features initialiser and the PPM initialiser, registers the lua.* interop entries, then calls (stlib.initialise). Extensions and initialisation order match shen-scheme 0.50.
  • The 290 standard-library declares are cached in a sidecar next to the kernel bytecode cache (<cache>.stlib), using the existing record/replay mechanism.
  • The standard-library setup runs with the JIT off. With the JIT on, timing varied between 10 ms and 700 ms depending on which traces were recorded. The JIT is restored afterwards only if it was on before.
• Standard library: the precompiled klambda/stlib.kl replaces the lib/StLib Shen sources. lib/StLib and the stdlib boot image are removed.
• prims.lua:
  • native get/put/unput/arity/GET_OR now work on the 42.2 shen.dict property vector;
  • the native fn cache is invalidated on every write of a shen.lambda-form property, the 42.0 requirement for redefinitions;
  • shen.lambda-entry is native from boot;
  • <-address bounds as above.
• Other files:
  • typecheck_native.lua: new record_sig hook for the cached replay.
  • build/make-bundle.lua: no longer embeds stdlib sources.
  • bin/yggdrasil-build.lua: expects kernel-version=42.2; older slices still build, with a warning.
  • shen-scm-1.rockspec: kernel 42.2, lib/ dropped.
  • PROVENANCE.md and klambda/README.md: rewritten to record the lineage choice, keeping the S42 history.
  • New CHANGELOG.md; README and walkthrough updated.
• Specs:
  • prolog-memory defaults to 10000 in 42.2 (was 1000), so the per-query limit tests now set 1000 first.
  • The boot-cache spec covers the signature sidecar.
  • The <-address spec now expects an error past the end.

Verification

• make test: 1088 pass, 0 fail across 22 specs. Also passes from a clean checkout with a fresh fasl dir.
• make certify (luajit run-kernel-tests.lua): 143/143. Same with SHEN_PROLOG_ENGINE=legacy SHEN_FASL=off, and on PUC Lua 5.4.7 (47 s).
• 42.2 extension suite (tests/extensions/runme.shen): 123 passed, 0 failed (same as shen-scheme 0.50).
• bench/golden_typecheck.lua compare: 27 pass, 0 fail.
• Bifrost go run . --impls shen-lua,shen-scheme (shen-scheme = the 0.50 reference): 63 pass, 3 FAIL:
  • version-expr-42: the golden still expects "42"; both ports print "42.2".
  • stdin-eof-clean-exit: shen-lua passes, shen-scheme fails.
  • hush-file-write: shen-lua writes the file as the case requires; shen-scheme doesn't.
• Yggdrasil --target lua with the current S42 slice: fib 20 = 6765. A hand-assembled 42.2 slice (full kernel, init=shen.initialise) prints the same.
• Spot checks against shen-scheme 0.50 agree on: version, arithmetic, lists, strings, closures, trap-error, the (head []) message, fn/arity of stdlib functions, prolog, macroexpand, and read/lineread at EOF without a trailing newline (from files and stdin). shen.*sigf* matches shen-scheme's 458 signatures plus the four lua.checked-* ones.
• Warm boot is about 100 ms CPU, down from about 135 ms on main.

Remaining work / known failures

• Bifrost: the version-expr-42 golden and the shen-lua ref/rock pin need updating (Track Shen 0.50 / ShenOSKernel 42.2 support across all ports bifrost#26).
• A new tag and luarocks release are needed (e.g. v0.12.0 / shen 0.12.0-1). Bifrost and Yggdrasil still pin v0.11.0 / 0.11.0-1. No release rockspec is added here.
• A shaken 42.2 slice is untested, because Yggdrasil still ships S42 (Retarget shaker and builders to Shen 0.50 / ShenOSKernel 42.2 yggdrasil#32).
• Known divergences from shen-scheme, unchanged from main:
  • numbers are IEEE doubles, so (* 123456789 987654321) loses precision;
  • (str [1 "a" b]) raises an error;
  • (get-time unix) returns a wrong value;
  • prolog recursion beyond about 5000 levels overflows the LuaJIT stack before the 10000 prolog-memory limit.

Closes #79

🤖 Generated with Claude Code

https://claude.ai/code/session_01X718xocsFckMvso6EAKoXR

---

Generated by Claude Code