Skip to content

bench+web: mirror the test-code trees, measure on WSL, and unbreak the docs - #13

Merged
MotherOfProgrammers merged 1 commit into
Project-Lithon:masterfrom
Tenuka22:feature/mirror-test-trees-wsl-bench
Oct 21, 2026
Merged

MotherOfProgrammers merged 1 commit into
Project-Lithon:masterfrom
Tenuka22:feature/mirror-test-trees-wsl-bench

Conversation

@Tenuka22

Copy link
Copy Markdown
Contributor

Summary

The Lithon test-code tree was reorganised into category folders, but the Python side was only half-migrated and the website still pointed at the old flat paths. This finishes the migration, makes the benchmark measure where the measurement is honest, and brings the docs back in line with the engine.

Three independent problems, one PR:

  1. The test-code trees no longer matched — test_codes_lithon/ had category subfolders, test_codes_python/ had only two of them, and every config and website reference still used the old flat paths.
  2. The benchmark measured the wrong thing on Windows — it picked up the repo's Windows lithon.exe from inside WSL, then died with a baffling "native runner not found at C:\...".
  3. The website's verification was failing and its docs were stale — verify:docs failed with 58 errors, the IR map was 5 commits out of date, and the docs documented a try/except that had landed in the engine but was never written down.

test_codes_python mirrors test_codes_lithon

Folder for folder, all seven categories:

test_codes_python/{Classes,DataTypes,Functions,Loops,NewTypes,Pointers,Structs}/*.py
test_codes_lithon/{Classes,DataTypes,Functions,Loops,NewTypes,Pointers,Structs}/*.lio

Every .lio program now has a .py twin one folder away. Where CPython has no equivalent — raw pointers, packed structs, width annotations — the twin keeps the shape of the program and says so in its docstring. The three programs that are deliberately pinned failures on the Lithon side (DataTypes/check_strings.lio, Pointers/pointer_error.lio, Classes/class_basics.lio) are left exactly as they are: they are the record of what is missing, not bugs to be fixed.

bench_vid_maker

Resolve the engine for the platform it will actually run on. A Windows .venv/Scripts/lithon.exe used to be picked up from inside WSL and then failed with "native runner not found at C:\...". A binary that cannot run here is now skipped, and the repo's cli.py is the last resort, so a cloned-but-not-installed repo still works.

Re-execute the whole run inside WSL from Windows (--wsl / --no-wsl, or BENCH_VID_MAKER_NO_WSL=1). The native tier is an x86-64 binary, CPython's loop speed differs under WSL, and CPU affinity only works on the Linux side — so the only honest place to measure is a Linux kernel with both engines native. This is not a convenience wrapper: the whole tool re-executes, so every number the UI reports is produced by a Linux process.

Fix a truncated-frame bug in the recorder. A ~35 MB frame written through a BufferedWriter could reach ffmpeg short, and ffmpeg then aborted the encode with Invalid buffer size, packet size N < expected frame_size M. The pipe is now unbuffered, every write is checked for a short count, and ffmpeg's stderr is surfaced instead of a bare exit code.

Four new configs (fib_30, find_e, fsum_1b_simd, multi_ops_1b) and a README that explains the WSL flow, the engine-resolution order, and the new tree layout.

web

Verification works again. verify:docs wrote its examples as .py, which the frontend now refuses (LITHON-E0114: Lithon source files use the .lio extension), and 24 sourceRef/expectedRef paths still named .py files that are now .lio. Both fixed.

docs verification: 29 pages · 97 examples · 76 compiled · 62 repo refs
ALL DOCS EXAMPLES VERIFIED against the real Lithon toolchain

The IR map is current and the interpreter understands it. Regenerated (31 → 48 blobs), and the JS IR interpreter was taught the IR the engine emits today: print_bare/print_nl instead of print, band/bor/bxor instead of and/or/xor, fieldstore/fieldload, the guardnonzero trap, and tryenter/tryleave/trycatch.

44/44 IR executions match the documented output

The docs describe the engine that exists. Added try/except over the built-in traps (the catchable set, the three deliberately-refused shapes, and the tier-3 trap→exception mapping), enumerate/zip, the string and list methods, and the full error-code table instead of three ranges. Every one of these was checked against the real engine first — in both tiers, not from the source.

The broken bits are fixed:

was now
/contributors 404'd from the search palette and the sitemap points at the homepage's contributors section, which now has an anchor
sitemap/robots.txt disagreed with the app about the canonical origin one origin, SITEMAP_HOST, shared by seo.ts and gen-seo.ts
public/manifest.json was Create-TanStack-App boilerplate naming two icons that do not exist a real Lithon manifest, linked from the document head
static assets were fetched from the site root, which 404s under the GitHub Pages sub-path assetUrl() prefixes the deployment base path
two different Discord invite links one

Also: the .py → .lio rename applied to every stale program path in prose and shell snippets, the duplicated search tags fixed, and the missing cross-links between doc pages added.

Verification

All run from web/:

bun run typecheck          2 successful, 2 total
bun run lint               2 successful, 2 total
bun run verify:docs        ALL DOCS EXAMPLES VERIFIED
bun scripts/verify-ir-exec 44/44 IR executions match

And in WSL, the benchmark runs both engines natively with identical output on both sides.

Notes for review

  • No new dependencies anywhere.
  • The try/except semantics documented here were verified by running the engine in both tiers (--interp and native), including the three shapes it refuses and the fact that a trap raised in a callee is not catchable by the caller.
  • tools/typecheck.py is a mirror of the older scalar-only static checks and does not know the container type language or try yet; the examples that use them are marked typecheck: false with a comment explaining why, exactly as the existing container examples are.
  • uv.lock and lithon.egg-info/ are regenerated build metadata, included because they were stale in the tree.

…e docs

The Lithon test-code tree was reorganised into category folders
(Classes, DataTypes, Functions, Loops, NewTypes, Pointers, Structs), but
the Python side was only half-migrated and the website still pointed at the
old flat paths. This commit finishes the migration, makes the benchmark
measure where the measurement is honest, and brings the docs back in line
with the engine.

test_codes_python now mirrors test_codes_lithon folder for folder: every
.lio program has a .py twin in the same category. The twins keep the shape
of the program where CPython has no equivalent (raw pointers, packed
structs, width annotations) and say so in their docstring. The three
programs that are deliberately pinned failures on the Lithon side
(check_strings, pointer_error, class_basics) are left alone -- they are
the record of what is missing, not bugs.

bench_vid_maker:

* Resolve the engine for the platform it will actually run on. A Windows
  .venv/Scripts/lithon.exe used to be picked up from inside WSL and then
  failed with a baffling "native runner not found at C:\...". A binary
  that cannot run here is now skipped, and the repo's cli.py is the last
  resort so a cloned-but-not-installed repo still works.
* Re-execute the whole run inside WSL from Windows (--wsl / --no-wsl).
  The native tier is an x86-64 binary, CPython's loop speed differs under
  WSL, and CPU affinity only works on the Linux side -- so the only
  honest place to measure is a Linux kernel with both engines native.
* Fix a truncated-frame bug in the recorder: a ~35 MB frame written
  through a BufferedWriter could reach ffmpeg short, and ffmpeg then
  aborted the encode with "Invalid buffer size". The pipe is now
  unbuffered, every write is checked for a short count, and ffmpeg's
  stderr is surfaced instead of a bare exit code.
* Four new configs (fib_30, find_e, fsum_1b_simd, multi_ops_1b) and a
  README that explains the WSL flow and the new tree layout.

web:

* verify:docs wrote its examples as .py, which the frontend now refuses
  (LITHON-E0114: Lithon source files use the .lio extension), and 24
  sourceRef/expectedRef paths still named .py files that are now .lio.
  Both fixed: 29 pages, 97 examples, 62 repo refs now verify against the
  real toolchain.
* Regenerate the IR map (31 -> 48 blobs) and teach the JS IR interpreter
  the IR the engine emits today: print_bare/print_nl instead of print,
  band/bor/bxor instead of and/or/xor, fieldstore/fieldload, the
  guardnonzero trap, and tryenter/tryleave/trycatch. 44/44 documented
  outputs now match.
* Document what the engine actually does: try/except over the built-in
  traps (the catchable set, the three refused shapes, and the tier-3
  mapping), enumerate/zip, the string and list methods, and the full
  error-code table instead of three ranges.
* Fix the broken bits: /contributors 404'd from the search palette and
  the sitemap; the sitemap and robots.txt disagreed with the app about
  the canonical origin; the PWA manifest was Create-TanStack-App
  boilerplate naming two icons that do not exist; static assets were
  fetched from the site root, which 404s under the GitHub Pages
  sub-path; the two Discord invite links disagreed.
* Add the missing cross-links between doc pages.

uv.lock / lithon.egg-info are regenerated build metadata.
@MotherOfProgrammers
MotherOfProgrammers merged commit 6bfe720 into Project-Lithon:master Oct 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants