Add real wasm32 smoke tests and CI job by cderv · Pull Request #109 · quarto-dev/q2

cderv · 2026-04-09T13:22:34Z

Replace the lost cfg proxy coverage (removed in #116) with 6 real WASM smoke tests
in crates/pampa/tests/wasm_lua.rs that compile and run on wasm32-unknown-unknown:

Restricted Lua VM creation (Lua::new_with())
Filter execution through the WASM code path (VFS + synthetic io/os)
Shortcode engine initialization
Error handling (validates panic_unwind works)
Synthetic io and os module registration

These run in a new wasm-tests CI job (Linux, nightly + Clang).

Infrastructure

crates/wasm-c-shim/ — shared C stdlib shim crate for wasm32 (no libc on wasm32,
C deps like tree-sitter and Lua need malloc, fprintf, etc. at link time)
.cargo/config.toml — wasm-bindgen-test runner config
Gate proptest, insta, tempfile, tokio dev-deps on not(wasm32) to avoid
pulling getrandom 0.3.4 which doesn't compile for wasm32
Gate pampa [[bin]] targets via required-features to prevent compilation during
WASM test runs (Add an option to skip building binaries when there are integration tests rust-lang/cargo#12980)
Use RUSTUP_TOOLCHAIN=nightly in CI to bypass rust-toolchain.toml sysroot conflict
with -Zbuild-std

Known blocker

The wasm-tests CI job currently fails because quarto-system-runtime/src/wasm.rs
has 4 raw_module extern blocks (/src/wasm-js-bridge/{template,sass,cache,fetch}.js)
that get baked into any wasm32 binary. wasm-bindgen-test-runner generates require()
calls for these absolute paths, which fail in Node.js.

Recommended fix: feature-gate the JS bridge in quarto-system-runtime — add a
js-bridge feature, gate the 4 extern blocks, provide stubs when off.
wasm-quarto-hub-client enables js-bridge; pampa test builds don't.

Depends on #116.

The four `raw_module` extern blocks in quarto-system-runtime/src/wasm.rs import absolute paths (/src/wasm-js-bridge/{template,sass,cache,fetch}.js) that hub-client serves through Vite at runtime. wasm-bindgen generates unconditional `require()` calls for these paths in the shim it produces, so any wasm32 binary that links quarto-system-runtime — including the new pampa wasm_lua tests — fails to load under Node.js with `MODULE_NOT_FOUND` before any test body runs. Add a `js-bridge` Cargo feature, default off. Gate the four extern blocks behind it, and provide stubs that return `Err("js-bridge feature not enabled")` / `false` when off so the SystemRuntime impl still compiles. wasm-quarto-hub-client opts in via `features = ["js-bridge"]`; pampa's wasm test build does not, so the require()s disappear from the shim. Native builds are unaffected (wasm.rs is `#![cfg(target_arch = "wasm32")]`). This clears the blocker documented in PR #109. A separate downstream issue is now visible: rust_lua_throw panics propagate as wasm RuntimeError instead of being caught by rust_lua_protected_call, so the 6 wasm tests still fail. That looks like a panic-strategy problem in wasm-c-shim, unrelated to the JS bridge fix.

gordonwoodhull · 2026-04-20T17:29:03Z

@cderv, thanks for writing these tests!

I was trying to understand why these paths weren't tested, and either misunderstood or miscommunicated this to Claude, testing them the wrong way! This was the mistake you fixed in

Fix WASM cfg proxy: remove test arm from wasm32 guards #116

I've fixed the remaining issues with this PR and it runs clean here:
https://github.com/quarto-dev/q2/actions/runs/24612984662

I also rebased onto latest main. May I force push, or do you have other work on this branch?

cderv · 2026-04-21T07:56:34Z

Oh thanks for chiming in ! I was going to continue with this this week. It needed to be rebased on main and it is also related to work I started in #125

My aim was that we have some wasm build test, but I was sorting out quarto-hub build from other wasm build.

interested to see your change !

Yes you can force push here. And maybe we can discuss quickly all this together if you have time today.

gordonwoodhull · 2026-04-21T12:04:14Z

Thanks! I've pushed. It's really just

the feature-gate fix you suggested above
build flags enabling the same error handling / panic strategy for wasm32-unknown-unknown tests as used for the lua-wasm build in wasm-quarto-hub-client
merge/rebase conflict resolution
docs

(I'm going for a hike today since I stupidly worked over the weekend, glad to discuss later in the week.)

Design spec (claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md) and plan (claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md) for replacing the lost cfg-proxy coverage (removed in #116) with real wasm32 smoke tests. Incorporates design-review findings: wasm-bindgen-cli version pinning via cargo xtask dev-setup, the C toolchain prerequisite, and Linux-CI-only scope (skipped on Windows).

Add 6 smoke tests in crates/pampa/tests/wasm_lua.rs that compile and run on wasm32-unknown-unknown: - Restricted Lua VM creation (Lua::new_with()) - Filter execution through the WASM code path (VFS + synthetic io/os) - Shortcode engine initialization - Error handling (validates panic_unwind works) - Synthetic io and os module registration They run in a new wasm-tests CI job (Linux, Clang), and locally on Linux/macOS — see dev-docs/wasm.md. Build infrastructure, including findings from the CI bring-up: - .cargo/config.toml: wasm-bindgen-test-runner for the wasm32 target - Gate proptest, insta, tempfile, tokio dev-deps on not(wasm32) — they pull in getrandom, which does not compile for wasm32 - Gate pampa [[bin]] targets via required-features so cargo skips them during WASM test runs (rust-lang/cargo#12980) - Absolute -isystem paths for the wasm sysroot; -fno-builtin - Docs: dev-docs/wasm.md, .claude/rules/wasm.md, pampa CLAUDE.md, claude-notes/instructions/testing.md

Move the ~980-line c_shim.rs from wasm-quarto-hub-client into a new wasm-c-shim crate. This provides #[no_mangle] C stdlib stubs (malloc, fprintf, snprintf, abort, etc.) needed by tree-sitter and Lua when compiled for wasm32-unknown-unknown, which has no libc. The crate is a no-op on native targets (all exports gated on wasm32). Both wasm-quarto-hub-client (production) and pampa WASM tests now depend on wasm-c-shim for the shared symbols.

…rategy The raw_module extern blocks in quarto-system-runtime/src/wasm.rs import absolute paths (/src/wasm-js-bridge/*.js) that hub-client serves through Vite at runtime. wasm-bindgen generates unconditional require() calls for these paths in the shim it produces, so any wasm32 binary that links quarto-system-runtime — including the pampa wasm_lua tests — fails to load under Node.js with MODULE_NOT_FOUND before any test body runs. Add a js-bridge Cargo feature, default off. Gate the extern blocks behind it, and provide stubs that return Err("js-bridge feature not enabled") / false when off so the SystemRuntime impl still compiles. wasm-quarto-hub-client opts in via features = ["js-bridge"]; pampa's wasm test build does not, so the require()s disappear from the shim. Native builds are unaffected (wasm.rs is #![cfg(target_arch = "wasm32")]). That exposed a downstream panic-strategy problem: rust_lua_throw panics propagated as wasm RuntimeError instead of being caught by rust_lua_protected_call. Fix by mirroring wasm-quarto-hub-client's rustflags (-C panic=unwind, +bulk-memory,+exception-handling, -Zwasm-c-abi=spec) in the workspace-root .cargo/config.toml for wasm32 builds, and moving the LuaThrow marker type into wasm-c-shim so the production build and the tests share it. Update design/plan docs and dev-docs/wasm.md accordingly.

posit-snyk-bot · 2026-07-02T01:04:41Z

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

…lchain This branch predates the 2026-04-20 wasm-shim-merge work and the bd-at72 toolchain pin; rebasing onto current main needed alignment beyond conflict resolution: - wasm-c-shim: the rebase carried main's evolved c_shim.rs content into shim.rs via rename detection; declare the wasm-printf-fmt dependency it uses for snprintf/vsnprintf. - Root Cargo.toml: patch tree-sitter-language to the local wasm-shim fork (same patch wasm-quarto-hub-client uses) so tree-sitter core does not compile upstream's wasm32 stdio/stdlib/string stubs, which collide with wasm-c-shim at link time. The fork has byte-identical Rust source and a no-op build.rs on native targets. - wasm-tests CI: run on the pinned toolchain from rust-toolchain.toml (now a nightly with rust-src and the wasm32 target) instead of RUSTUP_TOOLCHAIN=nightly — bd-at72 pinned the toolchain because later nightlies SIGSEGV in LLVM ThinLTO on wasm32 builds of this workspace. The E0152 duplicate-lang-item conflict the override worked around no longer reproduces on the pinned toolchain. - wasm_lua.rs: SourceInfo::for_test() (SourceInfo::default() is deprecated under -D deprecated), and pass the new attribution argument (None) to apply_lua_filters. - Regenerate Cargo.lock and crates/wasm-quarto-hub-client/Cargo.lock. Verified: cargo build + nextest workspace green (9862 passed); all 6 wasm tests pass under wasm-bindgen-test-runner (macOS, Homebrew LLVM clang, pinned nightly); production build-wasm.js succeeds.

gordonwoodhull · 2026-07-02T05:25:21Z

I dropped the unrelated stuff from

Modernize WASM CI workflows and clean up wasm-qmd-parser #125
(wasm-qmd-parser and rust toolchain)

and squashed this. I'll go ahead and merge it now.

Thanks @cderv!

cderv force-pushed the feature/wasm-testing-and-cleanup branch 2 times, most recently from 52923e7 to 60df573 Compare April 9, 2026 13:34

cderv mentioned this pull request Apr 9, 2026

Behavioral difference: dofile() script-dir tracking on WASM vs native #112

Closed

cderv force-pushed the feature/wasm-testing-and-cleanup branch from 8c6bfe2 to 89d90e0 Compare April 9, 2026 15:46

cderv mentioned this pull request Apr 14, 2026

Fix WASM cfg proxy: remove test arm from wasm32 guards #116

Merged

cderv changed the base branch from main to fix/wasm-cfg-proxy-and-cleanup April 14, 2026 12:30

cderv changed the title ~~Clean up WASM testing: remove dead crate, fix cfg proxy, add real wasm32 tests~~ Add real wasm32 smoke tests and CI job Apr 14, 2026

cderv force-pushed the feature/wasm-testing-and-cleanup branch from 267f27a to ba0ec15 Compare April 14, 2026 14:21

cderv force-pushed the fix/wasm-cfg-proxy-and-cleanup branch from 6009f60 to 2db6f1e Compare April 16, 2026 12:58

Base automatically changed from fix/wasm-cfg-proxy-and-cleanup to main April 16, 2026 15:45

gordonwoodhull force-pushed the feature/wasm-testing-and-cleanup branch from ba0ec15 to 901f9f7 Compare April 21, 2026 11:58

cderv mentioned this pull request Apr 24, 2026

Improve cargo xtask dev-setup: cmake detection and xtask self-build lock #136

Merged

5 tasks

cderv and others added 4 commits July 1, 2026 21:02

gordonwoodhull force-pushed the feature/wasm-testing-and-cleanup branch from 901f9f7 to b816828 Compare July 2, 2026 01:04

gordonwoodhull force-pushed the feature/wasm-testing-and-cleanup branch from b816828 to bf370a0 Compare July 2, 2026 01:16

gordonwoodhull marked this pull request as ready for review July 2, 2026 01:20

gordonwoodhull merged commit e26a8b8 into main Jul 2, 2026
8 checks passed

gordonwoodhull deleted the feature/wasm-testing-and-cleanup branch July 2, 2026 05:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add real wasm32 smoke tests and CI job#109

Add real wasm32 smoke tests and CI job#109
gordonwoodhull merged 5 commits into
mainfrom
feature/wasm-testing-and-cleanup

cderv commented Apr 9, 2026 •

edited

Loading

Uh oh!

gordonwoodhull commented Apr 20, 2026 •

edited

Loading

Uh oh!

cderv commented Apr 21, 2026

Uh oh!

gordonwoodhull commented Apr 21, 2026 •

edited

Loading

Uh oh!

posit-snyk-bot commented Jul 2, 2026 •

edited

Loading

Uh oh!

gordonwoodhull commented Jul 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

cderv commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Infrastructure

Known blocker

Uh oh!

gordonwoodhull commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cderv commented Apr 21, 2026

Uh oh!

gordonwoodhull commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

posit-snyk-bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Snyk checks have passed. No issues have been found so far.

Uh oh!

gordonwoodhull commented Jul 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cderv commented Apr 9, 2026 •

edited

Loading

gordonwoodhull commented Apr 20, 2026 •

edited

Loading

gordonwoodhull commented Apr 21, 2026 •

edited

Loading

posit-snyk-bot commented Jul 2, 2026 •

edited

Loading