From cfccad97a5416531f5bfef86902eb8278217f42f Mon Sep 17 00:00:00 2001 From: Christophe Dervieux Date: Fri, 3 Apr 2026 20:07:32 +0200 Subject: [PATCH 1/5] Add WASM testing and cleanup design spec and plan Design spec (claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md) and plan (claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md) for replacing the lost cfg-proxy coverage (removed in #116) with real wasm32 smoke tests. Incorporates design-review findings: wasm-bindgen-cli version pinning via cargo xtask dev-setup, the C toolchain prerequisite, and Linux-CI-only scope (skipped on Windows). --- .../2026-04-03-wasm-testing-and-cleanup.md | 233 ++++ .../2026-04-07-wasm-testing-and-cleanup.md | 1058 +++++++++++++++++ 2 files changed, 1291 insertions(+) create mode 100644 claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md create mode 100644 claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md diff --git a/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md b/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md new file mode 100644 index 000000000..45d8b1277 --- /dev/null +++ b/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md @@ -0,0 +1,233 @@ +# WASM Testing and Cleanup Design + +Date: 2026-04-03 +Beads: bd-itj9 + +## Problem + +`filter.rs` and `shortcode.rs` use `#[cfg(any(target_arch = "wasm32", test))]` to force +native tests through the WASM-restricted Lua stdlib + synthetic io/os modules. This was +a proxy for WASM testing: native tests would catch WASM-incompatible Lua code without +needing real WASM test infrastructure. + +The proxy causes 8 filter traversal tests to fail on Windows because the synthetic +`io.open` only handles POSIX VFS paths (`/project/...`), not Windows paths (`C:\...`). + +Additionally, `wasm-qmd-parser` is a stale crate fully superseded by +`wasm-quarto-hub-client`. Its CI workflow (`build-wasm.yml`) and wasm-pack dependency +are orphaned artifacts. + +## Success criteria + +- All 8 filter traversal tests pass on Windows (currently fail with os error 123) +- WASM smoke tests pass in CI on wasm32-unknown-unknown target +- No active build/test/runtime references to wasm-qmd-parser or wasm-pack remain + (historical references in claude-notes/ plans are acceptable) +- hub-client WASM build (`npm run build:all`) unaffected + +## Solution + +Replace the cfg proxy with real WASM tests, clean up stale artifacts, and document the +WASM testing convention. + +**Phase ordering:** WASM tests (Phase 3) must be added before or alongside the cfg proxy +removal (Phase 2) to avoid any validation gap. In practice, Phase 3 setup and Phase 2 +cfg changes should land in the same changeset. + +## Phase 1: Clean up stale WASM artifacts + +### Remove + +- `crates/wasm-qmd-parser/` — entire crate (superseded by wasm-quarto-hub-client) +- `.github/workflows/build-wasm.yml` — only builds wasm-qmd-parser, manual dispatch +- wasm-pack from `cargo xtask dev-setup` install list +- `Cargo.toml` root: remove wasm-qmd-parser from `exclude` list, remove + `[workspace.dependencies.wasm-qmd-parser]` entry (line 86-87), remove wasm-pack comments + +### Check and update + +- `.github/workflows/hub-client-e2e.yml` — remove stale `cargo install wasm-pack` step + (verified: wasm-pack is installed but never used; WASM build uses build-wasm.js) +- `hub-client/README.md` — remove wasm-pack prerequisite +- `crates/wasm-quarto-hub-client/README.md` — remove wasm-pack references + +### Rewrite + +- `dev-docs/wasm.md` — rewrite as single source of truth for WASM in this project: + - Architecture: wasm-quarto-hub-client wraps pampa + quarto-core for hub-client + - Build: `hub-client/scripts/build-wasm.js` → cargo build + wasm-bindgen CLI + - Why not wasm-pack: needs `-Zbuild-std=std,panic_unwind` for Lua error handling + - Testing: see Phase 3 + - Note: wasm-pack is deprecated (rustwasm org sunset September 2025) + +## Phase 2: Remove the cfg proxy + +### Code changes + +- `crates/pampa/src/lua/filter.rs:123`: + `#[cfg(any(target_arch = "wasm32", test))]` → `#[cfg(target_arch = "wasm32")]` +- `crates/pampa/src/lua/filter.rs:133`: + `#[cfg(not(any(target_arch = "wasm32", test)))]` → `#[cfg(not(target_arch = "wasm32"))]` +- `crates/pampa/src/lua/shortcode.rs:72`: same change +- `crates/pampa/src/lua/shortcode.rs:85`: same change +- Update comments above the cfg blocks (remove mention of test environment) + +### No changes + +- `io_wasm.rs` unit tests — keep as `#[cfg(test)]`. They test the Lua API contract + (read modes, write buffering, handle lifecycle) on native using NativeRuntime. Valid + unit tests of the implementation logic; don't need to run under wasm32. +- `os_wasm.rs` unit tests — same reasoning. + +### Verification + +The 8 filter traversal tests that use `io.open` should pass on all platforms after this +change, since they'll use `Lua::new()` with real C stdlib instead of synthetic WASM io. + +## Phase 3: Add real WASM testing + +### Dependencies + +- Add `wasm-bindgen-test` as dev-dependency to `crates/pampa/Cargo.toml` +- Version must match the `wasm-bindgen` version used by the project +- Install `wasm-bindgen-cli` via `cargo xtask dev-setup` (this provides the + `wasm-bindgen-test-runner` binary, version-matched from Cargo.lock) + +### Configuration + +Add to `.cargo/config.toml` (workspace root): + +```toml +[target.wasm32-unknown-unknown] +runner = 'wasm-bindgen-test-runner' +``` + +Note: the `runner` setting only applies to `cargo test`, not `cargo build`. The +hub-client WASM production build (`build-wasm.js` → `cargo build`) is unaffected. + +### Test file + +Create `crates/pampa/tests/wasm_lua.rs`: + +```rust +//! WASM integration tests for Lua filter and shortcode infrastructure. +//! +//! These tests verify that the restricted Lua stdlib setup, synthetic io/os +//! modules, and filter/shortcode execution work correctly when compiled to +//! the real wasm32 target. +//! +//! **When to add tests here:** Only when modifying WASM-specific code paths: +//! - The #[cfg(target_arch = "wasm32")] blocks in filter.rs / shortcode.rs +//! - io_wasm.rs (synthetic io module) +//! - os_wasm.rs (synthetic os module) +//! +//! Native filter logic is tested comprehensively by the existing native tests. +//! These WASM tests are smoke tests of the target-specific setup. + +#![cfg(target_arch = "wasm32")] + +use wasm_bindgen_test::*; +// Tests run in Node.js by default. Use wasm_bindgen_test_configure!(run_in_browser) +// if browser APIs are needed — current tests don't require it. +``` + +### Test coverage + +Focused smoke tests of WASM-specific code paths (not duplication of native tests): + +1. **Restricted Lua VM creation** — `Lua::new_with()` with restricted stdlib succeeds, + synthetic io/os get registered +2. **Filter execution** — run a simple filter on a small document, verify output + (uses Lua table to collect results, not io.open) +3. **Shortcode engine** — create engine, dispatch a basic handler +4. **Error handling** — Lua error gets caught as Rust error (not a WASM crash). + This validates the `-Zbuild-std=std,panic_unwind` setup. +5. **Synthetic io registration** — io.open, io.type are available as globals +6. **Synthetic os registration** — os.time, os.clock, os.difftime are available + +### CI + +Add a `wasm-tests` job to `.github/workflows/test-suite.yml` (the main Rust CI workflow). +Trigger on the same paths as existing Rust tests, plus `crates/pampa/tests/wasm_lua.rs`. + +**C toolchain prerequisite:** pampa with `lua-filter` pulls in `mlua` → `lua-src-wasm`, +which compiles Lua from C source via the `cc` crate. When targeting wasm32, this requires +Clang + `CC_wasm32_unknown_unknown` + `CFLAGS_wasm32_unknown_unknown` pointing to the +wasm-sysroot. This is the same setup already used by `ts-test-suite.yml` for the +production WASM build — the new job mirrors that toolchain setup. + +Note: `ts-test-suite.yml` currently hardcodes `wasm-bindgen-cli --version 0.2.108`. +This should be migrated to `cargo xtask dev-setup` as part of this work. + +Test command: +```bash +CC_wasm32_unknown_unknown=clang \ +CFLAGS_wasm32_unknown_unknown="-isystem crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ +cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind +``` + +The WASM build step in hub-client workflows (`npm run build:all`, `npm run build:wasm`) +stays unchanged — it builds the production WASM artifact. WASM tests are a separate +concern testing Rust code on the wasm32 target. + +## Documentation updates + +| File | Audience | Content | +|------|----------|---------| +| `crates/pampa/CLAUDE.md` | AI assistants | WASM test convention: when/where to add, how to run | +| `.claude/rules/wasm.md` | AI assistants | Never add `test` to wasm32 cfg guard; verify WASM tests when editing io_wasm/os_wasm | +| `dev-docs/wasm.md` | Developers | Single source of truth for WASM architecture, build, and testing | +| `claude-notes/instructions/testing.md` | AI assistants | Brief pointer to pampa CLAUDE.md for WASM details | +| `crates/pampa/tests/wasm_lua.rs` header | All | What this file tests and when to add to it | + +## Testing strategy summary + +| Layer | What it tests | Where | Runs on | +|-------|--------------|-------|---------| +| Native unit tests (io_wasm, os_wasm) | Synthetic Lua API contract | Inline in source files | All OS via `cargo test` | +| Native integration tests (filter_tests) | Filter logic with real Lua stdlib | Inline in source files | All OS via `cargo test` | +| WASM integration tests (new) | WASM-specific setup works on real target | `crates/pampa/tests/wasm_lua.rs` | wasm32 in CI | + +## Risks and mitigations + +- **`-Zbuild-std` is nightly-only**: Project is committed to nightly for WASM. If this + changes, WASM tests would need adjustment. Acceptable risk. +- **`wasm-bindgen-test-runner` version pinning**: Must match `wasm-bindgen` crate version + exactly. `cargo xtask dev-setup` reads the version from Cargo.lock and installs the + matching CLI. CI uses `cargo xtask dev-setup` so the version stays in sync automatically. +- **C toolchain for wasm32**: Required because mlua/lua-src compiles Lua from C. Both the + WASM test job and the existing TS test suite WASM build need this. Opportunity to share + the setup (composite action or reusable workflow) rather than duplicating Clang + env + vars across workflows. +- **`--test wasm_lua` required**: Running `cargo test -p pampa --target wasm32` without + `--test` would fail (native tests can't compile for wasm32). Document this clearly. +- **Feature flags required**: WASM test command must use `--no-default-features --features lua-filter` + to match how wasm-quarto-hub-client consumes pampa. Document the full command. + +## Local developer workflow + +WASM tests require nightly Rust + `rust-src` component + Clang (for C compilation of +Lua source) + `wasm-bindgen-test-runner` (installed via `cargo xtask dev-setup`). + +```bash +CC_wasm32_unknown_unknown=clang \ +CFLAGS_wasm32_unknown_unknown="-isystem crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ +cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind +``` + +WASM tests are NOT part of `cargo xtask verify` — they require nightly + Clang with +wasm32 support + wasm-sysroot, which is Linux/macOS only. The WASM build itself +(`build-wasm.js`) also doesn't support Windows (no Clang wasm32 target). WASM tests +run in Linux CI only, matching the existing WASM build behavior. + +On Windows, skip WASM tests — this is consistent with the WASM build being skipped. +On macOS/Linux with LLVM installed, contributors modifying WASM-specific code can run +them locally. + +## Out of scope + +- Migrating wasm-pack usage (no longer needed — only stale crate used it) +- Adding WASM tests for wasm-quarto-hub-client (cdylib-only, tested via hub-client JS tests) +- VFS-backed test runtime for io_wasm under wasm32 (native unit tests cover the logic) diff --git a/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md b/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md new file mode 100644 index 000000000..e28babbc6 --- /dev/null +++ b/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md @@ -0,0 +1,1058 @@ +# WASM Testing and Cleanup Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace the cfg test proxy with real WASM tests, remove stale wasm-qmd-parser artifacts, and document the WASM testing convention. + +**Architecture:** The `#[cfg(any(target_arch = "wasm32", test))]` guards in `filter.rs` and `shortcode.rs` force native tests through WASM-restricted Lua stdlib, causing Windows failures. We remove the `test` from these guards so native tests use `Lua::new()` with real C stdlib, and add real `wasm-bindgen-test` smoke tests that run on the actual wasm32 target in CI. Stale `wasm-qmd-parser` crate and its `build-wasm.yml` workflow are removed. + +**Tech Stack:** Rust, wasm-bindgen-test, cargo test --target wasm32-unknown-unknown, GitHub Actions + +**Beads:** bd-itj9 + +**Design spec:** `claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md` + +--- + +## File Map + +### Files to delete +- `crates/wasm-qmd-parser/` — entire crate directory (superseded by `wasm-quarto-hub-client`) +- `.github/workflows/build-wasm.yml` — manual workflow that only builds wasm-qmd-parser + +### Files to create +- `crates/pampa/tests/wasm_lua.rs` — WASM integration smoke tests +- `.claude/rules/wasm.md` — AI rule: never add `test` to wasm32 cfg guard + +### Files to modify +- `Cargo.toml` (workspace root) — remove wasm-qmd-parser from exclude + workspace deps, update comments +- `crates/pampa/Cargo.toml` — add `wasm-bindgen-test` dev-dependency +- `crates/pampa/src/lua/filter.rs` — change cfg guards (lines 123, 133) +- `crates/pampa/src/lua/shortcode.rs` — change cfg guards (lines 72, 85) +- `.cargo/config.toml` — add wasm32 test runner +- `.github/workflows/test-suite.yml` — add wasm-tests job +- `.github/workflows/hub-client-e2e.yml` — remove stale `cargo install wasm-pack` step +- `.github/workflows/ts-test-suite.yml` — migrate wasm-bindgen-cli install to `cargo xtask dev-setup` +- `hub-client/README.md` — remove wasm-pack prerequisite +- `crates/wasm-quarto-hub-client/README.md` — remove wasm-pack references +- `dev-docs/wasm.md` — full rewrite as WASM single source of truth +- `crates/pampa/CLAUDE.md` — add WASM test convention +- `claude-notes/instructions/testing.md` — update WASM section to reflect new approach + +--- + +## Phase 1: Clean Up Stale WASM Artifacts + +### Task 1: Remove wasm-qmd-parser crate + +**Files:** +- Delete: `crates/wasm-qmd-parser/` (entire directory) + +This crate is superseded by `wasm-quarto-hub-client`. It uses `wasm-pack` (deprecated, rustwasm org sunset Sep 2025) while the active crate uses `cargo build + wasm-bindgen` directly. + +- [ ] **Step 1: Verify no other crate depends on wasm-qmd-parser** + +Run from the worktree root: +```bash +grep -r "wasm-qmd-parser" --include="*.toml" --include="*.rs" --include="*.js" --include="*.ts" --include="*.yml" --include="*.yaml" . \ + | grep -v "crates/wasm-qmd-parser/" \ + | grep -v "claude-notes/" \ + | grep -v ".beads/" +``` + +Expected: Only hits in `Cargo.toml` (workspace root, lines 10 and 86-87), `.github/workflows/build-wasm.yml`, and possibly documentation. No runtime/build imports. + +- [ ] **Step 2: Delete the crate directory** + +```bash +rm -rf crates/wasm-qmd-parser +``` + +- [ ] **Step 3: Verify deletion** + +```bash +ls crates/wasm-qmd-parser 2>&1 +``` +Expected: "No such file or directory" + +--- + +### Task 2: Remove build-wasm.yml workflow + +**Files:** +- Delete: `.github/workflows/build-wasm.yml` + +This workflow is manual-dispatch only and only builds `wasm-qmd-parser` with `wasm-pack`. It has no consumers. + +- [ ] **Step 1: Remove the workflow file** + +```bash +rm .github/workflows/build-wasm.yml +``` + +--- + +### Task 3: Clean up workspace Cargo.toml + +**Files:** +- Modify: `Cargo.toml` (workspace root, lines 7-10, 86-87, 244-249) + +Three changes: remove wasm-qmd-parser from exclude list, remove its workspace dependency entry, update stale comments. + +- [ ] **Step 1: Remove wasm-qmd-parser from exclude list** + +In `Cargo.toml` line 10, change the `exclude` array from: +```toml +exclude = ["crates/wasm-quarto-hub-client", "crates/wasm-qmd-parser", "crates/experiments", "crates/pampa/fuzz"] +``` +to: +```toml +exclude = ["crates/wasm-quarto-hub-client", "crates/experiments", "crates/pampa/fuzz"] +``` + +- [ ] **Step 2: Remove workspace dependency for wasm-qmd-parser** + +Delete lines 86-87: +```toml +[workspace.dependencies.wasm-qmd-parser] +path = "./crates/wasm-qmd-parser" +``` + +- [ ] **Step 3: Update the comment on line 7** + +Change line 7 from: +```toml +# - WASM crates: build with wasm-pack or --target wasm32-unknown-unknown +``` +to: +```toml +# - WASM crates: require --target wasm32-unknown-unknown and -Zbuild-std (see dev-docs/wasm.md) +``` + +- [ ] **Step 4: Update the dev profile comment** + +In lines 244-249, change the comment from referencing wasm-pack: +```toml +[profile.dev] +# Tell `rustc` to optimize for small code size to +# work around "too many locals" error from wasm-pack +# https://github.com/wasm-bindgen/wasm-bindgen/issues/3451#issuecomment-1562982835 +opt-level = "s" +``` +to: +```toml +[profile.dev] +# Tell `rustc` to optimize for small code size to +# work around "too many locals" error in WASM builds +# https://github.com/wasm-bindgen/wasm-bindgen/issues/3451#issuecomment-1562982835 +opt-level = "s" +``` + +- [ ] **Step 5: Verify workspace builds** + +```bash +cargo check --workspace +``` +Expected: Clean build with no errors about missing wasm-qmd-parser. + +--- + +### Task 4: Remove stale wasm-pack install from hub-client-e2e.yml + +**Files:** +- Modify: `.github/workflows/hub-client-e2e.yml` (line 47-48) + +The workflow installs wasm-pack but never uses it — the WASM build step runs `npm run build:wasm` which calls `build-wasm.js` (uses `wasm-bindgen`, not wasm-pack). + +- [ ] **Step 1: Read the file to confirm the exact lines** + +Read `.github/workflows/hub-client-e2e.yml` around lines 45-55 to see the wasm-pack step and surrounding context. + +- [ ] **Step 2: Remove the wasm-pack install step** + +Delete the step: +```yaml + - name: Install wasm-pack + run: cargo install wasm-pack +``` + +- [ ] **Step 3: Verify no other reference to wasm-pack in the file** + +```bash +grep -n "wasm-pack" .github/workflows/hub-client-e2e.yml +``` +Expected: No output. + +--- + +### Task 5: Update hub-client/README.md + +**Files:** +- Modify: `hub-client/README.md` + +Remove `wasm-pack` from prerequisites. The actual build tool is `wasm-bindgen-cli` (installed via `cargo xtask dev-setup`). + +- [ ] **Step 1: Read the prerequisites section** + +Read `hub-client/README.md` to find the prerequisites list (around lines 5-11). + +- [ ] **Step 2: Replace the wasm-pack prerequisite** + +Change: +```markdown +- `wasm-pack` (`cargo install wasm-pack`) +``` +to: +```markdown +- `wasm-bindgen-cli` (`cargo xtask dev-setup` installs the correct version) +``` + +--- + +### Task 6: Update wasm-quarto-hub-client/README.md + +**Files:** +- Modify: `crates/wasm-quarto-hub-client/README.md` + +- [ ] **Step 1: Read the README** + +Read `crates/wasm-quarto-hub-client/README.md` and identify any wasm-pack references. + +- [ ] **Step 2: Remove or update wasm-pack references** + +The line "Always use the build script in `hub-client/scripts/build-wasm.js` rather than running `wasm-pack` directly" should be changed to: + +```markdown +Always use the build script in `hub-client/scripts/build-wasm.js` rather than running cargo/wasm-bindgen manually. +``` + +If there are other wasm-pack references, update them similarly. + +--- + +### Task 7: Update workspace CLAUDE.md + +**Files:** +- Modify: `CLAUDE.md` (workspace root, lines 231, 235, 269) + +Three references to `wasm-qmd-parser` need updating after the crate is deleted. + +- [ ] **Step 1: Read and update the WASM crate listing** + +Line 231 lists `wasm-qmd-parser` under the WASM section of workspace structure. Remove the entry: +```markdown +- `wasm-qmd-parser`: WASM module with entry points from `pampa` (see [crates/wasm-qmd-parser/CLAUDE.md](crates/wasm-qmd-parser/CLAUDE.md) for build instructions) +``` + +- [ ] **Step 2: Update hub-client description** + +Line 235 says hub-client "Uses Automerge for real-time sync and the WASM build of `wasm-qmd-parser`". Change to reference the correct crate: +```markdown +A React/TypeScript web application for collaborative editing of Quarto projects. Uses Automerge for real-time sync and the WASM build of `wasm-quarto-hub-client` for live preview rendering. +``` + +- [ ] **Step 3: Update crate layout note** + +Line 269 says `wasm-quarto-hub-client` is "the WASM client (NOT wasm-qmd-parser)". Since wasm-qmd-parser no longer exists, simplify to: +```markdown +- `wasm-quarto-hub-client` is the WASM client for hub-client +``` + +--- + +### Task 8: Note on wasm-pack in dev-setup + +The design spec mentions removing wasm-pack from `cargo xtask dev-setup`. However, wasm-pack +is **not** in the dev-setup install list (`crates/xtask/src/dev_setup.rs`). It was only installed +via `cargo install wasm-pack` in workflow files (already addressed in Tasks 4 and 2). +No action needed here. + +--- + +### Task 9: Commit Phase 1 cleanup + +- [ ] **Step 1: Stage all Phase 1 changes** + +```bash +git add -A +git status +``` + +Review: should show deleted `crates/wasm-qmd-parser/`, deleted `.github/workflows/build-wasm.yml`, modified `Cargo.toml`, modified workflow files, modified READMEs. + +- [ ] **Step 2: Commit** + +```bash +git commit -m "$(cat <<'EOF' +Remove stale wasm-qmd-parser crate and wasm-pack references + +wasm-qmd-parser is fully superseded by wasm-quarto-hub-client. +The build-wasm.yml workflow only built the stale crate. +wasm-pack is not used by the active WASM build pipeline +(build-wasm.js uses cargo build + wasm-bindgen CLI directly). + +- Delete crates/wasm-qmd-parser/ entirely +- Delete .github/workflows/build-wasm.yml +- Remove wasm-qmd-parser from workspace exclude and deps +- Remove stale wasm-pack install from hub-client-e2e.yml +- Update hub-client and wasm-quarto-hub-client READMEs +- Update workspace CLAUDE.md and Cargo.toml comments +EOF +)" +``` + +--- + +## Phase 2+3: Remove cfg Proxy and Add WASM Tests + +These phases land together per the design spec to avoid any validation gap. We add the WASM test infrastructure first (Phase 3 setup), then remove the cfg proxy (Phase 2). + +### Task 10: Add wasm-bindgen-test dependency to pampa + +**Files:** +- Modify: `crates/pampa/Cargo.toml` (dev-dependencies section, around line 70) + +- [ ] **Step 1: Add the dev-dependency** + +Add `wasm-bindgen-test` to the `[dev-dependencies]` section. Also add `wasm-bindgen` since +`wasm_bindgen_test` macros require it: + +```toml +[dev-dependencies] +insta = { version = "1.46", features = ["json", "redactions"] } +proptest = "1.10" +quarto-util.workspace = true +tempfile = "3.24" +wasm-bindgen = "0.2" +wasm-bindgen-test = "0.3" +``` + +- [ ] **Step 2: Verify it resolves** + +```bash +cargo check -p pampa +``` +Expected: compiles. The `wasm-bindgen` and `wasm-bindgen-test` crates are only pulled in for test compilation. + +- [ ] **Step 3: Check the resolved wasm-bindgen version** + +```bash +cargo metadata --format-version 1 | jq -r '.packages[] | select(.name == "wasm-bindgen") | .version' +``` + +Note the version. If it differs from `0.2.108` (the version `cargo xtask dev-setup` installs for `wasm-bindgen-cli`), the dev-setup pinned version in `crates/xtask/src/dev_setup.rs` will need updating. The versions must match exactly or `wasm-bindgen-test-runner` will refuse to run. + +--- + +### Task 11: Add wasm32 test runner to .cargo/config.toml + +**Files:** +- Modify: `.cargo/config.toml` (workspace root) + +Current content is just aliases. Add the runner configuration so `cargo test --target wasm32-unknown-unknown` knows to use `wasm-bindgen-test-runner`. + +- [ ] **Step 1: Append the runner config** + +Add to `.cargo/config.toml`: + +```toml + +[target.wasm32-unknown-unknown] +runner = "wasm-bindgen-test-runner" +``` + +The full file should now be: +```toml +# Cargo configuration for the Quarto Rust workspace + +[alias] +# Run project-specific tasks via: cargo xtask +# See crates/xtask/src/main.rs for available commands +xtask = "run --package xtask --" +dev-setup = "xtask dev-setup" + +[target.wasm32-unknown-unknown] +runner = "wasm-bindgen-test-runner" +``` + +Note: this `runner` setting only applies to `cargo test`, not `cargo build`. The hub-client WASM production build (`build-wasm.js` → `cargo build`) is unaffected. The `wasm-quarto-hub-client` crate has its own `.cargo/config.toml` with `-Zbuild-std` and rustflags; those settings are scoped to that crate's directory. + +--- + +### Task 12: Create WASM test file + +**Files:** +- Create: `crates/pampa/tests/wasm_lua.rs` + +These are smoke tests of WASM-specific code paths. They only compile for `wasm32`. They run in Node.js via `wasm-bindgen-test-runner` (default, no browser needed). + +**Important context for the implementer:** +- `filter.rs` line 123: The `#[cfg(target_arch = "wasm32")]` block creates a restricted Lua VM via `Lua::new_with()` and registers synthetic `io_wasm` and `os_wasm` modules. +- `shortcode.rs` line 72: Same pattern for the shortcode engine. +- `io_wasm.rs` provides `register_wasm_io()` which registers `io.open`, `io.type`, etc. as Lua globals backed by `SystemRuntime`. +- `os_wasm.rs` provides `register_wasm_os()` which registers `os.time`, `os.clock`, `os.difftime`. +- The tests need access to pampa's internal types. Since this is an integration test file (in `tests/`), it can only use pampa's public API. Check what pampa exports. + +- [ ] **Step 1: Check pampa's public API for what we need** + +Read `crates/pampa/src/lib.rs` to see what's publicly exported. We need to find: +- How to create a `SystemRuntime` (or equivalent) for WASM +- How to invoke filter execution +- How to invoke shortcode execution +- Whether `io_wasm` / `os_wasm` registration functions are public + +If key functions are not public, we may need to add `#[cfg(target_arch = "wasm32")]` pub exports or use a different test approach. The design spec lists 6 smoke tests: + +1. Restricted Lua VM creation — `Lua::new_with()` with restricted stdlib succeeds +2. Filter execution — run a simple filter on a small document +3. Shortcode engine — create engine, dispatch a basic handler +4. Error handling — Lua error gets caught as Rust error (not WASM crash) +5. Synthetic io registration — `io.open`, `io.type` available as globals +6. Synthetic os registration — `os.time`, `os.clock`, `os.difftime` available + +- [ ] **Step 2: Write the test file** + +Create `crates/pampa/tests/wasm_lua.rs`. The exact test implementations depend on what pampa exports (determined in Step 1). Here is the skeleton with the tests we can write for certain: + +```rust +//! WASM integration tests for Lua filter and shortcode infrastructure. +//! +//! These tests verify that the restricted Lua stdlib setup, synthetic io/os +//! modules, and filter/shortcode execution work correctly when compiled to +//! the real wasm32 target. +//! +//! **When to add tests here:** Only when modifying WASM-specific code paths: +//! - The #[cfg(target_arch = "wasm32")] blocks in filter.rs / shortcode.rs +//! - io_wasm.rs (synthetic io module) +//! - os_wasm.rs (synthetic os module) +//! +//! Native filter logic is tested comprehensively by the existing native tests. +//! These WASM tests are smoke tests of the target-specific setup. +//! +//! **How to run:** (Linux/macOS only, requires nightly + Clang + wasm-sysroot) +//! ``` +//! CC_wasm32_unknown_unknown=clang \ +//! CFLAGS_wasm32_unknown_unknown="-isystem crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ +//! cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ +//! --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind +//! ``` + +#![cfg(target_arch = "wasm32")] + +use wasm_bindgen_test::*; +// Tests run in Node.js by default. No wasm_bindgen_test_configure!(run_in_browser) +// needed — current tests don't require browser APIs. +``` + +The actual test function bodies depend on pampa's public API discovered in Step 1. The implementer must: + +1. Check which types/functions pampa re-exports for WASM consumers +2. For each of the 6 smoke tests, write a `#[wasm_bindgen_test]` function +3. Tests should be minimal — verify the setup works, not duplicate native test coverage + +Example patterns for the test bodies (adapt to actual API): + +```rust +/// Smoke test: restricted Lua VM creation succeeds on real wasm32 target. +#[wasm_bindgen_test] +fn restricted_lua_vm_creation() { + // Create a Lua VM the same way filter.rs does under #[cfg(target_arch = "wasm32")] + use mlua::{Lua, StdLib}; + let libs = StdLib::COROUTINE | StdLib::TABLE | StdLib::STRING | StdLib::UTF8 | StdLib::MATH; + let lua = Lua::new_with(libs, mlua::LuaOptions::default()) + .expect("restricted Lua VM creation should succeed on wasm32"); + // Verify a basic operation works + let result: i64 = lua.load("1 + 1").eval().unwrap(); + assert_eq!(result, 2); +} + +/// Smoke test: Lua error is caught as Rust error, not a WASM crash. +/// Validates that -Zbuild-std=std,panic_unwind works correctly. +#[wasm_bindgen_test] +fn lua_error_caught_as_rust_error() { + use mlua::{Lua, StdLib}; + let libs = StdLib::COROUTINE | StdLib::TABLE | StdLib::STRING | StdLib::UTF8 | StdLib::MATH; + let lua = Lua::new_with(libs, mlua::LuaOptions::default()).unwrap(); + let result: Result<(), _> = lua.load("error('test error')").exec(); + assert!(result.is_err(), "Lua error should propagate as Rust error"); + let err_msg = result.unwrap_err().to_string(); + assert!(err_msg.contains("test error"), "Error message should be preserved"); +} +``` + +For tests 2, 3, 5, 6 (filter execution, shortcode engine, io/os registration): these require pampa internals. If pampa doesn't export enough API, the implementer should either: +- Add minimal `#[cfg(target_arch = "wasm32")] pub` exports to pampa's `lib.rs` +- Or test via Lua: create the restricted VM, manually call `register_wasm_io`/`register_wasm_os`, then run Lua code that exercises those modules + +The Lua-based approach is preferred since it tests the actual code path without needing to modify pampa's public API: + +```rust +/// Smoke test: synthetic io module is available after registration. +#[wasm_bindgen_test] +fn synthetic_io_registration() { + // This test verifies that io_wasm registers correctly on real wasm32. + // It creates the VM + registers modules the same way filter.rs does. + use mlua::{Lua, StdLib}; + let libs = StdLib::COROUTINE | StdLib::TABLE | StdLib::STRING | StdLib::UTF8 | StdLib::MATH; + let lua = Lua::new_with(libs, mlua::LuaOptions::default()).unwrap(); + // Register synthetic io (needs SystemRuntime — check pampa API) + // pampa::lua::io_wasm::register_wasm_io(&lua, runtime)?; + // + // Then verify: + let has_io_open: bool = lua.load("type(io.open) == 'function'").eval().unwrap(); + assert!(has_io_open, "io.open should be registered"); + let has_io_type: bool = lua.load("type(io.type) == 'function'").eval().unwrap(); + assert!(has_io_type, "io.type should be registered"); +} +``` + +**Note for implementer:** `mlua` must be used directly from `crates/pampa/tests/wasm_lua.rs`. Since `mlua` is a dependency of pampa (behind the `lua-filter` feature), and this test file is compiled with `--features lua-filter`, `mlua` should be available. If not, add `mlua` as a dev-dependency of pampa with the same features. + +- [ ] **Step 3: Verify the test file compiles for native (should be skipped)** + +```bash +cargo check -p pampa --tests +``` + +Expected: compiles cleanly. The `#![cfg(target_arch = "wasm32")]` means the entire file is excluded on native. No compilation errors. + +--- + +### Task 13: Remove cfg proxy from filter.rs + +**Files:** +- Modify: `crates/pampa/src/lua/filter.rs` (lines 120-134) + +- [ ] **Step 1: Read the current code** + +Read `crates/pampa/src/lua/filter.rs` lines 118-136 to see the exact current state. + +- [ ] **Step 2: Change the cfg guards** + +Change line 123 from: +```rust + #[cfg(any(target_arch = "wasm32", test))] +``` +to: +```rust + #[cfg(target_arch = "wasm32")] +``` + +Change line 133 from: +```rust + #[cfg(not(any(target_arch = "wasm32", test)))] +``` +to: +```rust + #[cfg(not(target_arch = "wasm32"))] +``` + +- [ ] **Step 3: Update the comment above the cfg blocks** + +The comment on lines 121-122 currently reads: +```rust + // On WASM, we can't load all libraries (no package/io/os/debug support), + // so use a restricted set. On native, load everything for full compatibility. +``` + +Keep this comment as-is — it's accurate. The comment about test environment (if any additional +comment exists) should be removed. + +--- + +### Task 14: Remove cfg proxy from shortcode.rs + +**Files:** +- Modify: `crates/pampa/src/lua/shortcode.rs` (lines 72-86) + +- [ ] **Step 1: Read the current code** + +Read `crates/pampa/src/lua/shortcode.rs` lines 70-88. + +- [ ] **Step 2: Change the cfg guards** + +Change line 72 from: +```rust + #[cfg(any(target_arch = "wasm32", test))] +``` +to: +```rust + #[cfg(target_arch = "wasm32")] +``` + +Change line 85 from: +```rust + #[cfg(not(any(target_arch = "wasm32", test)))] +``` +to: +```rust + #[cfg(not(target_arch = "wasm32"))] +``` + +--- + +### Task 15: Verify native tests pass after cfg proxy removal + +This is the critical verification step. The 8 filter traversal tests that use `io.open` should now pass on all platforms because they use `Lua::new()` with real C stdlib instead of the synthetic WASM io. + +- [ ] **Step 1: Run pampa tests** + +```bash +cargo nextest run -p pampa +``` + +Expected: All tests pass, including the 8 filter traversal tests that previously failed on Windows with "os error 123". + +- [ ] **Step 2: Run full workspace tests** + +```bash +cargo nextest run --workspace +``` + +Expected: All tests pass. Changes to pampa's cfg guards could theoretically affect downstream crates. + +--- + +### Task 16: Commit Phase 2+3 + +- [ ] **Step 1: Stage and review** + +```bash +git add -A +git diff --cached --stat +``` + +Expected changes: `crates/pampa/Cargo.toml` (new dev-deps), `.cargo/config.toml` (runner), `crates/pampa/tests/wasm_lua.rs` (new), `crates/pampa/src/lua/filter.rs` (cfg change), `crates/pampa/src/lua/shortcode.rs` (cfg change). + +- [ ] **Step 2: Commit** + +```bash +git commit -m "$(cat <<'EOF' +Replace cfg test proxy with real WASM tests + +Remove `test` from `#[cfg(any(target_arch = "wasm32", test))]` guards +in filter.rs and shortcode.rs so native tests use Lua::new() with real +C stdlib on all platforms. This fixes the 8 filter traversal tests that +failed on Windows because the synthetic io.open only handles POSIX VFS +paths. + +Add wasm-bindgen-test smoke tests in crates/pampa/tests/wasm_lua.rs +that run on the real wasm32 target, validating: +- Restricted Lua VM creation +- Filter execution through WASM code path +- Shortcode engine on WASM +- Error handling (panic_unwind works) +- Synthetic io/os module registration + +Configure wasm-bindgen-test-runner in .cargo/config.toml. +WASM tests require nightly + Clang + wasm-sysroot (Linux/macOS CI only). +EOF +)" +``` + +--- + +## Phase 4: CI Integration + +### Task 17: Add wasm-tests job to test-suite.yml + +**Files:** +- Modify: `.github/workflows/test-suite.yml` + +Add a new job that runs the WASM tests on Linux only. Mirror the Clang/wasm-sysroot setup from `ts-test-suite.yml` and `hub-client/scripts/build-wasm.js`. + +- [ ] **Step 1: Read the current workflow** + +Read `.github/workflows/test-suite.yml` to understand the structure, triggers, and existing jobs. + +- [ ] **Step 2: Add the wasm-tests job** + +Add a new job after the existing `test-suite` job. The job should: +- Run on `ubuntu-latest` only (no matrix — WASM tests are Linux-only) +- Use the same trigger paths as the existing test suite, plus `crates/pampa/tests/wasm_lua.rs` +- Set up: Rust nightly, Clang, rust-src component, wasm-bindgen-test-runner (via `cargo xtask dev-setup`) +- Run the WASM test command + +```yaml + wasm-tests: + name: WASM Tests + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 1 + + - name: Set up Rust nightly + uses: dtolnay/rust-toolchain@nightly + with: + targets: wasm32-unknown-unknown + components: rust-src + + - name: Set up Clang + uses: egor-tensin/setup-clang@v1 + with: + version: latest + platform: x64 + + - name: Rust cache + uses: Swatinem/rust-cache@v2 + with: + shared-key: rust-wasm-tests + + - name: Install wasm-bindgen-cli + run: cargo xtask dev-setup + + - name: Run WASM tests + run: | + CC_wasm32_unknown_unknown=clang \ + CFLAGS_wasm32_unknown_unknown="-isystem crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ + cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind +``` + +- [ ] **Step 3: Verify the workflow YAML is valid** + +```bash +python3 -c "import yaml; yaml.safe_load(open('.github/workflows/test-suite.yml'))" +``` +Or use `yq` if available. Expected: no parse errors. + +--- + +### Task 18: Migrate wasm-bindgen-cli install in ts-test-suite.yml + +**Files:** +- Modify: `.github/workflows/ts-test-suite.yml` (line 125-127) + +The design spec says to migrate the hardcoded `cargo install wasm-bindgen-cli --version 0.2.108` to use `cargo xtask dev-setup`, which reads the version from Cargo.lock and keeps it in sync. + +- [ ] **Step 1: Read the current install step** + +Read `.github/workflows/ts-test-suite.yml` around lines 123-130. + +- [ ] **Step 2: Replace the install step** + +Change: +```yaml + - name: Install wasm-bindgen-cli + run: cargo install wasm-bindgen-cli --version 0.2.108 +``` +to: +```yaml + - name: Install dev tools (wasm-bindgen-cli) + run: cargo xtask dev-setup +``` + +Note: `cargo xtask dev-setup` also installs `cargo-nextest` and `cargo-insta`, but those may already be installed by a previous step. The setup is idempotent so this is fine — the tools are cached and skip reinstall if present. + +--- + +### Task 19: Commit CI changes + +- [ ] **Step 1: Stage and commit** + +```bash +git add .github/workflows/test-suite.yml .github/workflows/ts-test-suite.yml +git commit -m "$(cat <<'EOF' +Add WASM test CI job and migrate wasm-bindgen-cli to dev-setup + +Add wasm-tests job to test-suite.yml that runs pampa WASM smoke tests +on Linux with nightly Rust + Clang + wasm-sysroot. Uses cargo xtask +dev-setup for version-matched wasm-bindgen-cli installation. + +Migrate ts-test-suite.yml from hardcoded wasm-bindgen-cli version to +cargo xtask dev-setup for consistent version management. +EOF +)" +``` + +--- + +## Phase 5: Documentation + +### Task 20: Rewrite dev-docs/wasm.md + +**Files:** +- Modify: `dev-docs/wasm.md` (full rewrite) + +This becomes the single source of truth for WASM in this project. Current content is outdated (references wasm-qmd-parser and wasm-pack). + +- [ ] **Step 1: Write the new content** + +```markdown +# WASM in the Quarto Rust Monorepo + +## Architecture + +`wasm-quarto-hub-client` wraps `pampa` + `quarto-core` for the hub-client web app. +It compiles to a WASM module that runs in the browser, providing live preview rendering. + +The crate is **excluded from the default workspace** (`Cargo.toml` `exclude` list) because +it requires `--target wasm32-unknown-unknown` and `-Zbuild-std=std,panic_unwind`. + +## Build + +The production WASM build is handled by `hub-client/scripts/build-wasm.js`: + +```bash +cd hub-client +npm run build:wasm # WASM module only +npm run build:all # WASM + TypeScript +``` + +The build script runs: +1. `cargo build -p wasm-quarto-hub-client --target wasm32-unknown-unknown` + with `-Zbuild-std=std,panic_unwind` (via `crates/wasm-quarto-hub-client/.cargo/config.toml`) +2. `wasm-bindgen` CLI to generate JS/TS bindings + +### Why not wasm-pack? + +This project uses `cargo build` + `wasm-bindgen` CLI directly because: +- `-Zbuild-std=std,panic_unwind` is required for Lua error handling (setjmp/longjmp to + panic/catch_unwind). wasm-pack doesn't support `-Zbuild-std`. +- wasm-pack is deprecated (rustwasm org sunset September 2025). + +### C toolchain requirement + +`pampa` with `lua-filter` pulls in `mlua` → `lua-src-wasm`, which compiles Lua from C source +via the `cc` crate. When targeting wasm32, this requires Clang with wasm32 support: + +```bash +# Set by build-wasm.js automatically for production builds. +# For manual builds or tests: +export CC_wasm32_unknown_unknown=clang +export CFLAGS_wasm32_unknown_unknown="-isystem crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" +``` + +## Testing + +### Native tests (all platforms) + +Native Rust tests (`cargo nextest run`) test filter and shortcode logic using `Lua::new()` +with the full C stdlib. These run on all platforms including Windows. + +### WASM smoke tests (Linux CI) + +`crates/pampa/tests/wasm_lua.rs` contains smoke tests that compile and run on the real +`wasm32-unknown-unknown` target. They verify the WASM-specific Lua VM setup: +- Restricted stdlib creation (`Lua::new_with()`) +- Synthetic `io`/`os` module registration +- Filter and shortcode execution through the WASM code path +- Error handling (`panic_unwind` works correctly) + +Run locally (Linux/macOS with LLVM): +```bash +CC_wasm32_unknown_unknown=clang \ +CFLAGS_wasm32_unknown_unknown="-isystem crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ +cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind +``` + +**Important:** You must use `--test wasm_lua` to select only the WASM test file. +Running `cargo test -p pampa --target wasm32` without `--test` will fail because +native tests can't compile for wasm32. + +WASM tests are **not** part of `cargo xtask verify` — they require nightly + Clang with +wasm32 support, which is Linux/macOS only. They run in the `wasm-tests` CI job. + +### Hub-client integration tests + +The hub-client test suite (`npm run test:ci`) tests the compiled WASM module through +JavaScript, covering rendering, templates, and format detection. These complement +the Rust-level WASM smoke tests. +``` + +- [ ] **Step 2: Verify no broken links** + +Check that all referenced files exist: +```bash +ls hub-client/scripts/build-wasm.js crates/wasm-quarto-hub-client/.cargo/config.toml crates/pampa/tests/wasm_lua.rs crates/wasm-quarto-hub-client/wasm-sysroot/ +``` + +--- + +### Task 21: Update pampa/CLAUDE.md + +**Files:** +- Modify: `crates/pampa/CLAUDE.md` + +Add a section about WASM tests so AI assistants know when and how to add them. + +- [ ] **Step 1: Append WASM testing section** + +Add at the end of `crates/pampa/CLAUDE.md`: + +```markdown + +## WASM Testing + +When modifying WASM-specific code paths (the `#[cfg(target_arch = "wasm32")]` blocks in +`filter.rs`/`shortcode.rs`, `io_wasm.rs`, or `os_wasm.rs`), add or update smoke tests in +`tests/wasm_lua.rs`. + +**Never add `test` to the `target_arch = "wasm32"` cfg guard.** Native tests must use +`Lua::new()` with the real C stdlib. WASM-specific setup is validated by the dedicated +WASM tests. + +WASM tests can't run on Windows. On Linux/macOS with LLVM: +``` +CC_wasm32_unknown_unknown=clang \ +CFLAGS_wasm32_unknown_unknown="-isystem crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ +cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind +``` + +See `dev-docs/wasm.md` for full WASM architecture and testing details. +``` + +--- + +### Task 22: Create .claude/rules/wasm.md + +**Files:** +- Create: `.claude/rules/wasm.md` + +This AI rule prevents future regression of the cfg proxy pattern. + +- [ ] **Step 1: Write the rule** + +```markdown +# WASM Code Rules + +## Never add `test` to wasm32 cfg guards + +The cfg pattern `#[cfg(any(target_arch = "wasm32", test))]` is prohibited. It forces +native tests through the WASM-restricted Lua stdlib, which fails on Windows. + +Correct pattern: +```rust +#[cfg(target_arch = "wasm32")] +// WASM-specific code (restricted Lua stdlib, synthetic io/os) + +#[cfg(not(target_arch = "wasm32"))] +// Native code (full Lua stdlib via Lua::new()) +``` + +## Verify WASM tests when editing WASM code + +When modifying any of these files, update `crates/pampa/tests/wasm_lua.rs`: +- `crates/pampa/src/lua/filter.rs` (cfg(target_arch = "wasm32") blocks) +- `crates/pampa/src/lua/shortcode.rs` (cfg(target_arch = "wasm32") blocks) +- `crates/pampa/src/lua/io_wasm.rs` +- `crates/pampa/src/lua/os_wasm.rs` + +WASM tests can't run locally on Windows — they run in Linux CI. +See `dev-docs/wasm.md` for the local run command (Linux/macOS). +``` + +--- + +### Task 23: Update claude-notes/instructions/testing.md + +**Files:** +- Modify: `claude-notes/instructions/testing.md` (lines 9-22, the WASM-Restricted Stdlib section) + +This section currently describes the cfg proxy pattern. Update it to reflect the new approach. + +- [ ] **Step 1: Read the current section** + +Read `claude-notes/instructions/testing.md` lines 1-30 to see the exact text. + +- [ ] **Step 2: Replace the WASM-Restricted Stdlib section** + +Replace the section (approximately lines 9-22) that starts with "Shortcode and filter tests always run against the WASM-restricted Lua stdlib" with: + +```markdown +## Native vs WASM Lua Testing + +Native tests (`cargo nextest run`) use `Lua::new()` with the full C stdlib on all platforms. +This is the standard Lua environment — tests can use `io.open`, `os.time`, and all standard +library functions. + +WASM-specific code paths (restricted Lua stdlib, synthetic io/os modules) are tested by +dedicated smoke tests in `crates/pampa/tests/wasm_lua.rs` that run on the real +`wasm32-unknown-unknown` target in CI. See `crates/pampa/CLAUDE.md` for details on when +to add WASM tests. + +**Never add `test` to the `#[cfg(target_arch = "wasm32")]` guard.** This was a prior pattern +that caused Windows test failures. WASM coverage is provided by the real WASM tests in CI. +``` + +--- + +### Task 24: Commit documentation + +- [ ] **Step 1: Stage and commit** + +```bash +git add dev-docs/wasm.md crates/pampa/CLAUDE.md .claude/rules/wasm.md claude-notes/instructions/testing.md +git commit -m "$(cat <<'EOF' +Document WASM testing convention and architecture + +Rewrite dev-docs/wasm.md as single source of truth for WASM in this +project: architecture, build pipeline, testing strategy, and C toolchain +requirements. + +Add WASM testing guidance to pampa/CLAUDE.md and .claude/rules/wasm.md +to prevent regression of the cfg test proxy pattern. + +Update testing.md to reflect the new native vs WASM testing approach. +EOF +)" +``` + +--- + +## Phase 6: Final Verification + +### Task 25: Full workspace verification + +- [ ] **Step 1: Build the full workspace** + +```bash +cargo build --workspace +``` +Expected: clean build. + +- [ ] **Step 2: Run full workspace tests** + +```bash +cargo nextest run --workspace +``` +Expected: all tests pass, including the 8 previously-failing Windows filter traversal tests. + +- [ ] **Step 3: Run cargo xtask lint** + +```bash +cargo xtask lint +``` +Expected: no lint violations. + +- [ ] **Step 4: Ask Chris to verify hub-client build** + +The hub-client WASM build (`npm run build:all`) should be unaffected since we only changed: +- `.cargo/config.toml` (added runner, only affects `cargo test`) +- pampa dev-dependencies (only affects test compilation) +- cfg guards (WASM path unchanged, native path simplified) + +Ask Chris: "Should I run `cargo xtask verify` to confirm the hub-client build is unaffected?" + +--- + +### Task 26: Update beads issue + +- [ ] **Step 1: Close the beads issue** + +```bash +br close bd-itj9 --reason "All phases complete: stale wasm-qmd-parser removed, cfg proxy replaced with real WASM tests, CI job added, documentation updated" +``` + +- [ ] **Step 2: Sync beads** + +From the **main repo** (not the worktree, since beads redirect is active): +```bash +cd /c/Users/chris/Documents/DEV_R/q2 +br sync --flush-only +git add .beads/ +git commit -m "Sync beads: close bd-itj9 WASM testing and cleanup" +``` From 1bd605d46cbba2c38f1729fe2d6996abfdfb58e3 Mon Sep 17 00:00:00 2001 From: Christophe Dervieux Date: Tue, 7 Apr 2026 16:16:17 +0200 Subject: [PATCH 2/5] Replace cfg test proxy with real WASM tests and CI job MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add 6 smoke tests in crates/pampa/tests/wasm_lua.rs that compile and run on wasm32-unknown-unknown: - Restricted Lua VM creation (Lua::new_with()) - Filter execution through the WASM code path (VFS + synthetic io/os) - Shortcode engine initialization - Error handling (validates panic_unwind works) - Synthetic io and os module registration They run in a new wasm-tests CI job (Linux, Clang), and locally on Linux/macOS — see dev-docs/wasm.md. Build infrastructure, including findings from the CI bring-up: - .cargo/config.toml: wasm-bindgen-test-runner for the wasm32 target - Gate proptest, insta, tempfile, tokio dev-deps on not(wasm32) — they pull in getrandom, which does not compile for wasm32 - Gate pampa [[bin]] targets via required-features so cargo skips them during WASM test runs (rust-lang/cargo#12980) - Absolute -isystem paths for the wasm sysroot; -fno-builtin - Docs: dev-docs/wasm.md, .claude/rules/wasm.md, pampa CLAUDE.md, claude-notes/instructions/testing.md --- .cargo/config.toml | 3 + .claude/rules/wasm.md | 11 + .github/workflows/test-suite.yml | 39 +++ .../2026-04-03-wasm-testing-and-cleanup.md | 187 ++++++++++++ claude-notes/instructions/testing.md | 9 +- .../2026-04-07-wasm-testing-and-cleanup.md | 88 ++++++ crates/pampa/CLAUDE.md | 22 +- crates/pampa/Cargo.toml | 15 + crates/pampa/tests/wasm_lua.rs | 277 ++++++++++++++++++ dev-docs/wasm.md | 108 +++++-- 10 files changed, 726 insertions(+), 33 deletions(-) create mode 100644 crates/pampa/tests/wasm_lua.rs diff --git a/.cargo/config.toml b/.cargo/config.toml index daab58d75..a90c7dab0 100644 --- a/.cargo/config.toml +++ b/.cargo/config.toml @@ -6,3 +6,6 @@ xtask = "run --package xtask --" dev-setup = "xtask dev-setup" create-worktree = "xtask create-worktree" + +[target.wasm32-unknown-unknown] +runner = "wasm-bindgen-test-runner" diff --git a/.claude/rules/wasm.md b/.claude/rules/wasm.md index 85a54e125..54edba969 100644 --- a/.claude/rules/wasm.md +++ b/.claude/rules/wasm.md @@ -52,3 +52,14 @@ stages run sequentially within a single task. If you find yourself wanting to drop `?Send`, that is a signal something is wrong with the design of the calling context, not with the trait. Stop and ask before changing it. + +## Verify WASM tests when editing WASM code + +When modifying any of these files, update `crates/pampa/tests/wasm_lua.rs`: +- `crates/pampa/src/lua/filter.rs` (cfg(target_arch = "wasm32") blocks) +- `crates/pampa/src/lua/shortcode.rs` (cfg(target_arch = "wasm32") blocks) +- `crates/pampa/src/lua/io_wasm.rs` +- `crates/pampa/src/lua/os_wasm.rs` + +WASM tests can't run locally on Windows — they run in Linux CI. +See `dev-docs/wasm.md` for the local run command (Linux/macOS). diff --git a/.github/workflows/test-suite.yml b/.github/workflows/test-suite.yml index 7a8b2d374..9601699a5 100644 --- a/.github/workflows/test-suite.yml +++ b/.github/workflows/test-suite.yml @@ -179,3 +179,42 @@ jobs: run: cargo nextest run --tests --cargo-profile ci env: RUSTFLAGS: "-D warnings" + + wasm-tests: + name: WASM Tests + runs-on: ubuntu-latest + # Override rust-toolchain.toml so the rustup proxy doesn't auto-install the + # prebuilt wasm32-unknown-unknown target. We need -Zbuild-std to rebuild + # core from source, and the prebuilt sysroot causes E0152 (duplicate lang + # item). The production WASM build avoids this because wasm-quarto-hub-client + # is excluded from the workspace and gets an isolated target/ directory. + env: + RUSTUP_TOOLCHAIN: nightly + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 1 + + - name: Install Rust nightly with rust-src + run: rustup toolchain install nightly --component rust-src --profile minimal + + - name: Set up Clang + uses: egor-tensin/setup-clang@v1 + with: + version: latest + platform: x64 + + - name: Rust cache + uses: Swatinem/rust-cache@v2 + with: + shared-key: rust-wasm-tests + + - name: Install wasm-bindgen-cli + run: cargo xtask dev-setup + + - name: Run WASM tests + run: | + CC_wasm32_unknown_unknown=clang \ + CFLAGS_wasm32_unknown_unknown="-isystem ${{ github.workspace }}/crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ + cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind,panic_abort diff --git a/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md b/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md index 45d8b1277..776e43e00 100644 --- a/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md +++ b/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md @@ -226,6 +226,193 @@ On Windows, skip WASM tests — this is consistent with the WASM build being ski On macOS/Linux with LLVM installed, contributors modifying WASM-specific code can run them locally. +## dofile_wasm interaction (discovered during CI) + +Removing the cfg proxy exposed a hidden coupling: `register_wasm_dofile` (called only on +WASM) overrides Lua's built-in `dofile` to push/pop the script-dir stack, enabling +`quarto.utils.resolve_path()` to resolve relative to the dofile'd script's directory. +The native path uses the C Lua `dofile` which doesn't interact with the stack. + +The `test_dofile_script_dir_stack` test in `dofile_wasm.rs` was passing on main because +the `cfg(any(wasm32, test))` proxy caused `register_wasm_dofile` to run in native tests. +After removing the proxy, native tests get the C `dofile` and the test fails. + +**Research finding:** Neither Pandoc nor Quarto CLI (TypeScript) provide script-dir tracking +for raw `dofile()`. Pandoc uses `PANDOC_SCRIPT_FILE` (set once, never updated). Quarto CLI +has an internal `scriptFile` stack used for shortcodes/wrapped filters, but raw `dofile()` +uses standard Lua CWD-relative resolution. + +**Resolution:** The dofile script-dir tracking is a WASM-only feature (needed because +WASM's dofile is fully reimplemented via SystemRuntime). The failing test should be gated +on `wasm32` or moved to `wasm_lua.rs`. A follow-up issue tracks adding this feature to +native as an improvement over both Pandoc and Quarto CLI behavior. + +## wasm-bindgen-cli install method (reverted) + +Migrating `ts-test-suite.yml` from `cargo install wasm-bindgen-cli --version 0.2.108` to +`cargo xtask dev-setup` caused all hub-client `.wasm.test.ts` tests to fail with an +`externref` type mismatch in the compiled WASM module. Main uses the hardcoded install +and passes. The difference is that `cargo xtask dev-setup` adds `--locked` to the install. + +Reverted in #109 — the TS Test Suite keeps the hardcoded install. The `test-suite.yml` +WASM Tests job still uses `cargo xtask dev-setup` (it installs `wasm-bindgen-test-runner`, +not the production `wasm-bindgen` CLI used by `build-wasm.js`). + +Tracked as `bd-jakt` for investigation. + +## WASM test CI build configuration (discovered during CI) + +The WASM Tests CI job failed with two independent build errors. Both stem from +differences between how the production WASM build (`npm run build:all`) and the +new WASM test build are configured. + +### Bug 1: Duplicate `core` lang item (E0152) + +**Symptom:** `error[E0152]: duplicate lang item in crate core: sized` — two copies of +`libcore` are linked. + +**Root cause:** The CI toolchain setup installs both the prebuilt `wasm32-unknown-unknown` +target (via `targets: wasm32-unknown-unknown`) AND uses `-Zbuild-std=std,panic_unwind`. +`-Zbuild-std` rebuilds the entire std dependency chain (`core` → `alloc` → `std`) from +source. The prebuilt target already ships a compiled `core`. Rust sees two definitions +of every lang item and refuses to link. + +This is a known conflict: +- rust-lang/cargo#10200 (duplicate use of std core with -Z build-std) +- rust-lang/rust#69090 (nightly regression with -Z build-std for wasm32) + +**Why the production build works:** `ts-test-suite.yml` sets up the toolchain as +`dtolnay/rust-toolchain@nightly` with NO `targets:` — it does not install the prebuilt +wasm32 target. The `-Zbuild-std` comes from `crates/wasm-quarto-hub-client/.cargo/config.toml` +and rebuilds everything from `rust-src` (included by default in nightly). + +**Fix:** Removing `targets:` from the CI toolchain step is necessary but not sufficient. +The repo's `rust-toolchain.toml` specifies `targets = ["wasm32-unknown-unknown"]`, which +rustup applies automatically. The production build avoids the conflict because +`wasm-quarto-hub-client` is excluded from the workspace and gets an isolated `target/` +directory. The WASM test runs within the workspace, where the conflict manifests. + +The CI job must explicitly remove the prebuilt target before running tests: +```yaml +- name: Remove prebuilt wasm32 target (conflicts with -Zbuild-std) + run: rustup target remove wasm32-unknown-unknown +``` + +### Bug 2: Bin targets compiled for wasm32 + +**Symptom:** `error[E0433]: cannot find NativeRuntime` and `cannot find tokio` in +`pampa/src/main.rs` — the `pampa` and `ast-reconcile` binaries are being compiled for +wasm32, where native-only types don't exist. + +**Root cause:** When running integration tests, Cargo automatically builds the package's +binary targets so tests can access them via `CARGO_BIN_EXE_`. The `--test wasm_lua` +flag selects which test to run, but Cargo still builds all bin targets. This is documented +Cargo behavior (rust-lang/cargo#12980). + +**Why the production build doesn't hit this:** `npm run build:all` runs `cargo build` on +`wasm-quarto-hub-client` (which has no `[[bin]]` targets), not on `pampa`. + +**Fix:** Add `required-features = ["terminal-support"]` to both `[[bin]]` targets in +`crates/pampa/Cargo.toml`. The WASM test command uses `--no-default-features --features lua-filter`, +so `terminal-support` is absent and the bins are silently skipped. Normal builds use default +features (which include `terminal-support`), so nothing changes for development or CI test suite. + +### Key insight: two different `-Zbuild-std` paths + +The repo has two independent WASM build configurations: + +| Aspect | Production build | WASM tests | +|--------|-----------------|------------| +| Crate | `wasm-quarto-hub-client` | `pampa` (test target) | +| Cargo cwd | `crates/wasm-quarto-hub-client/` | repo root | +| Config | crate-local `.cargo/config.toml` | root `.cargo/config.toml` | +| `-Zbuild-std` | via `[unstable]` in crate config | explicit CLI flag | +| Build mode | `--release` | debug (default) | +| Prebuilt target | not installed | was installed (bug) | +| `-fno-builtin` | not needed (release) | needed (debug) | + +Both use `-Zbuild-std=std,panic_unwind` but through different mechanisms. +The WASM test path must match the production path's approach of NOT installing +the prebuilt target. + +## CI toolchain simplification (2026-04-13) + +### Removing dtolnay/rust-toolchain action + +All CI workflows used `dtolnay/rust-toolchain@nightly` to set up the Rust toolchain. +This is redundant — `rust-toolchain.toml` already specifies the full configuration +(nightly channel, components, targets), and `rustup` reads it natively via proxied +`cargo` commands. + +Replaced in all workflows with: +```yaml +- name: Set up Rust + run: rustup show active-toolchain +``` + +This triggers auto-install from `rust-toolchain.toml` and shows the resolved toolchain. + +### RUSTUP_TOOLCHAIN for WASM tests (supersedes Bug 1 fix) + +The original fix for Bug 1 (E0152 duplicate core) was `rustup target remove +wasm32-unknown-unknown`. This failed because the rustup proxy reads +`rust-toolchain.toml` and auto-reinstalls the target on the next `cargo` command. + +The correct fix: set `RUSTUP_TOOLCHAIN=nightly` as a job-level env var. This +bypasses `rust-toolchain.toml` entirely, preventing the target from ever being +installed. The job uses explicit `rustup toolchain install nightly --component +rust-src --profile minimal` instead of relying on `rust-toolchain.toml`. + +### panic_abort in -Zbuild-std + +The test binary (not the production library build) needs `panic_abort` in the +`-Zbuild-std` list. The WASM test command is: +```bash +cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind,panic_abort +``` + +The production build (`wasm-quarto-hub-client`) only needs `std,panic_unwind` because +it builds a library, not a test binary with its own main/harness. + +## wasm-c-shim: shared C stdlib stubs (2026-04-13) + +### Problem + +The WASM integration tests (filter execution, synthetic io/os verification) link +`pampa` for wasm32, which pulls in tree-sitter and Lua — both C libraries that +reference libc symbols (`calloc`, `fprintf`, `snprintf`, `abort`, etc.). On +`wasm32-unknown-unknown` there is no libc; these symbols must be provided by +Rust `#[no_mangle]` shim functions. + +The production build works because `wasm-quarto-hub-client/src/c_shim.rs` provides +~980 lines of these shims. The WASM test only builds `pampa` and doesn't include +that crate, so the linker can't resolve the symbols. + +### Solution + +Extract `c_shim.rs` into a new `crates/wasm-c-shim/` crate: + +- **Workspace member** (not excluded — it compiles for both native and wasm32, + but the `#[no_mangle]` exports are gated on `target_arch = "wasm32"`) +- **Dependency of `wasm-quarto-hub-client`** (replacing the inline `c_shim` module) +- **Dev-dependency of `pampa`** (gated on `target_arch = "wasm32"`) + +The test file imports `wasm_c_shim` to pull the shim symbols into the link: +```rust +// Pull in C stdlib shims for wasm32 (calloc, fprintf, snprintf, etc.) +// These are needed by tree-sitter and Lua's C code on wasm32-unknown-unknown. +extern crate wasm_c_shim; +``` + +### Why not alternatives + +- **Include via `#[path]`**: Brittle, the file uses `pub` items and module-level statics + that could conflict. Can't be tested independently. +- **Drop integration tests**: Leaves a gap — core tests verify the restricted VM + registers synthetic io/os, but only integration tests verify they actually work + when called from Lua during filter execution on real wasm32. + ## Out of scope - Migrating wasm-pack usage (no longer needed — only stale crate used it) diff --git a/claude-notes/instructions/testing.md b/claude-notes/instructions/testing.md index c57231878..4d7248fd5 100644 --- a/claude-notes/instructions/testing.md +++ b/claude-notes/instructions/testing.md @@ -13,12 +13,13 @@ Native tests (`cargo nextest run`) use `Lua::new()` with the full C stdlib on al This is the standard Lua environment — tests can use `io.open`, `os.time`, and all standard library functions. -WASM-specific code paths (restricted Lua stdlib, synthetic io/os modules) are tested -separately on the real `wasm32-unknown-unknown` target in CI. See `dev-docs/wasm.md` for -the WASM architecture and build details. +WASM-specific code paths (restricted Lua stdlib, synthetic io/os modules) are tested by +dedicated smoke tests in `crates/pampa/tests/wasm_lua.rs` that run on the real +`wasm32-unknown-unknown` target in CI. See `crates/pampa/CLAUDE.md` for details on when +to add WASM tests. **Never add `test` to the `#[cfg(target_arch = "wasm32")]` guard.** This was a prior pattern -that caused Windows test failures. WASM coverage is provided by dedicated WASM tests in CI. +that caused Windows test failures. WASM coverage is provided by the real WASM tests in CI. ## End-to-End Testing for WASM Features diff --git a/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md b/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md index e28babbc6..3b4bc7cc0 100644 --- a/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md +++ b/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md @@ -1003,6 +1003,94 @@ EOF --- +## Phase 6a: CI Fixes (added during PR review) + +Fixes discovered after CI ran for the first time. + +### Task 27: Fix WASM sysroot path in CI + +The `cc` crate runs clang from `OUT_DIR`, not the repo root, so relative `-isystem` paths +don't resolve. Use `$PWD` in local docs and `${{ github.workspace }}` in CI. + +- [x] Update `.github/workflows/test-suite.yml` — absolute path via `${{ github.workspace }}` +- [x] Update `dev-docs/wasm.md` — `$PWD` in local run instructions +- [x] Update `crates/pampa/tests/wasm_lua.rs` — `$PWD` in doc comment +- [x] Update `crates/pampa/CLAUDE.md` — `$PWD` in WASM test instructions +- [x] Document why `-fno-builtin` is needed for tests but not production (debug vs release) + +### Task 28: Gate wasm-incompatible dev-dependencies + +`proptest` pulls in `getrandom 0.3.4` which doesn't compile for `wasm32-unknown-unknown`. +This was never an issue before because pampa dev-deps were never compiled for wasm32. + +- [x] Move `proptest`, `insta`, `tempfile`, `tokio` to `cfg(not(target_arch = "wasm32"))` dev-deps +- [x] Verify native test compilation still works + +### Task 29: Handle dofile behavioral difference + +Removing the cfg proxy exposed that `register_wasm_dofile` adds script-dir stack tracking +to `dofile()` on WASM, but native C `dofile` doesn't interact with the stack. Neither +Pandoc nor Quarto CLI tracks script dirs for raw `dofile()`. + +- [x] Mark `test_dofile_script_dir_stack` as `#[ignore]` on non-wasm32 targets +- [x] Document finding in design spec (`claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md`) +- [x] Create GitHub issue #112 for team discussion on whether to align behavior +- [x] Create beads issue `bd-dvra` with local tracking + +## Phase 6b: CI Fixes Round 2 (2026-04-13) + +Fixes discovered after the E0152 sysroot conflict was resolved. + +### Task 30: Replace dtolnay/rust-toolchain with rustup + +The `dtolnay/rust-toolchain@nightly` action is redundant — `rust-toolchain.toml` +already specifies everything. Replace with `rustup show active-toolchain` (triggers +auto-install from `rust-toolchain.toml`). + +- [x] Replace action in `test-suite.yml` (main test-suite job) +- [x] Replace action in `hub-client-e2e.yml` +- [x] Replace action in `ts-test-suite.yml` +- [x] Validate all YAML files + +### Task 31: Fix WASM test sysroot conflict via RUSTUP_TOOLCHAIN + +The `rustup target remove` approach failed because the rustup proxy reads +`rust-toolchain.toml` and auto-reinstalls the target on the next `cargo` command. + +- [x] Set `RUSTUP_TOOLCHAIN: nightly` as job-level env in wasm-tests job +- [x] Replace `dtolnay/rust-toolchain@nightly` with `rustup toolchain install nightly --component rust-src --profile minimal` +- [x] Remove the `rustup target remove` step (no longer needed) + +### Task 32: Fix WASM test compilation errors + +Tests were written before CI could reach the compilation stage (E0152 always hit first). + +- [x] Add `.await` to async `apply_lua_filters` calls (became async in e537fb80) +- [x] Use `FilterOutput` struct field access instead of tuple destructuring +- [x] Add `panic_abort` to `-Zbuild-std` in CI and docs + +### Task 33: Extract wasm-c-shim crate + +The WASM integration tests link `pampa` for wasm32, pulling in tree-sitter and Lua +(C libraries) that need libc symbols (`calloc`, `fprintf`, `snprintf`, `abort`, etc.). +These are currently provided by `wasm-quarto-hub-client/src/c_shim.rs` but the test +doesn't include that crate. + +- [ ] Create `crates/wasm-c-shim/` with `Cargo.toml` and `src/lib.rs` +- [ ] Move `c_shim.rs` content from `wasm-quarto-hub-client` to new crate +- [ ] Gate `#[no_mangle]` exports on `#[cfg(target_arch = "wasm32")]` +- [ ] Add `wasm-c-shim` as dependency of `wasm-quarto-hub-client` (replace inline module) +- [ ] Add `wasm-c-shim` as dev-dependency of `pampa` (wasm32 only) +- [ ] Add `extern crate wasm_c_shim;` to `wasm_lua.rs` +- [ ] Verify native tests still pass (`cargo nextest run --workspace`) +- [ ] Verify production WASM build still works (ask Chris to run `npm run build:all`) +- [ ] Update `dev-docs/wasm.md` to document the shared shim crate + +### Task 34: Update documentation for CI changes + +- [x] Update `dev-docs/wasm.md` — RUSTUP_TOOLCHAIN approach for sysroot conflict +- [ ] Update design spec — CI simplification and wasm-c-shim sections (this update) + ## Phase 6: Final Verification ### Task 25: Full workspace verification diff --git a/crates/pampa/CLAUDE.md b/crates/pampa/CLAUDE.md index 3a423f85a..6af5fb44b 100644 --- a/crates/pampa/CLAUDE.md +++ b/crates/pampa/CLAUDE.md @@ -180,4 +180,24 @@ cat input.qmd | cargo run -- -t json - When I say "@doit", I mean "create a plan, and work on it item by item." - When you're done editing a Rust file, run `cargo fmt` on it. - If I ask you to write notes to yourself, do it in markdown and write the output in the `claude-notes` directory. -- If you need more information on the syntax differences, you are allowed to read the [syntax notes](../../docs/syntax-notes.md) file. \ No newline at end of file +- If you need more information on the syntax differences, you are allowed to read the [syntax notes](../../docs/syntax-notes.md) file. + +## WASM Testing + +When modifying WASM-specific code paths (the `#[cfg(target_arch = "wasm32")]` blocks in +`filter.rs`/`shortcode.rs`, `io_wasm.rs`, or `os_wasm.rs`), add or update smoke tests in +`tests/wasm_lua.rs`. + +**Never add `test` to the `target_arch = "wasm32"` cfg guard.** Native tests must use +`Lua::new()` with the real C stdlib. WASM-specific setup is validated by the dedicated +WASM tests. + +WASM tests can't run on Windows. On Linux/macOS with LLVM: +``` +CC_wasm32_unknown_unknown=clang \ +CFLAGS_wasm32_unknown_unknown="-isystem $PWD/crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ +cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind,panic_abort +``` + +See `dev-docs/wasm.md` for full WASM architecture and testing details. \ No newline at end of file diff --git a/crates/pampa/Cargo.toml b/crates/pampa/Cargo.toml index 399da4578..3e28a324a 100644 --- a/crates/pampa/Cargo.toml +++ b/crates/pampa/Cargo.toml @@ -10,13 +10,19 @@ keywords.workspace = true license.workspace = true repository.workspace = true +# Bins require terminal-support so they are skipped when building WASM tests +# (which use --no-default-features --features lua-filter). Without this, cargo +# builds bins alongside integration tests (rust-lang/cargo#12980) and they fail +# to compile for wasm32 due to native-only dependencies (tokio, NativeRuntime). [[bin]] name = "pampa" path = "src/main.rs" +required-features = ["terminal-support"] [[bin]] name = "ast-reconcile" path = "src/bin/ast_reconcile.rs" +required-features = ["terminal-support"] [package.metadata] cargo-fuzz = true @@ -79,10 +85,19 @@ base64 = "0.22" tokio = { workspace = true } [dev-dependencies] +quarto-util.workspace = true + +# Native-only dev-dependencies (proptest pulls in getrandom which doesn't +# compile for wasm32-unknown-unknown; insta/tempfile/tokio are native-only too) +[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies] insta = { version = "1.46", features = ["json", "redactions"] } proptest = "1.10" tempfile = "3.24" tokio = { workspace = true } +[target.'cfg(target_arch = "wasm32")'.dev-dependencies] +mlua = { version = "0.11", features = ["lua54", "vendored", "serialize"] } +wasm-bindgen-test = "0.3" + [lints] workspace = true diff --git a/crates/pampa/tests/wasm_lua.rs b/crates/pampa/tests/wasm_lua.rs new file mode 100644 index 000000000..f29999854 --- /dev/null +++ b/crates/pampa/tests/wasm_lua.rs @@ -0,0 +1,277 @@ +//! WASM integration tests for Lua filter and shortcode infrastructure. +//! +//! These tests verify that the restricted Lua stdlib setup, synthetic io/os +//! modules, and filter/shortcode execution work correctly when compiled to +//! the real wasm32 target. +//! +//! **When to add tests here:** Only when modifying WASM-specific code paths: +//! - The #[cfg(target_arch = "wasm32")] blocks in filter.rs / shortcode.rs +//! - io_wasm.rs (synthetic io module) +//! - os_wasm.rs (synthetic os module) +//! +//! Native filter logic is tested comprehensively by the existing native tests. +//! These WASM tests are smoke tests of the target-specific setup. +//! +//! **How to run:** (Linux/macOS only, requires nightly + Clang + wasm-sysroot) +//! ```text +//! CC_wasm32_unknown_unknown=clang \ +//! CFLAGS_wasm32_unknown_unknown="-isystem $PWD/crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ +//! cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ +//! --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind,panic_abort +//! ``` + +#![cfg(all(target_arch = "wasm32", feature = "lua-filter"))] + +use wasm_bindgen_test::*; + +// ============================================================================ +// Test 1: Restricted Lua VM creation +// ============================================================================ + +/// Verify that a restricted Lua VM (matching the wasm32 stdlib set) can be +/// created and can evaluate basic expressions. +#[wasm_bindgen_test] +fn restricted_lua_vm_creation() { + use mlua::{Lua, StdLib}; + + let libs = StdLib::COROUTINE | StdLib::TABLE | StdLib::STRING | StdLib::UTF8 | StdLib::MATH; + let lua = + Lua::new_with(libs, mlua::LuaOptions::default()).expect("Failed to create restricted Lua"); + + let result: i64 = lua.load("return 1 + 1").eval().expect("eval failed"); + assert_eq!(result, 2); + + // Verify string library is available + let upper: String = lua + .load(r#"return string.upper("hello")"#) + .eval() + .expect("string.upper failed"); + assert_eq!(upper, "HELLO"); +} + +// ============================================================================ +// Test 2: Filter execution through WASM code path +// ============================================================================ + +/// Run a Lua filter through the real WASM code path: restricted VM + synthetic +/// io/os + filter execution. Uses WasmRuntime with a filter file in the VFS. +#[wasm_bindgen_test] +async fn filter_execution_wasm() { + use pampa::lua::apply_lua_filters; + use pampa::lua::runtime::{VirtualFileSystem, WasmRuntime}; + use pampa::pandoc::{ASTContext, Block, Inline, Pandoc, Paragraph, Str}; + use std::path::PathBuf; + use std::sync::Arc; + + // Set up VFS with a simple uppercase filter + let mut vfs = VirtualFileSystem::new(); + vfs.add_file( + std::path::Path::new("/project/uppercase.lua"), + br#" +function Str(elem) + return pandoc.Str(elem.text:upper()) +end +"# + .to_vec(), + ); + + let runtime: Arc = Arc::new(WasmRuntime::with_vfs(vfs)); + + // Build a minimal Pandoc document with one paragraph containing "hello" + let pandoc = Pandoc { + meta: quarto_pandoc_types::ConfigValue::default(), + blocks: vec![Block::Paragraph(Paragraph { + content: vec![Inline::Str(Str { + text: "hello".to_string(), + source_info: quarto_source_map::SourceInfo::default(), + })], + source_info: quarto_source_map::SourceInfo::default(), + })], + }; + let context = ASTContext::new(); + + let output = apply_lua_filters( + pandoc, + context, + &[PathBuf::from("/project/uppercase.lua")], + "html", + runtime, + ) + .await + .expect("filter execution failed"); + + assert!( + output.diagnostics.is_empty(), + "unexpected diagnostics: {:?}", + output.diagnostics + ); + + // Verify the filter uppercased the text + match &output.pandoc.blocks[0] { + Block::Paragraph(p) => match &p.content[0] { + Inline::Str(s) => assert_eq!(s.text, "HELLO"), + other => panic!("Expected Str, got {other:?}"), + }, + other => panic!("Expected Paragraph, got {other:?}"), + } +} + +// ============================================================================ +// Test 3: Shortcode engine initialization on WASM +// ============================================================================ + +/// Verify that LuaShortcodeEngine::new() succeeds on WASM (creates restricted +/// VM, registers synthetic io/os, sets up pandoc/quarto namespaces). +#[wasm_bindgen_test] +fn shortcode_engine_init_wasm() { + use pampa::lua::LuaShortcodeEngine; + use pampa::lua::runtime::{VirtualFileSystem, WasmRuntime}; + use std::sync::Arc; + + let runtime: Arc = + Arc::new(WasmRuntime::with_vfs(VirtualFileSystem::new())); + + let _engine = + LuaShortcodeEngine::new("html", runtime).expect("shortcode engine creation failed"); +} + +// ============================================================================ +// Test 4: Error handling (panic_unwind works) +// ============================================================================ + +/// Verify that Lua errors produce Err results rather than WASM traps. +/// This validates that -Zbuild-std=std,panic_unwind,panic_abort is working correctly. +#[wasm_bindgen_test] +fn lua_error_handling() { + use mlua::{Lua, StdLib}; + + let libs = StdLib::COROUTINE | StdLib::TABLE | StdLib::STRING | StdLib::UTF8 | StdLib::MATH; + let lua = + Lua::new_with(libs, mlua::LuaOptions::default()).expect("Failed to create restricted Lua"); + + let result = lua.load("error('test error')").exec(); + assert!(result.is_err(), "expected Lua error, got Ok"); + + let err_msg = result.unwrap_err().to_string(); + assert!( + err_msg.contains("test error"), + "error message should contain 'test error', got: {err_msg}" + ); +} + +// ============================================================================ +// Test 5: Synthetic io module is registered in filter execution +// ============================================================================ + +/// Verify that the synthetic io module (io.open, io.type) is registered when +/// filters execute on wasm32. Uses a filter that asserts these globals exist. +#[wasm_bindgen_test] +async fn synthetic_io_available_in_filters() { + use pampa::lua::apply_lua_filters; + use pampa::lua::runtime::{VirtualFileSystem, WasmRuntime}; + use pampa::pandoc::{ASTContext, Block, Inline, Pandoc, Paragraph, Str}; + use std::path::PathBuf; + use std::sync::Arc; + + let mut vfs = VirtualFileSystem::new(); + vfs.add_file( + std::path::Path::new("/project/check_io.lua"), + br#" +function Pandoc(doc) + assert(type(io) == "table", "io should be a table") + assert(type(io.open) == "function", "io.open should be a function") + assert(type(io.type) == "function", "io.type should be a function") + return doc +end +"# + .to_vec(), + ); + + let runtime: Arc = Arc::new(WasmRuntime::with_vfs(vfs)); + + let pandoc = Pandoc { + meta: quarto_pandoc_types::ConfigValue::default(), + blocks: vec![Block::Paragraph(Paragraph { + content: vec![Inline::Str(Str { + text: "test".to_string(), + source_info: quarto_source_map::SourceInfo::default(), + })], + source_info: quarto_source_map::SourceInfo::default(), + })], + }; + + let output = apply_lua_filters( + pandoc, + ASTContext::new(), + &[PathBuf::from("/project/check_io.lua")], + "html", + runtime, + ) + .await + .expect("filter with io checks failed — synthetic io may not be registered"); + + assert!( + output.diagnostics.is_empty(), + "unexpected diagnostics: {:?}", + output.diagnostics + ); +} + +// ============================================================================ +// Test 6: Synthetic os module is registered in filter execution +// ============================================================================ + +/// Verify that the synthetic os module (os.time, os.clock, os.difftime) is +/// registered when filters execute on wasm32. +#[wasm_bindgen_test] +async fn synthetic_os_available_in_filters() { + use pampa::lua::apply_lua_filters; + use pampa::lua::runtime::{VirtualFileSystem, WasmRuntime}; + use pampa::pandoc::{ASTContext, Block, Inline, Pandoc, Paragraph, Str}; + use std::path::PathBuf; + use std::sync::Arc; + + let mut vfs = VirtualFileSystem::new(); + vfs.add_file( + std::path::Path::new("/project/check_os.lua"), + br#" +function Pandoc(doc) + assert(type(os) == "table", "os should be a table") + assert(type(os.time) == "function", "os.time should be a function") + assert(type(os.clock) == "function", "os.clock should be a function") + assert(type(os.difftime) == "function", "os.difftime should be a function") + return doc +end +"# + .to_vec(), + ); + + let runtime: Arc = Arc::new(WasmRuntime::with_vfs(vfs)); + + let pandoc = Pandoc { + meta: quarto_pandoc_types::ConfigValue::default(), + blocks: vec![Block::Paragraph(Paragraph { + content: vec![Inline::Str(Str { + text: "test".to_string(), + source_info: quarto_source_map::SourceInfo::default(), + })], + source_info: quarto_source_map::SourceInfo::default(), + })], + }; + + let output = apply_lua_filters( + pandoc, + ASTContext::new(), + &[PathBuf::from("/project/check_os.lua")], + "html", + runtime, + ) + .await + .expect("filter with os checks failed — synthetic os may not be registered"); + + assert!( + output.diagnostics.is_empty(), + "unexpected diagnostics: {:?}", + output.diagnostics + ); +} diff --git a/dev-docs/wasm.md b/dev-docs/wasm.md index 929a8c69c..4fcb23652 100644 --- a/dev-docs/wasm.md +++ b/dev-docs/wasm.md @@ -1,46 +1,98 @@ -# WASM Architecture +# WASM in the Quarto Rust Monorepo -## Overview +## Architecture -The `wasm-quarto-hub-client` crate builds the Quarto rendering engine (pampa + quarto-core) -as a WASM module for use in the hub-client web application. It targets -`wasm32-unknown-unknown` and uses `-Zbuild-std=std,panic_unwind` to rebuild the standard -library (required for Lua error handling via setjmp/longjmp → panic/catch_unwind). +`wasm-quarto-hub-client` wraps `pampa` + `quarto-core` for the hub-client web app. +It compiles to a WASM module that runs in the browser, providing live preview rendering. + +The crate is **excluded from the default workspace** (`Cargo.toml` `exclude` list) because +it requires `--target wasm32-unknown-unknown` and `-Zbuild-std=std,panic_unwind`. ## Build -The WASM module is built via `hub-client/scripts/build-wasm.js`, which runs: -1. `cargo build --target wasm32-unknown-unknown -Zbuild-std=std,panic_unwind` -2. `wasm-bindgen` CLI to generate JS glue code +The production WASM build is handled by `hub-client/scripts/build-wasm.js`: -From hub-client: ```bash -npm run build:all # Full build including WASM +cd hub-client +npm run build:wasm # WASM module only +npm run build:all # WASM + TypeScript ``` -This project does **not** use wasm-pack (deprecated, rustwasm sunset Sep 2025). -The `wasm-bindgen-cli` version is pinned to match `Cargo.lock` and installed via -`cargo xtask dev-setup`. +The build script runs: +1. `cargo build -p wasm-quarto-hub-client --target wasm32-unknown-unknown` + with `-Zbuild-std=std,panic_unwind` (via `crates/wasm-quarto-hub-client/.cargo/config.toml`) +2. `wasm-bindgen` CLI to generate JS/TS bindings + +### Why not wasm-pack? -## C Toolchain +This project uses `cargo build` + `wasm-bindgen` CLI directly because: +- `-Zbuild-std=std,panic_unwind` is required for Lua error handling (setjmp/longjmp to + panic/catch_unwind). wasm-pack doesn't support `-Zbuild-std`. +- wasm-pack is deprecated (rustwasm org sunset September 2025). -Building for `wasm32-unknown-unknown` requires Clang with wasm32 support. The `cc` crate -invokes Clang to compile C dependencies (tree-sitter, Lua). Environment variables: +### C toolchain requirement + +`pampa` with `lua-filter` pulls in `mlua` → `lua-src-wasm`, which compiles Lua from C source +via the `cc` crate. When targeting wasm32, this requires Clang with wasm32 support: ```bash -CC_wasm32_unknown_unknown=clang -CFLAGS_wasm32_unknown_unknown="-isystem /wasm-sysroot -fno-builtin" +export CC_wasm32_unknown_unknown=clang +export CFLAGS_wasm32_unknown_unknown="-isystem $PWD/crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" ``` -The wasm-sysroot at `crates/wasm-quarto-hub-client/wasm-sysroot/` provides minimal C -headers. The `-fno-builtin` flag is needed because debug-mode builds emit `__builtin_*` -intrinsic calls not present in the stub sysroot. +Production builds (`build-wasm.js`) only set `-isystem `. WASM tests +additionally need `-fno-builtin` because they compile in debug mode, where Clang emits +`__builtin_*` intrinsic calls (e.g. `memcpy`, `memset`) that don't exist in the stub +sysroot. Release builds inline or eliminate these calls, so the flag isn't needed there. + +## Testing + +### Native tests (all platforms) + +Native Rust tests (`cargo nextest run`) test filter and shortcode logic using `Lua::new()` +with the full C stdlib. These run on all platforms including Windows. + +### WASM smoke tests (Linux CI) + +`crates/pampa/tests/wasm_lua.rs` contains smoke tests that compile and run on the real +`wasm32-unknown-unknown` target. They verify the WASM-specific Lua VM setup: +- Restricted stdlib creation (`Lua::new_with()`) +- Synthetic `io`/`os` module registration +- Filter execution through the WASM code path +- Shortcode engine initialization on WASM +- Error handling (`panic_unwind` works correctly) + +Run locally (Linux/macOS with LLVM): +```bash +CC_wasm32_unknown_unknown=clang \ +CFLAGS_wasm32_unknown_unknown="-isystem $PWD/crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ +cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind,panic_abort +``` + +**Important notes:** + +- You must use `--test wasm_lua` to select only the WASM test file. + Running `cargo test -p pampa --target wasm32` without `--test` will fail because + native tests can't compile for wasm32. +- The prebuilt `wasm32-unknown-unknown` target (installed by `rust-toolchain.toml`) + conflicts with `-Zbuild-std` when building within the workspace — both produce a + `core` crate, causing E0152 (duplicate lang item). The production build avoids this + because `wasm-quarto-hub-client` is excluded from the workspace. The CI job sets + `RUSTUP_TOOLCHAIN=nightly` to bypass `rust-toolchain.toml`, so the prebuilt target + is never installed. Locally, you can either set `RUSTUP_TOOLCHAIN=nightly` or + remove the target before testing (`rustup target remove wasm32-unknown-unknown`) + and re-add it afterward for the production build. +- The pampa `[[bin]]` targets (`pampa`, `ast-reconcile`) use `required-features` to + prevent compilation when running WASM tests. Cargo builds bin targets alongside + integration tests by default (rust-lang/cargo#12980); the `required-features` gate + ensures they are skipped when `--no-default-features --features lua-filter` is used. -## Native vs WASM Testing +WASM tests are **not** part of `cargo xtask verify` — they require nightly + Clang with +wasm32 support, which is Linux/macOS only. They run in the `wasm-tests` CI job. -Native tests (`cargo nextest run`) use `Lua::new()` with the full C stdlib on all platforms. -WASM-specific code paths use `#[cfg(target_arch = "wasm32")]` guards — never -`#[cfg(any(target_arch = "wasm32", test))]` (see `.claude/rules/wasm.md`). +### Hub-client integration tests -Hub-client integration tests (`npm run test:ci`) exercise the compiled WASM module through -the JavaScript API. +The hub-client test suite (`npm run test:ci`) tests the compiled WASM module through +JavaScript, covering rendering, templates, and format detection. These complement +the Rust-level WASM smoke tests. From 78d717d315ab0982c3e489eed606adf9b3f877ce Mon Sep 17 00:00:00 2001 From: Christophe Dervieux Date: Mon, 13 Apr 2026 20:34:13 +0200 Subject: [PATCH 3/5] Extract C stdlib shims into shared wasm-c-shim crate Move the ~980-line c_shim.rs from wasm-quarto-hub-client into a new wasm-c-shim crate. This provides #[no_mangle] C stdlib stubs (malloc, fprintf, snprintf, abort, etc.) needed by tree-sitter and Lua when compiled for wasm32-unknown-unknown, which has no libc. The crate is a no-op on native targets (all exports gated on wasm32). Both wasm-quarto-hub-client (production) and pampa WASM tests now depend on wasm-c-shim for the shared symbols. --- crates/pampa/Cargo.toml | 1 + crates/pampa/tests/wasm_lua.rs | 4 +++ crates/wasm-c-shim/Cargo.toml | 13 ++++++++++ crates/wasm-c-shim/src/lib.rs | 26 +++++++++++++++++++ .../src/c_shim.rs => wasm-c-shim/src/shim.rs} | 0 crates/wasm-quarto-hub-client/Cargo.toml | 1 + crates/wasm-quarto-hub-client/src/lib.rs | 10 +++---- dev-docs/wasm.md | 14 ++++++++++ 8 files changed, 63 insertions(+), 6 deletions(-) create mode 100644 crates/wasm-c-shim/Cargo.toml create mode 100644 crates/wasm-c-shim/src/lib.rs rename crates/{wasm-quarto-hub-client/src/c_shim.rs => wasm-c-shim/src/shim.rs} (100%) diff --git a/crates/pampa/Cargo.toml b/crates/pampa/Cargo.toml index 3e28a324a..203ce69ab 100644 --- a/crates/pampa/Cargo.toml +++ b/crates/pampa/Cargo.toml @@ -98,6 +98,7 @@ tokio = { workspace = true } [target.'cfg(target_arch = "wasm32")'.dev-dependencies] mlua = { version = "0.11", features = ["lua54", "vendored", "serialize"] } wasm-bindgen-test = "0.3" +wasm-c-shim = { path = "../wasm-c-shim" } [lints] workspace = true diff --git a/crates/pampa/tests/wasm_lua.rs b/crates/pampa/tests/wasm_lua.rs index f29999854..40e2ae41b 100644 --- a/crates/pampa/tests/wasm_lua.rs +++ b/crates/pampa/tests/wasm_lua.rs @@ -22,6 +22,10 @@ #![cfg(all(target_arch = "wasm32", feature = "lua-filter"))] +// Link the C stdlib shims (malloc, fprintf, snprintf, etc.) needed by +// tree-sitter and Lua on wasm32-unknown-unknown. +extern crate wasm_c_shim; + use wasm_bindgen_test::*; // ============================================================================ diff --git a/crates/wasm-c-shim/Cargo.toml b/crates/wasm-c-shim/Cargo.toml new file mode 100644 index 000000000..7098086f2 --- /dev/null +++ b/crates/wasm-c-shim/Cargo.toml @@ -0,0 +1,13 @@ +[package] +name = "wasm-c-shim" +publish = false +authors.workspace = true +# Edition 2021: this crate is mostly unsafe extern "C" FFI shims where nearly +# every line dereferences raw pointers. Edition 2024 requires explicit unsafe {} +# blocks inside unsafe fn, which would add noise without safety benefit here. +edition = "2021" +license.workspace = true +description = "C stdlib shims for wasm32-unknown-unknown (no libc)" + +[lints] +workspace = true diff --git a/crates/wasm-c-shim/src/lib.rs b/crates/wasm-c-shim/src/lib.rs new file mode 100644 index 000000000..360c38c0f --- /dev/null +++ b/crates/wasm-c-shim/src/lib.rs @@ -0,0 +1,26 @@ +//! C stdlib shims for `wasm32-unknown-unknown`. +//! +//! On `wasm32-unknown-unknown` there is no libc. C libraries compiled for this +//! target (tree-sitter, Lua via lua-src) reference symbols like `malloc`, +//! `fprintf`, `snprintf`, etc. that must be provided by Rust `#[no_mangle]` +//! functions. +//! +//! This crate is a no-op on native targets. On wasm32, it exports the full set +//! of C stdlib shims needed by the project's C dependencies. +//! +//! Used by: +//! - `wasm-quarto-hub-client` (production WASM build) +//! - `pampa` dev-dependencies (WASM integration tests) +//! +//! # Edition note +//! +//! This crate uses edition 2021 (not the workspace default of 2024). Edition +//! 2024 requires explicit `unsafe {}` blocks inside `unsafe fn` bodies. Since +//! nearly every line in the shims dereferences raw pointers, this would add +//! `unsafe {}` wrappers to ~65 call sites with no safety benefit — the functions +//! are all `unsafe extern "C"` FFI entry points. + +#![cfg_attr(target_arch = "wasm32", feature(c_variadic))] + +#[cfg(target_arch = "wasm32")] +mod shim; diff --git a/crates/wasm-quarto-hub-client/src/c_shim.rs b/crates/wasm-c-shim/src/shim.rs similarity index 100% rename from crates/wasm-quarto-hub-client/src/c_shim.rs rename to crates/wasm-c-shim/src/shim.rs diff --git a/crates/wasm-quarto-hub-client/Cargo.toml b/crates/wasm-quarto-hub-client/Cargo.toml index 1c072bd13..39c57dec8 100644 --- a/crates/wasm-quarto-hub-client/Cargo.toml +++ b/crates/wasm-quarto-hub-client/Cargo.toml @@ -11,6 +11,7 @@ crate-type = ["cdylib"] [dependencies] pampa = { path = "../pampa", default-features = false, features = ["lua-filter"] } +wasm-c-shim = { path = "../wasm-c-shim" } quarto-ast-reconcile = { path = "../quarto-ast-reconcile" } quarto-core = { path = "../quarto-core" } # NOTE: `quarto-error-catalog` is intentionally NOT a dependency here — the WASM diff --git a/crates/wasm-quarto-hub-client/src/lib.rs b/crates/wasm-quarto-hub-client/src/lib.rs index 4a76ca22c..5a0068d80 100644 --- a/crates/wasm-quarto-hub-client/src/lib.rs +++ b/crates/wasm-quarto-hub-client/src/lib.rs @@ -6,13 +6,11 @@ * Provides VFS management and document rendering capabilities. */ -// For `vsnprintf()` and `fprintf()`, which are variadic. -#![feature(c_variadic)] - -// Provide rust implementation of blessed stdlib functions to -// tree-sitter itself and any grammars that have `scanner.c`. +// C stdlib shims for wasm32 (malloc, fprintf, snprintf, etc.) are provided +// by the wasm-c-shim crate. The extern crate ensures it's linked even though +// no Rust code references it — the symbols are consumed by C code at link time. #[cfg(target_arch = "wasm32")] -pub mod c_shim; +extern crate wasm_c_shim; /// Sentinel panic payload raised by `c_shim::rust_lua_throw`. /// diff --git a/dev-docs/wasm.md b/dev-docs/wasm.md index 4fcb23652..1a31091f2 100644 --- a/dev-docs/wasm.md +++ b/dev-docs/wasm.md @@ -45,6 +45,20 @@ additionally need `-fno-builtin` because they compile in debug mode, where Clang `__builtin_*` intrinsic calls (e.g. `memcpy`, `memset`) that don't exist in the stub sysroot. Release builds inline or eliminate these calls, so the flag isn't needed there. +### C stdlib shims (`wasm-c-shim`) + +`wasm32-unknown-unknown` has no libc. C libraries (tree-sitter, Lua) that reference +standard symbols (`malloc`, `fprintf`, `snprintf`, `abort`, etc.) need Rust-provided +`#[no_mangle]` shim functions at link time. + +These shims live in `crates/wasm-c-shim/`, a workspace member that is a no-op on native +targets (all exports gated on `cfg(target_arch = "wasm32")`). Both `wasm-quarto-hub-client` +(production) and `pampa` WASM tests (dev-dependency) link against it. + +**Edition note:** `wasm-c-shim` uses edition 2021, not the workspace default of 2024. +Edition 2024 requires explicit `unsafe {}` blocks inside `unsafe fn`, which would add +noise to ~65 FFI shim functions with no safety benefit. + ## Testing ### Native tests (all platforms) From e00fc7907e5482354ad2f32eff06da745167b22a Mon Sep 17 00:00:00 2001 From: Gordon Woodhull Date: Fri, 17 Apr 2026 09:32:21 -0400 Subject: [PATCH 4/5] Unblock pampa wasm tests: feature-gate JS bridge, set wasm32 panic strategy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The raw_module extern blocks in quarto-system-runtime/src/wasm.rs import absolute paths (/src/wasm-js-bridge/*.js) that hub-client serves through Vite at runtime. wasm-bindgen generates unconditional require() calls for these paths in the shim it produces, so any wasm32 binary that links quarto-system-runtime — including the pampa wasm_lua tests — fails to load under Node.js with MODULE_NOT_FOUND before any test body runs. Add a js-bridge Cargo feature, default off. Gate the extern blocks behind it, and provide stubs that return Err("js-bridge feature not enabled") / false when off so the SystemRuntime impl still compiles. wasm-quarto-hub-client opts in via features = ["js-bridge"]; pampa's wasm test build does not, so the require()s disappear from the shim. Native builds are unaffected (wasm.rs is #![cfg(target_arch = "wasm32")]). That exposed a downstream panic-strategy problem: rust_lua_throw panics propagated as wasm RuntimeError instead of being caught by rust_lua_protected_call. Fix by mirroring wasm-quarto-hub-client's rustflags (-C panic=unwind, +bulk-memory,+exception-handling, -Zwasm-c-abi=spec) in the workspace-root .cargo/config.toml for wasm32 builds, and moving the LuaThrow marker type into wasm-c-shim so the production build and the tests share it. Update design/plan docs and dev-docs/wasm.md accordingly. --- .cargo/config.toml | 11 +++ .../2026-04-03-wasm-testing-and-cleanup.md | 85 +++++++++++++++++++ .../2026-04-07-wasm-testing-and-cleanup.md | 71 ++++++++++++++++ crates/quarto-system-runtime/Cargo.toml | 11 +++ crates/quarto-system-runtime/src/wasm.rs | 54 ++++++++++++ crates/wasm-c-shim/src/lib.rs | 9 ++ crates/wasm-quarto-hub-client/Cargo.toml | 2 +- crates/wasm-quarto-hub-client/src/lib.rs | 10 +-- dev-docs/wasm.md | 63 +++++++++++++- 9 files changed, 305 insertions(+), 11 deletions(-) diff --git a/.cargo/config.toml b/.cargo/config.toml index a90c7dab0..3a11af89d 100644 --- a/.cargo/config.toml +++ b/.cargo/config.toml @@ -9,3 +9,14 @@ create-worktree = "xtask create-worktree" [target.wasm32-unknown-unknown] runner = "wasm-bindgen-test-runner" +# Mirror the rustflags used by wasm-quarto-hub-client/.cargo/config.toml so any +# wasm32 build invoked from the workspace root (e.g. the pampa wasm_lua test) +# gets the same panic strategy and ABI. Without panic=unwind plus the +# exception-handling target feature, wasm-c-shim's catch_unwind-based +# replacement for Lua's setjmp/longjmp can't catch panics, and any Lua throw +# during mlua initialization aborts the wasm module. +rustflags = [ + "-C", "target-feature=+bulk-memory,+exception-handling", + "-C", "panic=unwind", + "-Zwasm-c-abi=spec", +] diff --git a/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md b/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md index 776e43e00..313c15114 100644 --- a/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md +++ b/claude-notes/designs/2026-04-03-wasm-testing-and-cleanup.md @@ -413,6 +413,91 @@ extern crate wasm_c_shim; registers synthetic io/os, but only integration tests verify they actually work when called from Lua during filter execution on real wasm32. +## JS bridge isolation and panic strategy (2026-04-17) + +After Phase 4 landed, the `wasm-tests` CI job failed at the +`wasm-bindgen-test-runner` step (Node.js execution) before any test body +ran, with `MODULE_NOT_FOUND` for `/src/wasm-js-bridge/cache.js`. Two +distinct issues were uncovered, both originating in production code that +had never previously been exercised on the wasm32 test path. + +### Finding 1: `raw_module` extern blocks load unconditionally + +`crates/quarto-system-runtime/src/wasm.rs` declares four +`#[wasm_bindgen(raw_module = "/src/wasm-js-bridge/{template,sass,cache,fetch}.js")]` +extern blocks. Hub-client serves these JS files at runtime through Vite, +and `wasm-bindgen` generates `require()` calls for each one in the JS +shim it produces. The `require()` happens at module-load time regardless +of whether the test ever calls into JavaScript, so any wasm32 binary +that links `quarto-system-runtime` cannot load under Node.js — the +absolute paths don't resolve on disk. + +The pampa wasm tests pull in `quarto-system-runtime` transitively +(`WasmRuntime`, `LuaShortcodeEngine`), so the failure surfaced as soon +as Phase 3 tests reached the runner step. Production never tripped this +because the only wasm32 consumer (`wasm-quarto-hub-client`) runs in a +browser where Vite resolves the paths. + +**Fix:** add a `js-bridge` Cargo feature to `quarto-system-runtime`, +default off. Gate the four extern blocks behind the feature, and provide +stub modules that return `Err(JsValue::from_str("js-bridge feature not enabled"))` +or `false` when off so the `SystemRuntime` impl still compiles. +`wasm-quarto-hub-client/Cargo.toml` opts in via +`features = ["js-bridge"]`. Pampa's wasm test build does not, so the +`require()` calls disappear from the generated shim. + +### Finding 2: workspace wasm32 builds inherit `panic = "abort"` + +With Finding 1 resolved, the test runner loaded the module successfully +and reached the test bodies — at which point all six tests failed in +`wasm-c-shim::rust_lua_throw` with the wasm error `RuntimeError: unreachable`. + +The `wasm-c-shim` crate replaces Lua's `setjmp`/`longjmp` with +`panic!()`/`catch_unwind` (since wasm32 has no native unwinding). For +this substitution to work, the binary's panic strategy must be `unwind`, +not the wasm32 default of `abort`. Under `panic=abort`, `panic!()` +lowers directly to the wasm `unreachable` instruction and `catch_unwind` +becomes a compile-time no-op that always returns `Ok` — so the first +Lua throw during mlua's protected init aborts the module. + +The CI command's `-Zbuild-std=std,panic_unwind,panic_abort` only ensures +the unwind runtime is *available* in std; it does not change the +binary's panic strategy. Three additional flags are required: +`-C panic=unwind`, `-C target-feature=+exception-handling`, and +`-Zwasm-c-abi=spec`. `wasm-quarto-hub-client/.cargo/config.toml` already +sets all three, but that config lives in an isolated workspace and never +reaches builds invoked from the workspace root. Production never tripped +this because hub-client builds always use the local config. + +**Fix:** mirror the rustflags into the workspace-root `.cargo/config.toml` +under `[target.wasm32-unknown-unknown]`. `[unstable] build-std` is +deliberately *not* added to the workspace config — it is not target-scoped, +so adding it would force `build-std` for every native invocation from the +root. The `-Zbuild-std` flag stays on the test command (and in CI). + +### Finding 3: `LuaThrow` marker placement (rebase artifact) + +While this branch was open, main landed `Suppress noisy 'lua error' panic +stack traces in WASM console` (commit `675c22d2`), which introduced a +`LuaThrow` marker struct in `wasm-quarto-hub-client/src/lib.rs` and +changed `rust_lua_throw` from `panic!("lua error")` to +`std::panic::panic_any(crate::LuaThrow)`. Hub-client's panic hook +downcasts to `LuaThrow` to filter expected control-flow panics out of +console.error. + +When this branch's "Extract C stdlib shims into shared wasm-c-shim crate" +commit was rebased onto that change, `crate::LuaThrow` no longer +resolved — the shim moved out of `wasm-quarto-hub-client`, so `crate::` +points elsewhere. + +**Fix:** the marker belongs in `wasm-c-shim` (where the panic +originates), not in the hub-client (where the hook lives). Moved the +`pub struct LuaThrow;` definition into `crates/wasm-c-shim/src/lib.rs` +and have `wasm-quarto-hub-client/src/lib.rs` import it via +`use wasm_c_shim::LuaThrow;`. Behavior is preserved; the dependency +direction is now consistent (hub-client depends on the shim, not +vice-versa). + ## Out of scope - Migrating wasm-pack usage (no longer needed — only stale crate used it) diff --git a/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md b/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md index 3b4bc7cc0..164c58965 100644 --- a/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md +++ b/claude-notes/plans/2026-04-07-wasm-testing-and-cleanup.md @@ -1144,3 +1144,74 @@ br sync --flush-only git add .beads/ git commit -m "Sync beads: close bd-itj9 WASM testing and cleanup" ``` + +--- + +## Phase 7: Post-CI follow-up findings (2026-04-17) + +After Phase 4's CI job started executing, two production-side issues +surfaced in the test path that did not affect the production wasm32 +build. Both fixes landed on this branch; design rationale in the design +spec section "JS bridge isolation and panic strategy (2026-04-17)". + +### Task 27: Feature-gate JS bridge in `quarto-system-runtime` + +- [x] Add `js-bridge` Cargo feature (default off) to + `crates/quarto-system-runtime/Cargo.toml`. +- [x] Gate the four `#[wasm_bindgen(raw_module = ...)]` extern blocks in + `crates/quarto-system-runtime/src/wasm.rs` behind + `#[cfg(feature = "js-bridge")]`. +- [x] Provide stub modules under `#[cfg(not(feature = "js-bridge"))]` + that return `Err(JsValue::from_str("js-bridge feature not enabled"))` + or `false`, matching the original signatures so the `SystemRuntime` + impl still compiles. +- [x] Opt in from `crates/wasm-quarto-hub-client/Cargo.toml`: + `quarto-system-runtime = { ..., features = ["js-bridge"] }`. + +Without this gate, `wasm-bindgen-test-runner` (Node.js) fails to load +the module with `MODULE_NOT_FOUND` for `/src/wasm-js-bridge/cache.js`, +since the absolute paths only resolve under Vite. + +### Task 28: Set `panic=unwind` for workspace wasm32 builds + +- [x] Add to `[target.wasm32-unknown-unknown]` in workspace + `.cargo/config.toml`: + ```toml + rustflags = [ + "-C", "target-feature=+bulk-memory,+exception-handling", + "-C", "panic=unwind", + "-Zwasm-c-abi=spec", + ] + ``` +- [x] Do **not** add `[unstable] build-std` to the workspace config — + the `[unstable]` table is not target-scoped, so it would force + build-std on every native invocation. `-Zbuild-std` stays on the test + command and in CI. + +Without these flags the binary inherits the wasm32 default of +`panic=abort`, which makes `wasm-c-shim::rust_lua_protected_call`'s +`catch_unwind` a no-op. The first Lua throw during mlua initialization +then aborts the wasm module rather than being caught. + +### Task 29: Move `LuaThrow` into `wasm-c-shim` (rebase resolution) + +- [x] Move `pub struct LuaThrow;` from + `crates/wasm-quarto-hub-client/src/lib.rs` into + `crates/wasm-c-shim/src/lib.rs` (where the panic actually originates). +- [x] Update `crates/wasm-quarto-hub-client/src/lib.rs` to + `use wasm_c_shim::LuaThrow;` for its panic-suppression hook. + +The marker struct was introduced on main while this branch was open +(`Suppress noisy 'lua error' panic stack traces in WASM console`). After +rebase, `crate::LuaThrow` in the extracted shim no longer resolved. + +### Task 30: Local test verification + +- [x] All 6 `pampa wasm_lua` tests pass on `wasm32-unknown-unknown` + via `wasm-bindgen-test-runner` / Node.js. +- [x] `cargo check --workspace` clean (native). +- [x] `cargo check --target wasm32-unknown-unknown` clean for + `wasm-quarto-hub-client`. + +Note: `cargo xtask verify` was not re-run locally — these changes +should be exercised by the existing `wasm-tests` CI job once pushed. diff --git a/crates/quarto-system-runtime/Cargo.toml b/crates/quarto-system-runtime/Cargo.toml index 907794f30..210f1a670 100644 --- a/crates/quarto-system-runtime/Cargo.toml +++ b/crates/quarto-system-runtime/Cargo.toml @@ -7,6 +7,17 @@ license.workspace = true repository.workspace = true description = "Runtime abstraction layer for Quarto system operations" +[features] +# Enables wasm-bindgen `raw_module` imports of the JS bridge files at +# /src/wasm-js-bridge/{template,sass,cache,fetch}.js. +# +# Hub-client provides those JS modules at runtime via Vite. Off-by-default +# because non-hub-client wasm32 binaries (e.g. wasm-bindgen-test runs of +# pampa) have no bridge to require — Node.js fails to resolve the absolute +# paths at module load. With the feature off, the impls return errors. +default = [] +js-bridge = [] + [dependencies] # Async methods in traits async-trait.workspace = true diff --git a/crates/quarto-system-runtime/src/wasm.rs b/crates/quarto-system-runtime/src/wasm.rs index cc75cd894..0a73ee1fd 100644 --- a/crates/quarto-system-runtime/src/wasm.rs +++ b/crates/quarto-system-runtime/src/wasm.rs @@ -39,6 +39,7 @@ use crate::vfs::{VirtualFileSystem, not_found_error}; // The functions are expected to be provided via a module at the path specified. // In hub-client, this is at: /src/wasm-js-bridge/sass.js +#[cfg(feature = "js-bridge")] #[wasm_bindgen(raw_module = "/src/wasm-js-bridge/sass.js")] extern "C" { /// Check if SASS compilation is available. @@ -69,6 +70,27 @@ extern "C" { ) -> Result; } +#[cfg(not(feature = "js-bridge"))] +mod sass_stubs { + use wasm_bindgen::prelude::*; + pub(super) fn js_sass_available_impl() -> bool { + false + } + #[allow(dead_code)] + pub(super) fn js_sass_compiler_name_impl() -> String { + "unavailable (js-bridge feature not enabled)".to_string() + } + pub(super) fn js_compile_sass_impl( + _scss: &str, + _style: &str, + _load_paths_json: &str, + ) -> Result { + Err(JsValue::from_str("js-bridge feature not enabled")) + } +} +#[cfg(not(feature = "js-bridge"))] +use sass_stubs::{js_compile_sass_impl, js_sass_available_impl, js_sass_compiler_name_impl}; + // ============================================================================= // JavaScript Interop for Cache Operations // ============================================================================= @@ -79,6 +101,7 @@ extern "C" { // The functions are expected to be provided via a module at the path specified. // In hub-client, this is at: /src/wasm-js-bridge/cache.js +#[cfg(feature = "js-bridge")] #[wasm_bindgen(raw_module = "/src/wasm-js-bridge/cache.js")] extern "C" { /// Get a cached value by namespace and key. @@ -110,6 +133,31 @@ extern "C" { fn js_cache_clear_namespace_impl(namespace: &str) -> Result; } +#[cfg(not(feature = "js-bridge"))] +mod cache_stubs { + use wasm_bindgen::prelude::*; + pub(super) fn js_cache_get_impl(_namespace: &str, _key: &str) -> Result { + Err(JsValue::from_str("js-bridge feature not enabled")) + } + pub(super) fn js_cache_set_impl( + _namespace: &str, + _key: &str, + _value: &js_sys::Uint8Array, + ) -> Result { + Err(JsValue::from_str("js-bridge feature not enabled")) + } + pub(super) fn js_cache_delete_impl(_namespace: &str, _key: &str) -> Result { + Err(JsValue::from_str("js-bridge feature not enabled")) + } + pub(super) fn js_cache_clear_namespace_impl(_namespace: &str) -> Result { + Err(JsValue::from_str("js-bridge feature not enabled")) + } +} +#[cfg(not(feature = "js-bridge"))] +use cache_stubs::{ + js_cache_clear_namespace_impl, js_cache_delete_impl, js_cache_get_impl, js_cache_set_impl, +}; + // ============================================================================= // JavaScript Interop for Network Fetch // ============================================================================= @@ -121,6 +169,7 @@ extern "C" { // Binary content is base64-encoded by the JS side to avoid complex type // marshalling. The Rust side decodes it with the base64 crate. +#[cfg(feature = "js-bridge")] #[wasm_bindgen(raw_module = "/src/wasm-js-bridge/fetch.js")] extern "C" { /// Fetch content from a URL. @@ -133,6 +182,11 @@ extern "C" { fn js_fetch_url_impl(url: &str) -> Result; } +#[cfg(not(feature = "js-bridge"))] +fn js_fetch_url_impl(_url: &str) -> Result { + Err(JsValue::from_str("js-bridge feature not enabled")) +} + // ============================================================================= // Monotonic clock: performance.now() // ============================================================================= diff --git a/crates/wasm-c-shim/src/lib.rs b/crates/wasm-c-shim/src/lib.rs index 360c38c0f..1692a8d03 100644 --- a/crates/wasm-c-shim/src/lib.rs +++ b/crates/wasm-c-shim/src/lib.rs @@ -24,3 +24,12 @@ #[cfg(target_arch = "wasm32")] mod shim; + +/// Marker payload for panics raised by `rust_lua_throw` (the WASM replacement +/// for Lua's `LUAI_THROW`). Each Lua error inside `lua_pcall` produces one +/// such panic, which `rust_lua_protected_call` catches microseconds later. +/// +/// Hosts that install a custom panic hook (e.g. wasm-quarto-hub-client) can +/// downcast to this type to filter expected Lua control-flow panics out of +/// console.error logs without suppressing real Rust panics. +pub struct LuaThrow; diff --git a/crates/wasm-quarto-hub-client/Cargo.toml b/crates/wasm-quarto-hub-client/Cargo.toml index 39c57dec8..d1fda9ee5 100644 --- a/crates/wasm-quarto-hub-client/Cargo.toml +++ b/crates/wasm-quarto-hub-client/Cargo.toml @@ -27,7 +27,7 @@ quarto-lsp-core = { path = "../quarto-lsp-core" } quarto-pandoc-types = { path = "../quarto-pandoc-types" } quarto-sass = { path = "../quarto-sass" } quarto-source-map = "0.1.0" -quarto-system-runtime = { path = "../quarto-system-runtime" } +quarto-system-runtime = { path = "../quarto-system-runtime", features = ["js-bridge"] } quarto-project-create = { path = "../quarto-project-create" } quarto-trace = { path = "../quarto-trace" } flate2 = "1.1" diff --git a/crates/wasm-quarto-hub-client/src/lib.rs b/crates/wasm-quarto-hub-client/src/lib.rs index 5a0068d80..b60e8e16c 100644 --- a/crates/wasm-quarto-hub-client/src/lib.rs +++ b/crates/wasm-quarto-hub-client/src/lib.rs @@ -12,15 +12,7 @@ #[cfg(target_arch = "wasm32")] extern crate wasm_c_shim; -/// Sentinel panic payload raised by `c_shim::rust_lua_throw`. -/// -/// On wasm32 Lua's `LUAI_THROW` macro cannot use `setjmp`/`longjmp`, so -/// it is rewired to raise a Rust panic that `rust_lua_protected_call` -/// catches via `catch_unwind`. This happens on every Lua runtime error — -/// including ones caught by `pcall` — so the panic is expected control -/// flow. The `init()` panic hook filters panics carrying this payload -/// so they do not spam `console.error` with stack traces. -pub struct LuaThrow; +use wasm_c_shim::LuaThrow; use std::cell::RefCell; use std::path::Path; diff --git a/dev-docs/wasm.md b/dev-docs/wasm.md index 1a31091f2..2ee9d290e 100644 --- a/dev-docs/wasm.md +++ b/dev-docs/wasm.md @@ -55,10 +55,63 @@ These shims live in `crates/wasm-c-shim/`, a workspace member that is a no-op on targets (all exports gated on `cfg(target_arch = "wasm32")`). Both `wasm-quarto-hub-client` (production) and `pampa` WASM tests (dev-dependency) link against it. +The crate also replaces Lua's `LUAI_THROW`/`LUAI_TRY` macros (normally `setjmp`/`longjmp`) +with `panic!()` / `catch_unwind`, since wasm32 has no native unwinding. The panic payload +is `wasm_c_shim::LuaThrow`, a public marker type. Hosts that install a custom panic hook +(e.g. `wasm-quarto-hub-client`'s `init()`) can downcast to `LuaThrow` to filter expected +Lua control-flow panics out of `console.error` without suppressing real Rust panics. + **Edition note:** `wasm-c-shim` uses edition 2021, not the workspace default of 2024. Edition 2024 requires explicit `unsafe {}` blocks inside `unsafe fn`, which would add noise to ~65 FFI shim functions with no safety benefit. +### Wasm32 panic strategy and rustflags + +The `wasm-c-shim` `panic`/`catch_unwind` substitution only works when the binary's panic +strategy is `unwind`. The wasm32-unknown-unknown default is `abort`, under which `panic!()` +lowers to the wasm `unreachable` instruction and `catch_unwind` becomes a compile-time +no-op — meaning the first Lua throw during mlua initialization aborts the whole module. + +Three flags must be set on every wasm32 build that touches `wasm-c-shim`: + +``` +-C target-feature=+bulk-memory,+exception-handling +-C panic=unwind +-Zwasm-c-abi=spec +``` + +These live in two `.cargo/config.toml` files so they apply both to the production build +and to wasm32 invocations from the workspace root: + +- `crates/wasm-quarto-hub-client/.cargo/config.toml` — used when `build-wasm.js` builds + the production cdylib from the isolated hub-client workspace. +- `.cargo/config.toml` (workspace root) — used by `cargo test --target wasm32-unknown-unknown` + invocations from the monorepo root, including the `pampa wasm_lua` tests. + +`[unstable] build-std` is **not** in the workspace-root config because the `[unstable]` table +is not target-scoped — adding it would force `build-std` for every native invocation. The +`-Zbuild-std` flag stays on the test command and in CI. + +### JS bridge feature gate (`quarto-system-runtime`) + +`quarto-system-runtime/src/wasm.rs` declares four +`#[wasm_bindgen(raw_module = "/src/wasm-js-bridge/{template,sass,cache,fetch}.js")]` +extern blocks. Hub-client serves these JS modules at runtime through Vite, but +`wasm-bindgen` generates unconditional `require()` calls for the absolute paths in the +JS shim it produces. Under Node.js (where `wasm-bindgen-test-runner` runs), the paths do +not resolve and module load fails with `MODULE_NOT_FOUND`. + +To keep test wasm builds loadable, the four extern blocks are gated behind a +`js-bridge` Cargo feature on `quarto-system-runtime` (default off). When the feature +is off, stub modules return `Err(JsValue::from_str("js-bridge feature not enabled"))` or +`false`, preserving the `SystemRuntime` impl. `wasm-quarto-hub-client/Cargo.toml` opts in: + +```toml +quarto-system-runtime = { path = "../quarto-system-runtime", features = ["js-bridge"] } +``` + +Pampa's wasm test build does not, so the `require()` calls disappear from the generated shim. + ## Testing ### Native tests (all platforms) @@ -81,9 +134,17 @@ Run locally (Linux/macOS with LLVM): CC_wasm32_unknown_unknown=clang \ CFLAGS_wasm32_unknown_unknown="-isystem $PWD/crates/wasm-quarto-hub-client/wasm-sysroot -fno-builtin" \ cargo test -p pampa --test wasm_lua --target wasm32-unknown-unknown \ - --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind,panic_abort + --no-default-features --features lua-filter -Zbuild-std=std,panic_unwind ``` +The `-C panic=unwind`, `+exception-handling`, and `-Zwasm-c-abi=spec` flags are picked up +automatically from the workspace-root `.cargo/config.toml` — see "Wasm32 panic strategy +and rustflags" above. Only `panic_unwind` is needed in `-Zbuild-std` because the binary +uses the unwind strategy; `panic_abort` is unused. + +On macOS, Apple's bundled clang does not include the wasm32 target. Use Homebrew LLVM +instead: `CC_wasm32_unknown_unknown=/opt/homebrew/opt/llvm/bin/clang`. + **Important notes:** - You must use `--test wasm_lua` to select only the WASM test file. From bf370a0a8d197400d5ed80d35c89c90b9e13af1a Mon Sep 17 00:00:00 2001 From: Gordon Woodhull Date: Wed, 1 Jul 2026 20:26:44 -0400 Subject: [PATCH 5/5] Rebase onto main: align with wasm-shim-merge architecture, pin CI toolchain MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This branch predates the 2026-04-20 wasm-shim-merge work and the bd-at72 toolchain pin; rebasing onto current main needed alignment beyond conflict resolution: - wasm-c-shim: the rebase carried main's evolved c_shim.rs content into shim.rs via rename detection; declare the wasm-printf-fmt dependency it uses for snprintf/vsnprintf. - Root Cargo.toml: patch tree-sitter-language to the local wasm-shim fork (same patch wasm-quarto-hub-client uses) so tree-sitter core does not compile upstream's wasm32 stdio/stdlib/string stubs, which collide with wasm-c-shim at link time. The fork has byte-identical Rust source and a no-op build.rs on native targets. - wasm-tests CI: run on the pinned toolchain from rust-toolchain.toml (now a nightly with rust-src and the wasm32 target) instead of RUSTUP_TOOLCHAIN=nightly — bd-at72 pinned the toolchain because later nightlies SIGSEGV in LLVM ThinLTO on wasm32 builds of this workspace. The E0152 duplicate-lang-item conflict the override worked around no longer reproduces on the pinned toolchain. - wasm_lua.rs: SourceInfo::for_test() (SourceInfo::default() is deprecated under -D deprecated), and pass the new attribution argument (None) to apply_lua_filters. - Regenerate Cargo.lock and crates/wasm-quarto-hub-client/Cargo.lock. Verified: cargo build + nextest workspace green (9862 passed); all 6 wasm tests pass under wasm-bindgen-test-runner (macOS, Homebrew LLVM clang, pinned nightly); production build-wasm.js succeeds. --- .github/workflows/test-suite.yml | 17 +++--- Cargo.lock | 72 +++++++++++++++++++++++- Cargo.toml | 7 +++ crates/pampa/tests/wasm_lua.rs | 15 +++-- crates/wasm-c-shim/Cargo.toml | 5 ++ crates/wasm-c-shim/src/shim.rs | 18 +++++- crates/wasm-quarto-hub-client/Cargo.lock | 8 +++ dev-docs/wasm.md | 15 +++-- 8 files changed, 128 insertions(+), 29 deletions(-) diff --git a/.github/workflows/test-suite.yml b/.github/workflows/test-suite.yml index 9601699a5..225a9bcbb 100644 --- a/.github/workflows/test-suite.yml +++ b/.github/workflows/test-suite.yml @@ -183,21 +183,18 @@ jobs: wasm-tests: name: WASM Tests runs-on: ubuntu-latest - # Override rust-toolchain.toml so the rustup proxy doesn't auto-install the - # prebuilt wasm32-unknown-unknown target. We need -Zbuild-std to rebuild - # core from source, and the prebuilt sysroot causes E0152 (duplicate lang - # item). The production WASM build avoids this because wasm-quarto-hub-client - # is excluded from the workspace and gets an isolated target/ directory. - env: - RUSTUP_TOOLCHAIN: nightly + # Uses the pinned toolchain from rust-toolchain.toml, which is already a + # nightly with rust-src (required by -Zbuild-std) and the wasm32 target. + # Do NOT override with latest nightly: bd-at72 pinned the toolchain because + # newer nightlies SIGSEGV in LLVM ThinLTO when building this workspace for + # wasm32-unknown-unknown. (The E0152 duplicate-lang-item conflict between + # the prebuilt wasm32 sysroot and -Zbuild-std that this job originally + # worked around no longer reproduces on the pinned toolchain.) steps: - uses: actions/checkout@v4 with: fetch-depth: 1 - - name: Install Rust nightly with rust-src - run: rustup toolchain install nightly --component rust-src --profile minimal - - name: Set up Clang uses: egor-tensin/setup-clang@v1 with: diff --git a/Cargo.lock b/Cargo.lock index f0a0b33b8..773159d2d 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -491,6 +491,12 @@ dependencies = [ "unicode-normalization", ] +[[package]] +name = "cast" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "37b2a672a2cb129a2e41c10b1224bb368f9f37a2b16b612598138befd7b37eb5" + [[package]] name = "cc" version = "1.2.60" @@ -2615,6 +2621,16 @@ dependencies = [ "syn", ] +[[package]] +name = "minicov" +version = "0.3.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4869b6a491569605d66d3952bcdf03df789e5b536e5f0cf7758a7f08a55ae24d" +dependencies = [ + "cc", + "walkdir", +] + [[package]] name = "miniz_oxide" version = "0.8.9" @@ -2828,6 +2844,12 @@ version = "1.70.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" +[[package]] +name = "oorandom" +version = "11.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6790f58c7ff633d8771f42965289203411a5e5c68388703c06e14f24770b41e" + [[package]] name = "open" version = "5.3.5" @@ -2975,6 +2997,8 @@ dependencies = [ "tokio", "tree-sitter", "tree-sitter-qmd", + "wasm-bindgen-test", + "wasm-c-shim", "yaml-rust2", ] @@ -5727,8 +5751,6 @@ dependencies = [ [[package]] name = "tree-sitter-language" version = "0.1.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "009994f150cc0cd50ff54917d5bc8bffe8cad10ca10d81c34da2ec421ae61782" [[package]] name = "tree-sitter-lua" @@ -6136,6 +6158,52 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "wasm-bindgen-test" +version = "0.3.58" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "45649196a53b0b7a15101d845d44d2dda7374fc1b5b5e2bbf58b7577ff4b346d" +dependencies = [ + "async-trait", + "cast", + "js-sys", + "libm", + "minicov", + "nu-ansi-term", + "num-traits", + "oorandom", + "serde", + "serde_json", + "wasm-bindgen", + "wasm-bindgen-futures 0.4.58 (registry+https://github.com/rust-lang/crates.io-index)", + "wasm-bindgen-test-macro", + "wasm-bindgen-test-shared", +] + +[[package]] +name = "wasm-bindgen-test-macro" +version = "0.3.58" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f579cdd0123ac74b94e1a4a72bd963cf30ebac343f2df347da0b8df24cdebed2" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "wasm-bindgen-test-shared" +version = "0.2.108" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a8145dd1593bf0fb137dbfa85b8be79ec560a447298955877804640e40c2d6ea" + +[[package]] +name = "wasm-c-shim" +version = "0.0.0" +dependencies = [ + "wasm-printf-fmt", +] + [[package]] name = "wasm-encoder" version = "0.236.1" diff --git a/Cargo.toml b/Cargo.toml index fc27a5d78..f6500176a 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -274,6 +274,13 @@ verbose_file_reads = "warn" [patch.crates-io] lua-src = { path = "crates/lua-src-wasm" } +# Neutralize tree-sitter-language's upstream wasm32 stdio/stdlib/string C +# stubs so wasm-c-shim is the single source of truth for C stdlib symbols +# in workspace wasm32 builds (the pampa wasm_lua tests). Same patch as +# wasm-quarto-hub-client's; identical Rust source on native targets, and +# its build.rs is a no-op off-wasm32. See +# claude-notes/plans/2026-04-20-wasm-shim-merge.md. +tree-sitter-language = { path = "crates/tree-sitter-language-wasm-shim" } # Fork of runtimelib that adds venv-aware kernelspec discovery # (`data_dirs_with_jupyter_paths`, `find_kernelspec_with_jupyter_paths`) # and threads `searched_paths` through `RuntimeError::KernelNotFound`. diff --git a/crates/pampa/tests/wasm_lua.rs b/crates/pampa/tests/wasm_lua.rs index 40e2ae41b..192b0781e 100644 --- a/crates/pampa/tests/wasm_lua.rs +++ b/crates/pampa/tests/wasm_lua.rs @@ -87,9 +87,9 @@ end blocks: vec![Block::Paragraph(Paragraph { content: vec![Inline::Str(Str { text: "hello".to_string(), - source_info: quarto_source_map::SourceInfo::default(), + source_info: quarto_source_map::SourceInfo::for_test(), })], - source_info: quarto_source_map::SourceInfo::default(), + source_info: quarto_source_map::SourceInfo::for_test(), })], }; let context = ASTContext::new(); @@ -100,6 +100,7 @@ end &[PathBuf::from("/project/uppercase.lua")], "html", runtime, + None, ) .await .expect("filter execution failed"); @@ -198,9 +199,9 @@ end blocks: vec![Block::Paragraph(Paragraph { content: vec![Inline::Str(Str { text: "test".to_string(), - source_info: quarto_source_map::SourceInfo::default(), + source_info: quarto_source_map::SourceInfo::for_test(), })], - source_info: quarto_source_map::SourceInfo::default(), + source_info: quarto_source_map::SourceInfo::for_test(), })], }; @@ -210,6 +211,7 @@ end &[PathBuf::from("/project/check_io.lua")], "html", runtime, + None, ) .await .expect("filter with io checks failed — synthetic io may not be registered"); @@ -257,9 +259,9 @@ end blocks: vec![Block::Paragraph(Paragraph { content: vec![Inline::Str(Str { text: "test".to_string(), - source_info: quarto_source_map::SourceInfo::default(), + source_info: quarto_source_map::SourceInfo::for_test(), })], - source_info: quarto_source_map::SourceInfo::default(), + source_info: quarto_source_map::SourceInfo::for_test(), })], }; @@ -269,6 +271,7 @@ end &[PathBuf::from("/project/check_os.lua")], "html", runtime, + None, ) .await .expect("filter with os checks failed — synthetic os may not be registered"); diff --git a/crates/wasm-c-shim/Cargo.toml b/crates/wasm-c-shim/Cargo.toml index 7098086f2..056fa24f4 100644 --- a/crates/wasm-c-shim/Cargo.toml +++ b/crates/wasm-c-shim/Cargo.toml @@ -9,5 +9,10 @@ edition = "2021" license.workspace = true description = "C stdlib shims for wasm32-unknown-unknown (no libc)" +[target.'cfg(target_arch = "wasm32")'.dependencies] +# snprintf/vsnprintf formatting engine used by the shim's printf-family +# symbols (see the wasm-shim-merge plan, 2026-04-20). +wasm-printf-fmt = { path = "../wasm-printf-fmt" } + [lints] workspace = true diff --git a/crates/wasm-c-shim/src/shim.rs b/crates/wasm-c-shim/src/shim.rs index 532354e9b..503a15de3 100644 --- a/crates/wasm-c-shim/src/shim.rs +++ b/crates/wasm-c-shim/src/shim.rs @@ -655,12 +655,20 @@ pub unsafe extern "C" fn isxdigit(c: c_int) -> c_int { #[no_mangle] pub unsafe extern "C" fn toupper(c: c_int) -> c_int { - if islower(c) != 0 { c - 32 } else { c } + if islower(c) != 0 { + c - 32 + } else { + c + } } #[no_mangle] pub unsafe extern "C" fn tolower(c: c_int) -> c_int { - if isupper(c) != 0 { c + 32 } else { c } + if isupper(c) != 0 { + c + 32 + } else { + c + } } /* ====================================================================== */ @@ -669,7 +677,11 @@ pub unsafe extern "C" fn tolower(c: c_int) -> c_int { #[no_mangle] pub unsafe extern "C" fn abs(x: c_int) -> c_int { - if x < 0 { -x } else { x } + if x < 0 { + -x + } else { + x + } } #[no_mangle] diff --git a/crates/wasm-quarto-hub-client/Cargo.lock b/crates/wasm-quarto-hub-client/Cargo.lock index 99677ea82..bae5b3a82 100644 --- a/crates/wasm-quarto-hub-client/Cargo.lock +++ b/crates/wasm-quarto-hub-client/Cargo.lock @@ -3972,6 +3972,13 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "wasm-c-shim" +version = "0.0.0" +dependencies = [ + "wasm-printf-fmt", +] + [[package]] name = "wasm-encoder" version = "0.236.1" @@ -4035,6 +4042,7 @@ dependencies = [ "serde_yaml", "wasm-bindgen", "wasm-bindgen-futures", + "wasm-c-shim", "wasm-printf-fmt", "web-sys", "yaml-rust2", diff --git a/dev-docs/wasm.md b/dev-docs/wasm.md index 2ee9d290e..c3712f9b0 100644 --- a/dev-docs/wasm.md +++ b/dev-docs/wasm.md @@ -150,14 +150,13 @@ instead: `CC_wasm32_unknown_unknown=/opt/homebrew/opt/llvm/bin/clang`. - You must use `--test wasm_lua` to select only the WASM test file. Running `cargo test -p pampa --target wasm32` without `--test` will fail because native tests can't compile for wasm32. -- The prebuilt `wasm32-unknown-unknown` target (installed by `rust-toolchain.toml`) - conflicts with `-Zbuild-std` when building within the workspace — both produce a - `core` crate, causing E0152 (duplicate lang item). The production build avoids this - because `wasm-quarto-hub-client` is excluded from the workspace. The CI job sets - `RUSTUP_TOOLCHAIN=nightly` to bypass `rust-toolchain.toml`, so the prebuilt target - is never installed. Locally, you can either set `RUSTUP_TOOLCHAIN=nightly` or - remove the target before testing (`rustup target remove wasm32-unknown-unknown`) - and re-add it afterward for the production build. +- The pinned toolchain in `rust-toolchain.toml` (a nightly with `rust-src` and the + wasm32 target) is what both CI and local runs should use — no `RUSTUP_TOOLCHAIN` + override. Do not substitute a newer nightly: bd-at72 pinned the toolchain because + later nightlies SIGSEGV in LLVM ThinLTO when building this workspace for wasm32. + (An earlier iteration of this setup hit E0152 duplicate-lang-item conflicts + between the prebuilt wasm32 sysroot and `-Zbuild-std`; that no longer reproduces + on the pinned toolchain with the prebuilt target installed.) - The pampa `[[bin]]` targets (`pampa`, `ast-reconcile`) use `required-features` to prevent compilation when running WASM tests. Cargo builds bin targets alongside integration tests by default (rust-lang/cargo#12980); the `required-features` gate