Skip to content

Add WASM UDF support#2500

Open
sevencat2004 wants to merge 2 commits into
getdozer:mainfrom
sevencat2004:bounty-1653-wasm-udf
Open

Add WASM UDF support#2500
sevencat2004 wants to merge 2 commits into
getdozer:mainfrom
sevencat2004:bounty-1653-wasm-udf

Conversation

@sevencat2004
Copy link
Copy Markdown

@sevencat2004 sevencat2004 commented Jun 2, 2026

/claim #1653

Summary

This PR adds initial WebAssembly UDF support to Dozer SQL.

It introduces a new !Wasm UDF config variant:

udfs:
  - name: add_one
    config: !Wasm
      module: ./models/add_one.wasm
      return_type: int

The configured UDF can then be called from SQL:

SELECT add_one(value) FROM input;

The exported function name defaults to the configured UDF name. It can be overridden with function:

udfs:
  - name: score
    config: !Wasm
      module: ./models/fraud.wasm
      function: score_event
      return_type: boolean

Supported ABI in this first pass

  • int and uint map to WASM i64
  • float maps to WASM f64
  • boolean maps to WASM i32
  • uint arguments outside the current i64 ABI range and negative i64 returns for uint UDFs are rejected instead of silently wrapping
  • WASM functions must return exactly one value
  • Unsupported configured return types are rejected when the expression is built
  • WASM modules are loaded when the expression is built
  • Instances are created per evaluation for a simple and isolated first implementation

Not included yet

  • String, JSON, binary, date/time, decimal, and memory ABI support
  • WASM imports
  • Instance/store caching

These are intentionally left out to keep the first implementation small and reviewable. The scalar ABI covers the core issue path of configuring a WASM module and invoking an exported function from SQL.

Tests

Validated locally with:

C:\Users\Administrator\.cargo\bin\cargo.exe fmt -p dozer-types -p dozer-sql-expression -p dozer-sql
$env:PROTOC='D:\coin\protoc-35.0-win64\bin\protoc.exe'; C:\Users\Administrator\.cargo\bin\cargo.exe test -p dozer-sql-expression --features wasm wasm
$env:PROTOC='D:\coin\protoc-35.0-win64\bin\protoc.exe'; C:\Users\Administrator\.cargo\bin\cargo.exe test -p dozer-types udf_yaml_deserialize
$env:PROTOC='D:\coin\protoc-35.0-win64\bin\protoc.exe'; C:\Users\Administrator\.cargo\bin\cargo.exe test -p dozer-sql-expression wasm_builder_tests

Latest rerun results:

  • WASM runtime/builder tests: 6 passed, 0 failed
  • UDF YAML deserialization tests: 3 passed, 0 failed
  • Feature-off builder gate test: 1 passed, 0 failed
  • Full dozer-sql-expression --features wasm: 33 passed, 0 failed

cargo check -p dozer-cli --features wasm was attempted locally, but the Windows GNU environment is missing gcc.exe, which is required by the existing ring dependency during CLI compilation.

Dependency Notes

This adds wasmi as the optional WASM runtime dependency and wat as a dev-dependency for generating tiny test modules from WAT.

The local Cargo version is cargo 1.96.0, which rewrote Cargo.lock to version 4 and refreshed some compatible transitive dependency versions while adding the WASM-related packages. If maintainers prefer a smaller lockfile diff, regenerate the lockfile with the project's expected Cargo version before merging.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Jun 2, 2026

CLA assistant check
All committers have signed the CLA.

@sevencat2004
Copy link
Copy Markdown
Author

/claim #1653

@algora-pbc algora-pbc Bot mentioned this pull request Jun 3, 2026
@sevencat2004
Copy link
Copy Markdown
Author

Demo / verification evidence for the WASM UDF path:

  • The builder test generates a tiny WAT module exporting add_one(i64) -> i64.
  • The PR configures it as a !Wasm UDF with return_type: int.
  • SQL expression coverage builds SELECT add_one(value) FROM t.
  • Runtime evaluation with input Field::Int(41) returns Field::Int(42).
  • The feature-off path is also covered: a configured WASM UDF returns WasmNotEnabled when the wasm feature is disabled.

Local validation commands used:

C:\Users\Administrator\.cargo\bin\cargo.exe fmt -p dozer-types -p dozer-sql-expression -p dozer-sql
$env:PROTOC='D:\coin\protoc-35.0-win64\bin\protoc.exe'; C:\Users\Administrator\.cargo\bin\cargo.exe test -p dozer-sql-expression --features wasm wasm
$env:PROTOC='D:\coin\protoc-35.0-win64\bin\protoc.exe'; C:\Users\Administrator\.cargo\bin\cargo.exe test -p dozer-types udf_yaml_deserialize
$env:PROTOC='D:\coin\protoc-35.0-win64\bin\protoc.exe'; C:\Users\Administrator\.cargo\bin\cargo.exe test -p dozer-sql-expression wasm_builder_tests

Results:

  • WASM runtime/builder tests: 4 passed, 0 failed
  • UDF YAML deserialization tests: 3 passed, 0 failed
  • Feature-off builder gate test: 1 passed, 0 failed

I can add a short video walkthrough as a follow-up if maintainers want the bounty demo requirement satisfied with an actual recording rather than the reproducible test/demo path above.

@sevencat2004
Copy link
Copy Markdown
Author

sevencat2004 commented Jun 3, 2026

Follow-up pushed in d8b3727: tightened the WASM uint ABI boundary handling so unsigned values no longer silently wrap when crossing the current i64-based scalar ABI. uint arguments above i64::MAX and negative i64 returns for uint UDFs now return explicit WASM UDF errors, with focused tests for both cases.

Validation rerun after the follow-up:

  • cargo fmt -p dozer-sql-expression
  • PROTOC=D:\coin\protoc-35.0-win64\bin\protoc.exe cargo test -p dozer-sql-expression --features wasm wasm: 6 passed
  • PROTOC=D:\coin\protoc-35.0-win64\bin\protoc.exe cargo test -p dozer-sql-expression wasm_builder_tests: 1 passed
  • PROTOC=D:\coin\protoc-35.0-win64\bin\protoc.exe cargo test -p dozer-sql-expression --features wasm: 33 passed
  • git diff --check: no whitespace errors

@reneleonhardt
Copy link
Copy Markdown

This PR adds first-pass

Typo, maybe first-class?

@sevencat2004
Copy link
Copy Markdown
Author

sevencat2004 commented Jun 3, 2026

Thanks for catching that. I changed the summary wording to say "initial WebAssembly UDF support", which is more accurate for this intentionally scoped first implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants