Add build script for macOS & fix some cli/readme issues

[olilarkin]

Great to see you using ort-builder for this!

Here is some work to get it building on macOS. Will need to check that the linux and wasm builds are not affected by the changes

[olilarkin]

damn... ORT v1.16.3 fails on linux

[sevagh]

Thanks for making ort-builder in the first place. I feel like I would have definitely used it for Demucs if it were possible (rather than rewriting the universe).

Thanks for the PR! I can't test the OSX parts but as long as the other parts continue working for me, I'm fine with it.

[olilarkin]

Other than the STFT/ISTFT part (which i've seen you have in C++ already) do you see another reason why demucs couldn't run on onnxruntime? It would be interesting for your project to see if it worked via the onnxruntime webgpu execution provider

[sevagh]

I do believe it was the STFT/iSTFT. It's an interesting project idea. I feel like I'd have to:

1. Write a similar copy of the Demucs model architecture that leaves the STFT/iSTFT out of the weights/layers file and rewrite those to a new PyTorch weights file (call it Demucs-onnx or whatever)
2. Write the surrounding code to implement the STFT/iSTFT (as you say, I have this code already which is easy)
3. Convert Demucs-onnx to ONNX, then the ORT/ort-builder and see what happens

I'll note it down for a future curiosity.

[sevagh]

@olilarkin I got a (to be made public soon) version of demucs for onnx, like you said - take out the stft/istft code from the forward method and it's easy-peasy. I used ort-builder's scripts, but I avoided the model.h/model.c files which ended up blowing up to 1.2 GB 😱 I just pass in the ort model file from the outside (all details will be made public soon-ish)

Now I'm running into serious trouble understanding webgpu, webnn, jsep, etc. for onnxruntime

Building for webgpu says to use use_jsep (javascript execution provider), not the standalone use_webgpu flag which is mutually exclusive
Building for webnn says to use use_jsep + use_webnn combined

Then, when I compile my actual C++ code, when do I supply -s USE_WEBGPU=1 to emscripten flags? Etc.

[olilarkin]

That's great news. Yeah the bin2c stuff only works for small models unfortunately. I haven't compiled ORT for wasm with wgpu, webnn support etc. I am not sure if you have to pass the emscripten flag -s USE_WEBGPU=1, i think that will be done for onnxruntime by the onnxruntime cmake/build scripts if necessary.

https://onnxruntime.ai/docs/build/web.html

this seems relevant...

Q: I have a C++ project for web scenario, which runs a ML model using ONNX Runtime and generates WebAssembly as an output. Does ONNX Runtime Web support a static WebAssembly library, so that my application can link with it and make all pre/post processors to be compiled together into WebAssembly?

A: With --build_wasm, a build script generates .wasm and .js files for web scenarios and intermediate libraries are not linked properly with other C/C++ projects. When you build ONNX Runtime Web using --build_wasm_static_lib instead of --build_wasm, a build script generates a static library of ONNX Runtime Web named libonnxruntime_webassembly.a in output directory. To run a simple inferencing like an [unit test](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/test/wasm/test_inference.cc), what you need is three header files as follows and libonnxruntime_webassembly.a.

include/onnxruntime/core/session/onnxruntime_c_api.h
include/onnxruntime/core/session/onnxruntime_cxx_api.h
include/onnxruntime/core/session/onnxruntime_cxx_inline.h
One important note is that ONNX Runtime has dependencies on lots of 3rd party libraries such as protobuf, onnx, and others. You may need copy necessary header files to your project. You also take care of cases of library version conflicts or emsdk version conflicts between ONNX Runtime and your project.

[sevagh]

Check it out: https://github.com/sevagh/demucs.onnx