Skip to content

fix(embeddings): take the native ONNX Runtime from WireCat's own npm packages - #3117

Draft
leemour wants to merge 2 commits into
mainfrom
fix/ort-native-source
Draft

leemour wants to merge 2 commits into
mainfrom
fix/ort-native-source

Conversation

@leemour

@leemour leemour commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Task

Restore the native ONNX Runtime on Linux and macOS without any run-time dependency on Microsoft's hosting (owner's direction). Do not merge before the five packages are on npm: the "Native runtime" job downloads them and fails with 404 until then.

Why

Since about 22:20 UTC on 17.11.2026, github.com/microsoft/onnxruntime answers 404: the repository, the API and every release download (runtimewire). models download could not install the native library on Linux and macOS, so the models ran through WebAssembly. The "Native runtime" job failed on every PR.

What

  • Five npm packages, @wirecat/onnxruntime-{linux-x64,linux-arm64,darwin-arm64,win32-x64,win32-arm64} 1.30.0. Each holds only that library, plus Microsoft's LICENSE (MIT) and ThirdPartyNotices.txt from the nupkg, and sets os/cpu. Each README records the source URL, the source sha256 and the library sha256.
  • cli-core still downloads on demand into the shared models folder, from the package tarball on registry.npmjs.org. It uses the existing tarball path (the one npmTarball used: installFiles → gunzip → tarMember), with member package/<library>. The tarball and the library sha256 are both pinned. No new download code.
  • bin/publish-onnxruntime: one Node file. It downloads the NuGet Microsoft.ML.OnnxRuntime 1.30.0 nupkg (its SHA-512 matches NuGet's catalog) and the onnxruntime-node 1.30.0 tarball, checks both, and builds and packs the five packages into packages/onnxruntime/ (git-ignored). It refuses any tarball whose sha256 differs from the pin; npm pack gave the same bytes on two runs. With --publish, it publishes the packages that are not on npm yet. The npm token comes from the keyring.
  • Library sources: Linux from NuGet (byte for byte the old GitHub libraries). macOS from NuGet: same size and install name @rpath/libonnxruntime.1.dylib, new sha256 bffaa6ef…6879. Windows from onnxruntime-node as before (the DirectML ordinal), the same DLL sha.
  • packages/onnx/probe-native.mjs (monthly ONNX workflow) takes the same packages.
Download per platform before after (our package)
linux-x64 11.3 MB (GitHub) 11.3 MB
linux-arm64 10.3 MB (GitHub) 10.4 MB
darwin-arm64 42.4 MB (GitHub) 12.9 MB
win32-x64 113.5 MB (onnxruntime-node) 10.7 MB
win32-arm64 113.5 MB (onnxruntime-node) 10.8 MB

Testing

  • The first commit (NuGet Range slices) ran the "Native runtime" job green on all five systems: run 44365453619. macOS loaded the NuGet dylib there, and the darwin package carries those exact bytes.
  • On linux-x64, with the packed tarballs served in place of registry.npmjs.org (they are not published yet): downloadTextModel("e5-small") (what models download runs) installs from onnxruntime-linux-x64-1.30.0.tgz, and onnxRuntime().native is true. 64 passages embed in 272–292 ms native against 669 ms through WebAssembly. onnx-runtime.native.test.ts with NATIVE_RUNTIME_CHECK=native passes the same way.
  • npm publish --dry-run of a packed tarball. typecheck, lint, docs:check, and the unit tests for embeddings, speech and model-downloads.
  • After the packages are published: re-run this PR's "Native runtime" job.

🤖 Generated with Claude Code

…opy is gone

github.com/microsoft/onnxruntime answers 404 since 17.11.2026 (repo, API and
every release download), so `models download` could not install the native
library on Linux and macOS and the models fell back to WebAssembly. The
Native runtime CI job failed on every PR.

The library now comes from Microsoft's NuGet package Microsoft.ML.OnnxRuntime
1.30.0. One HTTP Range request fetches only the deflated bytes of this
system's library out of the 157 MB zip; installFiles checks the slice's size
and sha256, fflate inflates it, and the library's own sha256 is checked as
before. A published NuGet version never changes, and the package signature
nuget.org may replace is the zip's last file, so the pinned offsets hold.

- linux-x64, linux-arm64: the same library bytes as the GitHub release.
- darwin-arm64: NuGet's own build of 1.30.0, same size and install name
  (@rpath/libonnxruntime.1.dylib), new sha256.
- Windows keeps onnxruntime-node's npm tarball.

ModelFile gains an optional `offset`, sent as a Range header. The package's
probe-native.mjs takes the same slices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…packages

The owner wants no run-time dependency on Microsoft's hosting, GitHub or
NuGet. The library now comes from @wirecat/onnxruntime-<platform> on
registry.npmjs.org, one package per platform (linux-x64, linux-arm64,
darwin-arm64, win32-x64, win32-arm64), each holding only that library plus
Microsoft's LICENSE and ThirdPartyNotices.txt, with os/cpu set.

`models download` still fetches on demand into the shared models folder,
through the existing tarball path (installFiles, gunzip, tarMember): every
NATIVE entry now has the same shape, member `package/<library>`. Tarball and
library sha256 are both pinned. The NuGet Range code from the previous commit
is reverted: nothing needs it.

bin/publish-onnxruntime builds the packages once from the NuGet 1.30.0 nupkg
(Linux, macOS) and the onnxruntime-node 1.30.0 tarball (Windows, DirectML
build, same DLL sha as today), packs them with npm pack (reproducible here:
two runs, same sha256), refuses any tarball that differs from the pin, and
with --publish publishes those not yet on npm. Not published: a new npm
package needs the owner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leemour leemour changed the title fix(embeddings): fetch the native ONNX Runtime from NuGet, GitHub's copy is gone fix(embeddings): take the native ONNX Runtime from WireCat's own npm packages Nov 18, 2026
@leemour
leemour marked this pull request as draft November 18, 2026 18:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant