test(rag): skip transformers.js integration tests on Windows CI#119
Merged
Conversation
These tests download a model over the network and load the onnxruntime-node native backend, both flaky in Windows CI. Skip on Windows (in addition to the existing skip when @xenova/transformers is absent), matching the onnx provider test. A flaky failure here was previously mislabeled '@xenova/transformers is not installed' by an over-broad catch in loadPipeline, which also defeated an availability-only guard (the package was installed; model load was what failed). Applies to transformers-embedding-provider.integration.test.ts and the Transformers.js block of comparison.integration.test.ts.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #119 +/- ##
=======================================
Coverage 81.49% 81.49%
=======================================
Files 215 215
Lines 16795 16795
Branches 3214 3214
=======================================
Hits 13687 13687
Misses 3108 3108 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
Windows CI integration runs intermittently failed with
@xenova/transformers is not installed— even though the package was installed. The transformers.js integration tests run a real model download (~20MB over the network) and load theonnxruntime-nodenative backend, both of which are flaky in Windows CI. This skips them on Windows, matching the existingonnx-embedding-providertest (which already skips Windows for the same reason).Changes
packages/rag/test/integration/transformers-embedding-provider.integration.test.ts— add thetry { await import(...) } catch {}availability guard plus a Windows skip (skipTransformers = !transformersAvailable || isWindows) on bothdescribeblocks.packages/rag/test/integration/comparison.integration.test.ts— extend the Transformers.js block's guard to also skip Windows (!transformersAvailable || isWindows);isWindowswas already defined there for the ONNX block.CHANGELOG.md— entry under[Unreleased] > Fixed.No published/runtime code changes — test files + changelog only.
Why Windows-only, and why it surfaced now
comparison.testfailed inside itsdescribe.skipIf(!transformersAvailable)block, which only runs whentransformersAvailableis true — i.e. the package was installed; the real failure was downstream (model download / native backend).not installedmessage comes from an over-broadcatchin the provider'sloadPipelinethat wraps both the import and the model load. Tracked separately in rag: over-broad catch in loadPipeline mislabels all failures as '@xenova/transformers is not installed' #118.mainis green on Windows, and the PR that surfaced this (feat: first-class local HTML resources (#112) #116) changes nothing underpackages/rag. It just hit the flake.Testing
bun run test:system, 451s, exit 0).Follow-up
catchinloadPipelineso model/inference failures aren't mislabeled as "not installed".🤖 Generated with Claude Code