Repository navigation

Avoid ONNX session load for token counts - #640

Merged

joein merged 8 commits into

qdrant:mainfrom

Mirochill:fix-584-token-count-tokenizer-only

Oct 1, 2026

Merged

Avoid ONNX session load for token counts#640
joein merged 8 commits into
qdrant:mainfrom
Mirochill:fix-584-token-count-tokenizer-only

Commits on Sep 29, 2026
- Avoid ONNX session load for token counts
  
  Mirochill
  authored and
  joein
  committed
  7412375
- refactor: centralize lazy tokenizer loading for token_count
  joein
  committed
  c8ea84a
- new: use tokenizer hooks in siglip, if_splade, and bm42
  joein
  committed
  4d28155
- fix: build post-processing state lazily for bm42, minicoil, and colbert
  joein
  committed
  9c516b7
- fix: assign colbert and minicoil loader state only once complete
  joein
  committed
  a111933
- fix: reuse already-loaded tokenizer and minicoil state in load_onnx_model
  joein
  committed
  77400ef
- fix: build minicoil post-processing state only where it is used
  joein
  committed
  500880f
Commits on Oct 1, 2026
- refactoring: declare _model_dir once in OnnxModel
  joein
  committed
  af4333c