Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion packages/sie_server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,8 @@ pip install sie-server
```

- **Transformers 5 bundle** (LightOnOCR, GLM-OCR, GLiGuard, GLiNER2.5 entity and Decide models, Privacy Filter, and the TopK-Embed-V1
multi-vector models, and Perplexity decisions) — requires `transformers` 5.x, and is served with `-b transformers5`. The GLiNER2.5-Decide
multi-vector models, Perplexity decisions, and EmbeddingGemma 2) — requires `transformers` 5.x, and is served with `-b transformers5`.
`google/embeddinggemma-2` needs `EmbeddingGemma2Model`, which first shipped in `transformers` 5.19.0. The GLiNER2.5-Decide
and entity models also need `gliner2` 2.x. `sie-server` itself asks for `gliner2<2`, which the default bundle's GLiNER2
models need, so pip reports that conflict when the second command below installs 2.x; the transformers5
bundle's GLiNER2 models are pinned to 2.0.0:
Expand Down
8 changes: 6 additions & 2 deletions packages/sie_server/bundles/transformers5.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,19 @@ adapters:
- sie_server.adapters.pplx_decider.runtime_lora
# SAM 3 open-vocabulary detection (Sam3Model / Sam3Processor exist only in transformers 5.x).
- sie_server.adapters.sam3.adapter
# EmbeddingGemma 2 text embeddings (EmbeddingGemma2Model). The class first
# shipped in transformers 5.19.0; see models/google__embeddinggemma-2.yaml.
- sie_server.adapters.embedding_gemma2.adapter
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
deps:
# Transformers 5.x for models requiring newer architecture support
# (e.g., LightOnOCR-2-1B needs LightOnOcrForConditionalGeneration,
# GLM-OCR needs GlmOcrForConditionalGeneration). Floor raised to 5.14
# for the visual-SPLADE family (custom modeling code built on 5.13+),
# then 5.17 for Perplexity's pinned native Qwen decision checkpoint, then
# 5.19: Parakeet-TDT refuses to load below 5.18 (padded rows turn into NaN
# under eager attention), and 5.19.0 is the version the v0.10.x images
# already ship. The saved locks pin this floor exactly.
# under eager attention), EmbeddingGemma2Model first shipped in 5.19.0, and
# 5.19.0 is the version the v0.10.x images already ship. The saved locks
# pin this floor exactly.
transformers: '>=5.19,<6'
# Includes multimodal SparseEncoder (image documents), with the local-code
# trust boundary validated against this exact release.
Expand Down
58 changes: 58 additions & 0 deletions packages/sie_server/models/google__embeddinggemma-2.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# google/embeddinggemma-2 — Apache-2.0, ungated, 744,371,512 parameters.
# Released 2026-09-14. Native transformers class EmbeddingGemma2Model
# (model_type embedding_gemma2); the repo has no custom modeling code.
#
# The class first shipped in transformers 5.19.0 (2026-10-06). 5.18.x does
# not contain it. This profile is on the transformers5 bundle only, whose
# floor is >=5.19,<6 and whose saved locks pin 5.19.0. The root pin stays
# transformers 4.57.6. Do not serve this model from the default bundle.
#
# Sentence Transformers >=6.1.0 is the upstream wrapper. The bundle pins
# sentence-transformers==5.7.0, so the adapter applies the text recipe itself:
# mean pooling with the prompt included, then L2 normalization in float32.
# Query prompt: "task: search result | query: "
# Document prompt: "title: none | text: "
# Matryoshka prefixes, re-normalized after the slice: 128, 256, 512, and the
# native 768. Pass options.truncate_dim to request a shorter prefix. The
# catalog width stays 768.
#
# Text only. Image, audio, and video need EmbeddingGemma2Processor's soft
# tokens, video-token sampling, and Gemma 4 audio features. SIE's image, frame,
# and speech-PCM plumbing does not produce those inputs, and this profile
# drops both towers at load (vision_config=None, audio_config=None).
#
# Float16 overflows this model (NaN activations). The profile asks for
# bfloat16; off CUDA the adapter uses float32.
sie_id: google/embeddinggemma-2
hf_id: google/embeddinggemma-2
# Pinned to an immutable commit SHA. Resolved 2026-11-01 from the Hub
# model API (ungated, sha of the default revision).
hf_revision: f9972ac22e696a105c482202646979e33e4b5a1e
inputs:
text: true
image: false
audio: false
video: false
tasks:
encode:
dense:
dim: 768
sparse: null
multivector: null
score: null
extract: null
# Model card: one shared 8,192-token context. The tokenizer's
# model_max_length is an unbounded sentinel, so the cap lives here.
max_sequence_length: 8192
profiles:
default:
max_batch_tokens: 16384
compute_precision: bfloat16
adapter_path: sie_server.adapters.embedding_gemma2.adapter:EmbeddingGemma2Adapter
adapter_options:
loadtime: {}
runtime:
normalize: true
pooling: mean
query_template: 'task: search result | query: {text}'
doc_template: 'title: none | text: {text}'
Loading
Loading