Finding
The public Reranker::predict contract accepts text pairs and promises scores, but ModernBertCpuReranker returns Ok(Predictions) after mapping raw bytes modulo hidden_size, inserting zero-valued placeholder tokens, omitting real query/document pair framing, and running those invented IDs through the model. Its own documentation says this is structural validation only.
The sibling GteReranker correctly returns NotLoaded rather than dummy scores; the exported CPU implementation violates that same fail-loud contract.
Evidence
crates/rerank/src/reranker.rs:11-29,72-80 defines the production text-pair contract and explicitly rejects dummy results from the preflight implementation.crates/rerank/src/cpu_reranker.rs:194-239 performs placeholder byte tokenization, uses hidden width as the vocabulary modulus, creates fake CLS/SEP IDs, and returns success.crates/rerank/src/cpu_reranker.rs:207-212 admits real use requires tokenizer.json and calls the output structural.crates/rerank/src/lib.rs:26-36 exports the type and trait as the crate's public API.
Why this matters
Callers cannot distinguish real relevance logits from numerically valid output over meaningless token IDs. That is worse than an explicit stub: fabricated evidence can flow into retrieval ranking, evaluations, and later GPU parity tests as though it were model output.
Desired correction
Until the real tokenizer and checkpoint are owned by the implementation, return a precise unavailable/not-loaded error from the text-pair predict path. Keep structural projection tests behind test-only helpers. Then either make the production reranker own the pinned tokenizer and perform exact pair encoding, or expose a separate typed pre-tokenized API whose caller supplies validated IDs, masks, and pair boundaries. Add parity tests proving the public text path matches the reference tokenizer and logits.
Finding
The public
Reranker::predictcontract accepts text pairs and promises scores, butModernBertCpuRerankerreturnsOk(Predictions)after mapping raw bytes modulohidden_size, inserting zero-valued placeholder tokens, omitting real query/document pair framing, and running those invented IDs through the model. Its own documentation says this is structural validation only.The sibling
GteRerankercorrectly returnsNotLoadedrather than dummy scores; the exported CPU implementation violates that same fail-loud contract.Evidence
crates/rerank/src/reranker.rs:11-29,72-80defines the production text-pair contract and explicitly rejects dummy results from the preflight implementation.crates/rerank/src/cpu_reranker.rs:194-239performs placeholder byte tokenization, uses hidden width as the vocabulary modulus, creates fake CLS/SEP IDs, and returns success.crates/rerank/src/cpu_reranker.rs:207-212admits real use requirestokenizer.jsonand calls the output structural.crates/rerank/src/lib.rs:26-36exports the type and trait as the crate's public API.Why this matters
Callers cannot distinguish real relevance logits from numerically valid output over meaningless token IDs. That is worse than an explicit stub: fabricated evidence can flow into retrieval ranking, evaluations, and later GPU parity tests as though it were model output.
Desired correction
Until the real tokenizer and checkpoint are owned by the implementation, return a precise unavailable/not-loaded error from the text-pair
predictpath. Keep structural projection tests behind test-only helpers. Then either make the production reranker own the pinned tokenizer and perform exact pair encoding, or expose a separate typed pre-tokenized API whose caller supplies validated IDs, masks, and pair boundaries. Add parity tests proving the public text path matches the reference tokenizer and logits.