Context
The current optional ML path is a caller-provided DistilBERT-class token classifier. State of the art has moved: GLiNER2-PII (arXiv 2605.09973) offers multilingual, near-zero-shot PII extraction; fine-tuned DeBERTa reaches F1 0.97+ on fixed entity sets; ONNX-quantized variants run on CPU. These models improve monthly without any effort from this project.
The strategic read: never train models, curate them. Ride the commodity curve; keep the durable layers (policy, aliases, safe reports, typed locations, formats) as the product.
Proposal
- A
GLiNERPIIBackend beside LocalONNXPIIBackend, same contract: local ONNX inference, caller-provided artifacts, no implicit downloads, remote=False. GLiNER's span-based architecture also sidesteps some token-merging fragility of the current BIO pipeline.
- Verified model cards: a docs page listing 2-3 recommended open models with SHA-pinned artifact references and honestly measured per-entity numbers from the multi-source harness (see the evaluation proposal). "Bring your own model" stays; the cards remove the paralysis of choosing one.
- Keep the strict PII-only boundary from the vision: models are integrated solely for PII pseudonymization, never exposed as general-purpose NER.
Fit
- No new dependencies in the base wheel; GLiNER runtime needs stay inside the
ml extra (onnxruntime + tokenizers already cover the quantized exports).
- Testing follows the existing rule: real downloaded lightweight artifacts, cached outside version control, no mock inference.
🤖 Generated with Claude Code
https://claude.ai/code/session_012JiX3zWeEC28kmy5KmAXvf
Context
The current optional ML path is a caller-provided DistilBERT-class token classifier. State of the art has moved: GLiNER2-PII (arXiv 2605.09973) offers multilingual, near-zero-shot PII extraction; fine-tuned DeBERTa reaches F1 0.97+ on fixed entity sets; ONNX-quantized variants run on CPU. These models improve monthly without any effort from this project.
The strategic read: never train models, curate them. Ride the commodity curve; keep the durable layers (policy, aliases, safe reports, typed locations, formats) as the product.
Proposal
GLiNERPIIBackendbesideLocalONNXPIIBackend, same contract: local ONNX inference, caller-provided artifacts, no implicit downloads,remote=False. GLiNER's span-based architecture also sidesteps some token-merging fragility of the current BIO pipeline.Fit
mlextra (onnxruntime + tokenizers already cover the quantized exports).🤖 Generated with Claude Code
https://claude.ai/code/session_012JiX3zWeEC28kmy5KmAXvf