Problem
The OCR engine lineup no longer reflects measured quality/cost on the owner's hardware and born-digital academic workload. Several engines are mis-ranked or dead weight:
- DeepSeek-OCR scores ~0.085 (socOCRbench, near dead last) yet sits in
AUTO_ENGINE_ORDER and _LOCAL_ENGINE_ORDER.
- Mistral OCR (~0.45) is ranked as the premium rung above Gemini, but it is worse AND ~5x pricier than Gemini — strictly dominated.
- Figure description is hardwired to the Gemini API with no local-first path.
- judge / math-recovery defaults lean on slow/timeout-prone local models instead of the practical winner
qwen3.5:cloud.
Policy
Best cheap combination, local-first → Ollama Cloud → Gemini/Mistral edge-case only. Native PyMuPDF stays the real default for born-digital text and table values.
Measured winners (owner's 64GB Mac, real workload)
qwen3.5:cloud (Ollama Cloud, no extra key): ~0.57, ~49s/page, only engine to clear all 3 hard page types → practical workhorse VLM.
- Gemini 3.x: best quality (0.60-0.64) but paid + occasional empty pages → edge escalation only.
- DeepSeek-OCR / Mistral: drop from auto ladders.
Scope (this issue)
- New
AUTO_ENGINE_ORDER: QWEN → GEMINI → MARKER → GLM → NOUGAT (drop DeepSeek + Mistral from auto).
- Demote Mistral to manual-only fallback; drop DeepSeek from
_LOCAL_ENGINE_ORDER.
- Flip
judge_model / math_model defaults to qwen3.5:cloud.
- Add an LLM-friendly
MODELS.md documenting the routing policy per sub-task (text/formula, figure description, table layout).
Follow-ups (separate)
- Local-first figure description in
gemini_api.py.
- Split provider identity by engine+backend+model (QWEN local vs cloud) + fix
QwenEngine.is_available() cloud gating.
Design second opinion: Codex (gpt-5.5) — agreed with the lineup above.
Problem
The OCR engine lineup no longer reflects measured quality/cost on the owner's hardware and born-digital academic workload. Several engines are mis-ranked or dead weight:
AUTO_ENGINE_ORDERand_LOCAL_ENGINE_ORDER.qwen3.5:cloud.Policy
Best cheap combination, local-first → Ollama Cloud → Gemini/Mistral edge-case only. Native PyMuPDF stays the real default for born-digital text and table values.
Measured winners (owner's 64GB Mac, real workload)
qwen3.5:cloud(Ollama Cloud, no extra key): ~0.57, ~49s/page, only engine to clear all 3 hard page types → practical workhorse VLM.Scope (this issue)
AUTO_ENGINE_ORDER: QWEN → GEMINI → MARKER → GLM → NOUGAT (drop DeepSeek + Mistral from auto)._LOCAL_ENGINE_ORDER.judge_model/math_modeldefaults toqwen3.5:cloud.MODELS.mddocumenting the routing policy per sub-task (text/formula, figure description, table layout).Follow-ups (separate)
gemini_api.py.QwenEngine.is_available()cloud gating.Design second opinion: Codex (gpt-5.5) — agreed with the lineup above.