Repository navigation
Fix #1: let Ollama thinking models pass the local readiness probe - #17
Open
xiaoyao-666-coder wants to merge 2 commits into
Open
xiaoyao-666-coder wants to merge 2 commits into
xiaoyao-666-coder wants to merge 2 commits into
Conversation
…ess probe Add MEDGUIDE_LOCAL_LLM_PROVIDER (llama.cpp by default, ollama opt-in). With ollama, _local_request_options sends reasoning_effort='none' instead of the llama.cpp-only chat_template_kwargs.enable_thinking flag, covering both the readiness probe and generate requests; the flag previously left reasoning on, so a 16-token probe was consumed by reasoning and returned empty content, marking a usable model unavailable. Invalid values fail fast at startup. Docs updated. Closes zhengdaodeguL#1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1.
MEDGUIDE_MODE=local 下连接 Ollama 时,
_local_request_options()固定发送 llama.cpp 专用的chat_template_kwargs.enable_thinking=false,在 Ollama + gemma4:e2b 上没有关闭思考,16 token 的 readiness probe 全部用于 reasoning,message.content为空,模型被判为不可用。MEDGUIDE_LOCAL_LLM_PROVIDER:默认llama.cpp(保持现有行为);设为ollama时 probe 与正式生成请求都改发reasoning_effort="none"。非法值在构造时即抛 ValueError,不静默用错配置;非 local 模式下该变量被忽略。验证:
MEDGUIDE_MODE=test python -m pytest -q backend/tests529 passed(基线 521 + 8 新);前端未改动。说明:本机没有 Ollama,
reasoning_effort="none"能让 gemma4:e2b 返回 content "OK" 是 issue 报告人的实测结论,本次只验证了发出的请求正文正确,未在真实 Ollama 0.33.1 上跑过 probe。