Repository navigation
fix(agent): draft screening answers in chunks under the worker's own deadline (#410) — v1.68.7 - #461
Merged
Merged
Conversation
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 19 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
…deadline (#410) A packet's whole answer draft rode one chat completion under the api's Llm__TimeoutSeconds (120). Orbis's 22 model questions stalled at the endpoint twice, each time for the full deadline, and every answer was voided; the same prompt answered in 7 s on the next run. AnswerDrafter now sends the model's questions eight per completion. A completion that stalls or errors voids only its own chunk's answers and the chunks already drafted are kept. Two failed completions in a row stop the draft and flag the remaining questions for the person instead of waiting out another deadline per chunk. The worker's model calls run under Agent__LlmTimeoutSeconds (default 300; 0 inherits Llm__TimeoutSeconds). AgentWorker.WorkerLlmOptions copies the instance endpoint, model, key and cap under that deadline; the api's singleton keeps its request-sized 120 s. No max_tokens on the drafting call: a thinking model that writes its scratchpad into content would be cut off before the JSON, and StructuredCompleter already tolerates that scratchpad. Tests: 20 questions go out as 8/8/4 and all are kept; a stalled middle chunk voids four answers while the other eight are drafted and the third chunk is still attempted; two failures in a row stop the draft with the first chunk's work kept; WorkerLlmOptions keeps the endpoint and takes the worker's deadline, 0 inherits. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qGG2ojswVKiaTDH5SMZF
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0124qGG2ojswVKiaTDH5SMZF
CryptoJones
force-pushed
the
fix/410-chunked-drafting
branch
from
October 31, 2026 10:55
695b244 to
6ed6592
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #410
What
A packet's whole answer draft rode one chat completion under the api's
Llm__TimeoutSeconds(120). Orbis's 22 model questions stalled at the endpoint twice, each time for the full deadline, and every answer was recorded asno draft; the same prompt answered in 7 s on the third run. Two changes, both from the issue's own fix list:AnswerDraftersends the model's questions eight per completion (DefaultQuestionsPerCall). A completion that stalls or errors voids only its own chunk's answers; the chunks already drafted are kept and the later chunks are still attempted. Two failed completions in a row (GiveUpAfterConsecutiveFailures) stop the draft and flag the remaining questionsno draft: not attempted after 2 failed completions in a row (…)instead of waiting out another deadline per chunk. The 25-question cap and the per-question review reasons are unchanged.Agent__LlmTimeoutSeconds(default 300;0inheritsLlm__TimeoutSeconds).AgentWorker.WorkerLlmOptionscopies the instance endpoint, model, key and daily cap under that deadline; the api'sLlmOptionssingleton keeps its request-sized 120 s. Passed through in both compose files asAGENT_LLM_TIMEOUT, documented in the README env table and "Running it",.env.production.exampleand the quadletagent.env. pluto'sLlm__TimeoutSeconds=600override on the agent can come off once this is deployed.Not done, deliberately: no
max_tokenson the drafting call. A thinking model that writes its scratchpad intocontent(thepoolside/laguna-s-2.1case) would be cut off before the JSON, andStructuredCompleteralready scans past that scratchpad. Capping tokens would turn a slow success into a guaranteed failure.Tests
AnswerDrafterTests:Screening_questions_go_to_the_model_in_chunks_and_every_chunk_is_kept: 20 questions, chunk size 8 → 3 calls carrying 8/8/4, every id asked exactly once in order, 20 answers, nothing flagged.One_stalled_chunk_voids_only_its_own_answers_and_the_rest_of_the_packet_is_drafted: the middle chunk of three throwsLlmUnavailableException→ 8 answers kept, exactly q5–q8 flaggedno draft: could not reach the LLM endpoint…, and the third chunk was still sent.Two_failed_chunks_in_a_row_stop_the_draft_without_spending_another_timeout_per_chunk: chunks 2 and 3 fail → chunk 4 is never sent; the first chunk's 4 answers are kept; 8 questions carry the endpoint's error and 4 carry the "not attempted" reason.AgentLlmTimeoutTests(new): the worker copy keeps endpoint/model/key/cap and takes the worker's deadline, the api singleton is untouched,EffectiveLlmConfig.Resolvecarries it through;0and an equal value return the same instance; the default is longer than a request deadline.Run locally in a clean copy of
api/(so as not to disturb a fulldotnet testalready running for #460):AnswerDrafterTests+AgentLlmTimeoutTests63 passed;AgentWorkerTests+PacketBuilder*+AnswerBankTests+StructuredCompleter*76 passed. CI runs the full suite.Release
Two commits per the convention: the fix, then
chore: prepare v1.68.7 release(csproj, pyproject,__init__.py,uv.lock,.env.production.example, BACKLOG). Merge with a merge commit, then tagv1.68.7.Heads-up: PR #458 carries a v1.68.6 bump. If this lands first, #458's bump should move to 1.68.8 when it is rebased.
🤖 Generated with Claude Code
https://claude.ai/code/session_0124qGG2ojswVKiaTDH5SMZF
Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/