You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After the “rewrite LLM forecaster” refactor (6566923, 2026-06-26), metadata jobs call gpt-5-mini with low max_output_tokens and no reasoning.effort. Since the model defaults to medium reasoning, every request exhausts the budget and fails with status=incomplete, reason=max_output_tokens. After retries, tag_questions / validate_questions fall back to category="Other" and valid_question=True, silently mislabeling new questions.
A Kalshi tagging run made 120 requests; 0 succeeded, then hit the 30-minute timeout.
Questions processed through this path since 2026-06-26 may be contaminated.
Fix: Set reasoning={"effort": "minimal"} and raise the tagging/validation output budgets.
After the “rewrite LLM forecaster” refactor (
6566923, 2026-06-26), metadata jobs callgpt-5-miniwith lowmax_output_tokensand noreasoning.effort. Since the model defaults to medium reasoning, every request exhausts the budget and fails withstatus=incomplete, reason=max_output_tokens. After retries,tag_questions/validate_questionsfall back tocategory="Other"andvalid_question=True, silently mislabeling new questions.A Kalshi tagging run made 120 requests; 0 succeeded, then hit the 30-minute timeout.
Questions processed through this path since 2026-06-26 may be contaminated.
Fix: Set
reasoning={"effort": "minimal"}and raise the tagging/validation output budgets.