Repository navigation
fix(chat): rejection messages name the available views/functions so the model self-corrects - #27
Merged
Conversation
…model self-corrects Live QA after PR #26 found a real, separate bug: asked "What are the transactions available in this account?", the model queried `SELECT * FROM transactions` (a plausible-sounding guess matching the system prompt's own topic word "transactions") instead of the real view `v_transaction_history`, got validator_rejected, and gave up rather than retrying -- answering "I cannot access transaction details" even though the correct view was available and readable the whole time. Confirmed via information_schema.role_table_grants and a live query through trueup_chat_readonly against real data that this was never a permission/grant issue -- the DB layer works correctly. Confirmed from the actual chat_tool_call row for the live session that the rejection message did reach the model as a tool result, but named only what was rejected, never what was available, giving it nothing to correct toward. Fixed by having both sql_tool_validator rejection paths (off-allow-list relation, disallowed function) name the actual allow-list contents in the same tool result, plus a new system-prompt rule instructing the model to retry once with a corrected name before declining. Verified live against the restarted backend: the same question now correctly resolves to `SELECT * FROM v_transaction_history` on the first or corrected attempt. Also confirms a second finding from the same report: "what model portfolio is chosen" is not a bug -- no curated view covers model assignment, so declining is FR-54's mandated behavior, not a defect.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Live testing after PR #26 (which fixed the compound-WHERE and customer_id-hallucination bugs)
found chat still failing a real question: "What are the transactions available in this
account?" answered with "I cannot access transaction details at the moment."
Diagnosis (verified live, not guessed)
Ruled out permission/grant issues first, since that was the reporter's own hypothesis:
queried
v_transaction_historydirectly as thetrueup_chat_readonlyrole, withapp.role/app.customer_idset exactly as a real session does, against a real customer with realtransaction data — returns rows correctly, no permission error.
information_schema.role_table_grantsconfirms
SELECTon all 8 curated views. No permission change was needed.Root cause, from the actual
chat_tool_callrow of the live (post-restart) session:The model guessed the plain English word "transactions" (matching the system prompt's own topic
list) as a table name instead of the real view
v_transaction_history. The validator correctlyrejected it — traced the rejection message through
openai_agent_adapter.pyand confirmed itreally does reach the model as a tool result — but the message named only what was rejected, never
what was actually available, and the model gave up rather than retrying.
A second question in the same report ("what model portfolio is chosen") is not a bug — no
curated view covers model-portfolio assignment at all (S8/S9-domain data, never in S11's 8-view
allow-list), so declining is FR-54's mandated behavior. Confirmed still declining correctly (with
clearer wording) after this fix.
Fix
sql_tool_validator.py: both rejection paths (off-allow-list relation, disallowed function) nowname the actual allow-list contents in the same tool result, so the very next thing the model sees
already contains the correction.
chat_orchestration_service.py: one new system-prompt ruleinstructing the model to retry once with a corrected name from the rejection message before
declining.
Validation evidence
(
test_off_allow_list_relation_reason_names_the_available_views,test_disallowed_function_reason_names_the_available_functions).uv run pytest tests/unit/test_sql_tool_validator.py tests/integration/test_chat_security_perimeter.py tests/api/test_chat.py— 60 passed.uv run ruff check,uv run mypy --strict app,uv run lint-imports— clean, zero new baseline noise.customer, and re-asked the exact failing question through the real chat API. The
chat_tool_callrow now shows
SELECT * FROM v_transaction_historysucceeding on the first attempt, and theassistant correctly answers "There are currently no transactions available in this account"
(an honest answer for a test account with no transactions, not a false "I cannot access").
Security / DB / API impact
No migration, no grant/permission change, no new curated view, no new endpoint. Strictly improves
error-message content inside the existing ADR 19 safety perimeter.