Skip to content

Fix(chat): resolve response_complete event/DB inconsistency on attachments#457

Open
Jean-Regis-M wants to merge 1 commit intoGenAI-Security-Project:mainfrom
Jean-Regis-M:patch-51
Open

Fix(chat): resolve response_complete event/DB inconsistency on attachments#457
Jean-Regis-M wants to merge 1 commit intoGenAI-Security-Project:mainfrom
Jean-Regis-M:patch-51

Conversation

@Jean-Regis-M
Copy link
Copy Markdown
Contributor

Summary

Fixes #413

Resolves a data consistency bug where the response_complete event emitted to Redis
contained the original user_message, while the DB record saved effective_message
(the attachment-prefixed version). Any session with attachments produced mismatched
records between the event log and the database.


Context

stream_response constructs effective_message by prepending a FinDrive file reference
header when attachments are present:

effective_message = (
    f"[User attached FinDrive files: {file_refs}]\n\n{user_message}"
)

This effective_message is what gets:

  • passed to the LLM as actual input
  • saved to the DB via _save_message("user", effective_message)

However, the response_complete event was emitting user_message, the raw, unprefixed
original, making it diverge from every other record of what was actually processed.

The message_received event correctly and intentionally emits user_message to capture
what the user literally typed. The response_complete event should mirror what was
processed, which is effective_message.


Root Cause

# finbot/agents/chat.py  response_complete event (line 449)
event_data={
    ...
    "user_message": user_message,   # ← wrong: original, not what LLM received
    ...
}

effective_message was already in scope and correctly used everywhere else.
This was a straight omission the wrong variable was referenced.


Fix

finbot/agents/chat.py

 event_data={
     "response_length": len(full_response),
     "response_content": full_response,
     "duration_ms": duration_ms,
-    "user_message": user_message,
+    "user_message": effective_message,
     "vendor_id": self.session_context.current_vendor_id,
     "llm_model": self._model,
 },

effective_message is unconditionally assigned before this point:

  • No attachments: effective_message = user_message → identical value, zero regression
  • With attachments: effective_message carries the prefix → event now matches DB

Impact

  • Compliance audits comparing event logs to DB records will no longer find spurious
    mismatches on attachment sessions
  • Session replay and debug tooling using event logs will now reflect the exact input
    the LLM processed
  • No schema changes, no API contract changes, no behavioral change on attachment-free paths

Testing

pytest tests/integration/agents/test_chat_layer3.py::TestL3QAFindings::test_chat_l3_qa_002_event_data_logs_original_message_not_effective -v

Regression (no-attachment paths):

pytest tests/integration/agents/test_chat_layer3.py -v -k "not qa_002"

Tasks

  • Identified exact divergence point between DB write and event emission in stream_response
  • Confirmed effective_message is unconditionally in scope before the response_complete emit
  • Confirmed message_received event intentionally retains user_message left untouched
  • Verified no-attachment path is regression-safe (effective_message == user_message when attachments is falsy)
  • Single-line fix applied in response_complete event payload
  • Existing integration test test_chat_l3_qa_002 validates the corrected behaviour

Root cause:
stream_response built effective_message with attachment prefix but
passed user_message (original) to the response_complete event,
diverging from the DB record which correctly stores effective_message.

Solution:
Replace user_message with effective_message in the response_complete
event_data dict. effective_message equals user_message when no
attachments are present, so the no-attachment path is unaffected.

Impact:
Event log and DB record now consistent for all sessions.
Deterministic. Zero regression risk on attachment-free paths.

Signed-off-by: JEAN REGIS <240509606@firat.edu.tr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug_192_EVALUATE: CHAT-L3-QA-002 — Event log records original message, DB saves effective message — inconsistent on attachments

1 participant