[ibrahimmahaba]

Topic chats: find facts that only appear in the body of a topic's email threads

Problem

SearchMail only matched a thread's subject, summary and people. In a topic chat, a fact that appears
only in a message body (a date, a code, a decision) could not be found, and the assistant answered
"not found" or guessed from unrelated threads. Chats that belong to a topic also searched all of the
owner's mail instead of the topic.

What this changes

In a topic chat, SearchMail now reads the message text of the topic's own threads and returns the
passage that answers the question, with a quote, who wrote it, the date and a confidence.

1. Topic default. A chat with linked topics searches those topics by default. allTopics opts out;
   message text is then not searched.
2. Question kept whole. The owner's question is used as asked (trimmed, one line, max 500 chars).
   No words are split or dropped.
3. Read the whole topic. All messages of the topic's conversations are listed from the mailbox,
   including ones that were never imported (Graph $batch, 10 conversations per call, paged).
4. Rules first. Muted/hidden threads, excluded and never-ingest senders and folders are filtered
   before any model sees a message.
5. Classifier scores every message. COLLAB_CLASSIFIER_ENGINE_ID (a Jev / ITypeSafeEngine model)
   answers "does this message itself answer the question?" as a probability, in parallel.
6. Text model quotes the likeliest. COLLAB_LLM_ENGINE_ID reads the top 10 in batches of 4 (parallel) and
   returns relevant / confidence / quote / reason. A quote that is not in the message text is discarded.
7. One best message per thread: by judge confidence, then classifier score.
8. The result carries bodySearch.stats (mode, counts, classifiedTop, timings) and a note.

Also: topic briefs now show the thread count, the RelatedThreads agent tool is added, and the prompt tells
the agent to pass the question as asked, answer only from quoted text, and not compute dates itself.

Fallbacks

Failure	Behaviour
No classifier, no permission, or not a Jev model	Judged search with the text model
Classifier path fails (mailbox, all scores fail, timeout)	Judged search; reason in the note
Some classifier calls fail	Others used; status partial
Some conversations cannot be listed in Graph	Counted in stats.conversationsUnread; partial. If none can be read, falls back
One or more judge batches fail	Others used. If all fail, likeliest messages are shown unquoted with classifier confidence, and the note says why
Judged search fails or no text model	Word-based body search (old behaviour of this feature), noted as "matched by words only"
Everything fails	Status unavailable; the metadata results are still returned

SemossPixelException (login/permission) is always rethrown, never swallowed.

Configuration (RDF_Map.prop, all optional)

Setting	Default	Range	Meaning
COLLAB_TOPIC_SEARCH_MAX_MESSAGES	200	1-2000	Messages scored per question, newest first
COLLAB_TOPIC_SEARCH_PER_CONVERSATION	25	1-500	Messages read per conversation (paged)
COLLAB_TOPIC_SEARCH_THREADS	6	1-32	Parallel classifier/judge calls (read at startup)

Unset or non-numeric values use the default; out-of-range values are clamped.

Privacy and safety

• Only the owner's topic threads are read, and only after the same privacy check as before.
• Excluded and never-ingest content never reaches a model or the output.
• Model calls run on their own pool threads with their own Insight, so raw model JSON is never logged
  into the owner's chat room or the agent's context.
• Graph $filter values are quote-escaped.

Compatibility

• searchAll and non-topic search behave as before. The wider question-word list is applied only to the
  body search words, not to the subject/summary/people match.
• BrainMessageSource.conversationMessages is a new default method (returns null), so other sources are
  unaffected. The fixture source (COLLABORATION_FIXTURE_MAIL) implements it for offline testing.
• No schema changes.

Files

11 files, about +2,176 / -36. Main changes:

• BrainTopicJudge (new): classifier + text-model search, fallbacks, settings
• BrainTopicBodySearch (new): word-based fallback and shared context/gate/result types
• BrainThreadFinder: multi-topic search, body-search integration, QUESTION_WORDS
• BrainGraphMessageSource, BrainFixtureMessageSource, BrainMessageSource: conversationMessages
• BrainSearchThreadsReactor: topic default, allTopics, parameter description
• CollaborationPrompts, CollaborationAgentTools, BrainTopicBrief, BrainGetRelatedItemsReactor

How it was tested

Manually in the local Tomcat + UI against a fixture mailbox (~440 messages), with a topic of 40 messages whose
fact appears only in a message body and contradicts other threads. Question:
"What is the new date for the order-routing cutover".

• Found the answer in one SearchMail call, quoted the right message, and the agent answered from it.
• classify about 4s for 40 messages, judge about 5s, about 10s total.
• Not covered: no JUnit tests (by agreement), real Graph $batch and paging with bodies, topics larger
  than the 200-message cap, and the never-ingest sender test on a real mailbox.

Known limitations

• Only the newest COLLAB_TOPIC_SEARCH_MAX_MESSAGES messages of a topic are classified.
• Graph does not guarantee message order inside a conversation when a conversationId filter is used, so
  the per-conversation limit is applied to an unspecified subset when a conversation is longer.
• The Jev classifier separates relevant from irrelevant messages only weakly; the text model makes the
  final decision.
• Messages in conversations not yet linked to the topic are not searched.

Reviewer notes

• Needs H2 2.x databases (dev already requires this).
• Please verify on a real mailbox: a conversation of more than 25 messages, and a topic of more than 200.

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.