feat: add account-local conversation engine and native conversation reader

[Dragonk]

Summary

This PR adds an account-local Conversation Engine and a native conversation reader while retaining provider-native thread membership as the authoritative source for visible Reader cards.

Development history was consolidated into five logical implementation commits plus one durable CI-validation commit; the original integration branch remains preserved.

Why

MailFlow needs stable conversation identity without allowing messages from different managed accounts to merge. This change addresses:

• duplicate physical copies across folders;
• stable logical-message identity;
• account-local RFC Message-ID, provider-ID, reference, and parent resolution;
• optional native grouped list and Conversation Reader surfaces;
• shared, safe message-content rendering across readers;
• large-mailbox performance without eager per-row native-thread detail fetches.

Architecture

Conversation
└── LogicalMessage
    └── PhysicalCopy[]

• A folder belongs to a PhysicalCopy.
• Conversation and LogicalMessage identity is per email account, within the owning user boundary.
• Provider IDs and mappings are account-scoped.
• The Conversation Engine supplies persistence, stable identity, enrichment, evidence, rebuild, aliases, and manual operations.
• Native provider thread membership remains authoritative for visible Reader membership; stale CE-only cards are not appended to the UI.

User-facing behavior

The Grouping and Reader preferences are independent:

Grouping	Reader	Behavior
Off	Off	Flat native list + classic single MessagePane
On	Off	Native grouped list + same classic MessagePane
Off	On	Flat list + native-thread Conversation Reader for the selected physical message
On	On	Native grouped list + the same native-thread Conversation Reader

Grouping changes the list only; Reader changes the right pane only.

Desktop parent rows expand/collapse and navigate to the newest target. On mobile, multi-message parents expand/collapse only, singleton rows open directly, and child rows navigate to the exact physical message. Parent gestures act on thread scope; child gestures act on message scope. Opening a target marks only that copy read, preserving unread sibling state.

Mail rendering

• Shared MessageDetailContent keeps classic MessagePane and Conversation Reader content consistent.
• Attachments are rendered before the body.
• Context-menu behavior targets the correct physical copy in both surfaces.
• Safe email HTML/CSS, responsive layout behavior, remote-image privacy controls, and CSS background-image support are retained.

Performance

The regression coverage includes the release-blocking browser guard:

• 500 grouped rows issue 0 eager /api/mail/thread detail requests during initial render;
• singleton detail resolution remains lazy and occurs only on interaction;
• Reader target alignment uses bounded requestAnimationFrame/scroll work and does not continuously resnap after manual user scrolling.

The PostgreSQL CI gate additionally runs the repository’s 10k, 50k, and 100k scale/EXPLAIN harness.

Database / migrations

The PR adds migrations 0051–0062 after current upstream migration 0050:

• Conversation Engine tables, operations, evidence, and repair paths;
• tenant and account-local constraints;
• rebuild auditing and automated-series support;
• account-scoped logical-message and conversation identity;
• composite foreign keys, provider mapping constraints, and indexes.

CI validates both a fresh migration through 0062_conversation_account_identity and a real upgrade: it builds the actual current upstream main schema through 0050_message_sender, verifies it, then applies this PR’s migration runner through 0062 against the same PostgreSQL database. Populated legacy and 0061→0062 account-identity fixtures are also required.

Testing

GitHub Actions on GitHub-hosted Ubuntu runners validates:

• backend unit suite, lint, and plugin-boundary lint with PostgreSQL and Redis available;
• fresh migration, real upstream-0050 upgrade, populated migration fixtures, PostgreSQL integration/regression suites, and scale harness;
• frontend unit suite, lint, and production build;
• mocked Chromium browser suites on desktop, 390×844 mobile, and Pixel 7 412×915 mobile;
• real-app Playwright against isolated CI PostgreSQL, Redis, backend, and seeded test data;
• Playwright reports, traces, screenshots, and videos on failure.

Exact run results are recorded by the required GitHub checks for the final PR SHA.

Real-device UAT

Manual UAT previously passed on desktop browser and mobile device/browser. It covered:

• grouped/flat list and Reader 2×2 semantics;
• middle, early, and terminal target navigation/scroll alignment with the action toolbar visible;
• no post-user-scroll resnap;
• singleton mobile navigation and mobile parent expand/collapse;
• parent thread swipe and child message swipe;
• target-only read while unread siblings remain unread;
• HTML/newsletter rendering, attachments, and context menu parity;
• reply → Sent → IMAP sync with correct native-thread count/direction and no duplicate/ghost cards.

This manual UAT is distinct from the automated CI coverage above.

Screenshots

Desktop

Grouped conversation list + Conversation Reader

Expanded 1 message

##### Expanded 2 messages

New setting

Mobile

Expanded thread list

Conversation Reader

Reviewer notes

Please focus review on:

1. account-local identity and migration constraints;
2. native membership as the UI source versus CE persistence/enrichment;
3. shared message-content rendering and mail-content safety;
4. mobile/desktop interaction and read-state semantics;
5. lazy native resolution, bounded reader scrolling, and the 500-row performance guard.

[maathimself]

I'm going to close this rather than merge it, for two reasons:

• Conversation and thread identity is core product direction, not an implementation detail. CONTRIBUTING asks for an issue before anything larger than a focused fix, precisely so I can say early whether something fits. This landed as ~27k lines across 166 files with no prior discussion. That's the case the document is written for. The code itself is good; that isn't what decided this.
• Merging transfers the whole maintenance tail to me permanently: twelve sequential migrations, three new CI workflows, and roughly seven thousand lines of new services. That's a long-term commitment I have to be able to hold on my own.

Thank you.

[Dragonk]

Thank you for the clear explanation. I understand that the decision is about product direction and the long-term maintenance burden, not the quality of the implementation. I also acknowledge that I should have opened an issue before starting a change of this scope.

For context, this is not only an experimental branch anymore: my partner and I are already using this implementation in production, and the account-local conversation model plus the native reader have solved real threading problems for our mix of mail accounts.

Before I treat a maintained fork as the permanent path, would you be open to a separate, implementation-free discussion about the product direction for conversations?

In particular, it would help to know whether:

1. a Gmail-style conversation reader and account-local thread identity are capabilities you want MailFlow to have eventually;
2. you would consider a much smaller, maintainer-defined sequence of changes after agreeing on the architecture and boundaries first; or
3. this direction is intentionally out of scope for upstream, in which case I will continue it in my fork and avoid creating unnecessary follow-up proposals.

I am not asking you to reopen or merge this PR. I would just like to establish whether there is any acceptable upstream path before investing further in splitting or simplifying the work.

Thanks again for taking the time to review the scope and explain the decision.