Oversized message bodies are quarantined for the process lifetime and the limitation is unregistered

[tabmail-kmyi]

Current status — 2026-09-09

The stop-gap shipped through #105; the full fix in draft #103 remains parked. SwiftMail #215/#216/#219 landed through #226/#230/#228, but the separate metadata parser-limit train still waits for a released swift-nio-imap containing #849, followed by SwiftMail adoption and effective limit wiring. Updating the current SwiftMail pin is separate housekeeping, not completion of this issue.

Do not reserve a migration number now. On resume, choose the next free number from then-current main and update the tests and foreign-key-mode census together. No migration code changes, database resets, or migration execution are part of this pause.

The original report below is retained as historical evidence. Its attribution to an effective 4 MiB decoder limit was superseded by the investigation: on the inspected dependency version, the client decoder uses an 8,192-byte unread-buffer bound, and overflow depends on fragmentation. Partial body fetching does not remove this separate metadata limit.

Original report

A message whose IMAP FETCH response exceeds the SwiftMail fork's 4 MB NIO response-buffer limit fails with PayloadTooLargeError. BackfillBodyQueue.handlePayloadTooLarge handles the single-item case correctly and deliberately:

• it does not set bodyEmptyConfirmed — an oversized body is the opposite of "content confirmed gone", so Data Integrity rule 1 is respected and the row stays honestly bodyComplete = 0;
• it inserts the headerId into the process-lifetime set oversizedDeferredThisSession and calls storage.removeFromQueue(item);
• admit never re-admits an id in that set.

The consequence is a real, permanent-for-the-session user-visible limitation: that message never gets a body on this device until the app is restarted, it is absent from full-text search body content, and BackfillProgress.pendingBodyCount never reaches 0 for the account (which is why FastSyncView deliberately withholds the "Sync Complete" label — see the comments on keepScreenAwakeWhileWorking).

This behaviour is documented thoroughly in code comments (BackfillBodyQueue.handlePayloadTooLarge, ActiveBodyQueue.handlePayloadTooLarge, FastSyncView) but is not registered anywhere in the issue register. A search of Companion/Process/Current/KnownIssues/ and KNOWN_ISSUES.md for oversized, PayloadTooLarge, oversizedDeferredThisSession and Single item too large returns no entry. IOS-IMAP-006 mentions an oversized FETCH only in passing, about a different defect (post-COPY duplication).

Asking for a register entry (or an issue that stands in for one) so the limitation is discoverable from outside the source.

Evidence, and one thing this capture could not settle

A 1.7.12 device session shows the condition firing on a large-mailbox IMAP account:

[BackfillBody] Dispatching 2 items in 2 folder groups (deferred=0, activeBatches=0)
[BackfillBody] Batch START: 1 items in <dated archive folder A>
[BackfillBody] Batch START: 1 items in <dated archive folder B>
...
error … command FetchMessageInfoCommand<UID> [A005] failed: PayloadTooLargeError(); channelActive=true authenticated=true
error … command FetchMessageInfoCommand<UID> [A005] failed: PayloadTooLargeError(); channelActive=true authenticated=true
error … command LogoutCommand [A006] failed: PayloadTooLargeError()

Both batches were single-item, so both should have reached the items.count == 1 arm and logged [BackfillBody] Single item too large in … — deferring …. Neither that line, nor [BackfillBody] Batch DONE, nor [BackfillBody] Batch FAILED, nor [IMAP] fetchMessagesBatch START appears anywhere in the remaining 16 seconds of the capture.

I could not determine from this log whether the batches were quarantined as designed or left unresolved, and I am not claiming a wedge. The most likely explanation is console-capture loss rather than a code problem, on three grounds:

1. The capture contains six Logging Error: Failed to receive N log messages lines, so it is demonstrably lossy.
2. [IMAP] fetchMessagesBatch START is gated by the same DebugModeManager.isLoggingEnabled() flag as [IMAP] Created pinned connection …, which does appear throughout — so its absence is not a gating explanation, it is a dropped line.
3. withFolderConnection's unhealthy-release path does not await the logout (Task { try? await server.logout() } is fire-and-forget), so the throw is not delayed by teardown and the catch should have run promptly.

A capture taken via the downloadable boot log rather than the live console would settle it, since handlePayloadTooLarge also writes through BackgroundSyncLogger.logBackfill.

Baseline comparison: the identical PayloadTooLargeError on the same command shape appears in a 2026-08-12 device capture in this repo, there accompanied by its handler line ([BackfillBody] PayloadTooLarge for … — isolating 2 items for single-item testing). So the condition itself is pre-existing and the handler demonstrably runs; only this capture's disposition is unconfirmed.

Questions this entry should answer once written

• Is a restart genuinely the only release? oversizedDeferredThisSession is documented as having "three releases" — those should be named in the register entry, not only in a code comment.
• Should the user see anything? Today the message simply has no body and no explanation.
• Is 4 MB the right buffer size, or should the oversized case fall back to a part-by-part fetch rather than a bulk BODYSTRUCTURE?

References

• TabMail/Services/Sync/BackfillBodyQueue.swift — handlePayloadTooLarge, admit, oversizedDeferredThisSession, dispatchBatch's catch
• TabMail/Services/Sync/ActiveBodyQueue.swift — handlePayloadTooLarge
• TabMail/Providers/IMAPProvider.swift — fetchMessagesBatch, withFolderConnection, releaseFolderConnection
• TabMail/Views/Settings/FastSyncView.swift — keepScreenAwakeWhileWorking and the isAllComplete truth-claim comment

[tabmail-kmyi]

Reopening: this was auto-closed by GitHub's keyword parser matching #74 in PR #105's merge commit message ("Closes #104. #74 remains the tracker..."), despite that sentence saying explicitly it should stay open. The real fix (upstream parser bound raise) is still tracked here and is still unmerged as draft PR #103. No code or scope change — administrative correction only.

[tabmail-kmyi]

Re-triaged 2026-09-03 under the scale-up lens (the IMAP IDLE droplet fleet grows from one droplet to many; rotations and deploys become frequent): priority: P3 → priority: P2.

Two prerequisites now sit in the scale-out's path. Draft PR #103 registers v88_addBodyIndexingFailureReason while main already has v88_addBodyMetadataOversized (PR #105), so it must renumber to v89 and rebase per PR #105's stated order. And the 4 MiB bound is first-party — IMAPFetchMapping.responseBufferLimit — so the fix is a constant raise plus the sweep migration, not an upstream wait; the body's "exceeds the SwiftMail fork's limit" premise is stale.

[tabmail-kmyi]

Re-triaged under the new P0–P5 priority scheme (2026-09-04): P2 → P3 — oversized bodies are durably flagged and sync completes (PR #105); the remaining gap is non-ideal but acceptable. PR #103 still has to renumber its migration to v89 and rebase.

[tabmail-kmyi]

Housekeeping update (2026-09-09): keep this train parked. SwiftMail #215/#216/#219 were incorporated through merged #226/#230/#228. Apple swift-nio-imap #849 now exposes the required client decoder buffer parameter, but no containing release/tag is available yet. SwiftMail must adopt that release and wire the limit through before the oversized-metadata train resumes. Updating the current SwiftMail fork/app pin is separate from completing this fix.

Migration instruction: do not reserve a number now and do not change or run the migration during this pause. On resume, choose the next free number from the then-current main, update the migration tests and foreign-key-mode census, and address development databases that ran the old draft before launching it. This supersedes earlier comments prescribing a particular replacement number.

No rebase, database reset, migration execution, or issue closure is part of this housekeeping update.