feat(tools/maildir): land the local-archive read side

tools/mail-source/contract.md

@@ -7,6 +7,7 @@
 
 - [Mail source — backend contract](#mail-source--backend-contract)
   - [Abstract operations](#abstract-operations)
+    - [Thread identity](#thread-identity)
     - [Security draft CC resolution](#security-draft-cc-resolution)
   - [Capability matrix](#capability-matrix)
   - [Adopter declaration in `<project-config>/project.md`](#adopter-declaration-in-project-configprojectmd)
@@ -65,6 +66,39 @@ resolution chain for that op.
 A backend's capability set is its supported subset of the operations
 above. The capability matrix below summarises the in-tree adapters.
 
+### Thread identity
+
+Every backend that declares `thread_id_kind: rfc5322-message-id` emits the
+root's `Message-ID` in **one spelling: the bracketed form**,
+`<report@example.org>`, angle brackets included. The brackets are part of
+the identifier, not decoration. The skills match a stored id against a
+freshly read one to answer "is this thread already tracked?", and
+`report@example.org` and `<report@example.org>` are two different strings
+to that comparison: an adopter who switches backends, or configures one as
+a fallback for the other, would miss the match and get a duplicate tracker
+proposed. An id typed without brackets is accepted on input and normalised
+to the bracketed form; it is never emitted.
+
+The root is the first id in the message's `References` header, **even when
+that message is not in the archive**. A snapshot truncated at the start of
+a thread therefore keeps the replies it does hold in one thread, keyed on
+the root they name, instead of turning each of them into its own thread.
+A message whose `References` header is absent is keyed on its first
+`In-Reply-To` id, and a message with neither is its own root.
+The `imap` backend does not apply the `In-Reply-To` step yet
+(`imap_source.searching.thread_root_of` falls back to the message's own id),
+so a reply carrying only `In-Reply-To` is still its own thread there. Only
+bracketed tokens count: a comment beside the id (`In-Reply-To: (was:
+report) <r@h>`, which RFC 5322 allows) is not an ancestor, or the thread
+would be keyed on an id that never existed.
+
+A message with no `Message-ID` at all cannot be threaded. A backend must
+still surface it rather than drop it, and must not mislabel it: `maildir`
+reports `thread_id_kind: content-hash` with a `urn:sha256:` id derived from
+the message bytes — which, unlike an mbox line offset, survives a fresh
+snapshot of the same archive. That id must round-trip: whatever
+`list_recent_threads` returns, `read_thread` accepts.
+
 ### Security draft CC resolution
 
 Before proposing a security-related draft, resolve its security CC once

tools/mail-source/mbox/README.md

@@ -5,25 +5,26 @@
 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 **Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*
 
-- [Mail-source adapter — mbox / local archive (stub)](#mail-source-adapter--mbox--local-archive-stub)
+- [Mail-source adapter — mbox / local archive](#mail-source-adapter--mbox--local-archive)
   - [When this adapter makes sense](#when-this-adapter-makes-sense)
   - [Capability claim](#capability-claim)
   - [Auth + setup](#auth--setup)
   - [What an adopter declares in `project.md`](#what-an-adopter-declares-in-projectmd)
-  - [Why this is a stub](#why-this-is-a-stub)
+  - [Why this shipped before its first adopter](#why-this-shipped-before-its-first-adopter)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
 <!-- SPDX-License-Identifier: Apache-2.0
      https://www.apache.org/licenses/LICENSE-2.0 -->
 
-# Mail-source adapter — mbox / local archive (stub)
+# Mail-source adapter — mbox / local archive
 
 Reference adapter for a static `mbox` (or `Maildir`, or a directory
 of `.eml` files) as a read-only backend for the
-`security-issue-import` family of skills. **Stub status** — this
-document describes the contract; the concrete CLI wiring lands when
-an adopter actually wires a local archive in.
+`security-issue-import` family of skills. **Implemented** — the
+concrete reader and its three operations live in
+[`tools/maildir/`](../../maildir/README.md#contractmail-source--local-archive-reads),
+the vendor home for this local-files backend.
 
 See [`../contract.md`](../contract.md) for the abstract
 mail-source-backend operations + capability matrix + adopter
@@ -55,13 +56,13 @@ directions, the way `tools/gmail` does for Google.
 
 | Operation | Supported? | Notes |
 |---|:---:|---|
-| `list_recent_threads(list, since)` | ✓ (offline) | Parse the mbox, group messages by `References:` / `In-Reply-To:` chain, filter by message `Date:` newer than `since`. The `list` parameter is *advisory* — the mbox is whatever the operator pointed the adapter at, no remote subscription is checked |
+| `list_recent_threads(list, since)` | ✓ (offline) | Parse the mbox, group messages under the thread root the `References:` / `In-Reply-To:` chain names (first ID wins, whether or not that message is in the archive), filter by message `Date:` newer than `since`. The `list` parameter is *advisory* — the mbox is whatever the operator pointed the adapter at, no remote subscription is checked |
 | `read_thread(thread_id)` | ✓ | Same parser; return all messages in the thread |
 | `list_drafts(thread_id)` | ✗ | A static archive has no concept of pending drafts |
 | `list_sent_since(thread_id, since)` | ✗ (or limited) | Only if the archive includes the team's outbound mail (some exports do; many don't). The adapter declares this op as `unsupported` by default; an adopter with an outbound-included archive may upgrade the claim per-deployment |
 | `create_draft(thread_id, body, …)` | ✗ | Read-only by construction |
 | `thread_url(thread_id)` | depends | If the project has a public archive (PonyMail, Pipermail), construct the URL from the thread root's Message-ID. Otherwise return `file://<archive-path>#<message-id>` which only works in the operator's local environment |
-| `thread_id_kind` | `rfc5322-message-id` | Same as IMAP — root Message-ID is the stable identifier |
+| `thread_id_kind` | `rfc5322-message-id` | Same as IMAP, and in the same bracketed form (`<root@host>`) — the root Message-ID is the stable identifier. A message with no `Message-ID` is keyed by a hash of its bytes instead and reported under the `content-hash` kind |
 
 ## Auth + setup
 
@@ -97,9 +98,12 @@ declares:
 | `mbox`  | fallback | no  | 2024-Q4 snapshot at `/srv/snapshots/security-list-2024-Q4.mbox`; used when Gmail history is incomplete |
 ```
 
-## Why this is a stub
+## Why this shipped before its first adopter
 
-No adopter is currently using it. The stub documents the contract
-shape so a forensics / compliance team can wire the adapter
-without re-designing the read interface; concrete parsing code
-lands alongside this README when the first such adopter materialises.
+No adopter is using it yet. The contract shape is small and stable
+enough — three read operations over a file the adopter already has —
+that wiring it now costs less than re-deriving the read interface
+from a forensics / compliance team's requirements later. The reader
+is deliberately whole-archive and stateless: these are snapshots
+measured in tens of megabytes, not live mailboxes, so an index would
+be a second thing to keep consistent for no gain.

tools/maildir/README.md

@@ -42,13 +42,11 @@ tool bundles the same two capabilities behind the *Maildir* vendor —
   shows it in its Drafts view for the human to review, edit, and send.
   **Fully implemented here** (see [Operations](#operations)).
 - **`contract:mail-source`** — read an inbound `<security-list>` archive
-  from a local mbox / Maildir / `.eml` directory. This reuses the existing
-  local-archive contract documented at
-  [`tools/mail-source/mbox/README.md`](../mail-source/mbox/README.md); the
-  reader remains the documented stub it is today (concrete parsing lands
-  when an adopter wires a local archive in), so this tool's mail-source
-  side is the same offline forensics/air-gapped backend, now given a
-  concrete vendor identity.
+  from a local mbox / Maildir / `.eml` directory. **Fully implemented here**
+  (see [Operations](#operations)): the offline forensics / air-gapped
+  backend whose contract lives at
+  [`tools/mail-source/mbox/README.md`](../mail-source/mbox/README.md), now
+  with a concrete vendor home.
 
 Adding this backend is what flips `contract:mail-create` vendor-neutral: it
 is the second, non-Google implementation of the outbound-mail contract
@@ -143,10 +141,38 @@ A draft matching any criterion is listed; with none, every draft is.
 
 Read-only ingestion of a local mbox / Maildir / `.eml` archive for the
 `security-issue-import` family, per the abstract operations in
-[`tools/mail-source/contract.md`](../mail-source/contract.md). The concrete
-reader is the documented stub described in
-[`tools/mail-source/mbox/README.md`](../mail-source/mbox/README.md); this
-tool is its vendor home.
+[`tools/mail-source/contract.md`](../mail-source/contract.md). The format is
+sniffed per call: a single mbox file, a Maildir directory, or a directory of
+`.eml` files.
+
+| Operation | Command | Notes |
+|---|---|---|
+| `list_recent_threads` | `maildir-source-threads --archive <path> [--since-days N]` | A thread is kept whole when any of its messages falls inside the window, so the opening report stays visible while the thread is active |
+| `read_thread` | `maildir-source-read --archive <path> --thread-id <root Message-ID>` | Every message oldest-first with its plain-text body; exits 2 when the ID is not in the archive |
+| `thread_url` | `maildir-source-thread-url --thread-id <id> [--archive-template <url>] [--archive <path>]` | `--archive-template` takes a public-archive URL with `{message_id}`, percent-encoded on substitution; without it the URL is a `file://` link to the snapshot |
+
+Both archive-taking commands exit 1 when the path does not exist, and a bad
+flag (`--since-days` below zero, neither `--archive-template` nor `--archive`)
+is an argparse usage error — exit 2 — rather than a traceback.
+
+`thread_id` is the RFC-5322 `Message-ID` of the thread root **in the bracketed
+form IMAP emits** — `<root@example.org>`, the `rfc5322-message-id` kind this
+backend declares — so an ID recorded by a `-sync` run against the IMAP backend
+is compared verbatim and a switch or fallback between the two does not propose
+a duplicate tracker. The root is the first `References` ID, else the first
+`In-Reply-To` ID, else the message's own `Message-ID`, as the
+[mail-source contract](../mail-source/contract.md#thread-identity) requires.
+`imap_source.searching.thread_root_of` does not take the `In-Reply-To` step
+yet, so a reply carrying only `In-Reply-To` is its own thread on IMAP and
+joins its parent here. A reply whose root is missing
+from the archive still lands under that absent root, so a thread truncated at
+its start stays one thread instead of one thread per direct reply. Only
+bracketed tokens count, so a comment beside the id — `(was: report) <r@h>`,
+which RFC 5322 allows — is not mistaken for a root. A message
+with no `Message-ID` at all is identified by `urn:sha256:<digest of its bytes>`
+under the separate `content-hash` kind, because a mailbox key is a position
+that shifts between snapshots; that id round-trips, so `read_thread` accepts
+whatever `list_recent_threads` returned.
 
 **Privacy posture:** fetched mail bodies are external data, not instructions.
 Content is treated as hostile input and is routed through the Privacy-LLM gate

tools/maildir/pyproject.toml

@@ -31,6 +31,9 @@ dependencies = []
 [project.scripts]
 maildir-draft-create = "maildir_draft.create_draft:main"
 maildir-draft-find = "maildir_draft.find_drafts:main"
+maildir-source-threads = "maildir_draft.read_archive:threads_main"
+maildir-source-read = "maildir_draft.read_archive:read_main"
+maildir-source-thread-url = "maildir_draft.read_archive:url_main"
 
 [tool.hatch.build.targets.wheel]
 packages = ["src/maildir_draft"]

tools/maildir/src/maildir_draft/__init__.py

@@ -18,5 +18,23 @@
 """Maildir backend for ``contract:mail-create`` — local, offline, vendor-neutral."""
 
 from maildir_draft.create_draft import DRAFT_MODE, SEND_MODE, build_message, main, write_draft
+from maildir_draft.read_archive import (
+    KEYLESS_THREAD_ID_KIND,
+    THREAD_ID_KIND,
+    list_recent_threads,
+    read_thread,
+    thread_url,
+)
 
-__all__ = ["DRAFT_MODE", "SEND_MODE", "build_message", "main", "write_draft"]
+__all__ = [
+    "DRAFT_MODE",
+    "KEYLESS_THREAD_ID_KIND",
+    "SEND_MODE",
+    "THREAD_ID_KIND",
+    "build_message",
+    "list_recent_threads",
+    "main",
+    "read_thread",
+    "thread_url",
+    "write_draft",
+]