Harden sending, add mail stats, open the MCP to all staff

CLAUDE.md

@@ -13,6 +13,8 @@ Open source newsletter sender: Cloudflare Worker + D1 + R2, Amazon SES for deliv
 - Installation-specific values (organization, lists, senders, domain, D1 id) belong in `config/*.json`, never in `src/` or `cloudflare.config.ts`.
 - Schema changes go in a new file under `migrations/`, never by editing an applied one.
 - Pages, logs, stats, MCP tool output and the audit log never show an address or a name.
+- An assistant adds people only to lists it made (`src/lists.ts`). A newsletter from the config takes people only from forms and imports.
+- Opens and clicks keep only a machine-or-not flag (`src/tracking.ts`), never the IP address or user agent it was judged by.
 
 ## Code
 

README.md

@@ -8,12 +8,13 @@ You keep the list, the templates and the numbers. Amazon SES only delivers the m
 
 - **Sends** an HTML issue to a list, one mail per recipient, with the merge tags `*|FNAME|*`, `*|UNSUB|*` and `*|ARCHIVE|*`. A send that stops halfway resumes without mailing anyone twice.
 - **One-click unsubscribe** ([RFC 8058](https://www.rfc-editor.org/rfc/rfc8058)): the `List-Unsubscribe` headers that Gmail, Yahoo and Apple Mail act on, plus the footer link.
-- **Counts** clicks per link, and opens through an optional pixel per list, which feeds a resend to those who did not open.
+- **Counts** clicks per link, and opens through an optional pixel per list, which feeds a resend to those who did not open. Clicks by link scanners at mail servers and opens by Apple Mail's privacy proxy are counted apart from people's, so a B2B list does not look busier than it is.
+- **Numbers that matter:** open, click, click-to-open, unsubscribe, bounce and complaint rates per send; sending health over 30 days against the limits Amazon SES and Gmail hold senders to; and each list's growth by month.
 - **Bounces and spam complaints** arrive from SES through SNS; each message's signature is checked, and the address is suppressed.
 - **Imports** a Mailchimp audience export: subscribers with their opt-in time, unsubscribed and cleaned addresses as suppressions.
 - **Sign-ups** from your own sites: a browser form, a server route with a key, or a Cloudflare Worker over a service binding ([docs/signup.md](docs/signup.md)).
-- **A dashboard** for admins with lists, sends, rates, clicks per link and the audit log, and no personal data ([docs/dashboard.md](docs/dashboard.md)).
-- **An MCP server** for assistants (Claude, ChatGPT, Grok, Cursor, Codex …) with OAuth 2.1. An assistant can read stats, write drafts and send tests to the team; it can only *ask* to send to a list, and a person approves each send ([docs/mcp.md](docs/mcp.md)).
+- **A dashboard** for the team with sending health, lists and their growth, sends, rates, clicks per link and the audit log, and no personal data ([docs/dashboard.md](docs/dashboard.md)).
+- **An MCP server** for everyone at the company, through their own assistant (Claude, ChatGPT, Grok, Cursor, Codex …), with OAuth 2.1. An assistant reads the numbers, writes drafts, sends tests to the person using it, and makes lists for one mailing, such as an event's guests. It can only *ask* to send to a list: an admin gets the request by mail and approves each send ([docs/mcp.md](docs/mcp.md)).
 - **A web archive** page per issue, without names or tracking.
 
 ## Architecture
@@ -36,7 +37,7 @@ issue HTML ──► CLI ──/admin──► Worker ──SendEmail──► A
 1. `send … --dry-run` counts the audience (subscribers of the list, minus suppressed addresses), lists the links it will track, and writes a sample mail to `.dev/`.
 2. A live send needs `--confirm <count>`, the number the dry run printed. Without it nothing goes out.
 3. Creating a send writes one delivery row per recipient, each with a random token. Creating it again with the same content does nothing; other content under the same key is refused.
-4. The CLI asks the Worker for batches of up to 14 mails (SES's default rate), one batch a second. Each row is claimed before SES is called and settled after, so a stopped send resumes where it stopped (`resume <key>`). When SES's answer is lost or SES answers with a server error, the row becomes `unknown` and is not retried on its own ("rather none than two"); `retry-unknown` sends those again. An address SES refuses is `failed` (`retry-failed` puts those back in line). An account or settings error (access denied, sending paused, an unverified domain) stops the send with the rest pending: fix the cause, then `resume`. After the last batch the CLI says if any rows are still sending, unknown or failed, and what to run.
+4. The CLI asks the Worker for batches of up to 14 mails (SES's default rate), one batch a second. Each row is claimed before SES is called and settled as soon as SES answers, so a stopped send resumes where it stopped (`resume <key>`). When SES's answer is lost or SES answers with a server error, the row becomes `unknown` and is not retried on its own ("rather none than two"); `retry-unknown` sends those again. An address SES refuses is `failed` (`retry-failed` puts those back in line). An account or settings error (access denied, sending paused, an unverified domain or the SES sandbox, the day's quota spent, or SES refusing every mail of a batch the same way) stops the send with the rest pending: fix the cause, then `resume`. A row whose answer was lost turns `sent` by itself when SES's delivery event for it arrives. After the last batch the CLI says if any rows are still sending, unknown or failed, and what to run.
 5. Unsubscribes, bounces and complaints are checked again at send time.
 
 ## Getting started
@@ -74,7 +75,7 @@ Images next to the issue HTML (`src="hero.jpg"`) are uploaded to R2 and served f
 
 1. Copy `config/example.json` to `config/<you>.json` and fill it in: the `deploy` section (public URL, AWS region, D1, R2), your organization, lists, forms and admins.
 2. Follow [docs/setup.md](docs/setup.md) for SES, DNS, SNS, IAM, D1, R2, Cloudflare Access and the secrets.
-3. Deploy with `UTSKICK_CONFIG=config/<you>.json npx cf deploy`. Without `UTSKICK_CONFIG` a deploy stops, so nobody ships mail whose links point at someone else's Worker.
+3. Apply the migrations first (`cf d1 migrations apply <d1-id>`), then deploy with `UTSKICK_CONFIG=config/<you>.json npx cf deploy`. Code that ships before its migration answers readers' clicks with errors. Without `UTSKICK_CONFIG` a deploy stops, so nobody ships mail whose links point at someone else's Worker.
 
 More: [docs/signup.md](docs/signup.md) (forms), [docs/dashboard.md](docs/dashboard.md), [docs/mcp.md](docs/mcp.md) (assistants), [docs/examples/](docs/examples/) (connecting a Cloudflare Pages site).
 
@@ -85,10 +86,10 @@ More: [docs/signup.md](docs/signup.md) (forms), [docs/dashboard.md](docs/dashboa
 - **SNS webhook.** Nothing in a message is used before its signature checks out: SignatureVersion 1 or 2, the signing certificate fetched only from `sns.<the topic's region>.amazonaws.com` with the `SimpleNotificationService-…pem` path, and only the configured topic accepted. That includes a subscription confirmation's `SubscribeURL`.
 - **Reader tokens.** Each delivery has its own 128-bit random token, used in its unsubscribe, click and open URLs. A token identifies one mail, not a person, and the API never returns one. Like any newsletter footer link, a forwarded mail lets its new reader unsubscribe the original one.
 - **Unsubscribe.** A GET only shows a button, because mail scanners open links. A POST, the one RFC 8058 clients send, unsubscribes. In a test mail the POST is logged and takes nobody off a list.
-- **Bounces.** A hard bounce or a spam complaint suppresses the address on every list. A soft bounce does so when a second mail bounces.
-- **Sign-ups.** Browser forms are allowed by exact origin (CORS), server routes by a per-form key, sites on Cloudflare over a service binding. Each list's consent id must match the config, a honeypot field catches form bots, and requests are rate-limited per client (per /64 for IPv6). The answer is the same for a new, known or suppressed address. Without double opt-in, a sign-up never lifts a suppression and never changes an existing contact.
-- **Assistants (MCP).** OAuth 2.1 with PKCE S256 only, tokens bound to the `/mcp` resource, rotating refresh tokens, revocation per client, and least-privilege scopes (`stats.read`, `subscribers.read`, `drafts.write`, `test.send`). Sending to a list is never a scope: a person approves each send on a page that shows the list, the subject, the count, the text part, every link and anything hidden; the approval works once, for ten minutes. Every tool call is audited. Tokens, codes and secrets are stored only as hashes.
-- **Admins.** Sign-in for the dashboard, the consent page and approvals goes through Cloudflare Access; Utskick checks the Access JWT itself and allows only the addresses in `admins`. The development stand-in works only when both the configured and the requested host are local.
+- **Bounces.** A hard bounce or a spam complaint suppresses the address on every list. A full mailbox does so when a second mail bounces for it within 30 days; other soft bounces (a busy server, one slow to trust a new sender) do not. Only the mail's own recipient is suppressed, never other addresses a bounce report names.
+- **Sign-ups.** Browser forms are allowed by exact origin (CORS), server routes by a per-form key, sites on Cloudflare over a service binding. Each list's consent id must match the config, a honeypot field catches form bots, and requests are rate-limited per client (per /64 for IPv6). The answer is the same for a new, known or suppressed address. Without double opt-in, a sign-up never lifts a suppression and never changes an existing contact. A site over a service binding has the same rate limit.
+- **Assistants (MCP).** OAuth 2.1 with PKCE S256 only, tokens bound to the `/mcp` resource, rotating refresh tokens, revocation per client and per person, and least-privilege scopes (`stats.read`, `subscribers.read`, `drafts.write`, `test.send`, `lists.write`). Sending to a list is never a scope: an admin approves each send on a page that shows the list, the subject, the count, the text part, every link and anything hidden; the approval works once, within 24 hours by default. With `approval.notifyFrom` set, the admins get the approval link by mail and the assistant never holds it. An assistant can add addresses to a list it made, and no tool returns one. Every tool call is audited. Tokens, codes and secrets are stored only as hashes.
+- **Members and admins.** Sign-in for the dashboard, the consent page and approvals goes through Cloudflare Access; Utskick checks the Access JWT itself. `members` (addresses, or `@domain` for a whole company) may connect an assistant and read the dashboard; only `admins` approve sends. Every call an assistant makes checks that its person is still a member. The development stand-in works only when both the configured and the requested host are local.
 - **Pages.** Every reader and admin page is server-rendered without scripts. The dashboard and approval pages carry a strict Content-Security-Policy and cannot be framed; the web archive is served in a CSP sandbox.
 - **Limits on abuse.** At most one open per mail, and one click per mail and link, are logged an hour; request bodies are read only up to a small limit.
 - **Local stand-ins stay local.** `SES_ENDPOINT` (the SES mock) is ignored unless the public URL is a local host, and the CLI refuses to send the admin token over plain HTTP to anything but localhost.
@@ -98,7 +99,8 @@ Report a vulnerability as described in [SECURITY.md](SECURITY.md).
 ## Privacy
 
 - Leaving a list deletes the address from it, and the contact once it is on no list. A SHA-256 hash of the address stays on the suppression list, so an old import cannot sign the person up again; it is a pseudonym, not anonymous.
-- Opens and clicks are stored as "delivery N, link M, at time T", without IP address or user agent. Lists without `opens` get no pixel. In the EU, an open pixel may need consent (EDPB Guidelines 2/2023); say so where people sign up, or turn it off.
+- Opens and clicks are stored as "delivery N, link M, at time T, machine or not", without IP address or user agent. Whether a machine made it is decided from the network and the user agent when the request comes in, and only the yes or no is kept. Lists without `opens` get no pixel.
+- A list a colleague makes through an assistant holds addresses that colleague typed in, for one purpose the admin sees before any send. Deleting it deletes every address on no other list. In the EU, an open pixel may need consent (EDPB Guidelines 2/2023); say so where people sign up, or turn it off.
 - Pages, logs, stats, the dashboard and the audit log show no address or name.
 - D1 and R2 can be created in Cloudflare's EU jurisdiction, and SES runs in the AWS region you choose.
 

cli/utskick.ts

@@ -115,8 +115,17 @@ set and the sender's domain in SES, then run \`resume ${key}\`.`);
   }
 }
 
+// What the dry run found wrong with the mail. Older Workers send no warnings. One line each, whatever the text holds.
+function printWarnings(r: any) {
+  if (!Array.isArray(r.warnings)) return;
+  for (const w of r.warnings) console.log(`warning: ${String(w).replace(/[\s\x00-\x1f\x7f]+/g, ' ').trim()}`);
+}
+
 function printDryRun(r: any) {
   console.log(`${r.recipients} recipients${r.excluded ? `, ${r.excluded} subscribers left out (unsubscribed, bounced or complained)` : ''}`);
+  // Older Workers do not send the size either.
+  if (Number.isFinite(r.bytes)) console.log(`size: ${(r.bytes / 1024).toFixed(1)} KB (the sample's HTML)`);
+  printWarnings(r);
   console.log(`subject: ${r.sample.subject}\nfrom: ${r.sample.from}\nreply-to: ${r.sample.replyTo}`);
   console.log(`tracked links (${r.links.length}):`);
   r.links.forEach((l: string, n: number) => console.log(`  ${n}  ${l}`));
@@ -156,6 +165,8 @@ async function send(kind: 'live' | 'resend') {
     printDryRun(dry);
     fail(`\nNothing sent. This goes to ${dry.recipients} people on ${list()}. To send, run again with --confirm ${dry.recipients}`);
   }
+  // The count matches, so this goes out. Show once more what the dry run warned about.
+  printWarnings(dry);
   const r = await api('POST', '/admin/sends', body);
   console.log(`created ${key}: ${r.recipients} recipients`);
   await deliver(key);

config/example.json

@@ -34,9 +34,21 @@
   "admins": [
     "you@example.com"
   ],
+  "members": [
+    "@example.com"
+  ],
   "testAllowlist": [
     "you@example.com"
   ],
+  "adhoc": {
+    "from": "Example Org <hello@news.example.com>",
+    "opens": false,
+    "fname": "there"
+  },
+  "approval": {
+    "hours": 24,
+    "notifyFrom": "Utskick <utskick@news.example.com>"
+  },
   "access": {
     "teamDomain": "your-team.cloudflareaccess.com",
     "audience": "the-AUD-tag-of-your-Access-application"

config/sourceful.json

@@ -41,9 +41,22 @@
   "admins": [
     "fredrik@sourceful-labs.com"
   ],
+  "members": [
+    "@sourceful-labs.com",
+    "@sourceful.energy"
+  ],
   "testAllowlist": [
     "fredrik@sourceful-labs.com"
   ],
+  "adhoc": {
+    "from": "Sourceful Energy <hello@news.sourceful.energy>",
+    "opens": false,
+    "fname": "there"
+  },
+  "approval": {
+    "hours": 24,
+    "notifyFrom": "Utskick <utskick@news.sourceful.energy>"
+  },
   "access": {
     "teamDomain": "sourceful.cloudflareaccess.com",
     "audience": "83aeb3bf704fc1e26213a621c0c2ed12c788e21a502f2195fb8d3d5aa906d56f"