feat: Server-Side User Preferences Sync + Notification Delivery Channels

[koala73]

feat: Server-Side User Preferences Sync + Notification Delivery Channels

Revision 2026-03-29 — Updated from original 2026-03-24 plan. Corrects stale blocker language, fixes implementation paths, adds research findings from codebase audit and SpecFlow analysis.

Overview

WorldMonitor stores all user configuration in localStorage — panel layout, sources, watchlist, map layers, monitors, and every other personalisation — meaning it is browser-local, lost on a new device, and inaccessible server-side. This plan migrates user preferences to Convex-backed server storage for signed-in users and adds first-class notification delivery via Telegram, Slack, and email.

Multi-phase feature. Each phase ships value independently.

---

Auth Substrate Status (resolved — no blocker)

PR #1812 (feat/better-auth) merged 2026-03-26. Already in HEAD:

• convex/auth.config.ts:1 — Clerk JWT provider configured with CLERK_JWT_ISSUER_DOMAIN
• src/services/auth-state.ts:24 — user.id exposed via snapshotSession()
• src/services/clerk.ts:131 — getClerkToken() returns 50s-cached Clerk session token
• server/auth-session.ts:67 — validateBearerToken() JWKS RS256 verification; accepts both convex-template and standard session tokens

All phases can start now. Env validation needed: CLERK_JWT_ISSUER_DOMAIN and VITE_CLERK_PUBLISHABLE_KEY in Vercel + Railway + Convex dashboard.

---

Problem Statement

• User experience on a new browser/device starts from scratch — zero personalisation recovery
• No server knows what a user cares about, so no server-side notifications are possible
• Users cannot subscribe to conflict escalations, market moves, or breaking news and receive alerts externally
• Login is a gate for pro features only — not a value delivery mechanism in its own right

---

Architecture

Browser (Clerk session)
  │
  ├─ Preference writes ──► Convex userPreferences (via HTTP action, JWT-authenticated)
  │
  ├─ Preference reads  ◄── Convex userPreferences (on sign-in, merge with localStorage)
  │
  └─ Notification prefs ──► Convex alertRules + notificationChannels

breaking-news-alerts.ts (browser) dispatches wm:breaking-news
  └─ → POST /api/notify (new Vercel edge, Clerk-authenticated)
       └─ → PUBLISH wm:events:notify (Upstash REST) ← relay subscribes

scripts/notification-relay.cjs (new Railway service)
  ├─ SUBSCRIBE wm:events:notify via Upstash REST long-poll
  ├─ Queries Convex alertRules via ConvexHttpClient (or plain fetch HTTP action)
  └─ Fans out via token-bucket queues to:
       ├─ Telegram Bot API  (25 msg/s; chatId from notificationChannels)
       ├─ Slack webhook     (0.8 msg/s; URL AES-256-GCM encrypted, re-validated at send)
       └─ Resend email      (10 msg/s; email cached from Clerk at link time)

Critical gap resolved (SpecFlow): The transport from browser wm:breaking-news event to Railway relay is a new Vercel edge endpoint /api/notify. The browser POSTs to it with a Clerk token; the endpoint validates and PUBLISHes to wm:events:notify via Upstash REST. The relay subscribes. Without this bridge, the entire delivery chain is disconnected.

---

Key Design Decisions

Decision	Choice	Rationale
Sync conflict resolution	syncVersion wins (server-incremented monotonic counter)	Clock skew on updatedAt (client-supplied) causes stale prefs to silently win
Concurrent tabs	Last-write-wins by syncVersion; concurrent increments from same base are silently overwritten — acceptable for preferences (low stakes)	Full OCC adds complexity for no meaningful benefit
Sync scope	CLOUD_SYNC_KEYS allowlist in src/utils/sync-keys.ts (single source of truth)	Secrets in mcp-store.ts customHeaders and runtime-config.ts vault must never leave the device
Convex table name	userPreferences	Single canonical blob per userId+variant
Prefs schema type	v.any() (not v.string())	String blob bypasses Convex validation; migration after data exists is non-trivial
Alert delivery transport	wm:breaking-news → POST /api/notify → Upstash PUBLISH → relay subscribes	Avoids polling; <2s latency vs 60s
Notification relay	New scripts/notification-relay.cjs (separate Railway service)	Isolated crash domain; ais-relay.cjs is already 8K lines
Telegram pairing UX	Deep link with 15-min base64url token (43 chars)	Hex = exactly 64 chars at Telegram's limit; base64url has headroom
Pairing status detection	ConvexHttpClient.query polling every 3s, cleared after pairing or 5-min timeout	Class-based Preact with imperative rendering — no Convex reactive layer exists
Email source	Cached at link time from clerk.ts:130	Avoids Clerk API fan-out in hot relay path; rate limits would drop notifications
Alert rule scope	Per-variant	Tech-variant user should not receive conflict/OREF alerts from full-variant
Webhook URL storage	v1:<base64(iv+tag+ciphertext)> envelope	Enables key rotation without re-encrypting all rows
Sign-out behaviour	Preserve localStorage; mark sync state as signed-out	Do not silently delete user data
Feature flag	import.meta.env.VITE_CLOUD_PREFS_ENABLED === 'true'	Env gate, NOT a beta.ts localStorage clone (src/config/beta.ts is localStorage-only)
Desktop sync	Disabled — isDesktopRuntime() guard required	"Desktop NEVER connects to cloud for data" (project memory)

---

Open Questions

Q1/Q2/Q4/Q5/Q6 resolved 2026-03-29. Q3 deferred (not blocking Phase 0/1).

#	Question	Status	Resolution
Q1	Is cloud sync for all signed-in users or pro-only?	Resolved	All signed-in users; notification delivery channels are pro-only
Q2	Must initAuthState() move out of isProUser() gate?	Resolved	Yes — move unconditionally in App.ts
Q3	AES key source + rotation strategy for Slack webhook encryption?	Deferred	See options below — not blocking until Phase 3b
Q4	Concurrent tab: silent last-write-wins acceptable?	Resolved	Yes — low stakes, no personal data
Q5	TELEGRAM_BOT_TOKEN separate from MTProto TELEGRAM_API_ID/HASH/SESSION?	Resolved	Yes — separate Bot API token, separate .env.example section
Q6	Email source at send time: Clerk API or Convex cache?	Resolved	Convex notificationChannels.email (cached at link time); send via Resend. CLERK_SECRET_KEY NOT in relay

Q3 Options (deferred, choose before Phase 3b)

Option	Env vars	Rotation	Notes
A — single key (recommended for MVP)	NOTIFICATION_ENCRYPTION_KEY (Railway only)	User must re-enter all Slack webhooks	Zero overhead; envelope format supports upgrade to B later without schema changes
B — versioned keys	ENCRYPTION_KEY_V1, ENCRYPTION_KEY_V2 (Railway only)	Deploy new key; relay re-encrypts on next write	No user re-entry; ~20 extra lines in scripts/lib/crypto.cjs
C — KMS	AWS KMS / Cloudflare DEK	Automatic	Overkill unless compliance required

ENCRYPTION_KEY_* vars live in Railway env ONLY — never in Convex (Convex dashboard access would expose them).

---

Syncable Preferences Allowlist

Single source of truth lives at src/utils/sync-keys.ts (new file, Phase 0).
Both settings-persistence.ts SETTINGS_KEY_PREFIXES and cloud sync import from it.

Included in cloud sync:

Key	Service file
worldmonitor-panels	src/utils/settings-persistence.ts
worldmonitor-monitors	src/utils/settings-persistence.ts
worldmonitor-layers	src/utils/settings-persistence.ts
worldmonitor-disabled-feeds	src/utils/settings-persistence.ts
worldmonitor-panel-spans	src/utils/settings-persistence.ts
worldmonitor-panel-col-spans	src/utils/settings-persistence.ts
worldmonitor-panel-order	src/utils/settings-persistence.ts
worldmonitor-theme	src/utils/theme-manager.ts:6
worldmonitor-variant	src/config/variant.ts
worldmonitor-map-mode	src/utils/settings-persistence.ts
wm-breaking-alerts-v1	src/services/breaking-news-alerts.ts:23
wm-market-watchlist-v1	src/services/market-watchlist.ts:16
aviation:watchlist:v1	src/services/aviation/watchlist.ts:6
wm-pinned-webcams	src/services/webcams/pinned-store.ts:1
wm-map-provider	src/config/basemap.ts:49
wm-font-family	src/services/font-settings.ts
wm-globe-visual-preset	src/services/globe-render-settings.ts:113
wm-stream-quality	src/services/ai-flow-settings.ts:9
wm-ai-flow-cloud-llm	src/services/ai-flow-settings.ts
wm-analysis-frameworks	src/services/analysis-framework-store.ts
wm-panel-frameworks	src/services/analysis-framework-store.ts

Explicitly excluded (secrets / device-local):

Key	Reason
wm-mcp-panels	Contains customHeaders with API keys
wm-pro-key / wm-widget-key	API key credentials
worldmonitor-runtime-feature-toggles vault entries	26 RuntimeSecretKey API keys
worldmonitor-live-channels / worldmonitor-active-channel	Device-specific stream session state (src/config/variants/base.ts:79)
map-height / map-pinned / mobile-map-collapsed	Device-specific viewport state
wm-breaking-alerts-dedupe	Device-local dedup map — syncing would suppress alerts on new devices

---

Implementation Phases

Phase 0: Pre-Work (can start now, no auth dependency)

Goal: Primitives and shared infra needed by all subsequent phases.

Deliverables:

1. src/utils/sync-keys.ts — canonical CLOUD_SYNC_KEYS constant (single source of truth):

   export const CLOUD_SYNC_KEYS = [
     'worldmonitor-panels', 'worldmonitor-monitors', 'worldmonitor-layers',
     'worldmonitor-disabled-feeds', 'worldmonitor-panel-spans', 'worldmonitor-panel-col-spans',
     'worldmonitor-panel-order', 'worldmonitor-theme', 'worldmonitor-variant',
     'worldmonitor-map-mode', 'wm-breaking-alerts-v1', 'wm-market-watchlist-v1',
     'aviation:watchlist:v1', 'wm-pinned-webcams', 'wm-map-provider',
     'wm-globe-visual-preset', 'wm-stream-quality', 'wm-font-family',
     'wm-ai-flow-cloud-llm', 'wm-analysis-frameworks', 'wm-panel-frameworks',
   ] as const;

   Update src/utils/settings-persistence.ts SETTINGS_KEY_PREFIXES to import from this file.

2. convex/constants.ts — shared validators:

   export const channelTypeValidator = v.union(v.literal("telegram"), v.literal("slack"), v.literal("email"));
   export const sensitivityValidator = v.union(v.literal("all"), v.literal("high"), v.literal("critical"));
   export const CURRENT_PREFS_SCHEMA_VERSION = 1;
   export const MAX_PREFS_BLOB_SIZE = 65536; // 64KB

3. server/_shared/timing-safe.ts — constant-time comparison for webhook validation:

   export function timingSafeEqual(a: string, b: string): boolean {
     const aBuf = Buffer.from(a);
     const bBuf = Buffer.from(b);
     if (aBuf.length !== bBuf.length) return false;
     return crypto.timingSafeEqual(aBuf, bBuf);
   }

4. scripts/lib/crypto.cjs — AES-256-GCM encryption for Slack webhook URLs:

   // Envelope: "v1:<base64(iv[12] + tag[16] + ciphertext)>"
   const KEYS = { v1: Buffer.from(process.env.NOTIFICATION_ENCRYPTION_KEY, 'base64') };
   function encrypt(plaintext) { /* random IV, AES-256-GCM, prepend version */ }
   function decrypt(stored) { /* parse version, select key, decrypt */ }

5. .env.example additions:

   # Notification Delivery (Phase 4)
   TELEGRAM_BOT_TOKEN=           # @WorldMonitorBot token (SEPARATE from TELEGRAM_API_ID/HASH/SESSION)
   TELEGRAM_WEBHOOK_SECRET=      # secret for X-Telegram-Bot-Api-Secret-Token header
   NOTIFICATION_ENCRYPTION_KEY=  # 32-byte base64 AES key for Slack webhook URL encryption
   RESEND_API_KEY=                # for email delivery via Resend

6. scripts/package.json — add new deps for notification relay:

   • "convex": "^1" — ConvexHttpClient for alert rules queries
   • "resend": "^4" — email delivery (or use raw fetch to Resend API following api/contact.js pattern)

7. convex/crons.ts — cleanup expired pairing tokens:

   import { cronJobs } from "convex/server";
   const crons = cronJobs();
   crons.hourly("cleanup-expired-pairing-tokens", { minuteOffset: 0 }, internal.telegramPairingTokens.cleanupExpired);
   export default crons;

Convex deployment note: After schema changes, run BOTH:

npx convex dev --once   # dev environment (.env.local CONVEX_DEPLOYMENT)
npx convex deploy --yes # production

Missing one leaves prod schema out of sync; mutations fail with "Object contains extra field."

Phase 0 Acceptance Criteria:

• src/utils/sync-keys.ts exports CLOUD_SYNC_KEYS
• src/utils/settings-persistence.ts imports from sync-keys.ts (not a duplicate list)
• scripts/package.json includes convex and resend
• .env.example has all four new notification vars under "Notification Delivery" section
• convex/crons.ts defined (even if handler is a stub)

---

Phase 1: Convex Schema + Preferences API

Goal: Convex has the tables; app can write/read preferences for a signed-in user.

Startup sequencing (critical): App.ts runs in-place migrations at startup:

• v2.6 panel key renames at src/App.ts:416
• v2.5 layout reset at src/App.ts:550

Cloud sync must not read localStorage until both migrations have run and must not write to Convex until after the merge step. A first sign-in from a browser with pre-migration state must push the post-migration snapshot.

App.ts: run local migrations (v2.5, v2.6)
  → load local panel/layout state
  → cloud sync: fetch server prefs
  → merge (server syncVersion wins)
  → apply merged state

convex/schema.ts additions

import { channelTypeValidator, sensitivityValidator } from "./constants";

userPreferences: defineTable({
  userId: v.string(),      // ctx.auth.getUserIdentity().subject — NEVER from args
  variant: v.string(),
  data: v.any(),           // allowlist-filtered; NOT v.string()
  schemaVersion: v.number(),
  updatedAt: v.number(),   // display only — NOT used for conflict resolution
  syncVersion: v.number(), // server-incremented; PRIMARY conflict resolver
}).index("by_user_variant", ["userId", "variant"]),

notificationChannels: defineTable(
  v.union(
    v.object({ userId: v.string(), channelType: v.literal("telegram"), chatId: v.string(), verified: v.boolean(), linkedAt: v.number() }),
    v.object({ userId: v.string(), channelType: v.literal("slack"), webhookEnvelope: v.string(), verified: v.boolean(), linkedAt: v.number() }),
    v.object({ userId: v.string(), channelType: v.literal("email"), email: v.string(), verified: v.boolean(), linkedAt: v.number() }),
  )
).index("by_user", ["userId"])
 .index("by_user_channel", ["userId", "channelType"]),

alertRules: defineTable({
  userId: v.string(),
  variant: v.string(),
  enabled: v.boolean(),
  eventTypes: v.array(v.string()),
  sensitivity: sensitivityValidator,
  channels: v.array(channelTypeValidator),
  updatedAt: v.number(),
}).index("by_user", ["userId"])
 .index("by_user_variant", ["userId", "variant"])
 .index("by_enabled", ["enabled"]),  // REQUIRED: relay queries by enabled=true (NOT full table scan)

telegramPairingTokens: defineTable({
  userId: v.string(),
  token: v.string(),       // base64url, 43 chars (NOT hex — at Telegram's 64-char limit)
  expiresAt: v.number(),   // Unix ms, 15 minutes
  used: v.boolean(),
}).index("by_token", ["token"])
 .index("by_user", ["userId"]),

Convex module layout (three new files)

// convex/userPreferences.ts
export const getPreferences = query({ ... })    // by userId+variant; apply schemaVersion migration
export const setPreferences = mutation({ ... }) // upsert; accepts expectedSyncVersion; server-stamps updatedAt
// setPreferences rules:
// 1. ctx.auth.getUserIdentity() → userId (NEVER from args)
// 2. db.query(...).withIndex("by_user_variant", ...).unique() → existing
// 3. if existing && existing.syncVersion !== args.expectedSyncVersion → throw ConvexError("CONFLICT")
// 4. Validate args.data size ≤ MAX_PREFS_BLOB_SIZE
// 5. db.patch(existing._id, { data, schemaVersion, updatedAt: Date.now(), syncVersion: (existing?.syncVersion ?? 0) + 1 })

// convex/notificationChannels.ts
export const getChannels = query({ ... })            // by userId
export const setChannel = mutation({ ... })           // encrypt webhook server-side; test Slack; set verified=true
export const deleteChannel = mutation({ ... })        // remove + clean alertRules.channels[]
export const deactivateChannel = mutation({ ... })    // set verified=false (relay calls on 403/404)
export const createPairingToken = mutation({ ... })   // base64url token; expiresAt=now+15min; used=false
export const claimPairingToken = mutation({ ... })    // ATOMIC: check used+expiry, set chatId+verified+used in ONE mutation

// convex/alertRules.ts
export const getAlertRules = query({ ... })           // by userId
export const setAlertRules = mutation({ ... })        // upsert per userId+variant
export const getByEnabled = query({ ... })            // by_enabled index (NOT full table scan)

ConvexHttpClient call syntax: Uses string paths, not typed api object.

await client.mutation('notificationChannels:setChannel', { channelType: 'telegram', chatId })
// NOT: await client.mutation(api.notificationChannels.setChannel, ...)

Follow pattern at api/register-interest.js:244.

undefined vs null in Convex: JSON.stringify({ field: undefined }) → "{}". Field never arrives.
Use null on client to signal "clear field"; convert null → undefined in mutation handler before db.patch.

convex/http.ts additions

// POST /api/user-prefs — JWT-authenticated (Clerk Bearer token)
// OPTIONS /api/user-prefs — CORS preflight (paired route required; Authorization in Access-Control-Allow-Headers)
// POST /api/telegram-pair-callback — UNAUTHENTICATED (Telegram bot webhook)
//   Verifies X-Telegram-Bot-Api-Secret-Token with timingSafeEqual()
//   Verifies message.chat.type === 'private'
//   Verifies message.date within 900s
//   Calls claimPairingToken mutation
//   Always returns HTTP 200 (non-200 triggers Telegram retry storm)

Phase 1 Acceptance Criteria:

• convex/schema.ts has all four tables with indexes including by_enabled
• convex/userPreferences.ts exports getPreferences, setPreferences
• convex/notificationChannels.ts exports getChannels, setChannel, deleteChannel, deactivateChannel, createPairingToken, claimPairingToken
• convex/alertRules.ts exports getAlertRules, setAlertRules, getByEnabled
• npx convex deploy --yes applies schema without errors
• POST /api/user-prefs returns 200 for valid Clerk JWT
• Returns 401 for missing/invalid JWT
• Returns 409 with "CONFLICT" when expectedSyncVersion does not match
• claimPairingToken is atomic (one mutation, not read-then-write)
• Blob > 64KB rejected with 400
• convex/crons.ts hourly cleanup runs for expired pairing tokens

---

Phase 2: Frontend Preferences Sync

Goal: Sign-in recovers prefs from cloud. Pref changes sync back to Convex.

Feature flag: import.meta.env.VITE_CLOUD_PREFS_ENABLED === 'true'
— add VITE_CLOUD_PREFS_ENABLED=false to .env.example; flip in Vercel env when Phase 2 is QA-verified.

Desktop guard: Wrap all sync calls in:

if (isDesktopRuntime()) return; // Desktop NEVER syncs to cloud

Auth gate (Q1/Q2 resolved): Cloud sync is for ALL signed-in users. initAuthState() must be called unconditionally in App.ts — move it out of the isProUser() gate before Phase 2 ships.

Sync protocol

On sign-in (after App.ts migrations v2.5, v2.6 complete):
  1. Render immediately with localStorage (OPTIMISTIC — no blocking first paint)
  2. fetch cloud prefs via ConvexHttpClient.query in parallel
  3. If cloud.syncVersion > localStorage['wm-cloud-sync-version']:
       Apply schemaVersion migration if needed
       Apply cloud prefs to localStorage (cloud wins)
       Show 5-second undo toast on first-ever sign-in: "We restored your settings. [Undo]"
  4. Else: upload local prefs with expectedSyncVersion = wm-cloud-sync-version
  5. Store cloud.syncVersion in localStorage['wm-cloud-sync-version']

On preference change:
  → Debounce 5s for layout/display; 2s for alert rule changes
  → Build CLOUD_SYNC_KEYS-filtered blob (never syncs mcp-store or vault keys)
  → POST /api/user-prefs with { variant, data, expectedSyncVersion }
  → On success: store response.syncVersion in localStorage['wm-cloud-sync-version']
  → On 409: re-fetch, merge, retry once
  → On tab close: flush via navigator.sendBeacon() in visibilitychange+pagehide
    (NOT beforeunload — unreliable in bfcache/mobile)
  → Multi-tab: listen to storage event; cancel local debounce if another tab wrote newer syncVersion

On sign-out:
  → Clear only sync metadata keys (NOT prefs)
  → Set localStorage['wm-last-signed-in-as'] = userId

schemaVersion migration pattern

// src/utils/settings-persistence.ts
const CURRENT_PREFS_SCHEMA_VERSION = 1;
const MIGRATIONS: Record<number, (data: Record<string, unknown>) => Record<string, unknown>> = {};
function applyMigrations(data, fromVersion) {
  let result = data;
  for (let v = fromVersion + 1; v <= CURRENT_PREFS_SCHEMA_VERSION; v++) {
    result = MIGRATIONS[v]?.(result) ?? result;
  }
  return result;
}

New localStorage sync-state keys (never uploaded):

Key	Purpose
wm-cloud-sync-version	Last known cloud syncVersion
wm-last-sync-at	Server-returned Unix ms of last confirmed write (set ONLY after server success)
wm-cloud-sync-state	'synced' | 'pending' | 'syncing' | 'conflict' | 'offline' | 'signed-out' | 'error'

New Convex user pref endpoint: api/user-prefs.ts (new Vercel edge function)

• Follow pattern: api/contact.js:1-80 for CORS + JWT validation + Convex client

Phase 2 Acceptance Criteria:

• Sign in on Device A → panel layout from Device B loads correctly, no blocking first paint
• Pref change on Device A → sign in Device B → pref synced
• Tab close with pending debounce → flushed via sendBeacon
• Sign out → prefs preserved in localStorage
• Sync skips all RuntimeSecretKey values and wm-mcp-panels
• 409 CONFLICT triggers re-fetch + merge (no silent loss)
• VITE_CLOUD_PREFS_ENABLED=false → complete no-op (no Convex calls)
• Cloud sync indicator shows all 7 states in settings UI
• isDesktopRuntime() guard prevents sync on Tauri desktop build
• First sign-in with existing cloud prefs shows 5-second undo toast

---

Phase 3: Notification Channel Linking UI

Goal: Users link Telegram, Slack, and email in Preferences → Notifications.

Settings UI (no new standalone component):

• src/services/preferences-content.ts — add Notifications <details> group in renderPreferences()
• src/settings-main.ts — add event handlers for pairing/unlinking
• src/services/notification-channels.ts — new service wrapping ConvexHttpClient calls

Telegram pairing flow

1. User clicks "Connect Telegram"
2. Frontend: ConvexHttpClient.mutation('notificationChannels:createPairingToken', {variant})
   → returns {token, deepLink}; token = base64url 43 chars (NOT hex)
   deepLink = https://t.me/WorldMonitorBot?start=<token>
3. UI shows deepLink button + QR + 15-min countdown timer
4. Start 3s setInterval polling: ConvexHttpClient.query('notificationChannels:getChannels', {userId})
   Stop on: (a) pairing confirmed, (b) 15-min timeout, (c) settings panel cleanup (AbortController)
5. User taps deepLink → Telegram → /start <token> to @WorldMonitorBot
6. Bot webhook (Convex HTTP action POST /api/telegram-pair-callback):
   a. Verify X-Telegram-Bot-Api-Secret-Token with timingSafeEqual()
   b. Verify message.chat.type === 'private'
   c. Verify message.date within 900s
   d. Extract token: /^\/start(?:@\w+)?\s+([A-Za-z0-9_-]{1,64})$/
   e. claimPairingToken mutation (ATOMIC: chatId + verified=true + used=true in one mutation)
   f. Send: "WorldMonitor connected. You will receive alerts here."
   g. Return HTTP 200 always
7. Frontend poll detects verified=true → show success, clear setInterval

Note: TELEGRAM_BOT_TOKEN is separate from TELEGRAM_API_ID/HASH/SESSION (MTProto for OSINT ingestion). These must never be conflated.

Slack linking flow

1. User pastes Slack incoming webhook URL
2. Client validates: must match ^https://hooks\.slack\.com/services/[A-Z0-9]+/[A-Z0-9]+/[a-zA-Z0-9]+$
3. ConvexHttpClient.mutation('notificationChannels:setChannel', {type:'slack', webhookUrl})
4. Server-side (Convex mutation handler):
   a. Re-validate URL (same regex — prevent SSRF at write AND send time)
   b. Resolve DNS, verify NOT RFC-1918/link-local
   c. AES-256-GCM encrypt: "v1:<base64(iv+tag+ciphertext)>"
   d. Store webhookEnvelope (never return plaintext to client)
   e. Send test message: "WorldMonitor connected ✓"
   f. If 200/ok → set verified=true

Email linking

1. Auto-populated from getCurrentClerkUser().primaryEmailAddress (src/services/clerk.ts:130)
2. Only Clerk-verified emails accepted (emailAddress.verification.status === 'verified')
3. Cached in notificationChannels.email at link time

Phase 3 Acceptance Criteria:

• Telegram pairing completes end-to-end < 60s
• X-Telegram-Bot-Api-Secret-Token verified with timingSafeEqual() on every webhook call
• Group chat /start commands rejected (chat.type !== 'private')
• Expired pairing token shows "Link expired — please start over"
• Poll setInterval cleared in attach() cleanup (no leak after panel close)
• Slack webhook URL never returned to client after saving
• Slack test message sent; verified=true only if test succeeds
• Email auto-populated from Clerk; no user entry required
• Alert rules scoped per-variant and persist across sessions
• Deleting channel removes from alertRules.channels[]

---

Phase 4: Notification Delivery (Railway)

Goal: scripts/notification-relay.cjs listens for breaking events; delivers to verified channels.

Dependency additions to scripts/package.json:

"convex": "^1",
"resend": "^4"

New Railway service: Add entry to scripts/package.json scripts block:

"notification-relay": "node notification-relay.cjs"

Create a new Railway service pointing at the same scripts/ repo root with start command npm run notification-relay.

Transport: browser → relay

breaking-news-alerts.ts dispatches wm:breaking-news (browser CustomEvent)
  → new listener in breaking-news-alerts.ts POSTs to /api/notify (new Vercel edge)
       /api/notify validates Clerk token, publishes to wm:events:notify via Upstash REST
          ← notification-relay.cjs subscribes via Upstash REST long-poll

// api/notify.ts (new Vercel edge)
// Pattern: api/contact.js (CORS + JWT + Upstash REST PUBLISH)
// Upstash REST PUBLISH:
// POST ${UPSTASH_REDIS_REST_URL}/publish/wm:events:notify/${encodeURIComponent(msg)}

Relay subscribe + fan-out

Path A (recommended, no new Redis dep): Upstash REST long-poll

// scripts/notification-relay.cjs
async function subscribe() {
  const url = process.env.UPSTASH_REDIS_REST_URL;
  const token = process.env.UPSTASH_REDIS_REST_TOKEN;
  while (true) {
    try {
      const res = await fetch(`${url}/subscribe/wm:events:notify`, {
        headers: { Authorization: `Bearer ${token}` },
        signal: AbortSignal.timeout(35_000),
      });
      const { message } = await res.json();
      if (message) await processEvent(JSON.parse(message));
    } catch { await new Promise(r => setTimeout(r, 5000)); }
  }
}
process.on('SIGTERM', () => process.exit(0));

Path B (if native pub/sub required): Add ioredis to scripts/package.json and use UPSTASH_REDIS_URL (rediss:// connection string).

Delivery fan-out

// ConvexHttpClient usage in relay (PATH B ONLY — requires adding convex to scripts/package.json)
// Path A: use plain fetch to a Convex HTTP action that wraps getByEnabled
const { ConvexHttpClient } = require("convex/browser");
const convex = new ConvexHttpClient(process.env.CONVEX_URL);

async function processEvent(event) {
  const enabledRules = await convex.query('alertRules:getByEnabled', { enabled: true });
  const matching = enabledRules.filter(r =>
    r.eventTypes.includes(event.eventType) &&
    matchesSensitivity(r.sensitivity, event.severity)
  );
  for (const rule of matching) {
    // SET NX dedup: atomic, prevents double-delivery within 30-min window
    const eventHash = sha256Hex(JSON.stringify({ type: event.eventType, title: event.payload.title }));
    const dedupKey = `wm:notif:dedup:${rule.userId}:${eventHash}`;
    const isNew = await upstashRest('SET', dedupKey, '1', 'NX', 'EX', '1800');
    if (!isNew) continue;
    // fan out
    const channels = await convex.query('notificationChannels:getChannels', { userId: rule.userId });
    for (const ch of channels.filter(c => c.verified && rule.channels.includes(c.channelType))) {
      await deliveryQueue.enqueue({ channel: ch, event, userId: rule.userId });
    }
  }
}

Telegram delivery with error handling

async function sendTelegram(chatId, text) {
  const res = await fetch(`https://api.telegram.org/bot${process.env.TELEGRAM_BOT_TOKEN}/sendMessage`, {
    method: 'POST', headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ chat_id: chatId, text, parse_mode: 'HTML' }),
  });
  if (res.status === 403 || (res.status === 400 && (await res.json()).description?.includes('chat not found'))) {
    await convex.mutation('notificationChannels:deactivateChannel', { userId, channelType: 'telegram' });
    return;
  }
  if (res.status === 429) {
    const retryAfter = (await res.json()).parameters?.retry_after ?? 5;
    await sleep((retryAfter + 1) * 1000);
    return sendTelegram(chatId, text); // single retry
  }
}

Slack delivery with SSRF prevention

async function sendSlack(webhookEnvelope, text) {
  const webhookUrl = decrypt(webhookEnvelope); // scripts/lib/crypto.cjs
  if (!isValidSlackWebhook(webhookUrl)) throw new Error('Invalid webhook URL');
  const { address } = await dns.resolve4(new URL(webhookUrl).hostname);
  if (isPrivateIP(address)) throw new Error('Webhook URL resolves to private IP');
  const res = await fetch(webhookUrl, {
    method: 'POST', headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ text, unfurl_links: false }),
    signal: AbortSignal.timeout(10000),
  });
  if (res.status === 404 || res.status === 410) {
    await convex.mutation('notificationChannels:deactivateChannel', { userId, channelType: 'slack' });
  }
}

Phase 4 Acceptance Criteria:

• wm:breaking-news event → Telegram message delivered < 10s end-to-end
• Same event not delivered twice within 30 min (SET NX dedup)
• Two concurrent relay instances do not double-deliver (SET NX is atomic)
• Unverified channel produces no outbound request
• Telegram 403 (user blocked bot) deactivates channel, stops retrying
• Slack URL re-validated at send time AND write time
• Slack URL resolving to private IP rejected at send time
• SIGTERM handled cleanly (Railway restart safety)
• CLERK_SECRET_KEY NOT present in Railway env (email from Convex cache, not Clerk API)
• RESEND_API_KEY added to Railway env and .env.example

---

System-Wide Impact

Interaction Graph

User changes panel layout
  → localStorage write (existing)
  → debounced syncToCloud() fires (new, 5s debounce)
    → POST /api/user-prefs (new Vercel edge action)
      → setPreferences mutation (Convex, server-stamps syncVersion)
        → userPreferences upserted

wm:breaking-news dispatched (browser)
  → POST /api/notify (new Vercel edge)
    → Upstash REST PUBLISH wm:events:notify
      → notification-relay subscriber wakes
        → queries alertRules (by_enabled index)
          → dedup check (SET NX)
            → fan out via token-bucket queues

User signs in
  → App.ts migrations run (v2.5, v2.6)
    → render with localStorage immediately
    → getPreferences query (Convex, parallel)
      → if cloud.syncVersion > local: apply cloud prefs → re-render panels

Error Propagation

Error	Location	Handling
Convex 401	Frontend sync	Silent; set wm-cloud-sync-state = 'signed-out'
Convex 409 CONFLICT	Frontend sync	Re-fetch, merge, retry once
Convex 500	Frontend sync	Retry after 5s; then 'error' state
Convex unreachable	Frontend sync	'offline' state; resume on window.addEventListener("online")
Telegram 403	notification-relay	Deactivate channel; stop retrying
Telegram 429	notification-relay	Respect retry_after; single retry
Slack 404/410	notification-relay	Deactivate channel
Slack private IP	notification-relay	Reject at send time; log; do not deliver

---

Dependencies

Dependency	Status
PR #1812 (Clerk migration)	Merged 2026-03-26 — no longer a blocker
convex added to scripts/package.json	Phase 0
resend added to scripts/package.json	Phase 0
.env.example notification vars	Phase 0
NOTIFICATION_ENCRYPTION_KEY in Railway env	Phase 0
TELEGRAM_BOT_TOKEN in Railway + Convex env	Phase 0
RESEND_API_KEY in Railway env	Phase 0
Q1 resolved (sync scope: all users vs pro)	Phase 2
Q3 resolved (AES key rotation strategy)	Phase 3b

---

Issue Segmentation

#2173 (this) — umbrella
  └─► Phase0 (sync-keys.ts, constants, crypto, env)
  └─► Phase1 (Convex schema + API)
  └─► Phase2 (frontend cloud prefs sync)
  └─► Phase3a (Notification channel linking UI)
  └─► Phase3b (Telegram bot webhook + pairing endpoint)
  └─► Phase4 (notification-relay Railway service)

Phase	Scope	Deps	Status
Phase 0	Sync key list, crypto, env, Convex crons	None	Can start now
Phase 1	Convex schema + preferences HTTP API	Phase 0	Can start now
Phase 2	Frontend cloud preferences sync	Phase 1 + Q1/Q2 resolved	After Phase 1
Phase 3a	Notification channel linking UI	Phase 1	Parallel with Phase 3b
Phase 3b	Telegram bot webhook + pairing endpoint	Phase 1	Parallel with Phase 3a
Phase 4	Notification delivery relay (Railway)	Phase 1 + Phase 3b	After Phase 3b

---

Internal References

• Settings key prefixes (current): src/utils/settings-persistence.ts:16
• Breaking alert settings: src/services/breaking-news-alerts.ts:15
• Preferences UI render + attach: src/services/preferences-content.ts:79
• UnifiedSettings host component: src/components/UnifiedSettings.ts
• Convex schema: convex/schema.ts
• Convex auth config: convex/auth.config.ts
• Clerk browser client: src/services/clerk.ts:131
• Auth state service: src/services/auth-state.ts:24
• Server-side JWT validation: server/auth-session.ts:67
• Redis client (Vercel): server/_shared/redis.ts
• atomicPublish: scripts/_seed-utils.mjs:169
• Resend email pattern: api/contact.js:32
• Convex mutation pattern: api/register-interest.js:244
• Railway relay: scripts/ais-relay.cjs
• Feature flag pattern: import.meta.env.VITE_CLOUD_PREFS_ENABLED === 'true' — do NOT use src/config/beta.ts (localStorage toggle)
• Hash utilities: server/_shared/hash.ts (use sha256Hex() for dedup, NOT hashString())

External References

• Convex-Clerk native auth
• Convex HTTP actions + CORS
• Convex discriminated union tables
• Convex scheduled functions (crons)
• Telegram Bot API: setWebhook + secret_token
• Telegram Bot API: deep linking (base64url)
• Slack incoming webhooks
• Upstash REST pub/sub
• AES-256-GCM key versioning

Related Work

• PR feat(auth): integrate clerk.dev  #1812 (feat/better-auth) — merged 2026-03-26, no longer a blocker
• PR feat: Dodo Payments integration + entitlement engine & webhook pipeline #2024 (Dodo Payments) — DEPLOYMENT-PLAN.md merge order: feat(auth): integrate clerk.dev  #1812 (merged) before feat: Dodo Payments integration + entitlement engine & webhook pipeline #2024
• Issue feat: Server-Side User Preferences Sync + Notification Delivery Channels #2173 — source issue on GitHub (with full revision history)

[koala73]

Plan Revision — March 2026

Updating this plan to reflect the current HEAD state. Six issues addressed below.

---

1. Clerk blocker is resolved — reframe the plan

The prerequisite framing "cannot start until PR #1812 lands" is stale. As of HEAD:

• convex/auth.config.ts — Clerk JWT provider configured with CLERK_JWT_ISSUER_DOMAIN
• src/services/auth-state.ts:24 — user.id exposed via snapshotSession()
• src/services/clerk.ts:131 — getClerkToken() returns a 50s-cached Clerk session token
• server/auth-session.ts:67 — validateBearerToken() does full JWKS verification (RS256, issuer-checked, accepts both convex template tokens and standard session tokens)

Update the overview: replace "cannot start until the Clerk migration is merged" with "auth substrate is in place; the remaining work is adding the Convex preference and notification tables, and the delivery relay."

---

2. Correct the implementation paths

Two paths in the current plan reference structures that don't exist:

Relay: The plan references a relay/ app directory. The actual relay is scripts/ais-relay.cjs with package name worldmonitor-railway-relay (scripts/package.json:4). There is one Railway service, one entrypoint. The plan should decide explicitly:

Decision: extend scripts/ais-relay.cjs with a notification subscriber, OR add a second named entrypoint (e.g. scripts/notification-relay.cjs) within the same scripts/ package.

A dedicated notification-relay.cjs is architecturally cleaner (separate crash domain, independent deploy) and avoids growing ais-relay.cjs further past its current 8,048 lines. The scripts/package.json scripts block should gain a "notification-relay": "node notification-relay.cjs" entry and a second Railway service pointed at the same repo pointing to that entrypoint.

Settings UI: The plan references a standalone NotificationSettingsPanel.ts. The actual settings surface is:

• src/components/UnifiedSettings.ts — modal shell with tabs
• src/services/preferences-content.ts — renders all preferences as HTML strings (not React components)
• src/settings-main.ts:580+ — wires export/import buttons and settings event handlers

Notification channel setup (Telegram pairing, Slack webhook entry) must be added as a new <details> group inside renderPreferences() in preferences-content.ts, not as a new standalone component. Event handlers wire in settings-main.ts.

---

3. Establish a single cloud-syncable key source of truth before building

The current export/import in src/utils/settings-persistence.ts:16 and the issue's sync allowlist are already diverging. Specifically:

Export includes keys the issue marks device-local (must exclude from cloud sync):

• worldmonitor-live-channels — active stream session state
• worldmonitor-active-channel — active channel session state
• map-height, map-pinned, mobile-map-collapsed — viewport state

Active preference keys not in export, but should sync:

• wm-market-watchlist-v1 (src/services/market-watchlist.ts:16)
• aviation:watchlist:v1 (src/services/aviation/watchlist.ts:6)
• wm-pinned-webcams (src/services/webcams/pinned-store.ts:1)
• wm-map-provider (src/config/basemap.ts:49)
• wm-globe-visual-preset (src/services/globe-render-settings.ts:113)
• wm-ai-flow-cloud-llm, wm-stream-quality (src/services/ai-flow-settings.ts)

Before writing any cloud sync code, Phase 0 must produce a canonical CLOUD_SYNC_KEYS constant (in src/utils/settings-persistence.ts or a new src/utils/sync-keys.ts) that is the single source of truth imported by both export/import AND cloud sync. Keys not in this list never leave the device.

wm-mcp-panels (src/services/mcp-store.ts) contains customHeaders with API keys — must remain explicitly excluded regardless of other MCP panel config.

---

4. Replace React/hook language with the app's actual model

The app is class-based TypeScript with Preact. preferences-content.ts renders innerHTML strings. Convex is accessed via ConvexHttpClient (see api/contact.js:3, api/register-interest.js:3) — there is no React tree, no useQuery, no WebSocket subscription wired to component re-renders.

Replace all useQuery / WebSocket subscription language in the plan with:

• Preference reads on sign-in: ConvexHttpClient.query(api.userPreferences.getPreferences, ...) called once after getClerkToken() resolves, result merged into localStorage.
• Pairing status polling: setInterval calling ConvexHttpClient.query(api.userPreferences.getChannels, ...) every 3s for up to 5 minutes after the deep-link is shown, then cleared. No WebSocket subscription needed for a pairing flow.
• Preference writes: debounced ConvexHttpClient.mutation(api.userPreferences.setPreferences, ...) on change events, same pattern as existing Redis writes.

---

5. Specify cloud sync sequencing relative to startup migrations

src/App.ts runs two in-place localStorage migrations at startup before any data is consumed:

• v2.6 — panel key renames (live-youtube → live-webcams, pinned-webcams → windy-webcams)
• v2.5 — layout reset (clears worldmonitor-panel-spans, worldmonitor-panel-order)

Cloud sync must not read from localStorage until both migrations have run, and must not write back to Convex until after the merge step — otherwise a first sign-in from a browser that has v2.4 state will push stale/pre-migration keys to the server.

Required startup sequencing:

App.ts: run local migrations (v2.5, v2.6, ...) 
  → load local panel/layout state 
  → cloud sync: fetch server prefs 
  → merge (server syncVersion wins on conflict) 
  → apply merged state

Export/import flow in settings-main.ts:591 must also call the canonical CLOUD_SYNC_KEYS filter so that device-local keys are excluded from exported files, aligning file-export with cloud sync scope.

---

6. Fix the feature flag guidance

src/config/beta.ts:1 is localStorage.getItem('worldmonitor-beta-mode') === 'true' — a localStorage toggle, not an env gate. VITE_CLOUD_PREFS_ENABLED should follow the pattern of existing VITE_* boolean flags (check import.meta.env.VITE_CLOUD_PREFS_ENABLED === 'true'), not beta.ts. Add it to .env.example and document that it is false by default until Phase 2 is QA-verified in staging.

---

7. Convex module naming — pick one and stick to it

The issue mixes convex/userPreferences.ts with later references to api.notifications.* and api.alertRules.*. Canonical layout:

convex/
  schema.ts              — tables: userPreferences, notificationChannels, alertRules, telegramPairingTokens
  constants.ts           — shared validators
  userPreferences.ts     — getPreferences, setPreferences queries/mutations
  notificationChannels.ts — getChannels, setChannel, deleteChannel, claimPairingToken
  alertRules.ts          — getAlertRules, setAlertRules

All frontend and relay code references api.userPreferences.*, api.notificationChannels.*, api.alertRules.* — never the mixed api.notifications.* naming.

---

Summary of required plan changes

Item	Change
Blocker framing	Remove; Clerk auth substrate is live
Relay architecture	Add scripts/notification-relay.cjs as second Railway service in same scripts/ package
Settings UI	Extend preferences-content.ts + settings-main.ts; no new standalone panel component
Sync allowlist	Phase 0 must produce CLOUD_SYNC_KEYS constant shared by export/import and cloud sync
Frontend patterns	Replace useQuery/WebSocket with ConvexHttpClient + setInterval pairing poll
Startup sequencing	Cloud sync must run after local migrations complete in App.ts
Feature flag	import.meta.env.VITE_CLOUD_PREFS_ENABLED === 'true'; not a beta.ts clone
Convex modules	userPreferences.ts, notificationChannels.ts, alertRules.ts — no notifications.* alias