Skip to content

GA 1/11 · roadmap, model-router service-token auth, runtime spec plumbing, quota leak, tenant-scoped idempotency - #265

Closed
dshakes wants to merge 801 commits into
masterfrom
ga2/01-scheduler-modelrouter
Closed

dshakes wants to merge 801 commits into
masterfrom
ga2/01-scheduler-modelrouter

Conversation

@dshakes

@dshakes dshakes commented Oct 23, 2026

Copy link
Copy Markdown
Owner

Part 1 of 11 of the GA next-gen stack. Replaces #264/#255–#263: same changes, history rewritten to drop a 43 MB build binary committed by mistake, plus part 11 fixing every confirmed Codex cross-audit finding. Base: master. Merge in order with a merge commit (not squash).

Context, verification and needs-human list: #254 and docs/roadmap/2026-10-nextgen-sandbox-and-harness.md.

Commits in this part:

dshakes and others added 30 commits July 4, 2026 00:21
…efault theme

- The collapsed-sidebar expand button was positioned `absolute left-4 top-14`
  but its <aside>/parent aren't `relative`, so it anchored to the wrong
  ancestor and landed off-screen — no way to re-open a collapsed sidebar.
  Render it in-flow instead (centered below the logo in the flex-col brand
  block), larger hit target + focus ring.
- Default theme is now DARK (no-flash init falls back to 'dark'); the light
  theme stays available via the top-bar toggle.
…ency

- Type scale felt too small: set root `font-size: 17px` (scales the rem-based
  text-xs/sm/base + spacing) and bumped the smallest FIXED sizes one notch
  across the dashboard + components (text-[10px]→[11px], [11px]→[12px],
  [13px]→[14px]). ~600 tiny labels are now legible without changing layout.
- Consistency (review gate): the default theme is now DARK, so the no-flash
  comment, ThemeToggle comment, and CLAUDE.md theming note now say dark-default
  (light via the toggle) instead of the stale "light is default".
fix(dashboard): visible expand button when collapsed + dark default theme
…ook-to-calendar

New event-scout agent: weekly web_search-grounded scan (next 60 days near
the owner) for kids/family, fireworks, Indian community, circus, expo,
light-show, fair and county/DC events. Owner approves from self-chat
("book 1,3" / "book all") and picks land in Calendar.app with 1-day +
2-hour alarms.

- packages/bridge-core/src/event-scout.ts: pure module — batch prompts,
  tolerant JSON parse, curation (LLM ranking pass -> top picks + why),
  category grouping (scan-LLM labels, no keyword heuristics), 6-per-page
  pagination with global numbering, command grammar, calendar specs.
- Scan runs one LLM call per 3-category batch with a 480s per-call
  timeout: a single all-category turn exceeds the 180s SSE default
  (verified live: AbortError -> silent 0 events). AgentClient gains a
  per-call timeoutMs opt.
- Categories/location/audience are owner-editable state (events
  add/drop/categories/location) in bridge_state/<tenant>/event-scout.json
  (0600, 120-day seen TTL).
- Wired in BOTH bridges (both owner routing paths); scout sessions run
  under the registered event-scout agent so the dashboard shows them.
  WhatsApp scheduled scan defaults OFF (LANTERN_EVENT_SCOUT_WA=1) to
  avoid double weekly texts.
- bot-self prefixes 🎪/🎟 registered; 15 unit tests; validated live
  (scan complete found:19 sent:true in owner log).
…-record success

Root cause of 'agents run but I never see outcomes':

1. FAILURE PROPAGATION. A loop body error (e.g. dead Gmail OAuth token)
   was logged and swallowed — every run finalized 'succeeded, records: 0',
   indistinguishable from a quiet week. runLoopAgentIfPresent now returns
   the body error; the inline executor marks the run FAILED (usage still
   recorded) and writes a loop_failed journal event instead of
   loop_complete, so recovery re-drives re-execute the body.

2. AUTH-DEATH OWNER PAGE. isConnectorAuthError classifies expired/revoked
   credential errors; the owner gets ONE self-chat page per tenant per day
   (claimSideEffect dedup): '🔑 Gmail connection expired — … Reconnect.'
   Previously 7 email agents were blind for weeks with zero human signal.

3. WEEKLY BRIEF PUSH. The domain coach's brief (already composed per
   domain per ISO-week) was stored ONLY as an fyi commitment — visible
   solely when the owner texts '?', which nobody knew to do. It is now
   ALSO pushed to the owner self-chat once per (domain, week), deduped by
   claimSideEffect; commitment storage unchanged. sendSelfNote is the
   injectable delivery seam (tests never text the owner).

4. DASHBOARD LAST-OUTCOME. Agent cards now render a 'last: …' line
   summarizing the latest run's output (or its error, in red) so a wall
   of green pills can't hide zero-output runs again.

Tests: isConnectorAuthError table; once-per-day page dedup (+ non-fatal
delivery failure); once-per-week coach push incl. commitment row; dispatch
success still writes loop_complete. All DB-backed via the existing
openTestPool harness.
Owner feedback: grouped-by-category output read as 'dates not sorted' and
the dense one-liners were hard to scan. Display rework:

- STRICT date order everywhere: picks block date-sorted (why-lines stay
  paired through the sort), remainder date-sorted, pages date-sorted.
  Category grouping removed — category now appears as a short per-line
  tag instead ('Indian community events' without the parenthetical tail).
- Two-line layout per event: 'N. Fri, Jul 17 7pm — Title' then indented
  'Venue, City · $25 · category'. 24h times humanized (19:00 → 7pm),
  'unknown' cost dropped instead of displayed.
- Terse footers: 'book 1,3 → calendar · events more → next 6'.
feat(bridges): proactive family event scout — weekly curated scan + book-to-calendar
Server-side agent output (weekly domain briefs, morning-brief runs, the
🔑 connector-auth page) was delivered via the WhatsApp bridge only —
inconsistent for an owner living in iMessage self-chat. The iMessage
bridge exposes the identical /session/:tenant/send-self contract on
:3200, so deliverWhatsAppSelfNote now posts to both bridges (best-effort
each; success = at least one delivered). Existing side-effect dedup keys
are unchanged — one claim covers the fan-out.
fix(delivery): fan owner self-notes out to both bridges
…le Mail index

Two fixes born from a real miss ("Manasa's green card" — the doc existed,
the assistant said it didn't):

Personal-docs search (bridge-core):
- Per-source pool quota (PER_SOURCE=30) so the first root can no longer
  fill the whole candidate pool before later roots (iCloud!) are searched;
  POOL_CAP is now just a runaway guard.
- Descend one level into folders whose NAME matches a phrase — the target
  file often has an unrelated filename inside an aptly named folder
  ("Green Card/SeshamManasa_LatestGC.pdf").
- Cumulative phrase scoring (basename +30 / full-path +25 per phrase, no
  first-match break) so matching BOTH "manasa" and "green card" outranks
  matching one.
- ~/Downloads added to default roots (real doc clusters live there).
- Regression test proven red on the old code.

search_email tool (new):
- OAuth-free read-only search over Apple Mail's local "Envelope Index"
  SQLite — sender/subject/date of the ENTIRE synced mailbox (Gmail incl.),
  never stale (live WAL read, no copy), fails closed, LANTERN_MAIL_INDEX=0
  killswitch. Pure SQL-build/format in bridge-core (tested); reader in the
  iMessage bridge; POST /session/:t/mail/search endpoint; control-plane
  tool catalog + executor; owner-prompt guidance (email HISTORY →
  search_email; gmail_search only adds live/full-body detail).
- E2E-validated through the live session tool loop (real flight
  confirmations + USCIS history returned).

Also: launchd bridge wrapper now sources ~/.lantern/bridge.env (0600) on
every start — env overrides reload with a plain kickstart -k, no more
bootout/bootstrap for env changes.
A folder matching the query in its own name tied its contained file
(folder basename 30/phrase vs file path 25/phrase + type bonuses) and the
winner fell to platform-dependent mtime/locale tie-breaks — passed on
macOS, failed on the Linux QA runner. Folders now score their own name at
the path rate: breadcrumbs, not answers, so the real file always wins.
Verified in a node:22 Linux container + full macOS suite.
…and-docs

feat(personal): folder-aware doc search + local search_email over Apple Mail index
…eeps attention items

- workflow-engine child_run step now fails with typed ErrChildRunUnavailable
  instead of journaling a fake child_started/child_completed pair and
  returning a placeholder output (mirrors ErrModelRouterUnavailable pattern);
  regression tests added
- daily-digest deterministic fallback (buildDigest) now renders commitments,
  overdue contacts, and sleep hours — a compose failure used to silently drop
  exactly the attention items; also reuses pre-fetched drafts/nextEvent
  enrichment instead of re-fetching
- CLAUDE.md: loop AND subagent interpreter nodes are implemented (stale
  no-op claim removed); durable-engine child_run limitation admitted
…r Brief

The Brief (? / today / whats up) becomes the one ranked view of what needs
the owner across iMessage + WhatsApp:

- new pure module bridge-core/attention.ts: strict-JSON ranking prompt,
  tolerant parser (unknown ids dropped, whys clamped, garbage -> null),
  candidate-id allowlist — the LLM can only reorder/annotate existing items,
  never invent or drop them
- command-center Brief: unified needs-you list (drafts + commitments +
  NEW waiting-on-you threads), rank-aware ordering via rankedIds, per-item
  'why' suffixes, thread action grammar (n draft / draft n / n review /
  n skip) with verb-first parsing
- waiting-on-you threads: overdue contacts were gated out of real-time
  nudges AND absent from the Brief — they vanished. Now surfaced VIP-first
  with last-inbound previews
- both bridges: gatherWaitingForBrief + rankAttention (session key
  attention::rank, 20s timeout, ANY failure keeps deterministic order),
  thread draft action generates a reply in the owner's voice and stages it
  in the existing pendingDraftEdits drafts rail
- bot-self echo registry: register '👍 won't surface' prefix
- docs: CLAUDE.md + INTELLIGENCE-LAYER.md sections

Tests: bridge-core 1275 pass (incl. 7 new attention tests), imessage 27,
whatsapp 121; both bridges tsc clean
…in prod; gateway API-key auth

Three GA-hardening changes, each fail-safe:

- bridge-core secure-store.ts: AES-256-GCM line encryption (enc1: envelope,
  fresh IV per line, key auto-created 0600 at <stateDir>/state.key) wired
  into the highest-PII stores — episodes.jsonl, topic-index.jsonl,
  dislike-patterns.jsonl. Legacy plaintext lines still read and age out;
  public module APIs unchanged. Tamper/garbage lines skip, never throw.

- harness egress preflight: prod (LANTERN_ENV=prod/production/staging) is
  ALWAYS fail-closed when egress rules are declared and the iptables
  REDIRECT is absent — an explicit LANTERN_EGRESS_FAIL_CLOSED=0 is ignored
  with a WARN. Dev behavior unchanged. Pure resolve_fail_closed + tests.

- gateway X-API-Key path implemented (was fail-closed TODO): new
  control-plane POST /internal/auth/introspect-key guarded by the existing
  LANTERN_GRPC_SERVICE_TOKEN (constant-time compare; 403 fail-closed in
  prod when unset), reusing validateAPIKey. Gateway introspects with a 60s
  SHA-256-keyed cache, builds tenant claims, fails closed on any error.
  New gateway env: LANTERN_CONTROL_PLANE_URL. Secrets never logged.

Docs: CLAUDE.md (egress model, internal endpoint + env vars, state
encryption) + INTELLIGENCE-LAYER.md stores section.

Verified: bridge-core 1293 pass; harness cargo test 130 pass + clippy
clean; gateway cargo test 31 pass + clippy clean; control-plane introspect
tests 4/4 (DB-backed) + go vet clean; both bridges tsc clean
…box Personal tab

The Attention Engine grows its zero-typing surface:

- reaction-commands.ts: ReactableLog (ring buffer of recent center-view
  messages), kind-aware mapBriefReactionToAction (👍/❤️ confirm =
  draft→send / commitment→done / thread→draft; 👎 skip; ⏰ snooze; ❓
  review; auto_action never tap-triggered), tapbackToEmoji for iMessage
  codes. Brief semantics take precedence over the legacy emoji map for
  remembered messages; everything else unchanged. 20 new tests.
- WhatsApp: reaction handler resolves Brief reactions to executeCenterAction;
  Living Brief edits the sent Brief in place (strikethrough) as items
  resolve within the 14-min edit window; GET /session/:t/attention snapshot
  endpoint.
- iMessage: tapback branch with cross-device dedup executes top-item
  actions; sent-Brief GUID capture via pendingBriefEcho → poll harvest
  (mirrors the queueReplyMeta mechanism); same attention endpoint.
- dashboard /inbox: new Personal tab (deep-linkable ?tab=personal) merging
  both channels' attention snapshots — counts row, channel grouping, why
  lines, stale pills, per-channel soft-fail, 60s visible-only refresh,
  read-only with copy-command affordances (chat is the acting surface).
  Typed fetchAttention in lib/bridge-client.ts.

Verified: bridge-core 1313 pass; imessage tsc clean + 27 pass; whatsapp
tsc clean + 121 pass; apps/web tsc clean + next build ok
#159)

* feat(skill-forge): safe self-improvement loop — owner teaches recurring skills

The OpenClaw killer feature, done safely: the owner asks for a new recurring
capability in self-chat, the LLM drafts it, and it only activates on explicit
approval. A skill is a scheduled PROMPT, never code — no shell, no filesystem,
no new capability surface.

- bridge-core/skill-forge.ts (pure): matchSkillRequest/matchSkillCommand
  grammar, buildSkillSpecPrompt + tolerant parseSkillSpec (strict JSON,
  name/schedule validation, honest {error} refusal for non-schedulable
  requests), dueSkills owner-local schedule matcher (50-min window > tick so
  none are missed), formatting. 6 tests.
- both bridges: maybeHandleSkillForge (draft → proposal → 'approve skill'
  10-min TTL → activate; 'skills' list; 'drop skill <n>'), maybeRunSkills on
  the proactive tick (killswitch/mute/quiet-hours gated; lastFiredDay stamped
  BEFORE run so a crash drops one delivery, never double-fires; web_search on).
  State: bridge_state/<tenant>/skills.json (0600).
- bot-self: register 🛠 / '✅ skill live' echo prefixes.
- docs: CLAUDE.md + INTELLIGENCE-LAYER.md sections.

Verified: bridge-core 1319 pass (3x stable); both bridges tsc clean +
27/121 pass.

* test(memory): first test suite for the memory service (was 0 tests)

9 table-driven tests over internal/handlers covering the highest-value
behaviors: pgvector literal formatting, write-request validation, tenant
scoping/isolation, store/retrieve round-trips, windowing, and error paths.
Adds middleware.WithTenantID — a test-only context helper (production still
routes tenant id through UnaryTenantInterceptor; the trust boundary is
unchanged).

Verified: go test ./... (DB-backed) + go vet clean.
…dges (#160)

* feat(time-travel): 'what did I miss' narrative recap — both bridges

Ask 'what did I miss' / 'what happened while I was out' / 'catch me up' and the
assistant narrates (not lists) what landed since you were last around: contacts
waiting on you + the actions it took on your behalf. Leads with the 1-2 things
that matter, compresses the rest.

- bridge-core/time-travel.ts (pure): looksLikeRecapRequest (≤80-char guard so a
  real message isn't mistaken for a recap ask), parseRecapWindow (last N
  min/h/days · today · overnight; default 6h), buildRecapPrompt (weight-ranked,
  forbids invention), finalizeRecap deterministic fallback. 9 tests.
- both bridges: maybeHandleRecap gathers gatherAwaitingReply (messages) +
  recentActions (auto-actions, presence filtered) since the cutoff, LLM
  narrates via recap::synth session key, deterministic fallback on failure.
  Wired at the owner self-chat dispatch (both routing paths on iMessage).

Scoped down from the original Batch 7: dropped delivery-mode (would duplicate
the existing focus-guardian/quiet-hours gating; true voice delivery needs
unwired TTS) and deferred plan-preview-confirm — shipping the one feature that
works end-to-end rather than padding the batch.

Verified: bridge-core 1318 pass (incl. 9 new); both bridges tsc clean + 27/121.

* test(secure-store): fix flaky tamper test (byte-level mutation)

The tamper→null test flaked ~25% of runs: it substituted a base64 char in the
ciphertext, but that can be a no-op (char already equals its replacement) or a
base64 alias near '=' padding that decodes to the same bytes — leaving the
ciphertext intact so decryption legitimately succeeds. Proven: 755/3000 no-op
tampers. The AES-GCM code was always correct (real tampers all return null);
only the test's mutation was unreliable. Now mutates the decoded ciphertext
buffer (ct[0] ^= 0xff), which always breaks the GCM tag. 0 failures across 25
runs (was ~25%).
…-Mac onboarding (#161)

Three grounded design docs (ADRs, not code) for the enterprise-GA arc:

- 0018 SOC 2 Type II: controls matrix mapping CC/A1/C1 to actual code (RLS,
  gRPC service-token, AES-GCM secrets+PII, egress fail-closed, Ed25519
  receipts, OTel). #1 product gap = hash-chained org-wide audit_log with SIEM
  export; then mTLS + central RBAC. Rest are PROCESS gaps (IR, access reviews,
  pen-test) that gate the Type II window, not code.
- 0019 SSO (SAML 2.0) + SCIM 2.0: slots beside the existing OIDC
  oauthLoginProvider, terminates at the same generateToken→LanternClaims
  choke point. Central risk: tenant binding (resolve existing tenant by
  connection, never from assertion body / RelayState). Deprovision = disable,
  not delete.
- 0020 Managed no-Mac onboarding: agent-runtime is ALREADY no-Mac
  (control/data-plane); WhatsApp-personal CAN be de-Mac'd (Baileys on Linux +
  state to RLS Postgres/S3); iMessage CANNOT (Apple-account/TCC-bound) —
  stated as a hard limit.

Headers normalized to the repo ADR convention; docs-links gate green.
…ity (#163)

Two owner-self-chat improvements:

plan-preview-confirm (bridge-core/action-plan.ts, pure): a multi-action
request ('text Madhu I'll call, snooze the lease thing, add fireworks Sat 9pm')
is decomposed by the LLM into a typed step plan, shown as ONE compact numbered
preview, and executed on a single 'do it' via the EXISTING executors (calendar
marker / draft rail / snooze). matchPlanRequest is conservative (>=2 verb-led
clauses joined by a conjunction; normal chatter like 'hey how are you and the
kids' returns false); the maybeHandlePlan hook is wired LAST so genuine
single-action commands keep their normal path. Step-drop ('2 skip') + 10-min
TTL. 19 tests incl. anti-hijack negatives. bot-self prefix registered.

owner-reply brevity fix: the owner self-chat persona had a MANDATORY
'agentic follow-ups' checklist that offered to 'save as note' any number it
saw — so a 2FA/OTP code triggered 'want me to save this as a note?' (saving an
expiring one-time code = zero intelligence), and the '1-3 lines' rule was
buried. Replaced the mandatory checklist with reasoning: answer in one line
for identification questions, never offer to save/remind ephemeral values
(OTP/2FA/verification codes, login links), don't echo the code back. Applied
to both bridges' owner-assistant prompts.

Verified: bridge-core 1343 pass (2x stable); both bridges tsc clean.
Note: the brevity fix needs a live bridge restart to take effect.
…#162)

API-key scopes were stored but never enforced. Add per-endpoint scope
authorization, staged behind LANTERN_AUTHZ_ENFORCE (default OFF) exactly like
the RLS rollout so it cannot break the live bridges/dashboard until validated.

- Scope taxonomy (auth.go): coarse read/write/admin (backward-compatible) +
  fine-grained agents/runs/connectors/budgets/settings scopes; admin implies
  all, write implies read (scopeImplies).
- AuthHandler.WithScope(scope, handler) wraps ~30 mutating endpoints
  (agents/runs/sessions/connectors/deployments/schedules/budgets/settings/
  templates writes + executes). It authenticates via the SAME validateRequest
  the handlers already used, and caches claims in ctx so contextWithTenant
  reuses them (no double token-validation / API-key DB hit) — NOT a new auth
  surface.
- Default OFF: missing scope → debug log + PROCEED (zero behavior change).
  LANTERN_AUTHZ_ENFORCE=1 → 403. JWT owner/admin keep full access.
- authz_test.go: enforce-off allows scopeless; enforce-on allows-with-scope /
  403-without / admin-implies / JWT-owner; scopeImplies table test.

Verified: go build ./... OK; full internal/handlers suite green (33s);
go vet clean. Extends invariant #7 toward the fine-grained authZ ADR 0018
flags as a SOC 2 gap.
#164)

Adds the LangSmith-style observability the eval suites lacked, read-only over
the existing eval_runs history (no new tables, no writes):

- control-plane eval_observability.go:
  - GET /v1/eval-observability/failures — failing cases grouped across recent
    runs, worst-first (case, failures/seen, failRate, latest sampleError +
    expected/actual for triage, first/last seen). Pure clusterFailures.
  - GET /v1/eval-observability/trends — score/pass-rate/cost time series
    oldest-first for charting, with a regression signal (latest score vs mean
    of priors). Pure computeTrend.
  - Both tenant-scoped via WithTenant; limit clamped (clampLimit).
  - eval_observability_test.go: clusterFailures (ranking, latest-triage-wins,
    all-pass→empty), computeTrend (regressing/improving/single-point),
    clampLimit. 4 tests.
- dashboard /evaluations: an Observability section — quality-trend sparkline
  with a ▼regressing/▲improving pill, and a top-failing-cases list. Empty
  states + per-call soft-fail (api returns safe empty shapes on error).

Verified: go build + full internal/handlers suite (37s) + vet clean;
apps/web tsc clean + next build OK. Answers ADR 0018's evals/observability
parity gap.
…t re-answer old messages (#165)

Root cause of the post-restart 'pulling old convos' cascade: the send-dedup
state (recentBridgeSends — what the bot has already sent, used to skip its own
chat.db echoes) persists on a 5s DEBOUNCE and was NOT flushed on shutdown. On a
launchd/SIGTERM restart (kickstart -k), stop() ran without flushing, so the last
<=5s of sends were dropped from sent.json. Because the owner texts their own
number, every bot message has a fromMe:false received-copy that iCloud can
sync in AFTER the restart — those copies then miss the (stale) loaded dedup set
and get re-ingested as fresh owner queries, so the bot re-answers old messages.

Fix: stop() now calls flushBridgeSendsNow() — a synchronous write (writeFileSync,
completes before server.close→process.exit) that cancels the pending debounce
and persists the latest state. The SIGTERM/SIGINT handlers already call stop(),
so a clean restart now always preserves the dedup set. Extracted the shared
writeBridgeSends() so the timer and the flush share one code path.

Note: the FIRST restart to deploy this still runs the old (no-flush) code, so a
brief cascade may recur once; every restart after is clean.

Verified: iMessage bridge tsc clean + 27 tests pass. Real validation is a live
restart with no re-answered backlog (owner).
The vuln CI check (cargo audit) went red on master from a cluster of recent
RustSec advisories, blocking every PR. Cleared across all 4 Rust services:

Bumped (fix available, Cargo.lock-only, within existing semver specs):
- anyhow 1.0.102 -> 1.0.103 (RUSTSEC-2026-0190 unsound downcast_mut) — all services
- crossbeam-epoch 0.9.18 -> 0.9.20 (RUSTSEC-2026-0204 invalid ptr deref) — runtime-manager

Ignored with justification (.cargo/audit.toml; unmaintained or fix needs an
upstream major bump we don't control; each entry documents why + reachability
+ revisit condition):
- RUSTSEC-2026-0097 rand 0.8 — transitive via opentelemetry_sdk / kube; unsound
  only with a custom GlobalRng we never install.
- RUSTSEC-2025-0134 rustls-pemfile — unmaintained; needs tonic 0.13; no CVE.
- RUSTSEC-2025-0012 backoff — unmaintained; transitive via kube-runtime;
  pure-logic, no network/parse surface.
- RUSTSEC-2024-0384 instant — unmaintained; transitive via backoff; native
  delegates to std::time.

No source changes. Verified: cargo audit exit 0 + cargo build clean in all 4
services (gateway, model-router, runtime-manager, harness).
…s by rowid (#170)

Chains after search_email: the LLM picks a rowid from a search_email result,
read_email resolves the on-disk .emlx/RFC822 payload and returns a bounded
plaintext snippet. Owner-only, fails closed (throw/empty → unavailable),
mirroring the search_email contract. gmail_search now only ADDS live detail;
search_email + read_email stand on their own if it errors.

- bridge-core/mail-index.ts: readEmailBody(rowid) parser + tests (8 cases)
- imessage-bridge: POST /session/:tenantId/mail/read route + reader wiring
- control-plane: read_email tool def requiring positive-integer rowid
…167)

Same class as the Splash Day all-day bug (#166), swept across the agent paths:

1. HIGH — anticipation.ts birthday/anniversary nudges fired a day EARLY.
   daysUntilAnnual/nextAnnualMidnight/dayBucket/monthBucket/recurrenceYear used
   UTC parts; on the owner's Mac in a negative-offset zone, an evening-before
   instant (= next-day UTC) made the engine say 'your anniversary is today' a
   full day early — and the dedupe key then suppressed the real on-the-day
   ping. Switched to LOCAL parts (owner-Mac TZ), component-construction so it's
   DST-safe. Fixes the daily-dedupe buckets (overdue/commitment/dormant) too.

2. MEDIUM — calendar.ts (Google Calendar connector) all-day events read as
   midnight-UTC. new Date('2026-07-07') = Jul 6 8pm EDT → an all-day event
   tomorrow wrongly read as 'in a meeting' tonight (nextMeetingWindow) and
   formatWhen rendered 'Jul 6, 8:00 PM' (wrong day + fabricated time). Now skip
   all-day events for the imminent-meeting check and render date-only values as
   a bare local '(all day)' date.

3. LOW — daily-digest.ts msUntilNextDigest mixed the owner-TZ hour with
   process-local minutes → up to 30 min off in a half-hour zone (India +5:30).
   Read hour+minute+second from the same owner-TZ formatter.

Tests: 3 forced-TZ (America/New_York / Asia/Kolkata) regression files that
deterministically catch each bug in any CI zone. bridge-core 1349 pass; both
bridges tsc clean.
…t-UTC safe) (#168)

Server-side counterpart to the bridge timezone sweep. Two confirmed bugs, both
rooted in 'no per-tenant/schedule timezone server-side'. Default-UTC == today's
behavior, so this is a NO-OP until a timezone is configured (zero prod regression).

1. HIGH — cron schedules fired in server-UTC, not the schedule's zone.
   NextCronTime matched cron fields in the caller's Location; every caller
   passed time.Now() (UTC in prod), so '0 9 * * *' fired at 09:00 UTC, not 9am
   local. Added scheduler.ResolveLocation(tz) (schedule tz → LANTERN_DEFAULT_
   TIMEZONE → UTC, never nil), a nullable schedules.timezone column
   (migration 0011), an optional  field on POST/PUT (invalid → 400),
   and threaded now.In(ResolveLocation(...)) through the create/update handlers,
   the scheduler advance loop, and the 3 loop_agent NextCronTime calls.

2. MEDIUM — daily budget/usage day rolled at UTC midnight. A PST tenant's
   $50/day hard-fail could pass ~$100 in one LOCAL day. One usageDate(now)
   helper (ResolveLocation('')) now backs ALL FIVE agent_usage_daily.usage_date
   sites (CheckBudget, CheckToolBudget, RecordUsage, AdjustUsageCost, forecaster
   loadBudget) so read/write buckets stay in sync.

BUG 3 (runtime.go date_trunc, LOW) left as-is — noted, separate.

Verified (re-run, not trusted): go build + vet clean; migration 0011 applies
(full DB-backed handler suite green, 21s); full scheduler suite green 2x;
ResolveLocation default-UTC confirmed; all 5 usage_date sites use usageDate();
tz tests pure + green. (A pre-existing cross-test DB-isolation quirk shows only
when TestBadCronExprDisablesSchedule is run in an ad-hoc -run subset; it passes
isolated and in the full suite — unrelated to this change.)
…imezone shift) (#166)

* fix(calendar): all-day events fired bogus 'starts in N min' nudges (timezone shift)

A tomorrow all-day event ('Splash Day at Goddard (Ved)') fired a pre-meeting
nudge 'starts in 9 min' at 7:51pm. Root cause: the macOS Calendar store keeps
all-day events at midnight GMT (not a wall-clock time). readCalendarFromStore
converted that to a JS instant and, in a negative-offset zone (EDT, UTC-4), it
rendered as ~8pm the PRIOR day — so a tomorrow all-day event looked like it
started at 8pm today, inside the 15-min pre-meeting window.

Fix:
- new exported calendarStoreDate(sec, allDay): timed events return the exact
  instant; all-day events rebuild LOCAL midnight of the GMT calendar day, so
  they land on the correct day at 00:00 local (no prior-evening shift).
- CalendarEventRead gains an allDay flag; the store read selects ci.all_day.
- BOTH bridges filter all-day events out of the pre-meeting 'starts in N min'
  nudge gathering — an all-day event is never a timed meeting (defense in depth).
- regression test: an all-day event at GMT-midnight lands on local midnight of
  the correct day, not the prior evening; timed events pass through unchanged.

Verified: bridge-core 1344 pass; both bridges tsc clean.

* fix merge: restore master files dropped by corrupted merge (audit/Cargo.lock etc.)
…l guard (#171)

Owner-reported: a contact (Chinna Akula) greeted 'Hi Bava' and the bot
mirrored 'Hi bava, ...' back. The owner never addresses ANYONE as bava,
even contacts who call him that.

- add bava/baava to TELUGU_ADDRESS_PARTICLE_RE (vocative position only —
  mid-sentence third-person reference still allowed)
- extend the dialect prompt rule to forbid mirroring the greeting back
- regression test from the real failing reply
…eaking internal notes + fake sends

Two live embarrassments, same persona bug, opposite directions:

- Withheld: a contact asked 'is Madhu in CA' but the owner's people-graph
  (## Relationships, incl. 'Madhu: Dublin, CA') was never injected into
  contact replies, so the bot deflected on a recorded fact. Now inject
  knownPeople (relationshipsBlock) + a THIRD-PARTY-FACTS rule allowing
  benign facts about people the owner knows (where family/friends live,
  how they're related), distinct from the owner's OWN private facts.

- Leaked: a contact asked for 'pan and aadhar'; the bot parroted an
  internal to-do ('send pan and aadhar to Sarika') verbatim and faked
  'sending now' for ID docs it can't attach. Add CONTEXT-BLOCKS-ARE-PRIVATE
  (never quote internal notes/commitments to a contact) and
  YOU-SEND-TEXT-ONLY (never fake a file/document send; never promise to
  send PAN/Aadhaar/passport/SSN/cards) rules. Owner-audience prompt is
  excluded from both.

Wired into both WhatsApp + iMessage bridge contact-reply paths.
Regression test: natural-third-party-facts.test.ts (3 cases).
dshakes and others added 18 commits September 7, 2026 04:26
…for them; WhatsApp liveness line (ADR 0024 W1.3 + W5) (#243)

* fix(bridges): a contact reply now knows what the assistant already did for them; WhatsApp gets the liveness line (ADR 0024 W1.3 + W5)

W1.3 — the working-memory action log was owner-only, so in the very thread
it took an action for, the assistant was amnesiac ("did you add it?" →
"I'll add it"). contactActionsBlock() surfaces ONLY actions whose summary
names this contact (first name, word-bounded), last 6h, non-presence, and
tells the model these HAPPENED: past tense, never re-promise, never invent.
Injected into selfContextBlock on both bridges; never in a group.

W5 — the WhatsApp bridge had no reportSilentDrops twin, so a muted/killed
bridge looked like a quiet day. Mirrors iMessage: every 45 min (and once
~5 min after boot) a LIVENESS line reports kill switch, mute (with drop
count), paused contacts, and group sends blocked at the boundary.

Tests: contact-actions (scope, window, presence, other-person leak, short
names), wiring on both bridges; bridge-core 1536/1536, iMessage 91/91.

* docs: LANTERN_VOICE_REFINE (ADR 0024 W3.4)

* fix(bridges): contact action memory keeps the NEWEST six, and a shared first name needs the full name (review on #243)
… five-string greeting table (ADR 0024 W2.1) (#244)

When two drafts trip the bot-tell guard, the reply used to fall straight to
greetingReply() — five hash-indexed strings — or silence. Now there is ONE
more attempt first: the hint carries both rejection reasons, a hard shape
rule (one short line the way the owner texts, no offer, no question back, no
exclamation marks), and the owner's three most relevant real messages as the
target. It goes through the same guard, and when accepted it continues into
every later gate (commitment, tier, refine) like any other draft — never a
bypass; [[NO_REPLY]] stays silent. Groups skip it; the greeting table is now
the fallback only when that also fails.

Tests: hint content (reasons, shape rule, ≤3 exemplars), wiring order on both
bridges; bridge-core 1547/1547, iMessage 95/95, both bridges tsc clean.
…soned register when the table is silent (ADR 0024 W2.4) (#245)

* feat(bridges): Telugu and Hindi distress no longer reads as neutral — a reasoned register when the English table is silent (ADR 0024 W2.4)

The 115-lexeme table is English. The relative who wrote "tension ga undi,
dabbulu levu" scored NEUTRAL and was answered breezily. Now the table runs
first and, ONLY when it is silent on text that reads as non-English (native
script or romanized Telugu/Hindi cues), one purpose-keyed, 10s-boxed judgment
(`<jid>::register`) decides distress / frustration / excitement / neutral.
Accepted at ≥0.6 confidence; any failure, low confidence, or English text
keeps the table's verdict. Groups skip it.

Tests: gate fires exactly when table-silent + non-English, never on English;
tolerant parse; conservative acceptance; fail-safe; zero calls on English.
bridge-core 1549/1549, iMessage 97/97, both bridges tsc clean.

* docs: the ADR 0024 reply-pipeline gates, in order, with where each decision lives

* ci: re-trigger checks (the docs-only push was skipped by the paths filter)
…odel before a foreign-language reply mode engages (ADR 0024 W2.5) (#246)

The wordlist scorer stays the cheap first pass. In its weak romanized band
(0.4–0.7 — where "Brambleton, VA" once read as French) one purpose-keyed,
8s-boxed call (`<handle>::lang`) confirms the writer's language; a confident
'english' disengages the mode, a confident non-English answer raises the
score, anything weak or broken keeps the wordlist's verdict. Native script
and strong signals never spend a call. iMessage contact path (the WhatsApp
bridge has no wordlist language mode to confirm).

Tests: band gate, disengage/raise/keep, no call on strong verdicts, wiring;
bridge-core 1551/1551, iMessage 98/98, both bridges tsc clean.
…hip for contacts with no declared one (ADR 0024 W2.3 + W1.4) (#247)

* feat: cross-thread recall on the semantic index, and a reasoned relationship for contacts with no declared one (ADR 0024 W2.3 + W1.4)

W2.3 — related context from OTHER threads keyed on exact string equality
between a capitalized-word regex and a noun allowlist ("wedding" never met
"marriage"). New GET /v1/memory/search runs the same hybrid RRF (embedding +
full-text, rank-fused) as /v1/memory/context over every person in the tenant
except the one being answered; both bridges query it with the inbound text
and the local topic graph fills after the semantic hits (offline fallback).

W1.4 — a contact with no line in ## Relationships was a stranger to the
persona even when the thread said "bava" fifty times. One purpose-keyed
judgment (`<handle>::relationship`, 12s) over the real thread proposes a
label + evidence; cached 0600 for a week / 20 inbounds. The label feeds the
persona ONLY, marked inferred + unconfirmed; every security gate keeps the
owner-declared relationship (an inference never widens inner-circle,
location, ## Now, or contact sharing). The owner gets one 🧭 FYI so a
one-line teaching confirms it.

Tests: parseSearchWindow clamps (Go), mergeRelated (dedup/order/shape),
inference prompt/parse/cadence/store/label, wiring on both bridges;
bridge-core 1552/1552, iMessage 102/102, both bridges tsc, go test ok.

* docs: /v1/memory/search and the inferred-relationship layer (ADR 0024 W2.3 + W1.4)

* fix(bridges): restore firstNameIsShared's closing brace lost in a rebase conflict resolution
…e to 'Manjakka' and be staged one 'send' away (#248)

Live, 2026-09-07 06:18: the relative wrote "akka number" (elder sister's
number). The request extractor took "akka" as a name, the resolver's substring
match found "Manjakka Uppal", and her number was staged in the owner's hold
page as "I found it — reply send". Two fixes, both deterministic:
- extractContactRequests drops Telugu/Hindi/English kinship and role words
  (akka, anna, bava, mama, didi, bhai, mom, boss, …) — they name a
  relationship, not a person the bridge can resolve.
- both bridges stage numbers for a one-word send ONLY when every match is
  resolver-proven (exact name, one phone); a fuzzy hit is never one tap away.

Tests: kinship inputs extract nothing while a real name still does; wiring
pinned on both bridges. bridge-core 20/20 (gate), iMessage 104/104, tsc clean.
… longer clobbers a pause set on purpose (#249)

Live, 2026-09-07: the relative the owner had paused until 2027 was re-engaged
by the bot (three holds paged in one minute). Cause: every time the owner
types in a thread, the takeover pause re-arms with its short default TTL and
overwrote the year-long expiry. Both bridges now take the max of the existing
and the new expiry; lifting a pause stays an explicit act.
…l drafts for the owner instead of going silent (#250)

* fix(bridges): mute means don't SEND, not don't THINK — a muted channel drafts for the owner instead of going silent

The owner asked why the approval queue was empty. It could not populate: the
mute gate returned before the reply pipeline, so a muted channel produced
neither a reply nor a draft. 18 real contacts were dropped in one morning
(Bhramari 5, Carlas 4, Manu 2, Morgan 2, and three more) with nothing to
approve.

Mute now runs the pipeline and hands every contact reply to the owner as a
draft, reusing the machinery the non-English injection caution already uses:
tier forced LOW with reason -muted-draft-for-owner, then the existing draft
branch. It fails CLOSED — resolveHeldReply suppresses when the owner hand-off
fails, so the one channel the owner explicitly silenced can never auto-send.
The KILLSWITCH remains the do-nothing-at-all switch, and a muted message not
addressed to the owner is still skipped outright.

Also: the classify CI step no longer paints a PR red when haiku finishes
without structured output — the label step already defaults to 'core', so the
step is continue-on-error rather than a blocking failure on a routing nicety.

Tests: resolveHeldReply fails closed on a muted hold and is unchanged
otherwise; wiring pinned on both bridges. iMessage 111/111, bridge-core
1558/1558, both bridges tsc clean, actionlint clean.

Claude-Session: https://claude.ai/code/session_01Vwnsmx7xLXF1kQauAhnEnj

* fix(bridges): make mute a send-boundary invariant — muted group replies were falling through to a real send

Three reviewers on #250 caught the same regression, and they were right: the
draft branch is !isGroup-gated, so a muted monitored-group message that
targets the owner (or a celebratory wish) skipped the draft path entirely and
auto-sent to the group — the exact thing the change exists to prevent, and
worse than the old early return.

A branch condition is not an invariant. While muted, send()/sendMessage() now
block every destination that is not the owner, which covers the group path,
the contact-share direct-send shortcut, the greeting fallback and live-watch
follow-ups in one place. The boundary resolves the owner with the SAME helper
the draft hand-off uses, so it can never block a draft the bridge could
otherwise deliver. Muted group replies are additionally routed to the owner's
draft queue rather than silently blocked, and the confident contact-share
shortcut cannot fire while muted.

Tests: boundary present and after the sentinel guard on both bridges, muted
groups reach the draft path, share shortcut gated on mute. iMessage 117/117,
bridge-core 1558/1558, both bridges tsc clean.

Claude-Session: https://claude.ai/code/session_01Vwnsmx7xLXF1kQauAhnEnj
…+ world model from mail/calendar/self-chat/threads (#251)

* fix(bridges): a contact reply that ASSERTS something about the owner's life it cannot ground is HELD, not sent — grounding gate + memorial register

Two live incidents, both auto-sent as HIGH/MEDIUM:

  2026-09-18 07:02 WhatsApp, opening day of the owner's store.
    They: "Showtime today??"  → Bot: "not today, sep 10 is the big day 🎉"
    They: "18th anukuna"      → Bot: "haha no worries, it was 10th 😄"
    They: "Oh done ha"        → Bot: "ha, all done and rolling now"
  The only date the persona held was a `## Public` profile line eight days
  stale, rendered with a "do not contradict" directive; `## Now` had the
  right date but was expired and inner-circle-gated; no calendar or mail
  read fired because the calendar gate needed an availability verb. The
  bot corrected a human twice about the owner's own opening day, then
  invented the outcome. iMessage had seen "still on for the 18th" nine
  days earlier.

  2026-09-17 05:06 WhatsApp (replayed after quiet hours). A memorial-
  ceremony card ("invitation for a ceremony … photo of the person") was
  answered as an exciting party: the vision caption carried no tone, the
  register table had no memorial lexeme, the LLM register judge only ran
  on non-Latin text, and classifyConfidence never read inboundText.

What changed:
- packages/bridge-core/src/grounding-gate.ts (new, pure, tested): LLM
  judge (`${jid}::groundgate`) + deterministic backstop; held if either.
  Holds a fresh calendar date asserted in answer to a timing question, a
  date that contradicts the contact's, an unverifiable outcome claim, or
  a loss/memorial inbound answered in a celebratory register. Wired in
  BOTH bridges right after the commitment gate: forces LOW and the draft
  path even when draft-confirm is off; owner page "⚠️ HELD — … the draft
  claims … I can't verify"; fail-closed when the owner hand-off fails.
- owner-profile.ts: publicBlock(today) flags a Public fact whose date has
  passed (firstDateISO); natural.ts Public directive now defers to the
  contact on changed dates instead of "do not contradict".
- Contact calendar gate (both bridges) also fires on today / tomorrow /
  opening / showtime / still on anchors.
- Vision caption prompt (both bridges) names the event kind and tone; a
  card with a person's photo, dates and ceremony wording is a memorial.
- emotional-register.ts: memorial/condolence/rite lexemes (English +
  Telugu/Hindi); the model judges `[image …]` captions.
- confidence-tier.ts: grief on the CONTACT's side now raises risk
  (grief-inbound) — inboundText was declared and never read.
- scripts/introspect/PROMPT.md: the audit now covers "asserted a fact it
  couldn't ground / corrected the contact" and "wrong emotional register".
- CLAUDE.md: grounding-gate section.

Verified: bridge-core 1569/0; whatsapp-bridge 119/2 (the 2 failures in
__tests__/severe-fixes.test.ts fail identically on master, unrelated);
imessage-bridge 117/0 after the wiring test learned the new hold line;
tsc clean on both bridges.

* feat(bridges): the owner's present-tense world model is derived from mail + calendar on a cadence — dated Now/Public lines nobody has to type

Every fact the bot knew about the owner's life had to be TYPED into
owner-profile.md. On the store's real opening day the profile still said
the 10th: Apple Mail had a week of Square sales reports and vendor
confirmations, the device calendar had the day, an iMessage thread had
"still on for the 18th?", and nothing revised the line — so the bot
corrected a friend from an eight-day-stale fact.

- packages/bridge-core/src/world-model.ts (new, pure, tested): prompt,
  anti-hallucination parse (every item must cite a `source` that appears
  verbatim in the inputs and carry an `until:` in [today, +120d]),
  profile apply (upsert `## Now`; rebuild the bot-owned `## Public
  (managed)` wholesale, never the owner's hand-written `## Public`),
  owner FYI (`🧭 updated what I know …`, one line from a correction).
- iMessage bridge: refreshWorldModel rides the anticipation tick every
  LANTERN_WORLD_MODEL_HOURS (default 6; LANTERN_WORLD_MODEL=0 disables);
  inputs = last 7 days of envelope-index subjects (listRecentMail,
  read-only, never logged) + 30 days of device calendar + current lines;
  one purpose-keyed call (`owner::worldmodel`); state world-model.json.
- mail-index.ts: buildRecentMailQuery (no keyword filter); mail-reader:
  listRecentMail with the same fail-closed posture.
- owner-profile.ts: publicBlock(today) honors `| until:` on Public lines
  (expired lines vanish; live ones render "(through <date>)").
- upsertNowLine exported for the world-model writer. CLAUDE.md section.

Verified: bridge-core 1574/0; imessage-bridge 117/0; tsc clean on both
bridges. First live run is UNVERIFIED until deployed — check
imessage-bridge.out.log for "world model:".

* feat(bridges): the world model also reads the owner's self-chat and inbound contact threads on both channels

Mail subjects and the calendar were the only evidence; the two sources that
actually carried the opening-day correction were a self-chat note and a
friend's "still on for the 18th?" on iMessage. Now:

- world-model.ts: `selfChatLines` (highest authority — a correction here
  overrides mail and calendar) and `threadLines` (INBOUND contact messages;
  evidence about the OWNER's events only, never a contact's own life) are
  prompt inputs and count as valid `source` text. Pure readers
  `imessageEvidence` / `waHistoryEvidence` / `mergeEvidence`: bot-emitted
  self-chat lines are skipped (isBotSelfMessage) and the bot's own replies
  are excluded by construction — they may be the stale claim.
- chat-db.ts: `recentMessagesSince` (1:1 both directions, 7 days,
  attributedBody decoded, reactions/groups excluded).
- iMessage bridge reads WhatsApp's wa-history.jsonl directly
  (LANTERN_WA_HISTORY_FILE override; self-chat slice keyed by
  LANTERN_WA_OWNER_JID). Both channels feed one refresh.

Verified: bridge-core 1576/0; imessage-bridge 117/0; tsc clean on both
bridges. Live run still UNVERIFIED until deployed.
…ssages — parity with the WhatsApp guard (#252)

Found by the introspection loop on its first successful audit in ~4 weeks
(it wrote the fix + regression test before its run died on a network error,
leaving them uncommitted).

`maybeIngestUnknownInbound` was called without the `!isGroup` guard the
WhatsApp bridge already has at its call site, so an appointment-shaped
message in a GROUP could be claimed by the life-event/appointment ingester
and the contact reply suppressed — a silent drop in a group thread.
`isGroup` is already a parameter of the enclosing `handleInbound`; every
sibling branch around it is `!isGroup`-gated.

Verified: imessage-bridge 118/0 (incl. the new
unknown-inbound-group-scope regression test), tsc clean.
…— it was dead code for the one incident it was built for (#253)

Live finding, 24h after deploy: the world-model refresh had not run ONCE.
The iMessage bridge is muted, and `maybeRefreshWorldModel` sat below the
`proactivePaused()` early return — which is true whenever `this.muted` is
set. On a muted channel the whole refresh was unreachable.

That is exactly backwards. Mute means "don't SEND to contacts" (#250), not
"stop thinking". The refresh messages no one: it reads the owner's own mail,
calendar and self-chat notes and updates the OWNER's profile. And the bridge
WAS muted on 2026-09-18 — so the refresh that would have corrected the
stale grand-opening date never ran, on the very day the stale date made the
bot correct a friend twice. The feature shipped switched off by an unrelated
flag.

- `maybeRefreshWorldModel` moves ABOVE the pause gate, under the killswitch.
- Quiet hours and an explicit "quiet Nh" still defer it (its FYI would wake
  the owner); the next tick after the window picks it up.
- The killswitch — the do-nothing-at-all switch — still stops it.
- Regression test is structural, because the bug was WHERE the call sat:
  below that return it is dead code, and no behavioural test of the function
  itself would have caught it.

Verified: imessage-bridge 120/0, tsc clean.
…ped before it existed

A freeform self-chat message landing within milliseconds of a draft being
staged was treated as the owner's deliberate replacement text and forwarded
verbatim to the draft's contact. Live incident: an owner message about an
unrelated SLA digest ("morning, those are auto-nudges not real tasks") landed
11ms after a draft was staged for contact Safiya and got queued to send to her
as-is — blocked only because the channel happened to be muted at that moment.

classifyPendingReply now requires >=3s between the draft's issuedAt and the
reply before falling through to "replace" (approve/reject are unaffected).
Shared in bridge-core, so both bridges get the fix from one change.
…empotency keys, scheduler quota leak + tenant-scoped idempotency, SSE exec streaming
@dshakes

dshakes commented Oct 24, 2026

Copy link
Copy Markdown
Owner Author

Superseded: the history rewrite on this stack accidentally re-signed master's commits (base diverged). Replaced by a range-limited rewrite on ga3/* — see the new GA 1/11 PR.

@dshakes dshakes closed this Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant