Skip to content

fix(bridges): the bot was blind to its own commitments, and killed real replies - #224

Merged
dshakes merged 4 commits into
masterfrom
fix/bridge-self-context-and-tell-guard
Aug 24, 2026
Merged

dshakes merged 4 commits into
masterfrom
fix/bridge-self-context-and-tell-guard

Conversation

@dshakes

@dshakes dshakes commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Four defects found by auditing a week of production logs, chat.db, and the reply path. All four were silent — nothing errored, the bot just sounded like a bot.

1. It denied work it was actively doing

Owner: "Track above flight" → "I don't see a flight in our conversation." The watch was on disk with 4 completed checks.

Two things had to line up: WatchStore was never injected into the owner prompt, and the bot's own 📡 watching: announcement is stripped from history by isBotSelfMessage (correct — it's the echo-loop guard). So the only record of the commitment was erased and the store was never read. Not a reasoning failure; the context genuinely wasn't there.

watchContextBlock() renders active watches into the owner prompt. Verified by replaying the real stored watch at the exact failure timestamp.

2. The same blindness on the contact path — the general case

Every module documented "owner self-chat only" writes state the contact reply path never reads. working-memory has 9 write sites and zero reads there. The bot books Raju's event, then replies to Raju with no idea it happened.

contactWatchBlock() + a selfContextBlock field close it for watches — the case with per-contact attribution, and therefore the case that is safe to surface. Thread isolation is the load-bearing property and is tested: a watch started off one contact's message can never appear in another's thread. working-memory stays out until its actions carry a contact; dumping it would leak.

3. It suppressed real human drafts

AI_TELL_WORDS matched as a raw substring:

  • "as personal as it gets" → tripped "as per"
  • "kindlyn said she'd come" → tripped "kindly"

Both reproduced. A suppressed draft falls through to five hardcoded greeting strings, hash-indexed — so a contact texting "hey" gets the byte-identical reply forever — or to silence.

Fixed to word-boundary matching with inflections. Every genuine tell still suppresses; the test pins both directions.

4. Reply quality was structurally unmeasurable

Successful sends were never logged with their text. A full week of traffic left ~33 verbatim replies, all from sidecar files. Logged at the existing send choke point after delivery confirms, bot-self tagged rather than dropped so "real reply vs machinery" is answerable — 210 of 627 sent messages were machinery, and that ratio was invisible. Preview-capped at 160 chars.

Verification

  • 1496/1496 bridge-core tests pass
  • Both bridges typecheck clean
  • bridge-core has 972 pre-existing tsc errors (missing @types/node); this diff adds no new error class — verified against a stashed baseline

Deliberately not fixed

iMessage hardcodes disclosed: false while WhatsApp passes real state. The obvious one-word fix wouldn't compile (disclosedJids doesn't exist there) and the feature is inert anyway — env unset, set empty.

Known ceiling, not addressed

Memory has no embeddings. It's token-set intersection, so "any update on that thing we discussed?" scores zero against everything and returns the 6 newest irrelevant episodes.

…al replies

Four defects found by auditing a week of production logs, chat.db, and the
reply path. All four were silent — nothing errored, the bot just sounded
like a bot.

1. It denied work it was actively doing.

   Owner: "Track above flight" → "I don't see a flight in our conversation."
   The watch was on disk with 4 completed checks. Two things had to line up:
   WatchStore was never injected into the owner prompt, AND the bot's own
   "watching:" announcement is stripped from history by isBotSelfMessage
   (correct — it's the echo-loop guard). So the only record of the
   commitment was erased and the store was never read. Not a reasoning
   failure; the context genuinely wasn't there.

   watchContextBlock() renders active watches into the owner prompt.
   Verified by replaying the real stored watch at the exact failure
   timestamp — it now carries the flight, the contact, and the last result.

2. The same blindness on the contact path, which is the general case.

   Every module documented "owner self-chat only" writes state the contact
   reply path never reads. working-memory has 9 write sites and zero reads
   there. The bot books Raju's event, then replies to Raju with no idea it
   happened — the "no cross-app context" complaint, exactly.

   contactWatchBlock() + a selfContextBlock field on the persona prompt
   close it for watches, which is the case with per-contact attribution and
   therefore the case that is safe to surface. Thread isolation is the
   load-bearing property and is tested: a watch started off one contact's
   message can never appear in another's thread. working-memory stays out
   until its actions carry a contact — dumping it would leak.

3. It suppressed real human drafts.

   AI_TELL_WORDS matched as a raw substring, so "as personal as it gets"
   tripped "as per" and "kindlyn said she'd come" tripped "kindly". Both
   reproduced. A suppressed draft falls through to five hardcoded greeting
   strings, hash-indexed — so a contact texting "hey" gets the byte-identical
   reply forever — or to silence.

   Word-boundary matching with inflections. Every genuine tell still
   suppresses ("kindly let me know", "delve", "as per our discussion",
   "delved"), which the test pins from both directions.

4. Reply quality was structurally unmeasurable.

   Successful sends were never logged with their text. A full week of
   traffic left ~33 verbatim replies, all from sidecar files — too few to
   audit, and no way to tell whether any change helps. Logged at the
   existing send choke point after delivery confirms, bot-self tagged
   rather than dropped so "real reply vs machinery" is answerable (210 of
   627 sent messages were machinery; that ratio was invisible).
   Preview-capped at 160 chars like every other textPreview.

1496/1496 bridge-core tests pass. Both bridges typecheck clean; bridge-core
has 972 pre-existing tsc errors (missing @types/node) and this diff adds no
new error class — verified against a stashed baseline.

Not fixed, deliberately: iMessage hardcodes disclosed:false while WhatsApp
passes real state. The one-word fix wouldn't compile (disclosedJids doesn't
exist there) and the feature is inert anyway — env unset, set empty.

Known ceiling, not addressed here: memory has no embeddings. It is token-set
intersection, so "any update on that thing we discussed?" scores zero against
everything and returns the 6 newest irrelevant episodes.
@github-actions github-actions Bot added the domain:core Core library / business logic label Aug 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔎 Codex cross-audit (agent:audit)

Blocking

  • services/imessage-bridge/src/session.ts:5617 and services/whatsapp-bridge/src/session.ts:3433: this logs textPreview for every successful outbound send. That can include owner-only self-chat replies, document/ID answers, OTPs, private-vault-derived text, or sensitive contact replies. The repo invariant says secrets must never appear in logs; “preview-capped” does not make the first 160 chars safe. Redact, classify, or make this opt-in local debug only.

  • packages/bridge-core/src/live-watch.ts:295 and packages/bridge-core/src/live-watch.ts:324: topic, doneCondition, and lastSummary are untrusted LLM/web/contact-derived strings, but they are injected raw into high-priority prompt context. A contact or search result can persist text like “ignore previous instructions…” in the watch store, then have it replayed on later turns as “already tracking” context. Treat these fields as untrusted data: quote/escape/delimit them, strip instruction-like content, and add regression coverage for prompt-injection text in watch metadata.

Tests

I attempted the focused packages/bridge-core tests, but they could not run because tsx is missing in this checkout; all failures were module-loader errors before test code executed.

@compass-sdlc-bot compass-sdlc-bot Bot added the agent:reviewed-clean Reviewer found no Blocking issues this round label Aug 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔷 Gemini cross-audit (agent:audit-gemini)

Audit Report: Unified Diff Assessment (pr-diff.patch)


⚠️ Blocking / Correctness Regression

Dropped-E Verb Inflections Bypass AI-Tell Filter

The word-boundary regex generator aiTellRe(word) incorrectly assumes all suffixes append directly to the literal base word:

re = new RegExp(`\\b${escaped}(?:'?s|d|ed|ing)?\\b`, "i");

In standard English, verbs ending in a silent e drop the e before taking the -ing suffix (e.g., delve $\rightarrow$ delving, navigate $\rightarrow$ navigating, facilitate $\rightarrow$ facilitating, utilize $\rightarrow$ utilizing, elaborate $\rightarrow$ elaborating).

  • The Bug: For delve, the regex generated is \bdelve(?:'?s|d|ed|ing)?\b. This matches the non-existent "delveing" but fails to match the correct English word "delving".
  • Impact: Common AI-generated tell phrases (e.g., "delving deeper into...", "navigating this situation", "facilitating the process") will completely bypass the coherence filter, leaking obvious bot indicators to contacts.
  • Recommended Remedy: Adjust aiTellRe to handle silent-e dropping gracefully:
    function aiTellRe(word: string): RegExp {
      let re = aiTellReCache.get(word);
      if (!re) {
        let pattern = word.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
        if (word.endsWith("e")) {
          const base = pattern.slice(0, -1);
          pattern = `${base}(?:e|e'?s|ed|ing)`;
        } else {
          pattern = `${pattern}(?:'?s|d|ed|ing)?`;
        }
        re = new RegExp(`\\b${pattern}\\b`, "i");
        aiTellReCache.set(word, re);
      }
      return re;
    }

🔍 Non-Blocking Findings & Observations

1. Outbound Audit Logger PII Safety (Compliance with Invariant #10)

The addition of outbound message logging (textPreview: text.slice(0, 160)) in both IMessageSession and WhatsAppSession is highly effective for auditing quality while complying with Invariant 10. Slicing at 160 characters is consistent with other previews in the codebase, ensuring full message bodies (and potential bulk PII) are never logged verbatim.

2. Performance & Thread-Safety of aiTellReCache

The module-level caching const aiTellReCache = new Map<string, RegExp>(); is highly performant. Since AI_TELL_WORDS is a fixed, small list (14 entries), it has a bounded memory footprint. Because JavaScript/Node.js executes on a single-threaded event loop, there are zero concurrency/race hazards with this shared cache.

3. Isolation of contactWatchBlock and watchContextBlock

  • Contact-facing isolation is perfectly preserved by filtering against the specific JID (w.jid === jid), preventing cross-contact leaks.
  • Owner-facing visibility correctly gathers active watches globally, enabling the owner's briefing loop to accurately represent what the bot is actively tracking without hallucinations.

@github-actions

Copy link
Copy Markdown
Contributor

🛡️ Vuln scan — ❌ vulnerable dependency found

cd services/control-plane && govulncheck ./...
=== Symbol Results ===

No vulnerabilities found.

Your code is affected by 0 vulnerabilities.
This scan also found 1 vulnerability in packages you import and 15
vulnerabilities in modules you require, but your code doesn't appear to call
these vulnerabilities.
Use '-show verbose' for more details.
cd services/gateway && cargo audit
    Fetching advisory database from `https://github.com/RustSec/advisory-db.git`
      Loaded 1225 security advisories (from /home/runner/.cargo/advisory-db)
    Updating crates.io index
    Scanning Cargo.lock for vulnerabilities (272 crate dependencies)
Crate:     h2
Version:   0.4.13
Title:     h2 unbounded empty DATA frames
Date:      2026-08-17
ID:        RUSTSEC-2026-0258
URL:       https://rustsec.org/advisories/RUSTSEC-2026-0258
Solution:  Upgrade to >=0.4.16

error: 1 vulnerability found!
make: *** [Makefile:268: audit] Error 1

`vuln` was already red before this branch existed — the advisory published
2026-08-17 and cargo audit fails on unmodified master (reproduced locally).
h2 is transitive via hyper/tonic/axum, so it is a lockfile-only bump.

h2 0.4.13 -> 0.4.18. cargo audit clean, cargo check clean.

Not required by the branch ruleset (only `review` and `qa` are), and unrelated
to the bridge fixes on this branch — but a DoS advisory on the gateway's HTTP/2
stack is not something to merge past just because the gate would have let it.
@github-actions

Copy link
Copy Markdown
Contributor

🛡️ Vuln scan — ❌ vulnerable dependency found

cd services/control-plane && govulncheck ./...
=== Symbol Results ===

No vulnerabilities found.

Your code is affected by 0 vulnerabilities.
This scan also found 1 vulnerability in packages you import and 15
vulnerabilities in modules you require, but your code doesn't appear to call
these vulnerabilities.
Use '-show verbose' for more details.
cd services/gateway && cargo audit
    Fetching advisory database from `https://github.com/RustSec/advisory-db.git`
      Loaded 1225 security advisories (from /home/runner/.cargo/advisory-db)
    Updating crates.io index
    Scanning Cargo.lock for vulnerabilities (272 crate dependencies)
cd services/model-router && cargo audit
    Fetching advisory database from `https://github.com/RustSec/advisory-db.git`
      Loaded 1225 security advisories (from /home/runner/.cargo/advisory-db)
    Updating crates.io index
    Scanning Cargo.lock for vulnerabilities (251 crate dependencies)
Crate:     h2
Version:   0.4.13
Title:     h2 unbounded empty DATA frames
Date:      2026-08-17
ID:        RUSTSEC-2026-0258
URL:       https://rustsec.org/advisories/RUSTSEC-2026-0258
Solution:  Upgrade to >=0.4.16

error: 1 vulnerability found!
make: *** [Makefile:269: audit] Error 1

The gateway bump was necessary but not sufficient — `make audit` runs cargo
audit against three Rust services and the same advisory sat in the other two
(model-router 0.4.13, runtime-manager 0.4.14; the fix is >=0.4.16). CI proved
it: gateway went green and `vuln` still failed on the next service in the list.

Both to 0.4.18, lockfile-only (h2 is transitive via hyper/tonic). `make audit`
now exits 0 locally; cargo check clean on both.
…relevance

Two fixes to semantic recall. The embeddings were already there and working —
11,088 rows, 100% embedded, live across gmail/whatsapp/imessage/calendar — so
neither of these is "add embeddings"; both are about how the existing index is
queried.

A. Relevance was capped at 14 days, and shouldn't have been.

   `unifiedBlock` issued ONE windowed request and used it for two different
   questions: "what did we discuss recently" (genuinely time-scoped) and "what
   is relevant to what they just said" (not). So semantic recall could only
   ever match inside the window. Measured on this deployment: 1,622 of 1,850
   chat events — 88% — were unreachable no matter how relevant. A contact
   asking about something from last month got "what's that?" while the answer
   sat embedded and indexed.

   Split into two reads: the query pass runs UNWINDOWED (relevance decides,
   not age), the recency slice keeps its window (that block says "last N days"
   and would otherwise be a lie). Older hits render in their own block, dated,
   told not to assume they still hold — "we talked about this in June" is a
   different claim from "yesterday" and the model must not blur them. Rows
   already shown in the recency block are de-duped out.

B. Keyword and vector were either/or; now they fuse.

   Lexical ran ONLY as a fallback when embedding failed, so a query needing
   both got whichever won, never the union. Vector alone is weak on precisely
   what personal chat is made of — names, dates, order/flight numbers — while
   lexical alone misses "decorations" -> "balloon decor".

   Reciprocal Rank Fusion, k=60. Fusing by RANK is the point: cosine distance
   and ts_rank aren't comparable and normalizing them needs a corpus-wide
   calibration we don't have. A row found by BOTH lists outranks a row found
   by either alone. Each arm over-fetches (candidatePool) because fusing two
   lists only `limit` deep would hide exactly the rows that agree just past
   the cutoff. websearch_to_tsquery parses user text safely — no injection, no
   syntax errors on stray punctuation, no rows rather than an error on junk —
   so the vector arm still stands if the lexical arm matches nothing.

Verified live against the running API, not just unit tests:
  "what did we decide about the decorations" -> "Or option 3 with some ballon
  decor on the side" (no shared keyword) PLUS July rows the 14-day window had
  been discarding. Lexical-precision cases ("survey plot", "shloka gift") now
  return the exact rows.

go vet + go build clean, handler tests pass, 1496/1496 bridge-core tests pass.
@dshakes
dshakes merged commit 0b9990a into master Aug 24, 2026
12 checks passed
@dshakes
dshakes deleted the fix/bridge-self-context-and-tell-guard branch August 24, 2026 11:55
dshakes added a commit that referenced this pull request Sep 4, 2026
…sured reply loop (#231)

The plan behind PRs #224–#230, written down so it outlives the session that
produced it. Fourteen root-caused production failures from 2026-08-23 to
2026-09-04 are the Context; five workstreams derived from them are the
Decision.

The three findings the plan rests on:
  - none of the fourteen threw an error — every one was silent
  - the owner told the assistant the facts and they never reached the bot
  - static classifiers pre-empt the model at every gate a contact can see

And the three things the 2026 literature says NOT to do, so nobody relitigates
them: no more exemplars, no fine-tuning, no multi-agent fan-out on the reply
path. Sequenced so the deploy step comes first (three services ran stale code
for weeks) and the measurement loop comes before any voice change it would
have to judge.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:reviewed-clean Reviewer found no Blocking issues this round domain:core Core library / business logic

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant