feat(agent-core,webapp): add deterministic self-narration and duplicate message filters - #450
Merged
WinterYukky merged 2 commits intoAug 24, 2026
Conversation
…lters
Add deterministic output-layer filters that catch three categories of
low-value end-of-turn text the model emits despite prompt guidance:
- Cross-turn rehash (A-1): end-of-turn text restates what was already sent
- Self-narration after send (A-2): narrates a just-sent tool message
- No-information wake-up monologue (A-3): internal monologue on retrigger
Supporting modules:
- message-dedup: bigram-similarity based near-duplicate detection with
configurable time windows and length gates
- user-delivery-dedup: DDB-backed dedup log for user-facing deliveries
(sendMessageToUser/sendMessageToAgent) to prevent cross-turn rehash
- placeholder-detection: identifies placeholder end-of-turn text that
should be suppressed (".", "ok.", empty text, cancel boilerplate)
- rewind-filter: filters conversation history based on rewind state
(cutoff SK) for context window management
Also adds:
- messageSKFromTimestamp + getRecentMessages helpers in messages.ts
- TTL field on MessageItem schema (for ephemeral dedup log entries)
- rewindState field on SessionItem schema
Remove internal-only references from the self-narration/dedup test suite and comments so they are suitable for the public upstream: - drop internal session IDs and a private repo deploy workflow from fixtures - translate Japanese example prose in comments/fixtures to English - retain non-ASCII literals that are functional detection patterns (monologue and scaffolding-tag matching) with clarifying English comments No behavioural change; calibration thresholds preserved (228 tests pass).
remote-swe
Bot
force-pushed
the
feature/self-narration-dedup-1785860012
branch
from
August 24, 2026 06:03
e02bc5a to
455c373
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add deterministic, prompt-independent output-layer filters that suppress three
classes of low-value assistant text before delivery: cross-turn rehashes,
self-narration after a send, and no-information wake-up monologue. Also adds
near-duplicate/exact-duplicate delivery dedup and scaffolding-placeholder
detection, and consolidates the previous webapp-side dedup into agent-core so a
single source of truth serves every delivery path.
Motivation
Interrupted/auto-retriggered turns and timer/ack wake-ups frequently re-emit an
intro the session already sent, narrate a message it just delivered, or emit
internal monologue ("silent terminate", "routine progress"). Prompt guidance
alone did not eliminate these. A mechanical backstop at the single delivery
choke-point removes them without risking false positives on genuine reports.
Changes
message-dedup: char-bigram near-duplicate detection with conservativelength + time-window gates (short messages never deduped).
self-narration-filter: rehash/self-narration containment + relaxed-lengthsymmetric similarity, and a structurally-gated wake-up-monologue filter
(fires only on non-user triggers that ran no new work tool).
placeholder-detection: end-of-turn placeholder + scaffolding-tag detection,shared by the orchestrator and the send tool.
user-delivery-dedup: window-scoped delivery/redelivery dedup backed by ashort-lived DynamoDB log (fail-open on read errors).
rewind-filter: drop rewound/orphaned messages so no orphan toolUse remains.Testing
agent-coreunit tests, measured: message-dedup 21, self-narration-filter 43,placeholder-detection 25, user-delivery-dedup 20, rewind-filter 16
(125 tests for this feature; full agent-core suite 228 passing).
tscandprettier --checkpass for the agent-core workspace.Notes
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.