fix(memory): let a newer lesson supersede its near-duplicate - #6175
Conversation
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
1 similar comment
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Design Review (Fable 5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Sound fix with a disclosed gap: the authority guard covers only the semantic branch, and enforcing it there required mirroring the scan's predicates in a second pass. WatchThe guard's real coverage is narrower than the harm it names. Near-duplicates usually share keywords, so the source-blind topic-overlap branch (≥50% overlap, deletes unconditionally, per-row branch order ahead of the semantic test) claims most of them before the pre-pass authority check can ever apply — the diff itself pins this ( Suggestions
[DESIGN-REVIEWED] 8ce3abf |
First Principles Review (Fable 5, fork) — 🟡 CONCERNSPremise-level review of All base claims verified: the First-Principles-Verdict: CONCERNS The 120-line authority pre-pass re-spells three scan blocks to buy a deletes-nothing guarantee base never had — a post-scan check over Not justified as shipped
What this change shipsInventory (7 items) — 6 justifiedIntent: stop a user's shorter correction from being silently dropped while the stale lesson it retracts stays in effect — a FIX (repro in description; inverted pin fails on base).
Deferred sibling note: source-blind deletion remains at the substring and topic-overlap branches (2 siblings, counted) — declared and tracked in #9163, accepted-and-deferred. WatchThe pre-pass's stated invariant — "every refusal THIS change introduces is decidable before the scan mutates anything" — is self-imposed by this diff, documented nowhere else, and held only partially (lexical branches keep delete-as-you-go). The permanent cost is a second spelling of three matching blocks that must track the scan forever. SubtractionsDelete the authority pre-pass block (vector_memory.py, the [FIRST-PRINCIPLES-REVIEWED] 8ce3abf |
Opus 4.8 Review (fork) — ✅ no blocking findingsReviewed |
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
1 similar comment
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]: This PR has been inactive for 7+ days with failing CI. I've assessed the blockers and they appear resolvable — I'll push fixes directly to this branch as a co-author. Assessment: CI is fully green; the blockers are (1) one GPT blocking finding at If you'd prefer I don't touch this PR, add the |
02158aa to
0a6b502
Compare
|
Intent: Make the semantic lesson-dedup branch supersede on recency like its two sibling branches, so a user's correction replaces the stale near-duplicate it corrects instead of losing to it on character count — without letting an automated write displace a stored lesson that outranks it. |
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca] Drive-to-green update on behalf of the maintainers — original change by isotope14, credited as commit author; fixes below are amended into the single commit (head What changed since head
Verification: isort/flake8/mypy/black clean on changed files; full backend suite run locally at baseline parity with pristine |
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]
|
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]
|
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]
|
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]
|
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca] Round 7 push: head The reason-tailored FP's Subtraction (delete the pre-pass, decide authority at end-of-scan) is rebutted in its disposition: it concedes the delete-then-decline composition that the GPT lane blocked as security-class and the pre-pass exists to prevent. 80 surface tests + lint/format gates green locally. |
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]
|
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]
|
Human judgment recorded@NicholasRBowers marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
Human judgment recorded, but the following reviewer lane(s) could not be re-run automatically: GPT 5.6. Re-run the lane's latest workflow run for |
418db65 to
cc3c9e4
Compare
Semantic lesson dedup tie-broke on `len(rule) > len(existing_text)` and
DROPPED the submission when it lost, so character count decided which of
two near-identical rules was current.
Corrections land on the losing side of that comparison as a rule: a
retraction collapses a detailed stale claim ("tool X is available, 48
verbs, attaches on port 9224") into a short accurate one ("tool X is not
installed"). The shorter text lost, and the stale lesson stayed in
effect -- the one outcome that actively misleads the agent rather than
merely losing detail. Observed live: a measured correction was refused
while the claim it disproved kept applying, and the only recovery was a
blind learn_remove on a guessed substring.
The substring and topic-overlap branches directly above already
supersede the stored row unconditionally, so newest-wins makes the three
dedup paths agree instead of one of them arbitrating on length.
The learn_add reply had a matching problem: it printed the SUBMITTED rule
straight after "an existing stored lesson already covers it", which reads
as a quote of the stored lesson. A caller believed it had been shown the
winner, could not tell a correct dedup from a dropped correction, and was
never told the replace path. It now labels the text as what was DROPPED
and names the learn_list/learn_remove sequence.
Substring dedup still refuses -- a rule literally contained in a stored
one is genuinely covered.
Drive-to-green additions on top of isotope14's original change (rebase
onto current main plus review fixes):
- Authority pre-pass (settled after a review retrospective across three
rounds on this span): every refusal this change introduces is decided
BEFORE the scan takes its first deletion. A non-mutating pre-pass over
the same rows the semantic branch sees (shared lazy-backfill budget,
blobs memoized onto the row dicts) returns the old deduped outcome when
a stored near-duplicate outranks the write -- `user_explicit` over a
lower-authority source, or strictly higher stored confidence (the
onboarding import stores user lessons at confidence 1.0 under source
"import"). The pre-pass covers PURE semantic matches only: a row the
scan's substring or topic-overlap branch would claim first keeps main's
lexical outcome (those branches are source-blind by main's design).
The mutating semantic branch's own supersedes are DEFERRED and execute
only when the write proceeds, so a later `substring_covered` refusal
cannot strand a deletion newest-wins took where the old tie-break would
have refused. A write declined for any reason therefore deletes nothing
via the semantic branch and reports it honestly in `superseded`.
- The learn_add deduped reply labels the quoted text as the DROPPED
submission and names the learn_list -> learn_remove -> re-add recovery.
The `semantic_similarity` reason -- only reachable when the stored row
outranks the write -- deliberately withholds that coaching: following
it would delete the row the store just protected. A test locks the
reply to never name learn_remove.
- Rebase composition: keep main's `superseded` reporting (kirodotdev#8585) on the
supersede path, and log row ids rather than rule text, matching the
identities-not-content logging doctrine in this scan.
- Spec sync: the dedup table row in
docs/system-specs/modules/memory-skills-hooks.md drops "longer text
wins" for "newer replaces older" and notes the user_explicit
protection.
Co-authored-by: Kiro Crew <noreply@kirodotdev.github.io>
cc3c9e4 to
8ce3abf
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (5 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: memory lesson dedup fix -- a newer lesson now supersedes its near-duplicate, with the authority verdict settled in a non-mutating pre-pass so a declined write deletes nothing. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/memory-skills-hooks.md. CodeQL is not applicable on this fork PR (default-setup emits no check-run); SAST coverage is Semgrep only, latest run success with 0 annotations.
Problem / Motivation
Semantic lesson dedup tie-broke on
len(rule) > len(existing_text)and dropped the submission when it lost, so character count decided which of two near-identical rules is current.Corrections land on the losing side of that comparison as a rule: a retraction collapses a detailed stale claim ("tool X is available, 48 verbs, attaches on port 9224") into a short accurate one ("tool X is not installed"). The shorter text lost, and the stale lesson stayed in effect — the one outcome that actively misleads the agent rather than merely losing detail.
Observed live: a measured correction was refused while the claim it disproved kept applying, and the only recovery was a blind
learn_removeon a guessed substring.Why it matters
A stale lesson that survives its own correction keeps steering the agent wrong on every later session, and the write path reported the drop as a routine dedup. The user taught the agent the right thing and the store quietly kept the wrong thing.
What changed (motivation → approach → change)
vector_memory.py— the semantic branch now supersedes on recency, matching the substring and topic-overlap branches directly above it, so the three dedup paths agree instead of one of them arbitrating on length.One guard rides on top of newest-wins: a stored row that outranks the write —
user_explicitover a lower-authority source, or strictly higher stored confidence (the onboarding import writes user lessons at confidence 1.0 under sourceimport) — is never displaced by it. That write returns the olddedupedoutcome instead. Without this, a consolidation-extracted lesson arriving later would tombstone the user's own correction and replace it with automated guidance, inverting the authority order_write_semanticalready enforces for same-key writes. The supersede path keeps main'ssupersededreporting and logs row ids rather than rule text, per the identities-not-content logging doctrine in this scan.mcp_tools/learn.py— thededupedreply printed the submitted rule immediately after "an existing stored lesson already covers it", which reads as a quote of the stored lesson. A caller believed it had been shown the winner, could not tell a correct dedup from a dropped correction, and was never told the replace path. It now labels the text as what was DROPPED and names thelearn_list→learn_remove→ re-add sequence. The reply is tailored by reason code: after this changesemantic_similarityis only reachable when the stored row outranks the write, so that reply deliberately does not coach the remove-and-re-add path — an automated caller following it would delete the user's lesson, which is exactly what the store just refused to do.docs/system-specs/modules/memory-skills-hooks.md— the dedup table row drops "longer text wins" for "newer replaces older" and records the authority guard, keeping the spec in same-commit sync per AGENTS.md.Substring dedup still refuses: a rule literally contained in a stored one is genuinely covered.
Tests
test_semantic_dedup_rejects_the_shorter_ruleencoded the old behaviour and is inverted totest_semantic_dedup_keeps_the_newer_rule_when_it_is_shorter; its sibling is renamed to..._when_it_is_longerso the pair describes newest-wins rather than a length rule.Three regression tests pin the authority guard, each proven red without it: an inferred near-duplicate must not displace a
user_explicitlesson (returnsdeduped/semantic_similaritywith nothing deleted); auser_explicitcorrection still supersedes an inferred lesson; a confidence-1.0import-sourced lesson survives a confidence-0.9 consolidation write.learn_addwording assertions updated, plus a new one locking thesemantic_similarityreply to never namelearn_remove.Manual verification
N/A — unit coverage sufficient: the changed paths are pure store/reply logic with no UI or external service, and the full backend suite was run locally at baseline parity with
main.Related Issues
no linked issue: the observed live failure was never filed as an issue — this PR is the fix. The follow-up design decision on authority ordering at the other two dedup delete sites is tracked in #9163 (deferred-finding).
Pattern harvest
Rule candidate: review-prompt
Pattern: "tie-break on text length used as a recency/authority proxy — length never encodes which of two near-duplicates is current or who wrote it"
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)