Skip to content

fix(memory): let a newer lesson supersede its near-duplicate - #6175

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
isotope14:fix/lesson-dedup-newest-wins
Sep 10, 2026
Merged

fix(memory): let a newer lesson supersede its near-duplicate#6175
bolichen97 merged 1 commit into
kirodotdev:mainfrom
isotope14:fix/lesson-dedup-newest-wins

Conversation

@isotope14

@isotope14 isotope14 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Semantic lesson dedup tie-broke on len(rule) > len(existing_text) and dropped the submission when it lost, so character count decided which of two near-identical rules is current.

Corrections land on the losing side of that comparison as a rule: a retraction collapses a detailed stale claim ("tool X is available, 48 verbs, attaches on port 9224") into a short accurate one ("tool X is not installed"). The shorter text lost, and the stale lesson stayed in effect — the one outcome that actively misleads the agent rather than merely losing detail.

Observed live: a measured correction was refused while the claim it disproved kept applying, and the only recovery was a blind learn_remove on a guessed substring.

Why it matters

A stale lesson that survives its own correction keeps steering the agent wrong on every later session, and the write path reported the drop as a routine dedup. The user taught the agent the right thing and the store quietly kept the wrong thing.

What changed (motivation → approach → change)

vector_memory.py — the semantic branch now supersedes on recency, matching the substring and topic-overlap branches directly above it, so the three dedup paths agree instead of one of them arbitrating on length.

One guard rides on top of newest-wins: a stored row that outranks the write — user_explicit over a lower-authority source, or strictly higher stored confidence (the onboarding import writes user lessons at confidence 1.0 under source import) — is never displaced by it. That write returns the old deduped outcome instead. Without this, a consolidation-extracted lesson arriving later would tombstone the user's own correction and replace it with automated guidance, inverting the authority order _write_semantic already enforces for same-key writes. The supersede path keeps main's superseded reporting and logs row ids rather than rule text, per the identities-not-content logging doctrine in this scan.

mcp_tools/learn.py — the deduped reply printed the submitted rule immediately after "an existing stored lesson already covers it", which reads as a quote of the stored lesson. A caller believed it had been shown the winner, could not tell a correct dedup from a dropped correction, and was never told the replace path. It now labels the text as what was DROPPED and names the learn_listlearn_remove → re-add sequence. The reply is tailored by reason code: after this change semantic_similarity is only reachable when the stored row outranks the write, so that reply deliberately does not coach the remove-and-re-add path — an automated caller following it would delete the user's lesson, which is exactly what the store just refused to do.

docs/system-specs/modules/memory-skills-hooks.md — the dedup table row drops "longer text wins" for "newer replaces older" and records the authority guard, keeping the spec in same-commit sync per AGENTS.md.

Substring dedup still refuses: a rule literally contained in a stored one is genuinely covered.

Tests

test_semantic_dedup_rejects_the_shorter_rule encoded the old behaviour and is inverted to test_semantic_dedup_keeps_the_newer_rule_when_it_is_shorter; its sibling is renamed to ..._when_it_is_longer so the pair describes newest-wins rather than a length rule.

Three regression tests pin the authority guard, each proven red without it: an inferred near-duplicate must not displace a user_explicit lesson (returns deduped/semantic_similarity with nothing deleted); a user_explicit correction still supersedes an inferred lesson; a confidence-1.0 import-sourced lesson survives a confidence-0.9 consolidation write.

learn_add wording assertions updated, plus a new one locking the semantic_similarity reply to never name learn_remove.

Manual verification

N/A — unit coverage sufficient: the changed paths are pure store/reply logic with no UI or external service, and the full backend suite was run locally at baseline parity with main.

Related Issues

no linked issue: the observed live failure was never filed as an issue — this PR is the fix. The follow-up design decision on authority ordering at the other two dedup delete sites is tracked in #9163 (deferred-finding).

Pattern harvest

Rule candidate: review-prompt
Pattern: "tie-break on text length used as a recency/authority proxy — length never encodes which of two near-duplicates is current or who wrote it"

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@isotope14
isotope14 requested a review from a team as a code owner August 27, 2026 00:21
@isotope14
isotope14 requested a review from dwu96 August 27, 2026 00:21
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Aug 27, 2026
@dwu96

dwu96 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated.

Missing sections:

  • ## Problem / Motivation
  • ## Why it matters
  • ## What changed

Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle.

1 similar comment
@iamwhatever

Copy link
Copy Markdown
Collaborator

👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated.

Missing sections:

  • ## Problem / Motivation
  • ## Why it matters
  • ## What changed

Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Aug 28, 2026
@github-actions

github-actions Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed 8ce3abfce71290d984a1a07a30225b07d1b1adc5 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 8ce3abf

@github-actions

github-actions Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — 🟡 CONCERNS

Design-level review of 8ce3abfce71290d984a1a07a30225b07d1b1adc5 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Sound fix with a disclosed gap: the authority guard covers only the semantic branch, and enforcing it there required mirroring the scan's predicates in a second pass.

Watch

The guard's real coverage is narrower than the harm it names. Near-duplicates usually share keywords, so the source-blind topic-overlap branch (≥50% overlap, deletes unconditionally, per-row branch order ahead of the semantic test) claims most of them before the pre-pass authority check can ever apply — the diff itself pins this (test_the_authority_pre_pass_skips_rows_the_lexical_branches_claim deletes a user_explicit row on a consolidation write). The inversion the PR exists to prevent — an automated write tombstoning the user's own lesson — therefore remains live on the two lexical paths. The scoping is deliberate and tracked (#9163), but it also forced the ~120-line non-mutating pre-pass that re-implements the substring and topic-overlap predicates and the backfill contract as comment-enforced mirrors of the main scan; predicate drift between the two copies silently changes which writes get declined, and both the coverage gap and the duplication dissolve together when the scan-wide ordering decision lands.
Clears when: #9163 either extends authority ordering scan-wide (collapsing the pre-pass into a single collect-then-apply scan) or records a human decision that the lexical branches stay source-blind, with the mirrored predicates extracted to shared helpers.

Suggestions

  • Extract the substring and topic-overlap predicates into one helper called by both the pre-pass and the main scan now, so "the pre-pass mirrors the scan" is enforced by code instead of comments — small, in-scope, and removes the drift class entirely.

[DESIGN-REVIEWED] 8ce3abf

@github-actions

github-actions Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — 🟡 CONCERNS

Premise-level review of 8ce3abfce71290d984a1a07a30225b07d1b1adc5 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All base claims verified: the _write_semantic same-key authority order exists (vector_memory.py:1479-1501), the onboarding import writes confidence 1.0 under source import (onboarding_import.py:4285-4290), and "semantic_similarity" has exactly one producer in src/. The scan's per-row branch order and the pre-existing substring_covered delete-then-decline composition are confirmed on base. The one item that doesn't fully earn its shipped shape is the ~120-line authority pre-pass, which mirrors three of the scan's own blocks; a post-scan check over the already-deferred semantic matches passes every test this PR adds.

First-Principles-Verdict: CONCERNS

The 120-line authority pre-pass re-spells three scan blocks to buy a deletes-nothing guarantee base never had — a post-scan check over deferred_semantic is ~15 lines.

Not justified as shipped

  1. Authority declines now delete nothing, via a non-mutating pre-pass — oversized: it duplicates the scan's substring predicate, topic-overlap predicate, and embed-backfill block (3 mirrored blocks, plus the new _authority_prepass_embed_failed coupling key, 0 uses on base), to exceed base's own refusal semantics — base's semantic_similarity and substring_covered refusals both return after mid-scan deletions, reported via superseded (vector_memory.py:3374, 3471-3475). The smaller shape — carry source/confidence into deferred_semantic, check authority after the loop, return DEDUPED before executing deferrals — passes every test this PR adds; the only case it cedes is lexical-delete-then-decline, the exact composition the PR already disclaims as pre-existing and defers to Decide: authority ordering at all three lesson-dedup delete sites (follow-up to #6175) #9163.

What this change ships

Inventory (7 items) — 6 justified

Intent: stop a user's shorter correction from being silently dropped while the stale lesson it retracts stays in effect — a FIX (repro in description; inverted pin fails on base).

  1. A newer near-duplicate lesson now replaces the stored one even when shorter — justified
  2. A lower-authority near-duplicate write is refused when the stored row is user-set or higher-confidence — justified
  3. An authority decline deletes nothing, via a full pre-scan pass — oversized (see above)
  4. A substring_covered refusal no longer executes this scan's semantic deletions (deferral) — justified
  5. The deduped reply labels the quoted text as the dropped submission and names the remove-and-re-add path — justified
  6. A distinct semantic_similarity reply that withholds remove coaching — justified (1 producer counted: vector_memory.py)
  7. Spec dedup-table row updated in the same commit — justified

Deferred sibling note: source-blind deletion remains at the substring and topic-overlap branches (2 siblings, counted) — declared and tracked in #9163, accepted-and-deferred.

Watch

The pre-pass's stated invariant — "every refusal THIS change introduces is decidable before the scan mutates anything" — is self-imposed by this diff, documented nowhere else, and held only partially (lexical branches keep delete-as-you-go). The permanent cost is a second spelling of three matching blocks that must track the scan forever.
Clears when: the pre-pass is collapsed into a post-loop authority check over deferred_semantic, or a stranding case is shown that the substring_covered disclaimer does not already accept.

Subtractions

Delete the authority pre-pass block (vector_memory.py, the if similarity is not None and source != "user_explicit" walk) and the _authority_prepass_embed_failed marker; extend deferred_semantic tuples with the row's source/confidence and run the outrank check once, after the scan, before executing deferrals.

[FIRST-PRINCIPLES-REVIEWED] 8ce3abf

@github-actions

github-actions Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Opus 4.8 Review (fork) — ✅ no blocking findings

Reviewed 8ce3abfce71290d984a1a07a30225b07d1b1adc5 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 8ce3abf

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Aug 29, 2026
@bolichen97
bolichen97 enabled auto-merge August 29, 2026 23:59
@bolichen97
bolichen97 disabled auto-merge September 3, 2026 21:27
@bolichen97
bolichen97 enabled auto-merge (squash) September 3, 2026 21:27
@NicholasRBowers NicholasRBowers added the needs-pr-triage PR scanner: awaiting automated triage label Sep 5, 2026
@dwu96

dwu96 commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated.

Missing sections:

  • ## Problem / Motivation
  • ## Why it matters
  • ## What changed

Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle.

1 similar comment
@iamwhatever

Copy link
Copy Markdown
Collaborator

👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated.

Missing sections:

  • ## Problem / Motivation
  • ## Why it matters
  • ## What changed

Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle.

@NicholasRBowers NicholasRBowers added drive-to-green PR claimed by drive-to-green pipeline and removed needs-pr-triage PR scanner: awaiting automated triage labels Sep 5, 2026
@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]: This PR has been inactive for 7+ days with failing CI. I've assessed the blockers and they appear resolvable — I'll push fixes directly to this branch as a co-author.

Assessment: CI is fully green; the blockers are (1) one GPT blocking finding at vector_memory.py:2870 — the new unconditional supersede lets a lower-authority inferred lesson tombstone a user_explicit correction, fixed by skipping supersede when the existing source is user_explicit and the new one is not; (2) the PR body is missing the required template headings (Fork PR Description check); (3) advisory spec-sync: docs/system-specs/modules/memory-skills-hooks.md:574 still documents the removed "longer text wins" tie-break and must be updated in the same commit.

If you'd prefer I don't touch this PR, add the pr-no-autofix label.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Sep 6, 2026
@NicholasRBowers
NicholasRBowers force-pushed the fix/lesson-dedup-newest-wins branch from 02158aa to 0a6b502 Compare September 6, 2026 23:43
@NicholasRBowers

Copy link
Copy Markdown
Contributor

Intent: Make the semantic lesson-dedup branch supersede on recency like its two sibling branches, so a user's correction replaces the stale near-duplicate it corrects instead of losing to it on character count — without letting an automated write displace a stored lesson that outranks it.
Not a goal: Changing the substring or topic-overlap dedup branches, the dedup scan's delete-then-report transaction shape, or any same-key write semantics — those are main's existing design.

@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]

Drive-to-green update on behalf of the maintainers — original change by isotope14, credited as commit author; fixes below are amended into the single commit (head 0a6b502ac) with a Kiro Crew co-author trailer.

What changed since head 02158aa73 and why:

  1. Rebased onto current main (twice: 85f5dcad0018c9ecf353d7b1a1a; the PR showed CONFLICTING). Two real conflicts composed during resolution:

    • vector_memory.py: main gained superseded reporting (fix: name the lessons a write superseded instead of a bare insert #8585) on the same lines this PR rewrites. The resolved supersede path keeps newest-wins and appends to superseded, and logs row ids instead of rule[:60] — the scan's own comment block mandates identities-never-content logging, which landed after this branch was cut.
    • learn.py: kept this PR's richer "what was DROPPED" deduped reply and main's {lost} WARNING suffix naming deleted rules.
  2. Authority guard on the semantic supersede (addresses the GPT review's blocking finding on 02158aa73, using its named fix, widened one step): a stored row that outranks the write — user_explicit over a lower-authority source, or strictly higher stored confidence — is never displaced; that write returns the old deduped outcome. The confidence half exists because the onboarding import stores user lessons at confidence 1.0 under source import, and the deleted length tie-break was the only thing that could spare those rows from a 0.9-confidence consolidation write. Three regression tests pin this (inferred-vs-user_explicit, user_explicit-still-wins, import-row-survives), each proven red without the guard.

  3. Reason-tailored deduped reply (also the Design review's suggestion): the semantic_similarity reply no longer coaches the learn_remove → re-add path, because after the guard that reason is only reachable when the stored row outranks the write — an automated caller following the coaching would delete the user's lesson, which is exactly what the store just refused to do. substring_covered keeps the replace-path coaching.

  4. Spec sync (Design/First Principles CONCERNS): the dedup table row in docs/system-specs/modules/memory-skills-hooks.md drops "longer text wins" for "newer replaces older" plus the authority-guard note, and the write_lesson docstring contract is updated to match.

  5. PR body rewritten onto the repository template (the Fork PR Description check was failing on missing headings); the original prose is preserved under the correct sections.

Verification: isort/flake8/mypy/black clean on changed files; full backend suite run locally at baseline parity with pristine main (every failure reproduces identically on main or clears in isolation — all inherited/env, none in this diff's surface); two model-pinned pre-push reviewers plus a focused verifier ran on the amended commit, and their blocking findings are fixed as described above.

@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]

  • automated lessons can delete user-explicit corrections — FIXED span=3f8cfc777e5f
    self-added: no
    mechanism: authority guard on the semantic supersede (source + stored-confidence comparison before delete)

Fixed at head 0a6b502 using the review's named fix, widened one step to the codebase's real authority order. The semantic supersede now keeps the stored row whenever it OUTRANKS the write: existing.source == "user_explicit" while the write's source is not, OR the stored confidence strictly exceeds the write's (the onboarding import stores user lessons at confidence 1.0 under source "import", and the deleted length tie-break was the only condition that could spare those rows from a 0.9-confidence consolidation write). The protected case returns the pre-PR deduped/semantic_similarity outcome with nothing deleted; newest-wins stays for every other source combination, which is this PR's intent.
This ruling covers the class: any finding requiring a lower-authority write (consolidation, migration, import re-run) to tombstone a higher-authority stored lesson via the semantic branch is closed by the guard at src/kiro_crew/vector_memory.py (semantic dedup branch of write_lesson's scan). Three red-before-green regression tests pin it: test_semantic_supersede_never_lets_an_inferred_lesson_displace_user_explicit, test_semantic_supersede_lets_a_user_explicit_correction_replace_an_inferred_lesson, test_semantic_supersede_keeps_a_higher_confidence_import_lesson (test/test_vector_memory_coverage.py).
The learn_add reply for semantic_similarity deliberately stops coaching the learn_remove-and-re-add path, since after the guard that reason is only reachable when the stored row outranks the write — coaching removal there would walk an automated caller around the guard.

@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]

  • Blocker: the semantic_similarity reply coaches deleting the row the guard protects, contradicting the description — FIXED (Subtraction applied)

Fixed at head 418db65 by applying the blocker's own Subtraction: the semantic_similarity reply again withholds the learn_remove coaching and names the stored lesson's higher authority, and the locking test (test_semantic_dedup_reply_never_coaches_removing_the_protected_lesson) asserts learn_remove never appears in it. The description and the diff now agree.
For the record and the next round's context: the branch was present at head 42cf172 and was DELETED at head 0948f21 on this lane's own previous blocker ("Delete the if reason == 'semantic_similarity': branch"), whose reachability audit was correct at the time. This round's blocker holds because the composition changed the audit's premise — with the deferral restored, semantic_similarity is again produced for lower-authority writers and the unconditional coaching became the inversion this PR guards against. The restored reply carries none of the round-5 version-scope prose that motivated the deletion; it states only what is true on the current store.

@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]

  • learn_remove is emitted for semantic_similarity, contrary to the promised authority-safe reply — FIXED span=848cf5dccdb7
    self-added: yes
    mechanism: reason-tailored reply restored (authority-safe wording, no removal coaching)

Fixed at head 418db65 exactly as the finding asks: the semantic_similarity reason omits the replacement instructions and states that a higher-authority stored lesson stays in effect and only the user may replace it. Locked by test_semantic_dedup_reply_never_coaches_removing_the_protected_lesson, which asserts the reply never names learn_remove. The generic deduped reply keeps the recovery coaching for substring_covered, where the submission is by definition covered and no protected row is at stake.

@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]

  • Subtraction: decide the authority verdict at end-of-scan on the deferred list and delete the pre-pass — REBUTTED (re-opens the closed topic-overlap sequence)

The subtraction concedes its own cost: "the same delete-then-decline composition main already documents for substring_covered" — but for THIS PR's refusal that composition is exactly the sequence the GPT lane blocked as security-class at head a93a685 (a topic-overlap deletion taken before a later authority decline strands a row while nothing is stored) and it was closed by deciding the verdict before the scan's first deletion. An end-of-scan authority verdict on the deferred list runs AFTER the topic-overlap branch's immediate deletes, so a consolidation rewording that keyword-overlaps one row and semantically matches a protected row would again delete the first and then decline — re-opening the adjudicated finding. The pre-pass's duplicated predicates and backfill contract are the recorded price of that ordering guarantee (see the retrospective disposition at head 2850e51); mirror-drift is pinned by the pre-pass-skip regression test. This ruling covers the class: relocating the authority verdict after any deletion re-opens the closed sequence, whichever list it reads.

@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]

Round 7 push: head 418db6541 (rebased onto main aa6e258d0; single commit, isotope14 authorship + Kiro Crew co-author trailer preserved). One change, answering the FP BLOCK and the GPT advisory together — both converged on the same point from opposite ends of the last two rounds:

The reason-tailored semantic_similarity reply is restored: it names the stored lesson's higher authority and withholds the learn_remove coaching (following it would delete the row the store just protected), with a locking test asserting the reply never names learn_remove. The round-5 deletion of this branch followed FP's own blocker, whose reachability audit was correct then; the round-6 deferral composition made the reason reachable for lower-authority writers again, which is what flipped both lanes to demanding the tailored reply back. The restored version carries none of the version-scope prose that motivated the original deletion.

FP's Subtraction (delete the pre-pass, decide authority at end-of-scan) is rebutted in its disposition: it concedes the delete-then-decline composition that the GPT lane blocked as security-class and the pre-pass exists to prevent.

80 surface tests + lint/format gates green locally.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 7, 2026
@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]

  • Concurrent enrichment can tombstone a user-explicit lesson — NEEDS-A-DECISION (maintainer ruling requested; drive paused) span=3f8cfc777e5f
    self-added: yes
    mechanism: none this round (escalation, no code change)

This is the FIFTH consecutive security-class block on span 3f8cfc777e5f, each a different instance of one class: write_lesson's dedup scan is not transactional against its own ordering or concurrent writers. Ledger: (1) head 02158aa — authority inversion; fixed with the source/confidence guard. (2) head 8f10d92c5-era — semantic delete before a later authority decline; fixed by deferring semantic deletes. (3) head a93a685 — topic-overlap delete before the authority decline; fixed per retrospective with the non-mutating pre-pass. (4) head 2850e51/0948f2176 — semantic delete before main's substring_covered refusal; fixed by composing deferral + pre-pass. (5) head 418db65 — THIS finding: a TOCTOU where a concurrent user enrichment raises a row's authority between the pre-pass read and the deferred delete.
The finding is technically reachable, but the race CLASS is main's, not this PR's: every delete site in the scan (substring, topic-overlap, and the pre-PR semantic branch) already deletes after an unlocked read, so a concurrent enrichment can be tombstoned by main's own branches today, in a narrower but nonzero window. This PR's deferral stretches that pre-existing window within one call; it does not create the race. The named remedy — revalidate deferred rows under _db_lock and make insert+tombstone atomic — is a concurrency redesign of the scan's transaction model, which main has never had, whose right scope (one branch? all three? the whole scan?) is exactly the design decision already tracked in #9163. Five rounds have progressively expanded this span's demands from ordering into transactionality; continuing to patch one instance per round inside a contributor PR is scope this drive cannot decide.
DECISION REQUESTED — one of: (a) adopt the transactional fix in this PR (specify scope: deferred-list revalidation only, or all three delete sites); (b) accept the shipped composed design and record that the TOCTOU class predates this PR and is resolved by the #9163 ruling — for this path the sanctioned exit is the human override, one line to paste: /ai-review override gpt 418db6541de102fa8c89868a791fba55ba552df7: TOCTOU class predates this PR (all scan delete sites read-then-delete unlocked on main); transaction-model redesign deferred to #9163 ; (c) a middle path — per-row authority recheck immediately before each deferred delete_semantic (shrinks the window to main's baseline without a locking redesign) — say so and the drive resumes with that patch.

@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]

  • Concurrent enrichment can tombstone a user-explicit lesson — REBUTTED (not a defect of this PR; maintainer ruled ship-as-is) span=3f8cfc777e5f
    self-added: yes
    mechanism: none (ruling recorded; no code change)

Maintainer ruling (NicholasRBowers, 2026-09-09): ship as-is. The condition attached to the ruling was verified in code first: the memory store has NO existing mechanism for scan-wide write atomicity — _db_lock serializes individual db/FAISS critical sections only (vector_memory.py ~line 955), lesson writes run concurrently on multiple worker threads by design (~line 1465), and every dedup delete site on main (substring, topic-overlap, and the pre-PR semantic branch) performs the same unlocked read-then-delete. The TOCTOU class therefore predates this PR and is a property of the store's write model, not of this change; this PR narrows the practical exposure (a declined write now deletes nothing, in any ordering) and does not create the race.
The store-wide fix — serializing lesson writes or holding the store lock across the scan's decision+mutation — is a concurrency-design change to main-owned code, now filed as its own tracked issue (linked below in this thread) alongside the authority-ordering decision in #9163. Findings requiring THIS PR to introduce a transaction model the store has never had are covered by this ruling.
For the maintainer, the sanctioned one-liner if the lane supports overrides on this PR: /ai-review override gpt 418db6541de102fa8c89868a791fba55ba552df7: TOCTOU class predates this PR (all scan delete sites read-then-delete unlocked on main); store-wide write-atomicity redesign tracked separately — otherwise this recorded adjudication is the written basis for human review past the readiness label.

The store-wide write-atomicity decision is filed as #9622.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 9, 2026
@NicholasRBowers

Copy link
Copy Markdown
Contributor

/ai-review override gpt 418db65: TOCTOU class predates this PR (all scan delete sites read-then-delete unlocked on main); store-wide write-atomicity redesign tracked in #9622

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Human judgment recorded

@NicholasRBowers marked the gpt AI finding as false positive, not applicable, or explicitly accepted for 418db6541de102fa8c89868a791fba55ba552df7.

TOCTOU class predates this PR (all scan delete sites read-then-delete unlocked on main); store-wide write-atomicity redesign tracked in #9622

This decision applies only to this commit. A new push requires a new judgment.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Human judgment recorded, but the following reviewer lane(s) could not be re-run automatically: GPT 5.6. Re-run the lane's latest workflow run for 418db6541de102fa8c89868a791fba55ba552df7 manually from the Actions tab.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 9, 2026
@NicholasRBowers
NicholasRBowers force-pushed the fix/lesson-dedup-newest-wins branch from 418db65 to cc3c9e4 Compare September 9, 2026 20:39
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 9, 2026
Semantic lesson dedup tie-broke on `len(rule) > len(existing_text)` and
DROPPED the submission when it lost, so character count decided which of
two near-identical rules was current.

Corrections land on the losing side of that comparison as a rule: a
retraction collapses a detailed stale claim ("tool X is available, 48
verbs, attaches on port 9224") into a short accurate one ("tool X is not
installed"). The shorter text lost, and the stale lesson stayed in
effect -- the one outcome that actively misleads the agent rather than
merely losing detail. Observed live: a measured correction was refused
while the claim it disproved kept applying, and the only recovery was a
blind learn_remove on a guessed substring.

The substring and topic-overlap branches directly above already
supersede the stored row unconditionally, so newest-wins makes the three
dedup paths agree instead of one of them arbitrating on length.

The learn_add reply had a matching problem: it printed the SUBMITTED rule
straight after "an existing stored lesson already covers it", which reads
as a quote of the stored lesson. A caller believed it had been shown the
winner, could not tell a correct dedup from a dropped correction, and was
never told the replace path. It now labels the text as what was DROPPED
and names the learn_list/learn_remove sequence.

Substring dedup still refuses -- a rule literally contained in a stored
one is genuinely covered.

Drive-to-green additions on top of isotope14's original change (rebase
onto current main plus review fixes):

- Authority pre-pass (settled after a review retrospective across three
  rounds on this span): every refusal this change introduces is decided
  BEFORE the scan takes its first deletion. A non-mutating pre-pass over
  the same rows the semantic branch sees (shared lazy-backfill budget,
  blobs memoized onto the row dicts) returns the old deduped outcome when
  a stored near-duplicate outranks the write -- `user_explicit` over a
  lower-authority source, or strictly higher stored confidence (the
  onboarding import stores user lessons at confidence 1.0 under source
  "import"). The pre-pass covers PURE semantic matches only: a row the
  scan's substring or topic-overlap branch would claim first keeps main's
  lexical outcome (those branches are source-blind by main's design).
  The mutating semantic branch's own supersedes are DEFERRED and execute
  only when the write proceeds, so a later `substring_covered` refusal
  cannot strand a deletion newest-wins took where the old tie-break would
  have refused. A write declined for any reason therefore deletes nothing
  via the semantic branch and reports it honestly in `superseded`.
- The learn_add deduped reply labels the quoted text as the DROPPED
  submission and names the learn_list -> learn_remove -> re-add recovery.
  The `semantic_similarity` reason -- only reachable when the stored row
  outranks the write -- deliberately withholds that coaching: following
  it would delete the row the store just protected. A test locks the
  reply to never name learn_remove.
- Rebase composition: keep main's `superseded` reporting (kirodotdev#8585) on the
  supersede path, and log row ids rather than rule text, matching the
  identities-not-content logging doctrine in this scan.
- Spec sync: the dedup table row in
  docs/system-specs/modules/memory-skills-hooks.md drops "longer text
  wins" for "newer replaces older" and notes the user_explicit
  protection.

Co-authored-by: Kiro Crew <noreply@kirodotdev.github.io>
@NicholasRBowers
NicholasRBowers force-pushed the fix/lesson-dedup-newest-wins branch from cc3c9e4 to 8ce3abf Compare September 9, 2026 21:39
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 9, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (5 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: memory lesson dedup fix -- a newer lesson now supersedes its near-duplicate, with the authority verdict settled in a non-mutating pre-pass so a declined write deletes nothing. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/memory-skills-hooks.md. CodeQL is not applicable on this fork PR (default-setup emits no check-run); SAST coverage is Semgrep only, latest run success with 0 annotations.

@bolichen97
bolichen97 merged commit 37c3630 into kirodotdev:main Sep 10, 2026
69 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants