Memory Forgetting

[igordayen]

It appears that in cognitive science memory decay is not the only theory / factor in "forgetting" problem.

Two other theories are:

• Memory Interference;
• Memory Overwriting.

Source: "Cognitive Psychology" by Medin, Ross, Markman.

[igordayen]

[jimador]

Thanks for raising this — the Medin/Ross/Markman framing is a good reset. Decay tends to dominate how people talk about LLM “forgetting,” but that’s kind of the easy case, and DICE already handles it reasonably well (effective confidence with exponential decay, reinforcement on merge slowing that decay, and clean status transitions on contradiction).

Before we go too far into mechanisms, I think there’s a more fundamental question to pin down: what do we actually mean by forgetting in a long-horizon agent? In practice, it’s not a single thing.

Here’s a working taxonomy that’s been useful for me:

1. Out-of-context — fell out of the prompt window, but still in storage. Recoverable via retrieval.
2. Out-of-store — removed from persistence. Actually gone.
3. Demoted — still present, but its weight dropped enough that it no longer wins retrieval.
4. Superseded — still present, but explicitly replaced by a newer, more authoritative fact. Available for history, suppressed by default.
5. Compacted away — summarized into a coarser representation. Detail is gone, gist remains.
6. Interference-blocked — present and well-weighted, but crowded out by near-duplicates during retrieval, so it looks forgotten.
7. In-context but ignored — literally in the prompt, but not getting attention weight. This is the long-horizon attention failure that kicked off a lot of the activation-ranking work.

The textbook theories map pretty cleanly onto combinations of these:

• Decay → mostly (1) + (3), eventually (2)
• Overwriting → (4), sometimes (2)
• Interference → (6) + (7)

Lumping all of that under “forgetting” hides the important part: these aren’t the same failure mode, and they don’t have the same fix. Some of them aren’t even memory-layer problems.

Curious if this breakdown lines up with how others are thinking about it, or if I’m missing anything. I have some thoughts on what falls out once you separate these concerns, but it feels like getting alignment on the definition is the right first step.

[igordayen]

thanks for feedback, @jimador , taxonomy seems very accurate / relevant.

perhaps , would makes sense to come with interference practical modelling.

and combine all three "fogetting" theories using some weighting schema (with "natural decay" as just one factor).

Note, textbook is a bit outdated, there are few newer college textbook :

Cognitive Psychology: A Student's Handbook 7th Edition
by Michael W. Eysenck (Author), Mark T. Keane (Author)

Cognitive Psychology: Connecting Mind, Research, and Everyday Experience (MindTap Course List)
by E. Goldstein |

Special book for applications of cognitive science to Music:

https://www.aupress.ca/app/uploads/120227_99Z_Dawson_2013-Mind_Body_World.pdf

Would be good to empower embabel with the most advanced memory model:)