Skip to content

v2.5.0 - The best memory: temporal validity, contradiction resolution, team sync, brain packs, 83% LongMemEval#34

Merged
RodCor merged 28 commits into
mainfrom
release/v2.5.0
Jul 3, 2026
Merged

v2.5.0 - The best memory: temporal validity, contradiction resolution, team sync, brain packs, 83% LongMemEval#34
RodCor merged 28 commits into
mainfrom
release/v2.5.0

Conversation

@RodCor

@RodCor RodCor commented Jul 3, 2026

Copy link
Copy Markdown
Owner

v2.5.0 - "The best memory"

The release theme: a memory that is correct, not just retrievable. Stale facts stay out of retrieval, contradictions resolve to the current answer, and every claim ships with a measurement. It also lands the first slices of v3.0 (fleet safety, team sync, shareable brains) and a full docs/site refresh.

The brain (v2.5 flagships)

  • Temporal validity (F1): memories carry a validity window; retrieval is validity-aware. A changed fact collapses to its current value with the prior value invalidated-as-of, lineage preserved (schema v7).
  • Contradiction auto-resolution: write-time detection resolves by confidence x recency; surviving conflicts surface for review instead of silently winning.
  • Generation quality (F2): importance scoring, a quality filter, reflection, and confidence calibration on the write path.
  • Lifecycle & forgetting (F3): forget policy, regret review, proposal hygiene, and an invalidation taxonomy, scored by recall retained after a real forget pass.
  • Correctness evals (P0.1): stale-hit-rate 0.500 -> 0.091, resolution accuracy 0.364 -> 0.909, with no retrieval-quality regression.
  • Retrieval levels: one dial (basic | flexible | deep | advanced | custom); new projects default to deep.
  • Bulk ingest: kimetsu brain memory add-batch (one process, embedder loaded once).
  • Graph groundwork: model-free relates_to entity edges + a storage.backend selector (flat default, graph-lite opt-in).

v3.0 starts

  • Fleet write-safety (S3.A): IMMEDIATE write transactions + busy-retry, atomic ANN saves, per-event origin. Proven with threaded and multi-process tests.
  • Convergent team sync (S3.B): HLC clocks + total-order event replay; two brains converge without copying SQLite files.
  • Remote/team GA (S3.C): per-user write attribution on kimetsu-remote (token -> user), per-token rate limits, memory add --remote.
  • Shareable brain packs (S4, start): brain export always gzips and always security-scrubs (credentials + Luhn-gated PII); brain import merges (dedup, idempotent) or replaces (reversible, --yes) from a file, stdin, or URL, with pack provenance on every imported memory.

Benchmarks (measured on the verified embeddings build)

benchmark result
LongMemEval (_s) 83.0% (166/200, zero reader errors; ~80.9% population-weighted)
BrainBench quality index 80.0% (142 scenarios, reader-free)
BEAM 100K 73.3% (400 probes)
BEAM 1M 66.0% (300 probes), ahead of mem0's self-reported 62%
stale-hit rate 0.500 -> 0.091
resolution accuracy 0.364 -> 0.909

Methodology, caveats, and the honest comparison against mem0 / Cognee / Zep / Letta live in the new Memory Benchmark docs section. The harness (kbench) is public in the kimetsu-bench repo and now refuses to run on a lean (no-embeddings) build, the failure mode that produced every bad number we chased this cycle.

Docs & site

  • Docs site migrated from Docusaurus to Fumadocs (Next.js static export to GitHub Pages, Orama search, llms.txt).
  • Landing rebuilt: metrics band, tabbed "Under the hood" (how it works / retrieval levels / benchmarks / BrainBench), proactive and brain-sharing sections, CSS-only animations (reduced-motion safe).
  • Memory Benchmark split into a 6-page section; How Kimetsu Works trimmed 920 -> 556 lines and subdivided (adds Retrieval models + Configuration pages); ROI methodology retitled Kimetsu Algorithm.
  • README rewritten around the landing structure; ASCII diagram replaced with an SVG; demo GIF re-recorded (macOS window, current CLI flow, pack export beat).

Gates

  • fmt + clippy (-D warnings, lean and --features embeddings) clean across the workspace; kimetsu-brain 485 tests, kimetsu-cli 243 + 10 smoke tests pass.
  • Docs site builds green (all 17 pages prerendered).
  • Note: the docs deploy workflow triggers on push to main, so the site goes live when this merges.

RodrigoCordoba and others added 27 commits June 24, 2026 17:06
Stamp the version at the START of the release branch (lesson from v1.5.0).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add CaseKind enum (recall/knowledge_update/contradiction/temporal/
multi_session) and stale field to EvalCase (both #[serde(default)] for
back-compat). Implement two new pure metrics in eval.rs:
  stale_hit_rate(ranked, stale, k) -> f64  (lower is better)
  resolution_correct(ranked, relevant, stale) -> bool

Wire both metrics into brain_bench_single and the orchestrator summary
table (new columns: stale_hit_rate, resolution_accuracy).

Add bench/dataset-correctness.json (18 mem / 22 cases across all five
kinds). Baseline on flat retrieval (jina-v2-base-code +
ms-marco-tinybert-l-2-v2):
  recall@4 = 1.000, MRR = 0.841
  stale_hit_rate = 0.500  (expected high on flat)
  resolution_accuracy = 0.364  (expected poor — flagship target)

Existing dataset.json and dataset-100.json are unchanged:
stale_hit_rate=0.000, resolution_accuracy=N/A (no stale fields).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…etrieval

Schema v6→v7: add nullable valid_from/valid_to columns to memories with an
index on valid_to. All three retrieval paths (latest/FTS/ANN) and the
graph-lite backend now exclude expired memories (valid_to < now) by default.

A new `memory.temporal` event and `mark_memory_temporal` API stamp validity
windows event-source style — replay-safe via rebuild_in_place. Bench seeder
extended to seed valid_to and superseded_by_key from EvalMemory fixtures.

Bench result (jina-v2-base-code × ms-marco-tinybert-l-2-v2):
  stale_hit_rate: 0.500 → 0.091 (−82%)  resolution_accuracy: → 0.909
  dataset.json recall@4 0.977 MRR 0.941 — no regression
  dataset-100.json recall@4 0.949 MRR 0.914 — no regression

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tion

Story 1.3: Contradiction RESOLUTION (conflict.rs)
- Add resolve_conflicts_enabled() gate: KIMETSU_RESOLVE_CONFLICTS env >
  [ingestion] resolve_conflicts config (default true) > default true
- Add NEAR_TIE_BAND = 0.15 constant
- Add resolution_score(confidence, created_at) = confidence × recency_weight
  (30-day half-life exponential decay)
- Add detect_record_and_resolve_with_vec(): for each conflict hit, score both
  sides; if |Δ| >= 0.15 stamp loser's valid_to via mark_memory_temporal (event-
  sourced, rebuild-safe, NEVER deleted); if |Δ| < 0.15 queue to memory_conflicts
  for operator review (near-tie = no silent change)
- Add [ingestion] resolve_conflicts = true to IngestionSection / config.rs
- Update project.rs add_memory to call detect_record_and_resolve_with_vec
  when resolution is enabled, falling back to detect-only when disabled
- Tests: resolution_score ordering, loser stamping (winner/existing), near-tie
  queued not stamped, rebuild survives, resolve_conflicts_enabled env gate

Story 1.2: Temporal capture at generation (distiller.rs)
- Add optional valid_from / valid_to fields to Lesson struct (serde default None)
- Extend DISTILL_SYSTEM prompt to request temporal scope detection
- In distill_and_record: when lesson carries valid_from/valid_to, stamp the
  written memory via mark_memory_temporal (event-sourced); graceful skip when
  both absent (timeless lessons — the common case)
- Tests: parse_lessons_with_temporal_tags, parse_lessons_temporal_does_not_break_caps

Gates: cargo fmt --all, clippy -- -D warnings (lean + embeddings), cargo test
--workspace: 1053 passed, 0 failed.

Bench (stale_hit_rate ~0.09, resolution_accuracy ~0.9 — not regressed):
jina-v2-base-code × ms-marco-tinybert-l-2-v2: recall@2=1.000 recall@4=1.000
MRR=0.977 stale_hit_rate=0.091 resolution_accuracy=0.909

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Story 2.1 — Importance scoring at write time
  - Rule-based initial usefulness_score seeded into memory.accepted payload
    (kind weights: FailurePattern=0.3, Command=0.2, Convention=0.15,
     Fact=0.1, Preference=0.05) plus rarity bonus via corpus cosine scan
  - Rebuild-safe: projector reads `initial_usefulness` from event payload
  - Config-gated: `ingestion.initial_importance_scoring` (default true)

Story 2.2 — Quality-control filter in distiller
  - `quality_gate()` in distiller.rs: length sanity (10–500 chars),
    transience-marker heuristic, novelty gate (cosine ≥ threshold → drop)
  - Graceful degradation: cosine check skipped when embedder is noop
  - Config: `ingestion.quality_filter_enabled`,
    `quality_filter_novelty_threshold` (0.9),
    `quality_filter_min_len` (10), `quality_filter_max_len` (500)

Story 2.3 — Reflection synthesis
  - `consolidate::run_reflection`: clusters memories → principle synthesis
    via optional cheap-model call; emits `memory.proposed` events
  - `ModelProvider` trait (no Send bound) in consolidate.rs; adapter in
    main.rs bridges to kimetsu-agent without adding a crate dependency
  - CLI: `kimetsu brain reflect [--dry-run] [--workspace <path>]`

Story 2.4 — Confidence calibration from outcomes
  - Bayesian-ish posterior update in `apply_memory_usefulness_for_run`:
    cited memories nudge confidence toward 1.0 (run.finished) or 0.0
    (run.failed) with α=0.05, clamped [0.1, 0.99]
  - Rebuild-safe: runs during event replay inside existing projector path

All new config fields use `#[serde(default)]`.
Gates: cargo fmt, clippy (lean + embeddings), cargo test --workspace ✓

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Tests (the real proof of generation quality):
  - S2.1 initial_usefulness_seeds_score_and_survives_rebuild (importance-rank)
  - S2.2 quality_gate_drops_near_duplicate_passes_novel + length/transience
  - S2.3 run_reflection_creates_proposal_from_cluster + no-model report path
  - S2.4 confidence_calibration_rewards_success_and_survives_rebuild
…lter + reflection + confidence calibration) into release/v2.5.0
…3.4)

Story 3.1 — Active forgetting policy
- New `lifecycle` module in `kimetsu-brain` with `forget_brain(start, ForgetOptions)`
- Identifies stale low-usefulness memories (usefulness_floor, min_age_days,
  protect_use_count thresholds) and archives them via the existing
  `invalidate_memory` event path — fully event-sourced, rebuild-safe
- Opt-in only: `[lifecycle] forget_enabled = true` required (default false)
- `kimetsu brain forget` CLI: --dry-run, --yes, --force-enabled, --no-proposal-gc,
  per-run threshold overrides, graceful TTY/non-TTY detection

Story 3.2 — Regret-driven review
- `regret_flagged_memories(conn, threshold)` queries `retrieval.regret` events
  and returns memories with ≥ threshold regrets for human review (not auto-deleted)
- `brain status` surfaces regret-flagged count and a hint to run forget --dry-run
- Analytics `CorpusHealth.regret_flagged_count` computed at insight time

Story 3.3 — Proposal-queue hygiene
- `gc_proposals(start, ProposalGcOptions)` expires stale pending proposals via
  `reject_proposal(..., expired)` and optionally auto-accepts high-confidence
  proposals via `accept_proposal`
- Runs automatically at the end of `brain forget` (skippable with --no-proposal-gc)
- Thresholds: `proposal_expiry_days` (default 30), `proposal_auto_accept_confidence`
  (default 1.1 = disabled)

Story 3.4 — Structured invalidation taxonomy
- `InvalidationReason` enum (Obsolete, Superseded, Conflicted, Incorrect,
  Duplicate, Forgotten, Manual) with serde snake_case and `from_db()` back-compat
- `invalidations_by_reason(conn)` groups the `invalidated_reason` column
- Analytics `CorpusHealth.invalidations_by_reason` and `brain status` JSON output
  both include the breakdown

Config: `LifecycleSection` added to `ProjectConfig` with `#[serde(default)]` on
all fields — every existing project.toml loads cleanly with no migration needed.

All 7 new lifecycle tests pass; full workspace: 0 failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ew + proposal hygiene + invalidation taxonomy) into release/v2.5.0
New public docs/MEMORY-BENCHMARK.md: retrieval quality (recall@4 0.949 / MRR
0.914), the v2.5 memory-correctness wins (stale-hit 0.500->0.091, resolution
0.364->0.909, no retrieval regression), cost, how to reproduce, and the
LongMemEval driver status (built + dry-run-verified; scores pending a real run
with a model). Measured claims only. Wired into the docs site + linked from the
README Benchmarks section.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds `MemoryCommand::AddBatch` (`kimetsu brain memory add-batch <file>`)
that ingests many memories in a single process, paying project-open and
embedder-init costs exactly once regardless of entry count.

For LongMemEval-style benchmarks that add hundreds of memories per
instance, this collapses ~13 min/instance (one subprocess per entry) to
seconds.

## Implementation

- `BatchMemoryEntry` struct with `text`, `scope`, `kind`, `valid_from`,
  `valid_to` (temporal fields optional, `#[serde(default)]`-compatible).
- `add_memory_inner()` — private per-entry core factored out of
  `add_memory`.  Shared by both paths; zero logic duplication.
  Carries redaction, dedup, event emission, temporal stamping,
  embed_and_persist, rarity bonus, and conflict detection.
- `add_memories_batch(start, entries)` — opens project + resolves
  embedder ONCE, acquires lock ONCE, loops `add_memory_inner` for
  each entry.  GlobalUser entries route to the user brain (same as
  single-add); fallback to project DB when user brain is disabled.
- CLI: `add-batch <file>` reads JSONL or JSON-array; `-` reads stdin.
  `--scope` / `--kind` flags set defaults overridable per entry.
  `--json` emits `{added: N, ids: [...]}`.

## Tests (3 new)

- `add_memories_batch_present_retrievable_rebuild_safe`: 5 entries
  present, all ids returned, embedding_model consistent with single-add,
  survive `rebuild_in_place`, temporal bounds (valid_from/valid_to)
  round-trip through rebuild.
- `add_memories_batch_deduplicates`: same text twice returns same id,
  one DB row.
- `add_memories_batch_all_entries_same_embedding_model`: 8 entries all
  share the same `embedding_model` value (null in lean build, real
  model id in embeddings build) — structural proof that the embedder
  is loaded once.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tion

#2 — Knowledge graph (infrastructure, measured, not surfaced):
- memory.edge event + apply_memory_edge projector arm (reuses insert_memory_edge,
  rebuild-safe) + projector::add_memory_edges batch writer.
- kimetsu-brain::graph: extract_entities (reuses consolidate::parse_tags, splitting
  space-separated tag blocks) + build_relates_to_edges (links memories sharing an
  entity, fan-out capped). project::build_graph (+ active_memory_texts).
- CLI: `kimetsu brain graph build [--enrich] [--dry-run] [--json]` — rule layer by
  default; --enrich adds cheap-model typed edges via distiller::complete_simple.
- Measured via a new brainbench `graph` multi-hop dimension: graph-lite shows +0.00
  recall@k lift over flat (ANN already recovers the answer; graph-reached candidates
  enter at relevance 0.0 so they don't promote into top-k). Per "prove first, surface
  after", graph stays internal — NOT wired into retrieval levels.

#7 — Calibration scaled to release-grade (bench, local-only):
- calibration_gen dataset directive synthesizes ~100 deterministic cite/regret
  scenarios from the corpus + ~22 curated anchors; scorecard gains per-dimension
  n + 95% CI. Measured: 122 cases -> 99.7% +/- 0.3%.

fix(cli): retrieval-level override (pre-existing). `config set` of a level-managed
key (embedder.enabled/reranker) under a preset now drops retrieval.level to "custom"
so the explicit value survives apply_retrieval_level at load. Extracted config_set_text
shared by the command + integration test; pre-levels files unchanged.

Gate: fmt + clippy (lean and --features embeddings, -D warnings) clean;
kimetsu-brain 476, kimetsu-cli 251 tests pass (isolated).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…m sync

Slice A — fleet concurrent-write safety (multiple agents/hooks/Codex+Claude write
one local brain.db at once):
- IMMEDIATE write transactions (projector::with_write_txn) + bounded SQLITE_BUSY
  retry so concurrent writers serialize cleanly; migrate loop also IMMEDIATE +
  under-lock re-check.
- ANN sidecar atomic save (tmp+rename, manifest-last) — no torn .usearch across
  processes.
- Per-event origin `<machine>/<agent>` (schema v8 events.origin, nullable +
  backfill-safe), stamped from a process-global set in CLI run(); carried through
  insert/read/rebuild + SyncEvent.
- Proof: threaded test (6x25 cites -> no lost updates, deterministic rebuild) +
  #[ignore] multi-process test (4 procs x 8 cites -> use_count 32).

Slice B — convergent team sync (two teammates who exchange events reach identical
brain state):
- Hybrid Logical Clock (kimetsu-core/clock.rs): globally-sortable + causal HLC;
  Event.hlc (schema v9, backfilled from rowid); read_events_ordered now ORDER BY
  hlc -> deterministic total-order replay realizes per-field LWW.
- Rebuild-after-import in sync_dir + `brain sync import` so the merged log
  re-projects in HLC order (convergent).
- Concurrent supersede-to-different-survivor surfaced via a sync_conflicts
  projection + `brain sync --status`.
- Session-end auto-sync when [sync] dir configured ([sync] auto opt-out).
- Proof: two-brain convergence test (divergent supersedes/cites exchanged in
  opposite orders -> identical superseded_by/use_count/confidence + 1 conflict on
  both); HLC unit tests; v7->v9 migration test; E2E two-workspace `brain sync`.

Schema 7 -> 9 (forward-only, additive: events.origin/hlc nullable + backfilled).
Gate: fmt; clippy lean + --features embeddings (-D warnings); kimetsu-core 43,
kimetsu-brain 480, kimetsu-cli 251 tests pass (isolated).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ution)

Makes the shared LIVE brain on kimetsu-remote team-GA: a team writes to one
server-hosted brain and every event is attributed to the user who wrote it.
Write-safety already came from Slice A (per-request connection + IMMEDIATE txns);
the gap was per-user attribution on a multi-user server.

- Thread-local origin override (kimetsu-core/event.rs): OriginScope RAII guard;
  process_origin() prefers the thread-local. Each remote request runs on ONE
  spawn_blocking thread, so the per-request override is visible to Event::new and
  cleared on return (tokio reuses blocking threads).
- Token -> user identity (auth.rs): AuthConfig.token_names + user_for_token
  (configured name else a stable non-secret anon-<fingerprint>); tokens-file gains
  a [names] table (back-compat).
- Server attribution (lib/state/rpc): `serve --node <id>` sets the HLC node once
  at startup; dispatch_request wraps the blocking dispatch in
  OriginScope("<node>/user:<name>").
- CLI write-to-remote (kimetsu-cli/remote_client.rs + main.rs): `kimetsu brain
  memory add --remote <url> --repo <id> [--token | KIMETSU_REMOTE_TOKEN]` posts a
  JSON-RPC tools/call via reqwest.
- Metric: aggregate, unlabeled kimetsu_remote_memory_writes_total (the /metrics
  endpoint is intentionally unauthenticated -> no repo/user labels).

Proof: OriginScope / user_for_token / remote_client unit tests; a remote
integration test asserting the brain event origin == "remote/user:webuser"; E2E
(boot serve --node srv1, CLI memory add --remote -> event origin "srv1/user:alice",
hlc node srv1, memory_writes_total 1). Completes the #3 epic (A+B in 6061341).

Gate: fmt; clippy lean + --features embeddings + -p kimetsu-remote (-D warnings);
kimetsu-core 45, kimetsu-cli 243 + cli_smoke 10, kimetsu-remote 33 + integration
suites pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… scrub + merge/replace)

Prepares Kimetsu to produce and receive a shareable brain. The marketplace lives
in a separate repo; this is the pack I/O on top of the existing portable
export/import (MemoryExport + import_memories' normalized-text dedup = merge).

- Security scrub (redact.rs): factored the regex span engine; scrub_for_export =
  credentials + PII (email/phone/ssn/credit_card, Luhn-gated so ids/timestamps
  aren't touched). High precision. Credentials scrubbed at ingest, PII at export,
  defensive re-scrub on import.
- Compressed scrubbed export: export_memories scrubs every memory + returns a
  ScrubReport. Pack envelope {kimetsu_pack,name/version/description,memory_count,
  memories} + parse_pack_or_array (back-compat with the bare array). CLI
  `brain export` gains --name/--version/--description (-> pack), ALWAYS gzips,
  prints the scrub summary, --strict aborts on a finding. flate2 dep.
- Import/install: `brain import` reads a path, `-`, or an http(s) URL (reqwest),
  auto-detects gzip, parses envelope-or-array, --mode merge|replace (replace needs
  --yes). import_pack = merge | invalidate-in-scope-then-load (reversible) +
  provenance tagging (source:"pack" via a thread-local scope, like Slice C's
  OriginScope) so installed memories can later be listed/uninstalled/updated.

Gate: fmt; clippy lean + --features embeddings (-D warnings); kimetsu-brain 485,
kimetsu-cli 243 + cli_smoke 10 pass (isolated). E2E verified: gzip pack, scrub
report, merge dedup, replace --yes supersede, [REDACTED:email/phone] on import.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- npm audit fix (lockfile-only): bump the transitive js-yaml (via gray-matter)
  to a patched version, clearing the moderate DoS advisory (GHSA-h67p-54hq-rp68).
  npm audit now reports 0 vulnerabilities. serialize-javascript (7.0.6) and uuid
  (11.1.1) were already on patched versions.
- favicon: point Docusaurus at the Kimetsu square SVG logo (img/logo.svg) instead
  of the leftover Docusaurus default favicon.ico. The navbar + social card already
  used the Kimetsu logo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…titor comparison

- LongMemEval: 79.5% (200-q stratified) / ~77.2% population-weighted with per-type
  table, replacing the prior 86.7%/60-q slice.
- BEAM: new section — 100K bucket 62.3% (400 probes) and 1M bucket 66.0% (300 probes),
  per-ability tables, the retrieval-budget finding, and honest at-scale degradation.
- "How Kimetsu compares": local/free/model-free pipeline vs mem0/Zep's LLM-in-the-loop;
  the matched 1M bucket edges mem0 (66.0% vs 62%), with all caveats kept (slice sizes,
  vendor self-reported, reproduction gap).
- README: "matches the paid clouds — for free" bullet + public-benchmark comparison table.

All numbers reproducible via `kbench longmemeval` / `kbench beam` (codex reader/judge).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- New static-export Fumadocs site (Next.js 16 + Fumadocs 16, React 19) under
  website-fumadocs/, replacing the Docusaurus website/. Same content, same
  GitHub Pages URL (basePath /kimetsu), Orama static search, dark default.
- docs/*.md + README + CHANGELOG stay the single source of truth; a rewritten
  scripts/sync-docs.mjs regenerates content/docs/*.mdx + meta.json at build,
  with the same allowlist + security boundary (docs/superpowers/ never shipped).
- Branding ported: title/tagline, GitHub/crates.io/npm nav links, logo/favicon.
- Assets self-contained: favicon.ico + logo.svg moved into docs/assets/; the
  sync no longer depends on the retired website/.
- Remove the Docusaurus website/; docs.yml now builds website-fumadocs and
  publishes website-fumadocs/out.

Generated content/assets/node_modules/out are gitignored (rebuilt each run).
Static export builds clean: 10 pages under /kimetsu.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Model-free retrieval tuning on the graph-lite backend, verified on the
BEAM 100K bucket (full 20 convs / 400 probes, uncorrupted): 293/400 = 73.3%,
up from the 62.3% baseline.

- Lever 1: broker.abstain_min_score config (default 0.0, back-compat) so a
  min-score floor can gate low-relevance retrievals from the broker path.
- Lever 2: graph-discovered neighbours are earned supplements, not free-riders.
  graph_expand/petgraph_expand now return (id, hop) and candidate raw_relevance
  = seed_relevance * HOP_DECAY^hops (0.6) instead of 0.0. Holds recall gains
  (multi_session, knowledge_update) without the temporal/event-ordering noise.
- Lever 3: cap expansion (MAX_FAN_OUT 20 -> 12) to keep graph noise bounded.

Gate: fmt clean; clippy lean + --features graph both -D-warnings clean;
13 backend + 15 graph tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…/Zep/Letta)

Surface the uncorrupted full-20 BEAM 100K result (293/400 = 73.3%, graph-lite
backend) on the landing page and methodology docs, and compare against the other
memory systems.

- Landing (page.tsx): metrics band leads with 73.3% BEAM 100K; comparison rebuilt
  as a 5-column table (Kimetsu / mem0 / Cognee / Zep / Letta). Architecture rows
  where we win outright (model-free pipeline, $0 recall, runs local); accuracy
  rows only where we are on-par or ahead (BEAM 1M 66.0% vs mem0 62%, BEAM 100K at
  the prior public SOTA).
- MEMORY-BENCHMARK.md: 100K per-ability table updated to 293/400; graph-ranking
  note (flat baseline 62.3% -> 73.3%, model-free); "How Kimetsu compares" now
  carries Cognee's self-reported numbers (100K 79%, 10M 67%) as the full record,
  including where Cognee leads.
- README: comparison table and prose updated to match.

content/docs/*.mdx regenerate from these sources at build (prebuild sync-docs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ring, demo

Benchmark numbers (all measured on the verified embeddings build):
- LongMemEval: 83.0% (166/200, zero reader errors; ~80.9% population-weighted),
  supersedes 79.5%. Landing metrics band, README, and benchmark docs updated.
- BrainBench: Overall Brain Quality Index 80.0% (142 scenarios, reader-free);
  dedup 77% (was 54% on a lean build), forgetting 88%, importance 76%,
  retrieval 63%. Landing tab + docs updated.

Docs restructure:
- Memory Benchmark split into a 6-page sidebar section (overview, retrieval &
  correctness, BrainBench, LongMemEval, BEAM, comparison), mirroring the How
  Kimetsu Works pattern; prose roughly halved with no facts lost.
- How Kimetsu Works trimmed 920 -> 556 lines; two oversized pages subdivided
  (Interfaces -> Retrieval models; Operations -> Configuration).
- ROI methodology retitled "Kimetsu Algorithm" with a prose description
  (formula out of the frontmatter description).
- New "Sharing brains" section (Operations) and "Team writes and attribution"
  (Remote), documenting packs (gzip + scrub + merge/replace/URL) and per-user
  tokens; verified against the real CLI surface.
- Stale model ids updated (sonnet-4-7 -> sonnet-5, opus-4-7 -> opus-4-8).

Landing page:
- Sections reorganized: hero -> metrics -> tabbed "Under the hood" (how it
  works / retrieval levels / benchmarks / BrainBench) -> proactive -> features
  -> brain sharing -> CTA; full-width dividers with alternating backgrounds.
- New visuals: self-improving loop timeline, passive-vs-proactive contrast,
  retrieval-level ladder, BrainBench dimension cards, brain-sharing terminal.
- Benchmarks tab: architecture-edge table + BEAM-by-scale vs mem0/Cognee/
  Honcho (LongMemEval comparison removed by request; stat stays in the band).
- CSS-only entrance/scroll/hover animations, reduced-motion safe.

Assets:
- README "How it works" ASCII art replaced by docs/assets/how-it-works.svg
  (terminal-card style, 9 labels).
- demo.gif re-recorded: macOS window bar, 2.5.0, setup --no-setup flow,
  semantic recall, retrieval.level dial, scrubbed pack export (47s, 364KB);
  demo.tape updated and reproducible via the VHS Docker image.
- website-fumadocs/public/ fully gitignored (regenerated every build).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@RodCor RodCor changed the title v2.5.0 — The best memory: temporal validity, contradiction resolution, team sync, brain packs, 83% LongMemEval v2.5.0 - The best memory: temporal validity, contradiction resolution, team sync, brain packs, 83% LongMemEval Jul 3, 2026
…pty-index save

Two ubuntu embeddings-job failures on the release PR, both root-caused:

- apply_retrieval_level re-enabled a disabled embedder on every config load
  (level = "deep" clobbered an explicit [embedder] enabled = false), so
  vectors were written against the operator's opt-out. The off-switch now
  returns early before any preset applies; regression test covers all four
  levels. Reproduced on Windows too: the bug was platform-independent but
  only exercised by embeddings builds.

- usearch save() of a never-reserved empty index returns Ok without creating
  the file on Linux, so the Slice-A atomic rename failed with ENOENT.
  Reserving capacity 1 before an empty save makes the serializer emit a
  valid file on every platform.

Gate: 512 kimetsu-brain + 48 kimetsu-core tests pass with --features
embeddings (test-threads=1); clippy -D warnings clean; fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@RodCor
RodCor merged commit 2700a5f into main Jul 3, 2026
16 of 17 checks passed
@RodCor
RodCor deleted the release/v2.5.0 branch July 3, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants