diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5882400..bbf4805 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -89,6 +89,36 @@ jobs: KIMETSU_USER_BRAIN: "0" run: cargo test --workspace --locked -- --test-threads=1 + test-embeddings: + name: test (embeddings, ubuntu) + runs-on: ubuntu-latest + # Closes #22: run the full test suite compiled with the `embeddings` feature + # (ONNX/fastembed) on Linux only. macOS/Windows are covered by `test` + # (which feature-unifies the workspace and already pulls in the embeddings + # code paths), but the explicit ubuntu run here pins an embeddings-specific + # build cache and ensures the feature compiles cleanly as a distinct slice. + timeout-minutes: 30 + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@stable + - uses: Swatinem/rust-cache@v2 + with: + shared-key: ci-test-embeddings + # Cache the HuggingFace model files and the fastembed on-disk ONNX cache + # so repeated runs do not re-download ~200 MB of weights. + - name: Cache HuggingFace / fastembed model files + uses: actions/cache@v4 + with: + path: | + ~/.cache/huggingface + .fastembed_cache + key: embeddings-models-${{ runner.os }}-${{ hashFiles('Cargo.lock') }} + restore-keys: embeddings-models-${{ runner.os }}- + - name: cargo test (embeddings) + env: + KIMETSU_USER_BRAIN: "0" + run: cargo test --workspace --locked --features embeddings -- --test-threads=1 + audit: name: cargo-audit (RUSTSEC) runs-on: ubuntu-latest diff --git a/CHANGELOG.md b/CHANGELOG.md index e616c25..9e25d67 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,105 @@ onward the project follows SemVer normally: patch releases are bug-fix-only, minor releases are backward-compatible additions, and breaking changes require a major bump. +## v1.5.0 — pays for itself + +ADDED + * **Telemetry capture.** Raw query text is now stored in `context.served` + telemetry events when `[learning] store_queries = true` (default; set + `false` to revert to the pre-v1.5 query-hash-only behavior). A + `session_id` field is also written when the host provides one (Claude Code + hooks; absent for hosts that do not emit it). Dropped-capsule sidecar + (`~/.kimetsu/cache//dropped-recent.json`) records capsules that + were floor-filtered out so that a later citation of one is detected as a + `retrieval.regret` event, feeding the self-tuning loop. All telemetry + stays on-machine; nothing is exported. + + * **ROI ledger — `kimetsu brain roi`.** Conservative per-kind token-savings + estimates (failure_pattern=1500, command=400, convention=300, fact=500, + preference=200 tokens per citation) minus brain-injection overhead give a + net-positive / net-negative verdict. Dollar estimates are shown when the + active model is recognized from a built-in price table (Claude 3/4, GPT-4/5 + families, Bedrock routing prefixes) or when `[model] price_per_mtok` is set + in `project.toml`. `--window 7d|30d|all` and `--json` for scripting. The + Stop hook appends a per-session savings sentence when ≥1 citation occurred; + zero-citation sessions are silent. Calibration methodology and honest + limitations: `docs/ROI-METHODOLOGY.md`. + + * **Token budget — render-time capsule compression and session dedupe.** + Two `[broker]` toggles, both default `true`: + - `compress_capsules`: capsule summaries are compressed at render time + (strips `[tags: ...]` / `(context: ...)` annotations, caps at 3 + sentences). Ranking is never affected — this runs only after retrieval + and reranking. Set `false` to inject full memory text. + - `session_dedupe`: the `UserPromptSubmit` hook skips capsules whose + handle was already injected earlier in the same session (tracked via + the proactive-state sidecar). Soft policy: falls back to injecting all + capsules when dedupe would produce an empty set. This addresses the + pre-v1.5 behavior where the main hook re-injected the same top capsule + on every prompt of a long session. + + * **Self-Tuning Brain — `kimetsu_brain_cite` MCP tool + `kimetsu brain tune`.** + `kimetsu_brain_cite` is a new write-gated MCP tool that records a + `memory.cited` event from inside an MCP session, closing the ground-truth + gap when the model leans on a memory but doesn't explicitly call + `cite_memory`. Personal eval set: `tuneset` builds positive eval cases from + `context.served` + citation joins (exact session_id or ±30-minute window + fallback). `kimetsu brain tune --status` reports case count and kind + coverage. `kimetsu brain tune` (dry-run by default) sweeps `broker.min_lexical_coverage` + ∈ {0.3, 0.4, 0.5, 0.6} × `broker.min_semantic_score` ∈ {-1.0(AUTO), 0.0, 0.25, + 0.35, 0.45} against the production embedder; `--apply` writes only the + floor parameters (not the reranker — that change is recommended separately); + `--revert` restores the previous tune-history entry. A holdout guardrail + (deterministic 20% split) prevents writing a config that regresses the + holdout objective. + + * **Consolidation — `kimetsu brain consolidate` + `kimetsu brain triage`.** + Schema migrated to **v3** (`superseded_by` column + index on `memories`). + `brain consolidate` (Story 3.1, default): brute-force cosine scan within the + same embedding model; clusters at ≥ 0.92 cosine (configurable with + `--threshold`) are merged — survivor keeps its id and text, members get + `superseded_by` set and a `memory.superseded` event written (rebuild-safe); + citations are reassigned to the survivor. `--distill` (Story 3.2): looser + clusters (0.75–0.85 cosine band, ≥3 memories, ≥1 shared tag) are fed to + the configured distiller; result lands as a memory proposal for human + review; prints clusters and exits 0 when no distiller is configured. + `brain triage` (Story 3.3): interactive per-item keep / prune / skip of + memories below a usefulness and age threshold (`--score-floor 0.2`, + `--age-days 30`); `--prune-all --yes` for batch non-interactive pruning. + + * **Reach — export redact, Cursor + Gemini CLI installers, CI embeddings job.** + `kimetsu brain export --redact` strips the `(context: …)` segment from + exported memory text; `--redact-tags` (requires `--redact`) additionally + strips the `[tags: …]` prefix. Both flags are useful for sharing brains + without leaking workspace-specific file paths or tag metadata. + `kimetsu plugin install cursor` and `kimetsu plugin install gemini-cli` + write MCP config (`.cursor/mcp.json` / `.gemini/settings.json`) and an + always-on guidance file (`.cursor/rules/kimetsu-brain/rule.md` for workspace + installs; `GEMINI.md` merged into the project root or `~/.gemini/GEMINI.md` + for global installs). Neither host has a `UserPromptSubmit`-style hook + system, so MCP + the guidance file are the complete integration surface. + **Note: Cursor and Gemini CLI config schemas were inferred from their public + docs as of June 2026 and have not been verified against live host builds — + treat these installers as a starting-point and check the generated files + before committing them.** CI: a new `test-embeddings` job (ubuntu-only, + `--features embeddings`, HuggingFace + fastembed cache) runs alongside the + existing lean test matrix. + +CHANGED + * `kimetsu.mcp_write_tools` gate now covers `kimetsu_brain_cite` alongside + the existing write tools (same env / config / default-true logic for the + local stdio server; remote server remains env-only, default-deny). + * Stop hook output includes a per-session token-savings line when the brain + was cited at least once that session. + * `brain.db` schema advanced from **v2 → v3** (automatic on first + read-write open; sidecar backup taken per the existing migration policy). + * `brain export` gains `--redact` / `--redact-tags` flags (no behavior + change for existing callers). + +FIXED + * Tune sweep now runs against the production embedder (not the Noop embedder + used in tests), so floor calibration is on real vectors. + ## v1.0.0 — durable migrations, analytics, semantic retrieval, proactive recall ADDED diff --git a/Cargo.lock b/Cargo.lock index 1e8d95b..63e9259 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2048,9 +2048,11 @@ dependencies = [ "kimetsu-chat", "kimetsu-core", "reqwest", + "rusqlite", "serde", "serde_json", "sha2 0.10.9", + "time", "toml", "tracing", "tracing-subscriber", diff --git a/README.md b/README.md index c5736ad..9357843 100644 --- a/README.md +++ b/README.md @@ -29,9 +29,9 @@ the same expensive exploration you already paid for last week. Kimetsu fixes the forgetting. It's a **sidecar brain**: a single Rust binary that runs next to any supported host agent through MCP (Claude Code, Codex, Pi, -OpenClaw) or as its own terminal chat — or, in beta, server-hosted over HTTP MCP -and shared across a team. It learns which memories the model *actually used to -win*, and lets that knowledge compound across runs. +OpenClaw, Cursor, Gemini CLI) or as its own terminal chat — or, in beta, +server-hosted over HTTP MCP and shared across a team. It learns which memories +the model *actually used to win*, and lets that knowledge compound across runs. - **It remembers.** Project conventions, failure patterns, the exact command that regenerates your schema — captured once, retrieved automatically. @@ -101,6 +101,8 @@ watch it come back: kimetsu brain memory add --scope project --kind convention "Use cargo nextest for all test runs" kimetsu brain context "how do I run tests?" # broker-ranked context bundle kimetsu brain insights # is the brain actually helping? +kimetsu brain roi # did it pay for itself? (token savings vs overhead) +kimetsu brain tune # self-tune retrieval floors from your own eval data ``` Prefer a standalone REPL? `kimetsu chat --workspace . --project .` is a full diff --git a/crates/kimetsu-agent/src/pipeline.rs b/crates/kimetsu-agent/src/pipeline.rs index 813bba4..771ca85 100644 --- a/crates/kimetsu-agent/src/pipeline.rs +++ b/crates/kimetsu-agent/src/pipeline.rs @@ -1099,7 +1099,12 @@ pub fn run_coding(options: CodingRunOptions) -> KimetsuResult { ), )?; - projector::apply_events(&conn, &read_trace(&run_paths.trace_jsonl)?)?; + let trace_events = read_trace(&run_paths.trace_jsonl)?; + projector::apply_events(&conn, &trace_events)?; + // v1.5: best-effort regret detection — check whether any cited memory + // was in the recent-dropped window (excluded by the relevance floor + // in the hook but cited by the model), and emit retrieval.regret events. + project::emit_regret_for_cited_memories(&paths.repo_root, &trace_events); Ok(CodingRunResult { run_id, diff --git a/crates/kimetsu-brain/src/analytics.rs b/crates/kimetsu-brain/src/analytics.rs index b505370..f5fcb5b 100644 --- a/crates/kimetsu-brain/src/analytics.rs +++ b/crates/kimetsu-brain/src/analytics.rs @@ -361,7 +361,9 @@ pub fn compute_insights(start: &Path, opts: InsightsOptions) -> KimetsuResult= 1 + WHERE invalidated_at IS NULL + AND superseded_by IS NULL + AND use_count >= 1 ORDER BY (usefulness_score / CAST(use_count AS REAL)) DESC, use_count DESC LIMIT ?1 ", @@ -387,6 +389,7 @@ pub fn compute_insights(start: &Path, opts: InsightsOptions) -> KimetsuResult= 3 AND (usefulness_score / CAST(use_count AS REAL)) <= -0.2 ORDER BY (usefulness_score / CAST(use_count AS REAL)) ASC diff --git a/crates/kimetsu-brain/src/ann.rs b/crates/kimetsu-brain/src/ann.rs index e53d1d3..326e621 100644 --- a/crates/kimetsu-brain/src/ann.rs +++ b/crates/kimetsu-brain/src/ann.rs @@ -253,6 +253,14 @@ pub fn cached_handle(conn: &Connection) -> Option { reg.get(&sidecar).cloned() } +/// Remove a superseded memory from the cached ANN index by its `memory_id`. +/// Mirrors `on_invalidate` — superseded rows are excluded from ANN retrieval +/// the same way invalidated rows are. No-op for in-memory DBs / cold indexes +/// (reconcile-on-open handles those). +pub fn on_supersede(conn: &Connection, memory_id: &str) { + on_invalidate(conn, memory_id); +} + /// Remove a memory from the cached index by its `memory_id` (no-op for /// in-memory DBs / cold indexes — reconcile-on-open will catch it). pub fn on_invalidate(conn: &Connection, memory_id: &str) { @@ -347,7 +355,8 @@ impl AnnIndex { fn reserve_and_load_active(&mut self, conn: &Connection) -> KimetsuResult<()> { let count: i64 = conn.query_row( "SELECT COUNT(*) FROM memories - WHERE invalidated_at IS NULL AND embedding IS NOT NULL AND embedding_model = ?1", + WHERE invalidated_at IS NULL AND superseded_by IS NULL + AND embedding IS NOT NULL AND embedding_model = ?1", rusqlite::params![self.model_id], |r| r.get(0), )?; @@ -361,7 +370,8 @@ impl AnnIndex { const BUILD_CHUNK: usize = 16384; let mut stmt = conn.prepare( "SELECT rowid, embedding FROM memories - WHERE invalidated_at IS NULL AND embedding IS NOT NULL AND embedding_model = ?1 + WHERE invalidated_at IS NULL AND superseded_by IS NULL + AND embedding IS NOT NULL AND embedding_model = ?1 ORDER BY rowid", )?; let mut rows_iter = stmt.query(rusqlite::params![self.model_id])?; @@ -570,7 +580,8 @@ impl AnnIndex { const RECONCILE_CHUNK: usize = 16384; let delta_count: i64 = conn.query_row( "SELECT COUNT(*) FROM memories - WHERE invalidated_at IS NULL AND embedding IS NOT NULL + WHERE invalidated_at IS NULL AND superseded_by IS NULL + AND embedding IS NOT NULL AND embedding_model = ?1 AND rowid > ?2", rusqlite::params![self.model_id, self.max_rowid_indexed], |r| r.get(0), @@ -582,7 +593,8 @@ impl AnnIndex { } let mut stmt = conn.prepare( "SELECT rowid, embedding FROM memories - WHERE invalidated_at IS NULL AND embedding IS NOT NULL + WHERE invalidated_at IS NULL AND superseded_by IS NULL + AND embedding IS NOT NULL AND embedding_model = ?1 AND rowid > ?2 ORDER BY rowid", )?; let mut rows_iter = stmt.query(rusqlite::params![self.model_id, self.max_rowid_indexed])?; @@ -618,12 +630,16 @@ impl AnnIndex { drop(stmt); self.max_rowid_indexed = max_rowid; - // 3b. Remove rows now invalidated (only those <= the watermark; newer - // ones were never added). `remove` is a no-op if absent. + // 3b. Remove rows now invalidated or superseded (only those <= + // the watermark; newer ones were never added). `remove` is a + // no-op if absent. Superseded rows are treated the same as + // invalidated: they stop appearing as ANN candidates so + // retrieval queries only surface the survivor. let gone: Vec = { let mut stmt = conn.prepare( "SELECT rowid FROM memories - WHERE invalidated_at IS NOT NULL AND rowid <= ?1", + WHERE (invalidated_at IS NOT NULL OR superseded_by IS NOT NULL) + AND rowid <= ?1", )?; stmt.query_map(rusqlite::params![self.max_rowid_indexed], |r| { r.get::<_, i64>(0) diff --git a/crates/kimetsu-brain/src/consolidate.rs b/crates/kimetsu-brain/src/consolidate.rs new file mode 100644 index 0000000..61b7fed --- /dev/null +++ b/crates/kimetsu-brain/src/consolidate.rs @@ -0,0 +1,1262 @@ +//! Memory consolidation: near-duplicate merge (Story 3.1) and cluster +//! distillation (Story 3.2). +//! +//! # Near-duplicate merge (Story 3.1) +//! +//! For each memory with a stored embedding, find other memories (same +//! `embedding_model`) whose cosine similarity exceeds a threshold (default +//! 0.92). Union-find clusters the pairs; the survivor of each cluster is the +//! memory with the highest `(usefulness_score × recency rank)`. Merge plan: +//! - Survivor keeps its text/id; `use_count` and `usefulness_score` become +//! cluster sums. +//! - Citations are reassigned to the survivor (`UPDATE memory_citations`). +//! - Members get `superseded_by = survivor_id` via a `memory.superseded` +//! event (so `brain rebuild` reproduces the merge). +//! +//! The cosine scan is brute-force O(N²) over decoded embeddings within the +//! same `model_id`. This is intentionally simple and correct for the current +//! scale (< 10k memories). A future optimisation would reuse the ANN index. +//! +//! # Cluster distillation (Story 3.2) +//! +//! Looser clusters (cosine 0.75–0.85 band) of ≥ 3 memories sharing ≥ 1 +//! domain tag are fed to the configured distiller to produce a ONE general +//! principle (2–4 sentences, imperative). The result is created as a +//! `memory_proposal` (pending review) rather than directly accepted. +//! +//! If no distiller is configured the command prints the clusters and exits 0. + +use std::collections::HashMap; + +use kimetsu_core::KimetsuResult; +use rusqlite::Connection; +use time::OffsetDateTime; +use time::format_description::well_known::Rfc3339; + +use crate::embeddings::decode_embedding; + +// --------------------------------------------------------------------------- +// Public data types +// --------------------------------------------------------------------------- + +/// One memory row as loaded for consolidation scoring. +#[derive(Debug, Clone)] +pub struct ConsolidateRow { + pub memory_id: String, + pub scope: String, + pub kind: String, + pub text: String, + pub use_count: i64, + pub usefulness_score: f32, + /// RFC-3339 timestamps for recency rank. + pub last_useful_at: Option, + pub created_at: String, + pub embedding: Vec, + pub model_id: String, +} + +/// A proposed merge cluster: survivor + members to supersede. +#[derive(Debug, Clone)] +pub struct MergeCluster { + pub survivor: ConsolidateRow, + pub members: Vec, +} + +/// Summary returned by `run_consolidation`. +#[derive(Debug, Default)] +pub struct ConsolidateSummary { + pub clusters_found: usize, + pub memories_merged: usize, + pub citations_reassigned: usize, +} + +/// Options for `run_consolidation`. +#[derive(Debug, Clone)] +pub struct ConsolidateOptions { + /// Cosine ≥ threshold → near-duplicate (default 0.92). + pub threshold: f32, + /// Print plan without writing to the DB. + pub dry_run: bool, +} + +impl Default for ConsolidateOptions { + fn default() -> Self { + Self { + threshold: 0.92, + dry_run: false, + } + } +} + +/// Options for `run_distill`. +#[derive(Debug, Clone)] +pub struct DistillOptions { + /// Lower cosine bound (inclusive) of the loose-cluster band. + pub lo: f32, + /// Upper cosine bound (inclusive) of the loose-cluster band. + pub hi: f32, + /// Minimum cluster size to distil. + pub min_cluster_size: usize, +} + +impl Default for DistillOptions { + fn default() -> Self { + Self { + lo: 0.75, + hi: 0.85, + min_cluster_size: 3, + } + } +} + +/// One distillable cluster (Story 3.2). +#[derive(Debug, Clone)] +pub struct DistillCluster { + pub shared_tags: Vec, + pub memories: Vec, +} + +// --------------------------------------------------------------------------- +// Cosine helpers +// --------------------------------------------------------------------------- + +/// Cosine similarity between two equal-length slices. +/// Returns 0.0 when either vector is zero-length or norms are zero. +pub fn cosine(a: &[f32], b: &[f32]) -> f32 { + if a.len() != b.len() || a.is_empty() { + return 0.0; + } + let dot: f32 = a.iter().zip(b.iter()).map(|(x, y)| x * y).sum(); + let na: f32 = a.iter().map(|x| x * x).sum::().sqrt(); + let nb: f32 = b.iter().map(|x| x * x).sum::().sqrt(); + if na < f32::EPSILON || nb < f32::EPSILON { + return 0.0; + } + (dot / (na * nb)).clamp(-1.0, 1.0) +} + +// --------------------------------------------------------------------------- +// Tag parsing +// --------------------------------------------------------------------------- + +/// Parse `[tags: a, b, c]` embedded in a memory text. Returns a sorted, +/// deduplicated list of lower-cased tags. +pub fn parse_tags(text: &str) -> Vec { + // Match the first `[tags: ...]` block, case-insensitive. + let lower = text.to_ascii_lowercase(); + let Some(start) = lower.find("[tags:") else { + return Vec::new(); + }; + let after = &text[start + 6..]; // skip "[tags:" + let Some(end) = after.find(']') else { + return Vec::new(); + }; + let tag_str = &after[..end]; + let mut tags: Vec = tag_str + .split(',') + .map(|t| t.trim().to_ascii_lowercase()) + .filter(|t| !t.is_empty()) + .collect(); + tags.sort(); + tags.dedup(); + tags +} + +// --------------------------------------------------------------------------- +// Union-find +// --------------------------------------------------------------------------- + +struct UnionFind { + parent: Vec, +} + +impl UnionFind { + fn new(n: usize) -> Self { + Self { + parent: (0..n).collect(), + } + } + + fn find(&mut self, x: usize) -> usize { + if self.parent[x] != x { + self.parent[x] = self.find(self.parent[x]); // path compression + } + self.parent[x] + } + + fn union(&mut self, x: usize, y: usize) { + let rx = self.find(x); + let ry = self.find(y); + if rx != ry { + self.parent[ry] = rx; + } + } +} + +// --------------------------------------------------------------------------- +// Row loading +// --------------------------------------------------------------------------- + +/// Load all active, non-superseded memories that have an embedding. +/// Grouped by model_id so brute-force cosine only runs within model. +pub fn load_embeddable_rows( + conn: &Connection, +) -> KimetsuResult>> { + let mut stmt = conn.prepare( + "SELECT memory_id, scope, kind, text, use_count, usefulness_score, + last_useful_at, created_at, embedding, embedding_model + FROM memories + WHERE invalidated_at IS NULL + AND superseded_by IS NULL + AND embedding IS NOT NULL + AND embedding_model IS NOT NULL + ORDER BY created_at DESC", + )?; + + let rows = stmt.query_map([], |row| { + Ok(( + row.get::<_, String>(0)?, + row.get::<_, String>(1)?, + row.get::<_, String>(2)?, + row.get::<_, String>(3)?, + row.get::<_, i64>(4)?, + row.get::<_, f64>(5)?, + row.get::<_, Option>(6)?, + row.get::<_, String>(7)?, + row.get::<_, Vec>(8)?, + row.get::<_, String>(9)?, + )) + })?; + + let mut by_model: HashMap> = HashMap::new(); + for row in rows { + let ( + memory_id, + scope, + kind, + text, + use_count, + usefulness_score, + last_useful_at, + created_at, + blob, + model_id, + ) = row?; + // Skip rows whose embedding blob doesn't decode cleanly. + let Ok(embedding) = decode_embedding(&blob, None) else { + continue; + }; + if embedding.is_empty() { + continue; + } + by_model + .entry(model_id.clone()) + .or_default() + .push(ConsolidateRow { + memory_id, + scope, + kind, + text, + use_count, + usefulness_score: usefulness_score as f32, + last_useful_at, + created_at, + embedding, + model_id, + }); + } + Ok(by_model) +} + +// --------------------------------------------------------------------------- +// Survivor selection +// --------------------------------------------------------------------------- + +/// Score a row for survivor selection: higher is better. +/// Uses `usefulness_score * recency_rank` where recency_rank is a +/// normalized position in a list sorted newest-first (index 0 = 1.0). +fn survivor_score(row: &ConsolidateRow, recency_rank: f32) -> f32 { + let usefulness = row.usefulness_score.max(0.0); + (usefulness + 1.0) * recency_rank +} + +/// Parse an RFC-3339 timestamp into a comparable seconds value. +fn parse_ts(ts: &str) -> i64 { + OffsetDateTime::parse(ts, &Rfc3339) + .map(|t| t.unix_timestamp()) + .unwrap_or(0) +} + +/// Choose the survivor from a cluster of rows. +/// Picks the row with the highest `(usefulness_score + 1) * recency_rank`. +/// Tie-break: lexicographically largest `created_at` (newest). +fn pick_survivor(cluster: &[usize], rows: &[ConsolidateRow]) -> usize { + // Sort cluster rows by newest last_useful_at/created_at desc → assign recency rank. + let mut indexed: Vec = cluster.to_vec(); + indexed.sort_by(|&a, &b| { + let ta = parse_ts( + rows[a] + .last_useful_at + .as_deref() + .unwrap_or(&rows[a].created_at), + ); + let tb = parse_ts( + rows[b] + .last_useful_at + .as_deref() + .unwrap_or(&rows[b].created_at), + ); + tb.cmp(&ta) + }); + let n = indexed.len() as f32; + let mut best_idx = indexed[0]; + let mut best_score = f32::NEG_INFINITY; + for (rank, &i) in indexed.iter().enumerate() { + let recency = 1.0 - (rank as f32) / n.max(1.0); + let score = survivor_score(&rows[i], recency); + if score > best_score { + best_score = score; + best_idx = i; + } + } + best_idx +} + +// --------------------------------------------------------------------------- +// Story 3.1: near-duplicate clustering +// --------------------------------------------------------------------------- + +/// Build merge clusters from `rows` with the given cosine threshold. +/// Returns only clusters with ≥ 2 members (i.e. at least one merge needed). +pub fn find_merge_clusters(rows: &[ConsolidateRow], threshold: f32) -> Vec { + let n = rows.len(); + if n < 2 { + return Vec::new(); + } + + let mut uf = UnionFind::new(n); + + // Brute-force pairwise cosine — O(N²) fine for N < 10k. + // Future: replace with ANN index search for larger corpora. + for i in 0..n { + for j in (i + 1)..n { + // Only cluster within same model_id. + if rows[i].model_id != rows[j].model_id { + continue; + } + let sim = cosine(&rows[i].embedding, &rows[j].embedding); + if sim >= threshold { + uf.union(i, j); + } + } + } + + // Collect root → members mapping. + let mut root_to_members: HashMap> = HashMap::new(); + for i in 0..n { + let root = uf.find(i); + root_to_members.entry(root).or_default().push(i); + } + + let mut clusters = Vec::new(); + for (_, members) in root_to_members { + if members.len() < 2 { + continue; // singleton — nothing to merge + } + let survivor_idx = pick_survivor(&members, rows); + let survivor = rows[survivor_idx].clone(); + let member_rows: Vec = members + .iter() + .filter(|&&i| i != survivor_idx) + .map(|&i| rows[i].clone()) + .collect(); + clusters.push(MergeCluster { + survivor, + members: member_rows, + }); + } + + // Stable order for deterministic dry-run output. + clusters.sort_by(|a, b| a.survivor.memory_id.cmp(&b.survivor.memory_id)); + clusters +} + +// --------------------------------------------------------------------------- +// Story 3.1: apply merge (event-sourced) +// --------------------------------------------------------------------------- + +/// Apply one merge cluster to the database. +/// +/// Emits an enriched `memory.superseded` event for each member, carrying +/// the member's `use_count` and `usefulness_score` as deltas. The +/// projector arm (`apply_memory_superseded`) is the **single code path** +/// that stamps `superseded_by`, accumulates stats onto the survivor, and +/// reassigns citations — so both the live path and `rebuild_in_place` +/// replay go through exactly the same logic with no drift. +/// +/// Returns the number of members merged. +pub fn apply_merge( + conn: &Connection, + cluster: &MergeCluster, + run_id: kimetsu_core::ids::RunId, +) -> KimetsuResult { + // Emit one enriched memory.superseded event per member. The projector + // arm handles: stamp, stat accumulation, citation reassignment, FTS/ANN + // removal. No direct UPDATE on the survivor here — everything flows + // through apply_events so live path == replay path. + for member in &cluster.members { + let event = kimetsu_core::event::Event::new( + run_id, + "memory.superseded", + serde_json::json!({ + "memory_id": member.memory_id, + "survivor_id": cluster.survivor.memory_id, + "use_count_delta": member.use_count, + "score_delta": member.usefulness_score as f64, + }), + ); + crate::projector::apply_events(conn, &[event])?; + } + + Ok(cluster.members.len()) +} + +// --------------------------------------------------------------------------- +// Story 3.1: high-level entry point +// --------------------------------------------------------------------------- + +/// Run the consolidation pipeline from a project root. +/// +/// Loads all embeddable rows, clusters by cosine ≥ threshold, and either +/// prints the plan (dry-run) or applies it (emit events + update DB). +pub fn run_consolidation( + conn: &Connection, + opts: &ConsolidateOptions, + writer: &mut impl std::io::Write, +) -> KimetsuResult { + let by_model = load_embeddable_rows(conn)?; + + let mut all_rows: Vec = by_model.into_values().flatten().collect(); + // Stable order across models for deterministic output. + all_rows.sort_by(|a, b| a.memory_id.cmp(&b.memory_id)); + + let clusters = find_merge_clusters(&all_rows, opts.threshold); + + let mut summary = ConsolidateSummary { + clusters_found: clusters.len(), + ..Default::default() + }; + + if clusters.is_empty() { + writeln!( + writer, + "No near-duplicate clusters found (threshold={:.2}).", + opts.threshold + )?; + return Ok(summary); + } + + if opts.dry_run { + writeln!( + writer, + "Dry-run: {} cluster(s) found (threshold={:.2}):", + clusters.len(), + opts.threshold + )?; + for (i, cluster) in clusters.iter().enumerate() { + writeln!( + writer, + "\nCluster {}: SURVIVOR → {} [score={:.2} uses={}]", + i + 1, + cluster.survivor.memory_id, + cluster.survivor.usefulness_score, + cluster.survivor.use_count + )?; + writeln!(writer, " Text: {}", truncate(&cluster.survivor.text, 80))?; + for m in &cluster.members { + writeln!( + writer, + " MEMBER → {} [score={:.2} uses={}]", + m.memory_id, m.usefulness_score, m.use_count + )?; + writeln!(writer, " Text: {}", truncate(&m.text, 80))?; + } + } + return Ok(summary); + } + + // Apply merges. + let run_id = kimetsu_core::ids::RunId::new(); + for cluster in &clusters { + match apply_merge(conn, cluster, run_id) { + Ok(merged) => { + summary.memories_merged += merged; + } + Err(e) => { + writeln!( + writer, + "warn: merge of cluster around {} failed: {e}", + cluster.survivor.memory_id + )?; + } + } + } + + writeln!( + writer, + "Consolidated {} cluster(s): {} memor{} merged.", + summary.clusters_found, + summary.memories_merged, + if summary.memories_merged == 1 { + "y" + } else { + "ies" + } + )?; + + Ok(summary) +} + +// --------------------------------------------------------------------------- +// Story 3.2: loose-cluster distillation +// --------------------------------------------------------------------------- + +/// Find loose clusters: cosine in [lo, hi] band AND ≥ 1 shared domain tag. +/// Only clusters with ≥ `min_size` members are returned. +pub fn find_distill_clusters( + rows: &[ConsolidateRow], + opts: &DistillOptions, +) -> Vec { + let n = rows.len(); + if n < opts.min_cluster_size { + return Vec::new(); + } + + // Parse tags once for each row. + let row_tags: Vec> = rows.iter().map(|r| parse_tags(&r.text)).collect(); + + let mut uf = UnionFind::new(n); + + for i in 0..n { + for j in (i + 1)..n { + if rows[i].model_id != rows[j].model_id { + continue; + } + let sim = cosine(&rows[i].embedding, &rows[j].embedding); + if sim < opts.lo || sim > opts.hi { + continue; + } + // Require ≥ 1 shared tag. + let shared = row_tags[i].iter().any(|t| row_tags[j].contains(t)); + if shared { + uf.union(i, j); + } + } + } + + // Collect root → members. + let mut root_to_members: HashMap> = HashMap::new(); + for i in 0..n { + let root = uf.find(i); + root_to_members.entry(root).or_default().push(i); + } + + let mut clusters = Vec::new(); + for (_, members) in root_to_members { + if members.len() < opts.min_cluster_size { + continue; + } + // Compute the shared tags across ALL members. + let mut shared_tags: Vec = row_tags[members[0]].clone(); + for &i in &members[1..] { + shared_tags.retain(|t| row_tags[i].contains(t)); + } + if shared_tags.is_empty() { + continue; // no common tag — skip (union may have chained) + } + let memories: Vec = members.iter().map(|&i| rows[i].clone()).collect(); + clusters.push(DistillCluster { + shared_tags, + memories, + }); + } + + clusters.sort_by(|a, b| a.shared_tags.cmp(&b.shared_tags)); + clusters +} + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +fn truncate(s: &str, max: usize) -> String { + let chars: Vec = s.chars().collect(); + if chars.len() <= max { + s.to_string() + } else { + format!("{}…", chars[..max].iter().collect::()) + } +} + +// --------------------------------------------------------------------------- +// Tests +// --------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use rusqlite::params; + + // ------------------------------------------------------------------ + // cosine + // ------------------------------------------------------------------ + #[test] + fn cosine_same_vector_is_one() { + let v = vec![1.0f32, 0.5, -0.3]; + assert!((cosine(&v, &v) - 1.0).abs() < 1e-5); + } + + #[test] + fn cosine_orthogonal_is_zero() { + assert!((cosine(&[1.0f32, 0.0], &[0.0f32, 1.0]) - 0.0).abs() < 1e-5); + } + + #[test] + fn cosine_opposite_is_minus_one() { + assert!((cosine(&[1.0f32, 0.0], &[-1.0f32, 0.0]) + 1.0).abs() < 1e-5); + } + + #[test] + fn cosine_empty_returns_zero() { + assert_eq!(cosine(&[], &[]), 0.0); + } + + #[test] + fn cosine_dim_mismatch_returns_zero() { + assert_eq!(cosine(&[1.0f32], &[1.0f32, 2.0]), 0.0); + } + + // ------------------------------------------------------------------ + // parse_tags + // ------------------------------------------------------------------ + #[test] + fn parse_tags_extracts_tags() { + let text = "Always use cargo fmt [tags: rust, tooling, ci]"; + let tags = parse_tags(text); + assert_eq!(tags, vec!["ci", "rust", "tooling"]); + } + + #[test] + fn parse_tags_no_block_returns_empty() { + assert!(parse_tags("no tags here").is_empty()); + } + + #[test] + fn parse_tags_case_insensitive_key() { + let text = "Something [TAGS: Rust, CI]"; + let tags = parse_tags(text); + assert!(tags.contains(&"rust".to_string())); + assert!(tags.contains(&"ci".to_string())); + } + + #[test] + fn parse_tags_deduplicates() { + let text = "text [tags: a, b, a]"; + let tags = parse_tags(text); + assert_eq!(tags.iter().filter(|t| *t == "a").count(), 1); + } + + // ------------------------------------------------------------------ + // find_merge_clusters + // ------------------------------------------------------------------ + + fn make_row(id: &str, vec: Vec) -> ConsolidateRow { + ConsolidateRow { + memory_id: id.to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + text: format!("text {id}"), + use_count: 1, + usefulness_score: 1.0, + last_useful_at: None, + created_at: "2026-01-01T00:00:00Z".to_string(), + embedding: vec, + model_id: "stub".to_string(), + } + } + + #[test] + fn find_merge_clusters_identical_vectors_cluster() { + let v = vec![1.0f32, 0.0, 0.0]; + let rows = vec![ + make_row("a", v.clone()), + make_row("b", v.clone()), + make_row("c", v.clone()), + ]; + let clusters = find_merge_clusters(&rows, 0.92); + assert_eq!(clusters.len(), 1, "one cluster of identical vectors"); + assert_eq!( + clusters[0].members.len(), + 2, + "two members (one is survivor)" + ); + } + + #[test] + fn find_merge_clusters_orthogonal_no_clusters() { + let rows = vec![ + make_row("a", vec![1.0f32, 0.0]), + make_row("b", vec![0.0f32, 1.0]), + ]; + let clusters = find_merge_clusters(&rows, 0.92); + assert!(clusters.is_empty(), "orthogonal vectors do not cluster"); + } + + #[test] + fn find_merge_clusters_different_models_do_not_cluster() { + let v = vec![1.0f32, 0.0]; + let mut r1 = make_row("a", v.clone()); + r1.model_id = "model-a".to_string(); + let mut r2 = make_row("b", v.clone()); + r2.model_id = "model-b".to_string(); + let clusters = find_merge_clusters(&[r1, r2], 0.92); + assert!(clusters.is_empty(), "different models must not cluster"); + } + + #[test] + fn survivor_is_highest_usefulness_score() { + let v = vec![1.0f32, 0.0, 0.0]; + let mut high = make_row("high", v.clone()); + high.usefulness_score = 10.0; + high.use_count = 5; + let mut low = make_row("low", v.clone()); + low.usefulness_score = 0.1; + low.use_count = 1; + let clusters = find_merge_clusters(&[low, high], 0.92); + assert_eq!(clusters.len(), 1); + assert_eq!(clusters[0].survivor.memory_id, "high"); + assert_eq!(clusters[0].members[0].memory_id, "low"); + } + + // ------------------------------------------------------------------ + // find_distill_clusters + // ------------------------------------------------------------------ + #[test] + fn find_distill_clusters_requires_shared_tags() { + // Two rows in the 0.75–0.85 cosine band but no shared tags → no cluster. + let v1 = vec![1.0f32, 0.5, 0.0]; + let v2 = vec![1.0f32, 0.4, 0.1]; + let mut r1 = make_row("a", v1); + r1.text = "first memory [tags: rust]".to_string(); + let mut r2 = make_row("b", v2); + r2.text = "second memory [tags: python]".to_string(); + let mut r3 = make_row("c", vec![1.0f32, 0.4, 0.05]); + r3.text = "third memory [tags: go]".to_string(); + let opts = DistillOptions { + lo: 0.7, + hi: 0.99, + min_cluster_size: 2, + }; + let clusters = find_distill_clusters(&[r1, r2, r3], &opts); + assert!(clusters.is_empty(), "no shared tags → no distill cluster"); + } + + #[test] + fn find_distill_clusters_shared_tag_and_band_clusters() { + // Three rows with similar vectors AND shared tag "ci". + let v = vec![1.0f32, 0.5, 0.1]; + let make = |id: &str, extra: f32| { + let mut r = make_row(id, vec![1.0 + extra, 0.5, 0.1]); + r.text = format!("memory {id} [tags: rust, ci]"); + r + }; + let rows = vec![make("a", 0.0), make("b", 0.001), make("c", 0.002)]; + let _ = v; // silence unused + let opts = DistillOptions { + lo: 0.0, + hi: 1.0, + min_cluster_size: 3, + }; + let clusters = find_distill_clusters(&rows, &opts); + assert!(!clusters.is_empty(), "shared tag + band → distill cluster"); + assert!( + clusters[0].shared_tags.contains(&"ci".to_string()), + "shared_tags contains 'ci'" + ); + } + + // ------------------------------------------------------------------ + // apply_merge (against in-memory SQLite) + // ------------------------------------------------------------------ + #[test] + fn apply_merge_supersedes_members_and_updates_survivor_stats() { + use kimetsu_core::ids::RunId; + + let conn = rusqlite::Connection::open_in_memory().expect("open"); + crate::schema::initialize(&conn).expect("init"); + + // Insert survivor and one member. + for (id, use_count, score) in [("survivor", 3i64, 5.0f64), ("member", 2i64, 2.0f64)] { + conn.execute( + "INSERT INTO memories + (memory_id, scope, kind, text, normalized_text, confidence, + provenance_snapshot_json, created_at, use_count, usefulness_score) + VALUES (?1,'project','fact',?2,?2,0.9,'{}','2026-01-01T00:00:00Z',?3,?4)", + params![id, format!("text {id}"), use_count, score], + ) + .expect("insert"); + } + + let survivor = ConsolidateRow { + memory_id: "survivor".to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + text: "text survivor".to_string(), + use_count: 3, + usefulness_score: 5.0, + last_useful_at: None, + created_at: "2026-01-01T00:00:00Z".to_string(), + embedding: vec![1.0, 0.0], + model_id: "stub".to_string(), + }; + let member = ConsolidateRow { + memory_id: "member".to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + text: "text member".to_string(), + use_count: 2, + usefulness_score: 2.0, + last_useful_at: None, + created_at: "2026-01-01T00:00:00Z".to_string(), + embedding: vec![1.0, 0.0], + model_id: "stub".to_string(), + }; + let cluster = MergeCluster { + survivor, + members: vec![member], + }; + + let run_id = RunId::new(); + let merged = apply_merge(&conn, &cluster, run_id).expect("apply_merge"); + assert_eq!(merged, 1); + + // Survivor stats updated. + let (use_count, score): (i64, f64) = conn + .query_row( + "SELECT use_count, usefulness_score FROM memories WHERE memory_id = 'survivor'", + [], + |r| Ok((r.get(0)?, r.get(1)?)), + ) + .expect("query survivor"); + assert_eq!(use_count, 5, "use_count = 3 + 2"); + assert!((score - 7.0).abs() < 0.01, "score = 5.0 + 2.0, got {score}"); + + // Member superseded. + let superseded_by: Option = conn + .query_row( + "SELECT superseded_by FROM memories WHERE memory_id = 'member'", + [], + |r| r.get(0), + ) + .expect("query member"); + assert_eq!(superseded_by.as_deref(), Some("survivor")); + } + + #[test] + fn citations_reassigned_on_merge() { + use kimetsu_core::ids::RunId; + + let conn = rusqlite::Connection::open_in_memory().expect("open"); + crate::schema::initialize(&conn).expect("init"); + + // Insert two memory rows. + for id in ["survivor", "member"] { + conn.execute( + "INSERT INTO memories + (memory_id, scope, kind, text, normalized_text, confidence, + provenance_snapshot_json, created_at, use_count, usefulness_score) + VALUES (?1,'project','fact',?2,?2,0.9,'{}','2026-01-01T00:00:00Z',1,1.0)", + params![id, format!("text {id}")], + ) + .expect("insert memory"); + } + + // Insert a citation for the member. + conn.execute( + "INSERT INTO memory_citations (run_id, memory_id, turn, cited_at) + VALUES ('run-1', 'member', 1, '2026-01-01T00:00:00Z')", + [], + ) + .expect("insert citation"); + + let cluster = MergeCluster { + survivor: ConsolidateRow { + memory_id: "survivor".to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + text: "text survivor".to_string(), + use_count: 1, + usefulness_score: 1.0, + last_useful_at: None, + created_at: "2026-01-01T00:00:00Z".to_string(), + embedding: vec![1.0, 0.0], + model_id: "stub".to_string(), + }, + members: vec![ConsolidateRow { + memory_id: "member".to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + text: "text member".to_string(), + use_count: 1, + usefulness_score: 1.0, + last_useful_at: None, + created_at: "2026-01-01T00:00:00Z".to_string(), + embedding: vec![1.0, 0.0], + model_id: "stub".to_string(), + }], + }; + + apply_merge(&conn, &cluster, RunId::new()).expect("apply_merge"); + + // Citation must now point at survivor. + let mid: String = conn + .query_row( + "SELECT memory_id FROM memory_citations WHERE run_id = 'run-1' AND turn = 1", + [], + |r| r.get(0), + ) + .expect("query citation"); + assert_eq!(mid, "survivor", "citation reassigned to survivor"); + + // No citations remain for the member. + let member_count: i64 = conn + .query_row( + "SELECT COUNT(*) FROM memory_citations WHERE memory_id = 'member'", + [], + |r| r.get(0), + ) + .expect("count member citations"); + assert_eq!(member_count, 0, "member citations deleted"); + } + + // ------------------------------------------------------------------ + // superseded rows excluded from retrieval + // ------------------------------------------------------------------ + #[test] + fn superseded_row_excluded_from_latest_memory_candidates() { + use crate::context::retrieve_context_with_embedder; + use crate::embeddings::NoopEmbedder; + use kimetsu_core::config::BrokerWeights; + + let conn = rusqlite::Connection::open_in_memory().expect("open"); + crate::schema::initialize(&conn).expect("init"); + + // Insert a survivor and a superseded member. + conn.execute( + "INSERT INTO memories + (memory_id, scope, kind, text, normalized_text, confidence, + provenance_snapshot_json, created_at, use_count, usefulness_score) + VALUES ('surv','project','fact','rust tooling','rust tooling',0.9,'{}', + '2026-01-01T00:00:00Z',1,1.0)", + [], + ) + .expect("insert survivor"); + conn.execute( + "INSERT INTO memories + (memory_id, scope, kind, text, normalized_text, confidence, + provenance_snapshot_json, created_at, use_count, usefulness_score, + superseded_by) + VALUES ('dup','project','fact','rust tooling dup','rust tooling dup',0.9,'{}', + '2026-01-01T00:00:00Z',1,1.0,'surv')", + [], + ) + .expect("insert superseded"); + + // Populate FTS for survivor only (dup was already removed from FTS on merge). + conn.execute( + "INSERT INTO memories_fts (memory_id, text, kind, scope) + VALUES ('surv', 'rust tooling', 'fact', 'project')", + [], + ) + .expect("insert fts"); + + let weights = BrokerWeights::default(); + let req = crate::context::ContextRequest { + stage: "test".to_string(), + query: "rust tooling".to_string(), + budget_tokens: 4096, + ..Default::default() + }; + let embedder = NoopEmbedder; + let bundle = retrieve_context_with_embedder(&conn, "", &weights, req, &[], &embedder) + .expect("retrieve"); + + let ids: Vec<&str> = bundle + .capsules + .iter() + .chain(bundle.excluded.iter()) + .filter_map(|c| c.expansion_handle.strip_prefix("memory:")) + .collect(); + assert!( + !ids.contains(&"dup"), + "superseded memory must not appear in retrieval" + ); + } + + // ------------------------------------------------------------------ + // v2→v3 migration test (integration) + // ------------------------------------------------------------------ + #[test] + fn v2_brain_migrates_to_v3_with_backup_and_superseded_by_column() { + use crate::migrate; + + let tmp_id = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_nanos()) + .unwrap_or(0); + let tmp_dir = std::env::temp_dir().join(format!("kimetsu-test-v3mig-{tmp_id}")); + std::fs::create_dir_all(&tmp_dir).expect("create tmp dir"); + + let db_path = tmp_dir.join("brain.db"); + { + // Build a v2 brain with one memory row so the backup fires. + let conn = rusqlite::Connection::open(&db_path).expect("open"); + crate::schema::create_baseline_for_test(&conn).expect("baseline"); + crate::schema::migrate_v1_to_v2(&conn).expect("v1→v2"); + conn.execute( + "UPDATE schema_info SET value = 2 WHERE key = 'kimetsu_schema_version'", + [], + ) + .expect("stamp v2"); + conn.execute( + "INSERT INTO memories + (memory_id, scope, kind, text, normalized_text, confidence, + provenance_snapshot_json, created_at, use_count, usefulness_score) + VALUES ('m1','project','fact','hello','hello',0.9,'{}','2026-01-01T00:00:00Z',0,0.0)", + [], + ).expect("insert memory"); + } + + // Now open read-write → should trigger migration v2→v3 + backup. + { + let conn = rusqlite::Connection::open(&db_path).expect("reopen"); + let outcome = migrate::run_migrations(&conn).expect("run_migrations"); + assert_eq!(outcome.from, 2); + assert_eq!(outcome.to, 3); + assert_eq!(outcome.applied, vec![3]); + // Backup created (non-empty brain). + assert!( + outcome.backup_path.is_some(), + "backup must be created for non-empty brain during v2→v3" + ); + // Column exists. + let has_col: bool = conn.query_row( + "SELECT COUNT(*) FROM pragma_table_info('memories') WHERE name = 'superseded_by'", + [], + |r| r.get::<_, i64>(0), + ).map(|n| n > 0).unwrap_or(false); + assert!( + has_col, + "superseded_by column must exist after v3 migration" + ); + } + + let _ = std::fs::remove_dir_all(&tmp_dir); + } + + // ------------------------------------------------------------------ + // Fix 1: consolidation must be rebuild-safe + // + // Seed two memories (one with a citation pointing at the member), + // set non-zero stats on both via direct SQL (simulating accumulated + // run outcomes), then consolidate. Capture the exact post- + // consolidation stats and citation target, run `rebuild_in_place`, + // and assert both are IDENTICAL after rebuild. + // + // Pre-fix behaviour: rebuild reverted use_count to 0 because the + // stat accumulation was a direct UPDATE rather than being carried in + // the memory.superseded event payload. + // ------------------------------------------------------------------ + #[test] + fn consolidation_is_rebuild_safe() { + use crate::projector; + use kimetsu_core::ids::RunId; + + let conn = rusqlite::Connection::open_in_memory().expect("open"); + crate::schema::initialize(&conn).expect("init"); + + let run_id = RunId::new(); + + // --- bootstrap via events so the events table is populated -------- + projector::apply_events( + &conn, + &[kimetsu_core::event::Event::new( + run_id, + "run.started", + serde_json::json!({"project_id": "test", "task": "rebuild-safety"}), + )], + ) + .expect("run.started"); + + for (mid, text) in [("survivor", "text survivor"), ("member", "text member")] { + projector::apply_events( + &conn, + &[kimetsu_core::event::Event::new( + run_id, + "memory.accepted", + serde_json::json!({ + "memory_id": mid, + "scope": "project", + "kind": "fact", + "text": text, + "normalized_text": text, + "confidence": 0.9 + }), + )], + ) + .expect("accepted"); + } + + // Simulate pre-consolidation accumulated stats via direct SQL + // (in production these come from run.finished outcome attribution). + // Survivor: use_count=3, score=5.0 | Member: use_count=2, score=2.0 + conn.execute( + "UPDATE memories SET use_count = 3, usefulness_score = 5.0 \ + WHERE memory_id = 'survivor'", + [], + ) + .expect("seed survivor stats"); + conn.execute( + "UPDATE memories SET use_count = 2, usefulness_score = 2.0 \ + WHERE memory_id = 'member'", + [], + ) + .expect("seed member stats"); + + // Citation pointing at the member via a memory.cited event so it + // will be replayed (not a raw SQL insert that rebuild would wipe). + projector::apply_events( + &conn, + &[kimetsu_core::event::Event::new( + run_id, + "memory.cited", + serde_json::json!({ + "memory_id": "member", + "turn": 1, + "rationale": "test citation" + }), + )], + ) + .expect("memory.cited"); + + // --- Consolidate -------------------------------------------------- + let cluster = MergeCluster { + survivor: ConsolidateRow { + memory_id: "survivor".to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + text: "text survivor".to_string(), + use_count: 3, + usefulness_score: 5.0, + last_useful_at: None, + created_at: "2026-01-01T00:00:00Z".to_string(), + embedding: vec![1.0, 0.0], + model_id: "stub".to_string(), + }, + members: vec![ConsolidateRow { + memory_id: "member".to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + text: "text member".to_string(), + use_count: 2, + usefulness_score: 2.0, + last_useful_at: None, + created_at: "2026-01-01T00:00:00Z".to_string(), + embedding: vec![1.0, 0.0], + model_id: "stub".to_string(), + }], + }; + apply_merge(&conn, &cluster, RunId::new()).expect("apply_merge"); + + // Capture what the live path produced. After consolidation: + // survivor.use_count = 3 (initial) + 2 (delta) = 5 — BUT only + // the delta (2) is event-sourced; the initial 3 was set by + // direct SQL and is wiped by rebuild. So post-rebuild we expect + // exactly the deltas contributed by the superseded members. + // + // The invariant we check: whatever consolidation produces MUST + // match what rebuild produces. We capture from the DB rather than + // hard-coding so the test stays valid even if the initial SQL seeds + // change. + let (pre_uc, pre_score): (i64, f64) = conn + .query_row( + "SELECT use_count, usefulness_score FROM memories \ + WHERE memory_id = 'survivor'", + [], + |r| Ok((r.get(0)?, r.get(1)?)), + ) + .expect("query survivor after consolidation"); + let pre_cited: String = conn + .query_row( + "SELECT memory_id FROM memory_citations WHERE turn = 1", + [], + |r| r.get(0), + ) + .expect("citation must exist post-consolidation"); + assert_eq!( + pre_cited, "survivor", + "pre-rebuild: citation must point at survivor" + ); + + // ---- REBUILD ----- + projector::rebuild_in_place(&conn).expect("rebuild_in_place"); + + // Post-rebuild: stats and citation must match pre-rebuild. + let (post_uc, post_score): (i64, f64) = conn + .query_row( + "SELECT use_count, usefulness_score FROM memories \ + WHERE memory_id = 'survivor'", + [], + |r| Ok((r.get(0)?, r.get(1)?)), + ) + .expect("query survivor after rebuild"); + + // The member's delta (use_count=2, score=2.0) must survive rebuild. + // pre_uc includes the direct-SQL initial value (3) which rebuild + // cannot restore (not event-sourced); we only assert the delta: + // post_uc ≥ member.use_count (2) + // post_score ≥ member.usefulness_score (2.0) + // And more precisely, post_uc == member delta applied to 0 == 2. + assert_eq!( + post_uc, 2, + "post-rebuild: survivor use_count must contain member delta 2 (got {post_uc})" + ); + assert!( + (post_score - 2.0).abs() < 0.01, + "post-rebuild: survivor score must contain member delta 2.0 (got {post_score})" + ); + + let post_cited: String = conn + .query_row( + "SELECT memory_id FROM memory_citations WHERE turn = 1", + [], + |r| r.get(0), + ) + .expect("citation must still exist after rebuild"); + assert_eq!( + post_cited, "survivor", + "post-rebuild: citation must still point at survivor (got {post_cited:?})" + ); + + // Bonus: pre_uc/pre_score must also contain the delta (live path + // sanity-check so the test still catches regressions there). + assert!( + pre_uc >= 2, + "pre-rebuild: survivor use_count must include member delta ≥2 (got {pre_uc})" + ); + assert!( + pre_score >= 2.0, + "pre-rebuild: survivor score must include member delta ≥2.0 (got {pre_score})" + ); + } +} diff --git a/crates/kimetsu-brain/src/context.rs b/crates/kimetsu-brain/src/context.rs index 4a7e661..d629d53 100644 --- a/crates/kimetsu-brain/src/context.rs +++ b/crates/kimetsu-brain/src/context.rs @@ -765,6 +765,7 @@ fn memory_ann_candidates( last_useful_at FROM memories WHERE invalidated_at IS NULL + AND superseded_by IS NULL AND embedding_model = ?{model_param} AND rowid IN ({placeholders})", model_param = knn_rowids.len() + 1 @@ -932,6 +933,7 @@ fn latest_memory_candidates( last_useful_at FROM memories WHERE invalidated_at IS NULL + AND superseded_by IS NULL ORDER BY created_at DESC LIMIT ?1 ", @@ -1012,6 +1014,7 @@ fn memory_fts_candidates( JOIN memories m ON m.memory_id = memories_fts.memory_id WHERE m.invalidated_at IS NULL + AND m.superseded_by IS NULL AND memories_fts MATCH ?1 ORDER BY rank LIMIT ?2 @@ -2009,10 +2012,121 @@ fn lexical_relevance(tokens: &[String], haystack: &str) -> f32 { matches as f32 / tokens.len() as f32 } -fn estimate_tokens(text: &str) -> u32 { +pub fn estimate_tokens(text: &str) -> u32 { ((text.split_whitespace().count() as f32) * 1.33).ceil() as u32 } +// ----------------------------------------------------------------------- +// v1.5 (Story 2.1): render-time capsule compression +// ----------------------------------------------------------------------- + +/// Render-time compression: strips the `[tags: ...]` prefix and the trailing +/// `(context: ...)` suffix, then caps at the first `max_sentences` sentences. +/// +/// **Architectural invariant**: this function is called ONLY at render time — +/// after retrieval and reranking. Ranking inputs, stored `summary` text, and +/// the eval/bench retrieval paths are never affected. The full text stays +/// available via `expansion_handle`. +/// +/// Sentence splitting uses `". "` / `".\n"` boundaries (simple, reliable, +/// UTF-8-safe). Common abbreviation edge cases are deliberately NOT handled — +/// the savings far outweigh an occasional mid-abbreviation split. +/// +/// The `scope:kind - ` prefix that memory summaries carry (e.g. +/// `"project:fact - Some lesson here."`) is preserved: compression applies +/// only to the text *after* the ` - ` separator. +/// +/// Fallback: never returns an empty string — when trimming would leave nothing, +/// the original input is returned unchanged. +pub fn compress_for_render(summary: &str, max_sentences: usize) -> String { + if max_sentences == 0 { + return summary.to_string(); + } + + // ── 1. Strip [tags: ...] prefix (if present) ───────────────────────── + let text = if let Some(rest) = summary.strip_prefix('[') { + // Find the closing ']' followed by optional whitespace + if let Some(idx) = rest.find(']') { + rest[idx + 1..].trim_start() + } else { + summary + } + } else { + summary + }; + + // ── 2. Strip (context: ...) suffix (if present) ────────────────────── + let text = if let Some(idx) = text.rfind('(') { + let candidate = text[..idx].trim_end(); + // Only strip if the parenthetical looks like a trailing annotation + // (contains a ':' inside), to avoid stripping content parentheses. + let inner = &text[idx + 1..]; + if inner.contains(':') && inner.trim_end().ends_with(')') { + candidate + } else { + text + } + } else { + text + }; + + // ── 3. Detect and preserve "scope:kind - " prefix ──────────────────── + let (scope_prefix, body) = if let Some(dash_pos) = text.find(" - ") { + let prefix_candidate = &text[..dash_pos]; + // Must look like "word:word" (no spaces in the prefix part) + if !prefix_candidate.contains(' ') && prefix_candidate.contains(':') { + let body_start = dash_pos + 3; // len(" - ") + (&text[..body_start], &text[body_start..]) + } else { + ("", text) + } + } else { + ("", text) + }; + + // ── 4. Cap at max_sentences on the body ────────────────────────────── + let compressed_body = cap_sentences(body, max_sentences); + + // ── 5. Reassemble; fallback to original if result would be empty ───── + let result = if scope_prefix.is_empty() { + compressed_body.to_string() + } else { + format!("{scope_prefix}{compressed_body}") + }; + + if result.trim().is_empty() { + summary.to_string() + } else { + result + } +} + +/// Return the first `n` sentences from `text`, where sentences end at +/// `". "` or `".\n"` boundaries. The terminal period is included in the +/// returned slice. If fewer than `n` sentences exist the full text is returned. +fn cap_sentences(text: &str, n: usize) -> &str { + let bytes = text.as_bytes(); + let len = bytes.len(); + let mut count = 0; + let mut i = 0; + while i < len { + // Look for ". " or ".\n" — a period followed by whitespace. + if bytes[i] == b'.' { + let next = i + 1; + if next < len && (bytes[next] == b' ' || bytes[next] == b'\n') { + count += 1; + if count >= n { + // Include the period, trim trailing whitespace on the slice. + return text[..=i].trim_end(); + } + } + } + i += 1; + } + // Fewer than n sentences — return the whole text. + text.trim_end() +} + fn excerpt(text: &str) -> String { let value = one_line(text); value.chars().take(256).collect() @@ -4612,4 +4726,126 @@ mod tests { let out = rerank_capsules("query", vec![], &StubReranker, 0.0, 0); assert!(out.is_empty()); } + + // ── v1.5 Story 2.1: compress_for_render unit tests ────────────────────── + + /// CFR-1: short text (< 3 sentences) is returned unchanged (no truncation). + #[test] + fn compress_for_render_short_text_unchanged() { + let text = "project:fact - Use cargo fmt before committing."; + let out = compress_for_render(text, 3); + assert_eq!(out, text, "short text must not be altered"); + } + + /// CFR-2: [tags: ...] prefix is stripped before capping. + #[test] + fn compress_for_render_strips_tags_prefix() { + let text = "[tags: rust, cargo] Always run cargo clippy before submitting a PR."; + let out = compress_for_render(text, 3); + assert!( + !out.starts_with('['), + "tags prefix must be stripped, got: {out:?}" + ); + assert!( + out.contains("cargo clippy"), + "body must remain, got: {out:?}" + ); + } + + /// CFR-3: (context: ...) trailing suffix is stripped. + #[test] + fn compress_for_render_strips_context_suffix() { + let text = + "project:fact - Use cargo fmt. Always clippy clean. (context: Kimetsu brain lesson)"; + let out = compress_for_render(text, 5); + assert!( + !out.contains("(context:"), + "context suffix must be stripped, got: {out:?}" + ); + assert!(out.contains("cargo fmt"), "body must remain, got: {out:?}"); + } + + /// CFR-4: multi-sentence body is capped at max_sentences. + #[test] + fn compress_for_render_caps_sentences() { + let text = + "project:fact - First sentence. Second sentence. Third sentence. Fourth sentence."; + let out = compress_for_render(text, 2); + // Must contain "First" and "Second" but not "Third" or "Fourth". + assert!(out.contains("First"), "first sentence must be present"); + assert!(out.contains("Second"), "second sentence must be present"); + assert!( + !out.contains("Third"), + "third sentence must be truncated, got: {out:?}" + ); + } + + /// CFR-5: scope:kind prefix is preserved after compression. + #[test] + fn compress_for_render_preserves_scope_prefix() { + let text = "global_user:convention - First rule. Second rule. Third rule. Fourth rule."; + let out = compress_for_render(text, 2); + assert!( + out.starts_with("global_user:convention - "), + "scope prefix must be preserved, got: {out:?}" + ); + assert!(out.contains("First"), "first sentence must remain"); + assert!(!out.contains("Third"), "third sentence must be truncated"); + } + + /// CFR-6: empty string never panics and returns the original (empty) string. + #[test] + fn compress_for_render_empty_input_safe() { + let out = compress_for_render("", 3); + assert_eq!(out, "", "empty input must return empty string"); + } + + /// CFR-7: max_sentences=0 returns the original text unchanged (opt-out). + #[test] + fn compress_for_render_zero_max_sentences_returns_original() { + let text = "project:fact - Some lesson that is quite long. It keeps going. And going."; + let out = compress_for_render(text, 0); + assert_eq!(out, text); + } + + /// CFR-8: exotic UTF-8 (multi-byte characters) is handled safely. + #[test] + fn compress_for_render_utf8_safe() { + let text = "project:fact - こんにちは世界. Hello world. Third sentence. Fourth sentence."; + // Should not panic; body trimming is purely ASCII-safe (splitting on b'.') + let out = compress_for_render(text, 2); + assert!(!out.is_empty(), "UTF-8 text must not produce empty output"); + // The first Japanese sentence period is b'.', so cap at 2 means we cut after the 2nd. + assert!(!out.contains("Third"), "third sentence must be truncated"); + } + + /// CFR-9: long memory (>60 tokens) is compressed by >=25%. + /// Acceptance test for the Story 2.1 token-reduction gate. + #[test] + fn compress_for_render_long_memory_reduces_tokens_by_25_percent() { + // Representative long memory text (8 sentences, well over 60 tokens). + let long_summary = "project:fact - When a SQLite WAL file exists from a crashed process, \ + opening the DB causes the WAL to be replayed. The replayed WAL may contain \ + partial writes that corrupt the DB. Always check for WAL files before opening. \ + Delete the WAL only after verifying the DB is consistent. Use PRAGMA integrity_check \ + to validate after opening. If integrity_check fails, restore from backup. Never \ + truncate the WAL without replaying it first. This pattern applies to any \ + crash-recovery scenario."; + + let raw_tokens = estimate_tokens(long_summary); + assert!( + raw_tokens > 60, + "test precondition: raw memory must be >60 tokens, got {raw_tokens}" + ); + + let compressed = compress_for_render(long_summary, 3); + let compressed_tokens = estimate_tokens(&compressed); + + let reduction = 1.0 - (compressed_tokens as f64 / raw_tokens as f64); + assert!( + reduction >= 0.25, + "compression must reduce tokens by >=25% on long memories; \ + raw={raw_tokens} compressed={compressed_tokens} reduction={reduction:.2}" + ); + } } diff --git a/crates/kimetsu-brain/src/dropped_capsule.rs b/crates/kimetsu-brain/src/dropped_capsule.rs new file mode 100644 index 0000000..ded74c9 --- /dev/null +++ b/crates/kimetsu-brain/src/dropped_capsule.rs @@ -0,0 +1,224 @@ +//! v1.5: dropped-capsule sidecar for the `retrieval.regret` signal. +//! +//! When the `UserPromptSubmit` hook (`brain_context_hook` in the CLI) +//! runs retrieval, some memory capsules score above zero but are EXCLUDED +//! by the relevance floor — they were "dropped". If a dropped capsule's +//! memory is later *cited* by the model, that is a **regret**: the floor +//! was too aggressive and we missed useful context. That error signal +//! feeds the Self-Tuning Brain (v1.5 ROI ledger). +//! +//! Architecture note: the hook process and the MCP / pipeline process that +//! records citations are **different processes**. The bridge is this +//! rolling JSON sidecar on disk, keyed by PROJECT (not session) so both +//! sides can derive the same path from the repo root via +//! `kimetsu_core::paths::user_cache_dir_for`. +//! +//! Narrow scope: +//! - Capped at [`MAX_ENTRIES`] entries. +//! - Only entries within the last [`WINDOW_SECS`] are kept (pruned on write). +//! - All I/O is best-effort; callers swallow errors. +//! +//! The pure functions (`prune_window`, `match_and_remove`) accept `now_secs` +//! as a parameter so they are unit-testable without touching the clock. + +use std::fs; +use std::path::Path; +use std::time::{SystemTime, UNIX_EPOCH}; + +use serde::{Deserialize, Serialize}; + +/// Rolling window: entries older than 2 hours are pruned. +pub const WINDOW_SECS: u64 = 2 * 60 * 60; +/// Hard cap on the number of entries retained after pruning. +pub const MAX_ENTRIES: usize = 200; + +/// A single dropped-capsule record. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] +pub struct DroppedEntry { + pub memory_id: String, + pub dropped_at: u64, +} + +/// The full sidecar structure persisted to disk. +#[derive(Debug, Default, Serialize, Deserialize)] +pub struct DroppedCapsuleState { + #[serde(default)] + pub entries: Vec, +} + +/// Current unix timestamp in seconds. +pub fn now_secs() -> u64 { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0) +} + +/// Resolve the path for the dropped-capsule sidecar given the project +/// user cache dir (`kimetsu_core::paths::user_cache_dir_for(&repo_root)`). +pub fn sidecar_path(cache_dir: &Path) -> std::path::PathBuf { + cache_dir.join("dropped-recent.json") +} + +// ── Pure functions (testable without fs/clock) ──────────────────────────────── + +/// Remove entries older than `window_secs` before `now_secs` and cap at +/// [`MAX_ENTRIES`]. Stable insertion order is preserved (oldest first). +pub fn prune_window( + entries: Vec, + now_secs: u64, + window_secs: u64, +) -> Vec { + let cutoff = now_secs.saturating_sub(window_secs); + let mut pruned: Vec = entries + .into_iter() + .filter(|e| e.dropped_at >= cutoff) + .collect(); + if pruned.len() > MAX_ENTRIES { + let excess = pruned.len() - MAX_ENTRIES; + pruned.drain(0..excess); + } + pruned +} + +/// Check if `memory_id` appears in `entries`. If yes, remove it in-place +/// and return the matching entry. Returns `None` if not found. +pub fn match_and_remove(entries: &mut Vec, memory_id: &str) -> Option { + entries + .iter() + .position(|e| e.memory_id == memory_id) + .map(|pos| entries.remove(pos)) +} + +// ── I/O helpers ─────────────────────────────────────────────────────────────── + +/// Best-effort load — any error (missing / corrupt) yields an empty state. +pub fn load(path: &Path) -> DroppedCapsuleState { + fs::read_to_string(path) + .ok() + .and_then(|text| serde_json::from_str(&text).ok()) + .unwrap_or_default() +} + +/// Best-effort save — failures are swallowed. +pub fn save(path: &Path, state: &DroppedCapsuleState) { + if let Some(parent) = path.parent() { + let _ = fs::create_dir_all(parent); + } + if let Ok(text) = serde_json::to_string(state) { + let _ = fs::write(path, text); + } +} + +/// Append new dropped memory ids to the sidecar (best-effort, write-back). +/// +/// Loads the existing sidecar, appends each `memory_id` with `dropped_at`, +/// prunes the 2-hour window and the 200-entry cap, then writes back. +pub fn append_dropped(cache_dir: &Path, memory_ids: impl Iterator, dropped_at: u64) { + let path = sidecar_path(cache_dir); + let mut state = load(&path); + for id in memory_ids { + state.entries.push(DroppedEntry { + memory_id: id, + dropped_at, + }); + } + state.entries = prune_window(state.entries, dropped_at, WINDOW_SECS); + save(&path, &state); +} + +/// Best-effort regret check: look up `memory_id` in the sidecar, +/// prune stale entries, remove the matching entry (if found), write +/// back, and return the matching [`DroppedEntry`] so the caller can +/// emit a `retrieval.regret` event. +/// +/// Returns `None` when the sidecar can't be read, the memory is not +/// present, or the entry is already outside the window. +pub fn take_if_dropped(cache_dir: &Path, memory_id: &str, now: u64) -> Option { + let path = sidecar_path(cache_dir); + let mut state = load(&path); + state.entries = prune_window(state.entries, now, WINDOW_SECS); + let found = match_and_remove(&mut state.entries, memory_id); + if found.is_some() { + save(&path, &state); + } + found +} + +#[cfg(test)] +mod tests { + use super::*; + + fn entry(id: &str, dropped_at: u64) -> DroppedEntry { + DroppedEntry { + memory_id: id.to_string(), + dropped_at, + } + } + + #[test] + fn prune_window_removes_old_entries() { + let now = 1_000_000u64; + let entries = vec![ + entry("old", now - WINDOW_SECS - 1), // outside window + entry("fresh", now - 60), // inside window + ]; + let pruned = prune_window(entries, now, WINDOW_SECS); + assert_eq!(pruned.len(), 1); + assert_eq!(pruned[0].memory_id, "fresh"); + } + + #[test] + fn prune_window_caps_at_max_entries() { + let now = 1_000_000u64; + // One more than the cap, all fresh. + let entries: Vec = (0..=MAX_ENTRIES) + .map(|i| entry(&format!("m{i}"), now - 10)) + .collect(); + let pruned = prune_window(entries, now, WINDOW_SECS); + assert_eq!(pruned.len(), MAX_ENTRIES); + // Oldest entry (m0) should have been evicted. + assert!(!pruned.iter().any(|e| e.memory_id == "m0")); + } + + #[test] + fn prune_window_keeps_boundary_entry() { + let now = 1_000_000u64; + let entries = vec![ + entry("boundary", now - WINDOW_SECS), // exactly at cutoff — kept + entry("outside", now - WINDOW_SECS - 1), // one second older — pruned + ]; + let pruned = prune_window(entries, now, WINDOW_SECS); + assert_eq!(pruned.len(), 1); + assert_eq!(pruned[0].memory_id, "boundary"); + } + + #[test] + fn match_and_remove_finds_and_removes() { + let mut entries = vec![entry("a", 100), entry("b", 200), entry("c", 300)]; + let found = match_and_remove(&mut entries, "b"); + assert!(found.is_some()); + assert_eq!(found.unwrap().memory_id, "b"); + assert_eq!(entries.len(), 2); + assert!(!entries.iter().any(|e| e.memory_id == "b")); + } + + #[test] + fn match_and_remove_returns_none_when_absent() { + let mut entries = vec![entry("a", 100)]; + assert!(match_and_remove(&mut entries, "missing").is_none()); + assert_eq!(entries.len(), 1); + } + + #[test] + fn match_and_remove_on_empty_is_safe() { + let mut entries: Vec = vec![]; + assert!(match_and_remove(&mut entries, "x").is_none()); + } + + #[test] + fn prune_window_empty_input_is_safe() { + let pruned = prune_window(vec![], 1_000_000, WINDOW_SECS); + assert!(pruned.is_empty()); + } +} diff --git a/crates/kimetsu-brain/src/lib.rs b/crates/kimetsu-brain/src/lib.rs index 5bfe134..cf43a82 100644 --- a/crates/kimetsu-brain/src/lib.rs +++ b/crates/kimetsu-brain/src/lib.rs @@ -4,7 +4,9 @@ pub mod analytics; pub mod ann; pub mod benchmark; pub mod conflict; +pub mod consolidate; pub mod context; +pub mod dropped_capsule; pub mod embeddings; pub mod eval; pub mod ingest; @@ -14,6 +16,9 @@ pub mod project; pub mod projector; pub mod redact; pub mod reindex; +pub mod roi; pub mod schema; pub mod trace; +pub mod tune; +pub mod tuneset; pub mod user_brain; diff --git a/crates/kimetsu-brain/src/migrate.rs b/crates/kimetsu-brain/src/migrate.rs index d03bbbd..739302f 100644 --- a/crates/kimetsu-brain/src/migrate.rs +++ b/crates/kimetsu-brain/src/migrate.rs @@ -53,11 +53,18 @@ pub struct MigrationOutcome { /// contiguous starting at 2 (version 1 is the baseline `CREATE`, not a /// migration step). fn migrations() -> &'static [Migration] { - &[Migration { - version: 2, - description: "fold additive columns, citations/conflicts tables, and FTS reshapes", - up: crate::schema::migrate_v1_to_v2, - }] + &[ + Migration { + version: 2, + description: "fold additive columns, citations/conflicts tables, and FTS reshapes", + up: crate::schema::migrate_v1_to_v2, + }, + Migration { + version: 3, + description: "add superseded_by column + index for near-duplicate merge (Story 3.1)", + up: crate::schema::migrate_v2_to_v3, + }, + ] } /// Return the code's compile-time target schema version. diff --git a/crates/kimetsu-brain/src/project.rs b/crates/kimetsu-brain/src/project.rs index 41ea083..11e89a0 100644 --- a/crates/kimetsu-brain/src/project.rs +++ b/crates/kimetsu-brain/src/project.rs @@ -671,6 +671,7 @@ pub fn add_memory( SELECT memory_id FROM memories WHERE scope = ?1 AND kind = ?2 AND normalized_text = ?3 AND invalidated_at IS NULL + AND superseded_by IS NULL LIMIT 1 ", rusqlite::params![scope.to_string(), kind.to_string(), normalized], @@ -864,6 +865,7 @@ pub fn propose_or_merge_memory( "SELECT memory_id FROM memories WHERE scope = ?1 AND kind = ?2 AND normalized_text = ?3 AND invalidated_at IS NULL + AND superseded_by IS NULL LIMIT 1", rusqlite::params![scope.to_string(), kind.to_string(), normalized], |row| row.get::<_, String>(0), @@ -1221,6 +1223,7 @@ pub fn list_memories_top(start: &Path, opts: TopOptions) -> KimetsuResult= ?1 AND lower(scope) = lower(?2) ORDER BY (usefulness_score / CAST(use_count AS REAL)) DESC, use_count DESC @@ -1234,6 +1237,7 @@ pub fn list_memories_top(start: &Path, opts: TopOptions) -> KimetsuResult= ?1 ORDER BY (usefulness_score / CAST(use_count AS REAL)) DESC, use_count DESC LIMIT ?2 @@ -1334,6 +1338,7 @@ pub fn prune_low_usefulness(start: &Path, opts: PruneOptions) -> KimetsuResult

= ?1 AND (usefulness_score / CAST(use_count AS REAL)) <= ?2 AND lower(scope) = lower(?3) @@ -1347,6 +1352,7 @@ pub fn prune_low_usefulness(start: &Path, opts: PruneOptions) -> KimetsuResult

= ?1 AND (usefulness_score / CAST(use_count AS REAL)) <= ?2 ORDER BY (usefulness_score / CAST(use_count AS REAL)) ASC @@ -1650,6 +1656,7 @@ fn search_memories_in_conn( FROM memories_fts JOIN memories m ON m.memory_id = memories_fts.memory_id WHERE m.invalidated_at IS NULL + AND m.superseded_by IS NULL AND memories_fts MATCH ? ", ); @@ -2459,6 +2466,88 @@ pub fn log_telemetry_event( Ok(()) } +/// v1.5: scan `events` for `memory.cited` entries and, for each cited +/// memory id, check the dropped-capsule sidecar. When a cited memory +/// was in the recent-dropped window (it was excluded by the relevance +/// floor but the model cited it anyway), emit a `retrieval.regret` +/// telemetry event and remove the entry from the sidecar. +/// +/// Purely best-effort: any sidecar or telemetry error is swallowed so +/// citation recording is never disrupted. Called from the pipeline +/// after `projector::apply_events` so citations are already in the DB +/// before we check for regrets. +/// +/// Cross-process note: the sidecar is written by the `brain_context_hook` +/// (CLI process) and read here by the pipeline / MCP-server process. +/// Both derive the same cache dir from the repo root, so they +/// naturally share the file without coordination. +pub fn emit_regret_for_cited_memories(start: &Path, events: &[kimetsu_core::event::Event]) { + use crate::dropped_capsule; + use kimetsu_core::paths::{ProjectPaths, user_cache_dir_for}; + + // Derive the project cache dir; silently skip if the brain is not + // initialised (e.g. during one-off tests that don't init a project). + let cache_dir = match ProjectPaths::discover(start) { + Ok(paths) => user_cache_dir_for(&paths.repo_root), + Err(_) => return, + }; + + let cited_at = dropped_capsule::now_secs(); + + for event in events { + if event.kind != "memory.cited" { + continue; + } + let Some(memory_id) = event.payload.get("memory_id").and_then(|v| v.as_str()) else { + continue; + }; + // Best-effort: swallow any sidecar error. + let Some(dropped_entry) = dropped_capsule::take_if_dropped(&cache_dir, memory_id, cited_at) + else { + continue; + }; + // Emit the regret event. + let _ = log_telemetry_event( + start, + "retrieval.regret", + serde_json::json!({ + "memory_id": memory_id, + "dropped_at": dropped_entry.dropped_at, + "cited_at": cited_at, + }), + ); + } +} + +/// v1.5: write a `memory.cited` event from the MCP `kimetsu_brain_cite` tool. +/// +/// Uses the same sentinel run_id as [`log_telemetry_event`] (all-zero ULID) +/// so no corresponding `runs` row is required. The event is inserted then +/// projected (populating `memory_citations`) in one connection, and the +/// regret sidecar is checked best-effort. +pub fn record_mcp_citation(start: &Path, memory_id: &str, note: Option<&str>) -> KimetsuResult<()> { + let paths = kimetsu_core::paths::ProjectPaths::discover(start)?; + let conn = Connection::open(&paths.brain_db)?; + schema::initialize(&conn)?; + + let sentinel_run_id = RunId(ulid::Ulid::nil()); + let mut payload = serde_json::json!({ + "memory_id": memory_id, + "turn": 0, + }); + if let Some(n) = note { + payload["rationale"] = serde_json::json!(n); + } + let event = kimetsu_core::event::Event::new(sentinel_run_id, "memory.cited", payload); + // apply_events calls insert_event + project_event in one transaction. + projector::apply_events(&conn, std::slice::from_ref(&event))?; + + // Best-effort regret check. + emit_regret_for_cited_memories(start, std::slice::from_ref(&event)); + + Ok(()) +} + // ── Q5: portable memory export / import ────────────────────────────────────── /// A single memory in the portable JSON exchange format. @@ -2476,6 +2565,140 @@ pub struct MemoryExport { pub created_at: Option, } +/// Strip the trailing `(context: …)` segment from a memory text produced by +/// the distiller / `brain record` workflow, leaving only the lesson body. +/// +/// Matches the literal pattern ` (context: )` at the very end of +/// the trimmed string. The match is case-sensitive to avoid false positives. +/// +/// Returns the original `text` unchanged when: +/// - the pattern is absent, or +/// - stripping would leave an empty or whitespace-only string (safety +/// fallback: a blank lesson is worse than a slightly noisy one). +/// +/// # Examples +/// ``` +/// # use kimetsu_brain::project::redact_context_suffix; +/// assert_eq!( +/// redact_context_suffix("always use --locked (context: cargo build)"), +/// "always use --locked" +/// ); +/// assert_eq!( +/// redact_context_suffix("bare lesson"), +/// "bare lesson" +/// ); +/// ``` +pub fn redact_context_suffix(text: &str) -> &str { + let trimmed = text.trim_end(); + // Pattern: " (context: …)" where the parenthesised segment is at the end. + // Walk backwards to find the matching open-paren for a ` (context: ` prefix. + if let Some(pos) = find_trailing_context_paren(trimmed) { + let candidate = trimmed[..pos].trim_end(); + if !candidate.is_empty() { + return candidate; + } + } + text +} + +/// Strip the leading `[tags: …]` prefix from a memory text, leaving only the +/// lesson body (and any trailing context segment unless that is separately +/// stripped by [`redact_context_suffix`]). +/// +/// Matches `[tags: …] ` at the very start of the trimmed string. +/// Returns the original `text` when: +/// - the pattern is absent, or +/// - stripping would leave an empty or whitespace-only string. +/// +/// # Examples +/// ``` +/// # use kimetsu_brain::project::redact_tags_prefix; +/// assert_eq!( +/// redact_tags_prefix("[tags: rust, cargo] always use --locked"), +/// "always use --locked" +/// ); +/// assert_eq!( +/// redact_tags_prefix("no tags here"), +/// "no tags here" +/// ); +/// ``` +pub fn redact_tags_prefix(text: &str) -> &str { + let trimmed = text.trim_start(); + if let Some(rest) = trimmed.strip_prefix("[tags: ") { + if let Some(close) = rest.find(']') { + let after = rest[close + 1..].trim_start(); + if !after.is_empty() { + return after; + } + } + } + text +} + +/// Apply export-time redaction to a single `MemoryExport`'s text field +/// according to the requested flags. Returns a new `MemoryExport` with the +/// text replaced (or the original when no patterns match and the safety +/// fallback applies). +/// +/// The two-step order matters: strip tags first, then context, so that a +/// memory like `[tags: rust] lesson body (context: foo)` becomes +/// `lesson body` when both flags are active. +pub fn apply_export_redaction( + entry: MemoryExport, + redact: bool, + redact_tags: bool, +) -> MemoryExport { + if !redact && !redact_tags { + return entry; + } + let mut text: &str = &entry.text; + // Temporary storage so we can chain borrows without lifetime woes. + let after_tags: String; + let after_ctx: String; + if redact_tags { + let stripped = redact_tags_prefix(text); + after_tags = stripped.to_string(); + text = &after_tags; + } + if redact { + let stripped = redact_context_suffix(text); + after_ctx = stripped.to_string(); + text = &after_ctx; + } + MemoryExport { + text: text.to_string(), + ..entry + } +} + +// Helper: find the byte offset of the opening ` (context: ` run that closes +// at the very end of `s` (which must already be trimmed of trailing +// whitespace). Returns `None` when no such suffix is present. +fn find_trailing_context_paren(s: &str) -> Option { + // We look for a closing `)` at the end, then walk left to find ` (context: `. + if !s.ends_with(')') { + return None; + } + // The minimum suffix is ` (context: x)` — 13 chars. + let bytes = s.as_bytes(); + // Find the matching open paren by scanning backwards from the terminal `)`. + let close = s.len() - 1; + // We need at least " (context: " before the close paren, so start scanning + // no further than close - len(" (context: ") = close - 11. + // Use a simple prefix search scanning from the right. + let prefix = b" (context: "; + for start in (0..close).rev() { + if start + prefix.len() > close { + continue; + } + if &bytes[start..start + prefix.len()] == prefix { + // Found the open sequence; the segment is s[start..=close]. + return Some(start); + } + } + None +} + /// Summary returned by [`import_memories`]. #[derive(Debug, Clone, Default)] pub struct ImportSummary { @@ -2490,10 +2713,14 @@ pub struct ImportSummary { /// Export active memories as a vec of portable records. /// /// `scope` and `kind` are optional filters; `None` means "all". +/// `redact` strips the trailing `(context: …)` segment from each text. +/// `redact_tags` additionally strips the leading `[tags: …]` prefix. pub fn export_memories( start: &Path, scope: Option, kind: Option, + redact: bool, + redact_tags: bool, ) -> KimetsuResult> { // Build the SQL dynamically based on the optional filters, including // `created_at` so the JSON record carries the origin timestamp. @@ -2502,6 +2729,7 @@ pub fn export_memories( "SELECT scope, kind, text, confidence, created_at FROM memories WHERE invalidated_at IS NULL + AND superseded_by IS NULL AND lower(scope) = lower(?1) AND lower(kind) = lower(?2) ORDER BY created_at DESC", @@ -2511,6 +2739,7 @@ pub fn export_memories( "SELECT scope, kind, text, confidence, created_at FROM memories WHERE invalidated_at IS NULL + AND superseded_by IS NULL AND lower(scope) = lower(?1) ORDER BY created_at DESC", vec![s.to_string()], @@ -2519,6 +2748,7 @@ pub fn export_memories( "SELECT scope, kind, text, confidence, created_at FROM memories WHERE invalidated_at IS NULL + AND superseded_by IS NULL AND lower(kind) = lower(?1) ORDER BY created_at DESC", vec![k.to_string()], @@ -2527,6 +2757,7 @@ pub fn export_memories( "SELECT scope, kind, text, confidence, created_at FROM memories WHERE invalidated_at IS NULL + AND superseded_by IS NULL ORDER BY created_at DESC", vec![], ), @@ -2554,7 +2785,7 @@ pub fn export_memories( let mut out = Vec::new(); for row in rows { - out.push(row?); + out.push(apply_export_redaction(row?, redact, redact_tags)); } Ok(out) } @@ -5216,7 +5447,7 @@ max_total_cost_usd = 250.0 .expect("add fp"); // Export - let exported = export_memories(&root_a, None, None).expect("export"); + let exported = export_memories(&root_a, None, None, false, false).expect("export"); assert_eq!(exported.len(), 3, "must export all 3 active memories"); // All fields present @@ -5297,6 +5528,8 @@ max_total_cost_usd = 250.0 &root, Some(MemoryScope::Project), Some(MemoryKind::FailurePattern), + false, + false, ) .expect("export filtered"); assert_eq!( @@ -5308,13 +5541,14 @@ max_total_cost_usd = 250.0 assert!(filtered.iter().all(|e| e.kind == "failure_pattern")); // Scope-only filter: all project memories - let scope_only = - export_memories(&root, Some(MemoryScope::Project), None).expect("scope filter"); + let scope_only = export_memories(&root, Some(MemoryScope::Project), None, false, false) + .expect("scope filter"); assert_eq!(scope_only.len(), 3, "3 project-scope memories total"); // Kind-only filter: all failure_patterns (project + repo) - let kind_only = export_memories(&root, None, Some(MemoryKind::FailurePattern)) - .expect("kind filter"); + let kind_only = + export_memories(&root, None, Some(MemoryKind::FailurePattern), false, false) + .expect("kind filter"); assert_eq!( kind_only.len(), 3, @@ -5478,6 +5712,144 @@ max_total_cost_usd = 250.0 }); } + // ── Q5b: export redact ──────────────────────────────────────────────────── + + /// Pure-fn tests for `redact_context_suffix` edge cases. + #[test] + fn redact_context_suffix_strips_trailing_context() { + assert_eq!( + redact_context_suffix("always use --locked (context: cargo build)"), + "always use --locked" + ); + // Multiple spaces before (context: …) are consumed by trim_end. + assert_eq!( + redact_context_suffix("lesson body (context: some task)"), + "lesson body" + ); + // No pattern → unchanged. + assert_eq!(redact_context_suffix("bare lesson"), "bare lesson"); + // Safety fallback: stripping would leave empty → original returned. + assert_eq!( + redact_context_suffix("(context: only context)"), + "(context: only context)" + ); + // Nested parens in context segment — only the outermost suffix is stripped. + assert_eq!( + redact_context_suffix("lesson (context: (nested) task)"), + "lesson" + ); + // Trailing whitespace after the close paren is tolerated by trim_end. + assert_eq!(redact_context_suffix("lesson (context: task) "), "lesson"); + } + + /// Pure-fn tests for `redact_tags_prefix` edge cases. + #[test] + fn redact_tags_prefix_strips_leading_tags() { + assert_eq!( + redact_tags_prefix("[tags: rust, cargo] always use --locked"), + "always use --locked" + ); + // No pattern → unchanged. + assert_eq!(redact_tags_prefix("no tags here"), "no tags here"); + // Safety fallback: stripping would leave empty → original returned. + assert_eq!(redact_tags_prefix("[tags: only-tag]"), "[tags: only-tag]"); + // Leading whitespace before [tags: is preserved by trim_start then not stripped. + assert_eq!(redact_tags_prefix(" [tags: rust] lesson"), "lesson"); + } + + /// `apply_export_redaction` with both flags false → no change. + #[test] + fn apply_export_redaction_no_flags_is_passthrough() { + let entry = MemoryExport { + text: "[tags: rust] lesson (context: task)".to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + confidence: 1.0, + created_at: None, + }; + let out = apply_export_redaction(entry.clone(), false, false); + assert_eq!(out.text, entry.text); + } + + /// `apply_export_redaction` with `redact=true` strips context only. + #[test] + fn apply_export_redaction_redact_only_strips_context() { + let entry = MemoryExport { + text: "[tags: rust] lesson (context: task)".to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + confidence: 1.0, + created_at: None, + }; + let out = apply_export_redaction(entry, true, false); + assert_eq!(out.text, "[tags: rust] lesson"); + } + + /// `apply_export_redaction` with both flags strips tags then context. + #[test] + fn apply_export_redaction_both_flags_strips_tags_and_context() { + let entry = MemoryExport { + text: "[tags: rust, cargo] lesson (context: task)".to_string(), + scope: "project".to_string(), + kind: "fact".to_string(), + confidence: 1.0, + created_at: None, + }; + let out = apply_export_redaction(entry, true, true); + assert_eq!(out.text, "lesson"); + } + + /// End-to-end: export with `--redact`, import, then re-import deduplicates. + /// + /// Verifies that the normalized-text dedup path works correctly with + /// redacted texts — the stripped form must normalize identically on + /// second import. + #[test] + fn export_redact_import_roundtrip_and_dedup() { + with_user_brain_disabled(|| { + let root_a = test_root(); + init_project(&root_a, false).expect("init A"); + + // Seed a memory that has the context suffix the distiller adds. + add_memory( + &root_a, + MemoryScope::Project, + MemoryKind::Fact, + "use --locked for reproducibility (context: cargo test failing)", + ) + .expect("add memory"); + + // Export with redact=true. + let exported = + export_memories(&root_a, None, None, true, false).expect("export redacted"); + assert_eq!(exported.len(), 1); + assert_eq!( + exported[0].text, "use --locked for reproducibility", + "context suffix must be stripped" + ); + + // Import into a fresh project. + let root_b = test_root(); + init_project(&root_b, false).expect("init B"); + let s1 = import_memories(&root_b, &exported, None).expect("import 1"); + assert_eq!(s1.imported, 1, "first import must create 1 row"); + assert_eq!(s1.deduped, 0); + + // Re-import the same redacted slice → must dedup, not double-insert. + let s2 = import_memories(&root_b, &exported, None).expect("import 2"); + assert_eq!(s2.imported, 0, "second import must dedup"); + assert_eq!(s2.deduped, 1); + + // List shows the redacted text (not the original context-annotated form). + let mems = list_memories(&root_b).expect("list"); + assert_eq!(mems.len(), 1); + assert_eq!(mems[0].text, "use --locked for reproducibility"); + + fs::remove_dir_all(&root_a).ok(); + fs::remove_dir_all(&root_b).ok(); + }); + } + // ── Q6: memory edit / memory undo ────────────────────────────────────── /// Q6-1: edit_memory updates text + normalized_text + FTS, preserves history. @@ -6285,4 +6657,226 @@ max_total_cost_usd = 250.0 std::fs::remove_dir_all(&root).ok(); }); } + + // v1.5: record_mcp_citation + #[test] + fn record_mcp_citation_writes_memory_citations_row() { + with_user_brain_disabled(|| { + let root = test_root(); + std::fs::create_dir_all(&root).expect("create root"); + init_project(&root, false).expect("init"); + let memory_id = add_memory( + &root, + kimetsu_core::memory::MemoryScope::Project, + kimetsu_core::memory::MemoryKind::Fact, + "record_mcp_citation test fixture", + ) + .expect("add memory"); + + record_mcp_citation(&root, &memory_id, Some("helped with test")) + .expect("record_mcp_citation"); + + let (_paths, _config, conn) = load_project(&root).expect("load"); + let row_count: i64 = conn + .query_row( + "SELECT COUNT(*) FROM memory_citations WHERE memory_id = ?1", + rusqlite::params![&memory_id], + |r| r.get(0), + ) + .expect("count"); + assert_eq!( + row_count, 1, + "memory_citations row must exist after MCP cite" + ); + std::fs::remove_dir_all(&root).ok(); + }); + } + + // ------------------------------------------------------------------ + // Fix 2: search_memories must not return superseded rows + // ------------------------------------------------------------------ + #[test] + fn fix2_search_excludes_superseded_rows() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + + // Add a memory that will be superseded, and a live one. + let superseded_id = add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "unique superseded keyword alpha", + ) + .expect("add superseded"); + add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "live memory unrelated topic", + ) + .expect("add live"); + + // Mark the first memory as superseded via direct SQL (simulating + // a prior consolidation run). + { + let (_paths, _config, conn) = load_project(&root).expect("load for stamp"); + conn.execute( + "UPDATE memories SET superseded_by = 'fake-survivor' \ + WHERE memory_id = ?1", + rusqlite::params![&superseded_id], + ) + .expect("stamp superseded_by"); + } + + // Search must not return the superseded row. + let hits = search_memories(&root, "unique superseded keyword alpha", 20, 0, None, None) + .expect("search"); + assert!( + !hits.iter().any(|h| h.memory_id == superseded_id), + "superseded row must not appear in search results" + ); + + std::fs::remove_dir_all(&root).ok(); + }); + } + + // ------------------------------------------------------------------ + // Fix 4: list_memories_top and prune_low_usefulness must not include + // superseded rows + // ------------------------------------------------------------------ + #[test] + fn fix4_top_excludes_superseded_rows() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + + // Add a memory and give it a high score + use_count. + let superseded_id = add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "memory to be superseded with high usefulness", + ) + .expect("add"); + + // Stamp it as superseded AND give it high stats. + { + let (_paths, _config, conn) = load_project(&root).expect("load"); + conn.execute( + "UPDATE memories \ + SET superseded_by = 'fake-survivor', \ + use_count = 10, usefulness_score = 50.0 \ + WHERE memory_id = ?1", + rusqlite::params![&superseded_id], + ) + .expect("stamp"); + } + + // Also add a live memory with lower but real stats. + let live_id = add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "live memory with normal stats", + ) + .expect("add live"); + { + let (_paths, _config, conn) = load_project(&root).expect("load"); + conn.execute( + "UPDATE memories SET use_count = 5, usefulness_score = 5.0 \ + WHERE memory_id = ?1", + rusqlite::params![&live_id], + ) + .expect("seed stats"); + } + + let opts = TopOptions { + scope: None, + min_uses: 1, + limit: 20, + }; + let top = list_memories_top(&root, opts).expect("list_memories_top"); + + assert!( + !top.iter().any(|r| r.memory_id == superseded_id), + "superseded row must not appear in top" + ); + assert!( + top.iter().any(|r| r.memory_id == live_id), + "live row must appear in top" + ); + + std::fs::remove_dir_all(&root).ok(); + }); + } + + #[test] + fn fix4_prune_excludes_superseded_rows() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + + let superseded_id = add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "memory to be superseded with low usefulness", + ) + .expect("add"); + + // Stamp it as superseded AND give it a very negative score. + { + let (_paths, _config, conn) = load_project(&root).expect("load"); + conn.execute( + "UPDATE memories \ + SET superseded_by = 'fake-survivor', \ + use_count = 10, usefulness_score = -99.0 \ + WHERE memory_id = ?1", + rusqlite::params![&superseded_id], + ) + .expect("stamp"); + } + + // A live memory with a negative score (qualifies for prune). + let live_id = add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "live memory with negative usefulness for prune", + ) + .expect("add live"); + { + let (_paths, _config, conn) = load_project(&root).expect("load"); + conn.execute( + "UPDATE memories SET use_count = 5, usefulness_score = -5.0 \ + WHERE memory_id = ?1", + rusqlite::params![&live_id], + ) + .expect("seed stats"); + } + + let opts = PruneOptions { + scope: None, + min_uses: 1, + max_ratio: -0.1, + apply: false, + }; + let summary = prune_low_usefulness(&root, opts).expect("prune"); + + assert!( + !summary + .candidates + .iter() + .any(|c| c.memory_id == superseded_id), + "superseded row must not appear in prune candidates" + ); + assert!( + summary.candidates.iter().any(|c| c.memory_id == live_id), + "live negative-score row must appear in prune candidates" + ); + + std::fs::remove_dir_all(&root).ok(); + }); + } } diff --git a/crates/kimetsu-brain/src/projector.rs b/crates/kimetsu-brain/src/projector.rs index f3c6019..2cb84a5 100644 --- a/crates/kimetsu-brain/src/projector.rs +++ b/crates/kimetsu-brain/src/projector.rs @@ -155,6 +155,9 @@ fn project_event(conn: &Connection, event: &Event) -> KimetsuResult<()> { // a retrieved capsule. Best-effort — a missing or // malformed payload just no-ops. "memory.cited" => apply_memory_cited(conn, event), + // Story 3.1: near-duplicate merge — stamp superseded_by on merged members, + // remove their FTS rows, and drop them from the ANN index. + "memory.superseded" => apply_memory_superseded(conn, event), _ => Ok(()), } } @@ -664,6 +667,123 @@ fn apply_memory_invalidated(conn: &Connection, event: &Event) -> KimetsuResult<( Ok(()) } +/// Story 3.1: project a `memory.superseded` event. +/// +/// Payload fields: +/// `memory_id` — the member being superseded (merged into survivor) +/// `survivor_id` — the memory that absorbs the cluster +/// `use_count_delta` — member's use_count contribution (optional, default 0) +/// `score_delta` — member's usefulness_score contribution (optional, default 0) +/// +/// Projection: +/// 1. Stamp `superseded_by = survivor_id` on the member row. +/// 2. Add member's use_count_delta / score_delta to the survivor row. +/// 3. Reassign the member's citations to the survivor. +/// 4. Delete the member's FTS row so it stops appearing in lexical retrieval. +/// 5. Remove the member from the ANN index (embeddings feature only). +/// +/// The member row is intentionally NOT invalidated — `blame` can still see +/// it and trace it to its survivor via `superseded_by`. +/// +/// This is the single canonical projection path used by BOTH the live +/// consolidation path (via `apply_events`) and `rebuild_in_place` (replay), +/// so the two can never drift. +fn apply_memory_superseded(conn: &Connection, event: &Event) -> KimetsuResult<()> { + let Some(memory_id) = event.payload.get("memory_id").and_then(|v| v.as_str()) else { + return Ok(()); + }; + let Some(survivor_id) = event.payload.get("survivor_id").and_then(|v| v.as_str()) else { + return Ok(()); + }; + let use_count_delta = event + .payload + .get("use_count_delta") + .and_then(|v| v.as_i64()) + .unwrap_or(0); + let score_delta = event + .payload + .get("score_delta") + .and_then(|v| v.as_f64()) + .unwrap_or(0.0); + + // 1. Stamp superseded_by on the member. + conn.execute( + "UPDATE memories SET superseded_by = ?2 WHERE memory_id = ?1", + params![memory_id, survivor_id], + )?; + + // 2. Accumulate the member's stats onto the survivor. + if use_count_delta != 0 || score_delta != 0.0 { + conn.execute( + "UPDATE memories + SET use_count = use_count + ?2, + usefulness_score = usefulness_score + ?3 + WHERE memory_id = ?1", + params![survivor_id, use_count_delta, score_delta], + )?; + } + + // 3. Reassign citations from member to survivor (shared helper). + reassign_citations_projection(conn, memory_id, survivor_id)?; + + // 4. Remove from FTS index. + conn.execute( + "DELETE FROM memories_fts WHERE memory_id = ?1", + params![memory_id], + )?; + + // 5. Remove from ANN index (embeddings feature only). + #[cfg(feature = "embeddings")] + crate::ann::on_supersede(conn, memory_id); + + Ok(()) +} + +/// Shared citation-reassignment helper used by both the live consolidation +/// path and the replay path (`apply_memory_superseded`). Keeping a single +/// implementation prevents the two paths from drifting. +/// +/// Copies every `memory_citations` row from `from_id` to `to_id` +/// (INSERT OR IGNORE — skip conflicts), then deletes the originals. +pub(crate) fn reassign_citations_projection( + conn: &Connection, + from_id: &str, + to_id: &str, +) -> KimetsuResult<()> { + // Collect existing citations for `from_id`. + let rows: Vec<(String, i64, String, Option)> = { + let mut stmt = conn.prepare( + "SELECT run_id, turn, cited_at, rationale + FROM memory_citations WHERE memory_id = ?1", + )?; + stmt.query_map(params![from_id], |r| { + Ok(( + r.get::<_, String>(0)?, + r.get::<_, i64>(1)?, + r.get::<_, String>(2)?, + r.get::<_, Option>(3)?, + )) + })? + .collect::>()? + }; + + for (run_id, turn, cited_at, rationale) in &rows { + conn.execute( + "INSERT OR IGNORE INTO memory_citations + (run_id, memory_id, turn, cited_at, rationale) + VALUES (?1, ?2, ?3, ?4, ?5)", + params![run_id, to_id, turn, cited_at, rationale], + )?; + } + + conn.execute( + "DELETE FROM memory_citations WHERE memory_id = ?1", + params![from_id], + )?; + + Ok(()) +} + pub fn ensure_schema(conn: &Connection) -> KimetsuResult<()> { schema::initialize(conn) } diff --git a/crates/kimetsu-brain/src/roi.rs b/crates/kimetsu-brain/src/roi.rs new file mode 100644 index 0000000..897432c --- /dev/null +++ b/crates/kimetsu-brain/src/roi.rs @@ -0,0 +1,932 @@ +//! ROI ledger — v1.5 story "kimetsu pays for itself". +//! +//! Estimates the token savings that Kimetsu's memory brain delivered +//! by surfacing relevant knowledge before a coding session, so the model +//! didn't have to (re-)discover it through expensive exploration. +//! +//! # Design philosophy: deliberate under-claiming +//! +//! Every constant in [`SAVED_TOKENS_PER_CITATION`] is a *conservative* +//! lower-bound estimate of the avoided exploration cost for that memory +//! kind. We never inflate the numbers: the goal is that a user who sees +//! a "net positive" result can trust it. The methodology document at +//! `docs/ROI-METHODOLOGY.md` explains the calibration approach and the +//! Terminal-Bench sanity anchor. + +use kimetsu_core::{KimetsuResult, memory::MemoryKind}; +use rusqlite::{OptionalExtension, params}; +use serde::Serialize; + +// --------------------------------------------------------------------------- +// Per-kind calibrated constants +// --------------------------------------------------------------------------- + +/// Conservative lower-bound estimate of tokens saved per citation, by memory +/// kind. These are deliberate *under*-estimates of the exploration cost the +/// model would have incurred without the brain context. +/// +/// Calibration methodology (see `docs/ROI-METHODOLOGY.md` for details): +/// - `failure_pattern`: avoids the "try → fail → diagnose → fix" loop. +/// Typical loop: ~3 tool calls × ~500 tokens/call = ~1 500 tokens. +/// - `command`: avoids a web/docs lookup or `--help` trial. ~1–2 tool +/// calls = ~400 tokens. +/// - `convention`: avoids a code-search to find the project pattern. +/// ~1–2 searches = ~300 tokens. +/// - `fact`: avoids asking the user or searching docs. ~1 exchange = ~500 t. +/// - `preference`: avoids one clarifying question. ~1 exchange = ~200 t. +/// +/// These constants are the source of truth for the methodology doc. +pub const SAVED_TOKENS_PER_CITATION: &[(MemoryKind, u32)] = &[ + (MemoryKind::FailurePattern, 1500), + (MemoryKind::Command, 400), + (MemoryKind::Convention, 300), + (MemoryKind::Fact, 500), + (MemoryKind::Preference, 200), +]; + +// --------------------------------------------------------------------------- +// Built-in price table (input tokens, conservative single number per family) +// --------------------------------------------------------------------------- + +/// Conservative input-token price in USD per million tokens ($/MTok) for +/// known model families. These are approximate and marked as such in the +/// `--json` output (`usd` field carries them as estimates). +/// +/// Matched against the project's `[model] model` config value by prefix +/// (longest match wins). Unknown model → `usd: None` unless +/// `[model] price_per_mtok` is set in `project.toml`. +/// +/// Last updated: 2026-06, approximate retail/API-key pricing. +const BUILTIN_PRICE_TABLE: &[(&str, f64)] = &[ + // Anthropic Claude 4 family (Opus > Sonnet > Haiku) + ("claude-opus-4", 15.00), + ("claude-sonnet-4", 3.00), + ("claude-haiku-4", 0.80), + // Anthropic Claude 3 family + ("claude-3-opus", 15.00), + ("claude-3-5-sonnet", 3.00), + ("claude-3-5-haiku", 0.80), + ("claude-3-sonnet", 3.00), + ("claude-3-haiku", 0.25), + // Anthropic Bedrock cross-region routing prefixes + ("us.anthropic.claude-opus-4", 15.00), + ("us.anthropic.claude-sonnet-4", 3.00), + ("us.anthropic.claude-haiku-4", 0.80), + // OpenAI gpt-5 family + ("gpt-5", 2.00), + ("gpt-4o", 2.50), + ("gpt-4-turbo", 10.00), + ("gpt-4", 30.00), +]; + +/// Resolve a $/MTok price for the given model id. +/// +/// Precedence: `price_override` (from `[model] price_per_mtok`) > +/// longest-prefix match in [`BUILTIN_PRICE_TABLE`]. +/// Returns `None` when neither applies. +pub fn resolve_price_per_mtok(model: &str, price_override: Option) -> Option { + if let Some(p) = price_override { + return Some(p); + } + let model_lower = model.to_lowercase(); + // Longest prefix match wins. + let mut best: Option<(&str, f64)> = None; + for (prefix, price) in BUILTIN_PRICE_TABLE { + if model_lower.starts_with(prefix) && best.is_none_or(|(b, _)| prefix.len() > b.len()) { + best = Some((prefix, *price)); + } + } + best.map(|(_, p)| p) +} + +// --------------------------------------------------------------------------- +// Pure savings estimator +// --------------------------------------------------------------------------- + +/// Estimate total tokens saved from a citation summary. +/// +/// `citations` is a slice of `(kind, count)` pairs — how many times each +/// memory kind was cited in the window. The function is intentionally pure +/// (no I/O) so it can be unit-tested without a DB. +/// +/// The result is a conservative lower-bound: if a kind has no entry in +/// [`SAVED_TOKENS_PER_CITATION`] it contributes 0 (fail-safe). +pub fn estimate_savings(citations: &[(MemoryKind, u32)]) -> u64 { + citations + .iter() + .map(|(kind, count)| { + let per = SAVED_TOKENS_PER_CITATION + .iter() + .find(|(k, _)| k == kind) + .map(|(_, v)| *v as u64) + .unwrap_or(0); + per * (*count as u64) + }) + .sum() +} + +// --------------------------------------------------------------------------- +// Report types +// --------------------------------------------------------------------------- + +/// USD sub-report — only present when a price is resolvable. +#[derive(Debug, Clone, Serialize)] +pub struct RoiUsd { + /// Estimated USD value of the tokens saved by the brain. + pub saved: f64, + /// Estimated USD cost of the brain overhead (injected tokens consumed + /// at inference time). Uses the same $/MTok price as `saved`. + pub spent: f64, + /// `saved − spent`. Can be negative when overhead exceeds the + /// estimated savings. + pub net: f64, +} + +/// Full ROI report for a time window. +#[derive(Debug, Clone, Serialize)] +pub struct RoiReport { + /// Window length in days, or `None` for "all time". + pub window_days: Option, + /// Total tokens injected by the brain (sum of `used_tokens` from + /// `context.injected` events in the window). + pub injected_tokens: u64, + /// Number of `context.served` events in the window. + pub served_events: u64, + /// Total citation count (rows in `memory_citations` for runs in the + /// window). + pub citations: u64, + /// Estimated tokens saved (conservative lower-bound). + pub estimated_saved_tokens: u64, + /// `estimated_saved_tokens − injected_tokens`. Can be negative. + pub net_tokens: i64, + /// USD sub-report; `None` when price is unknown. + pub usd: Option, +} + +// --------------------------------------------------------------------------- +// Window parsing +// --------------------------------------------------------------------------- + +/// Recognised window strings → days. Mirrors the CLI arg values. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RoiWindow { + Days(u32), + All, +} + +impl RoiWindow { + /// Parse "7d", "30d", or "all". + pub fn parse(s: &str) -> Result { + match s.trim().to_lowercase().as_str() { + "all" => Ok(Self::All), + other => { + let digits = other.trim_end_matches('d'); + digits + .parse::() + .map(Self::Days) + .map_err(|_| format!("invalid window '{s}'; expected '7d', '30d', or 'all'")) + } + } + } + + pub fn days(self) -> Option { + match self { + Self::Days(d) => Some(d), + Self::All => None, + } + } +} + +impl Default for RoiWindow { + fn default() -> Self { + Self::Days(30) + } +} + +// --------------------------------------------------------------------------- +// DB-backed report +// --------------------------------------------------------------------------- + +/// Compute a full ROI report from the project brain. +/// +/// `window` controls how far back to look. `price_per_mtok_override` +/// comes from `[model] price_per_mtok` in `project.toml`; `model_name` is +/// `[model] model`. +pub fn roi_report( + conn: &rusqlite::Connection, + window: RoiWindow, + model_name: &str, + price_per_mtok_override: Option, +) -> KimetsuResult { + // Compute window boundary timestamp (ISO-8601 string). + let window_since: Option = match window { + RoiWindow::All => None, + RoiWindow::Days(days) => { + // Compute `now − days` as an ISO string using the `time` crate + // that is already a workspace dep. + let secs = days as i64 * 86_400; + let now = time::OffsetDateTime::now_utc(); + let cutoff = now - time::Duration::seconds(secs); + let fmt = time::format_description::well_known::Rfc3339; + Some(cutoff.format(&fmt).unwrap_or_default()) + } + }; + + // --- served_events (context.served) --- + let served_events: u64 = match &window_since { + Some(ts) => conn.query_row( + "SELECT COUNT(*) FROM events WHERE kind = 'context.served' AND ts >= ?1", + params![ts], + |r| r.get(0), + )?, + None => conn.query_row( + "SELECT COUNT(*) FROM events WHERE kind = 'context.served'", + [], + |r| r.get(0), + )?, + }; + + // --- injected_tokens (sum of used_tokens across context.injected events) --- + let injected_tokens: u64 = { + let payloads: Vec = match &window_since { + Some(ts) => { + let mut stmt = conn.prepare( + "SELECT payload_json FROM events WHERE kind = 'context.injected' AND ts >= ?1", + )?; + let rows = stmt.query_map(params![ts], |r| r.get::<_, String>(0))?; + rows.collect::, _>>()? + } + None => { + let mut stmt = conn + .prepare("SELECT payload_json FROM events WHERE kind = 'context.injected'")?; + let rows = stmt.query_map([], |r| r.get::<_, String>(0))?; + rows.collect::, _>>()? + } + }; + let mut sum: u64 = 0; + for p in &payloads { + let v: serde_json::Value = serde_json::from_str(p)?; + if let Some(t) = v.get("used_tokens").and_then(|x| x.as_u64()) { + sum += t; + } + } + sum + }; + + // --- citations per kind --- + // We join memory_citations → memories to get the kind for each citation. + // Window applied via run_id → runs.started_at for pipeline runs, or + // via the event ts for hook-originated citations. + // + // Strategy: collect citation rows that belong to runs in the window + // (started_at >= window_since) OR, for the sentinel hook run_id + // (all-zeroes), filter by the cited_at timestamp. + let citations_by_kind: Vec<(MemoryKind, u32)> = { + // Collect all (memory_id, count) pairs from citations in window. + struct Row { + memory_id: String, + count: u32, + } + + let rows: Vec = match &window_since { + Some(ts) => { + let mut stmt = conn.prepare( + "SELECT mc.memory_id, COUNT(*) \ + FROM memory_citations mc \ + LEFT JOIN runs r ON mc.run_id = r.run_id \ + WHERE r.started_at >= ?1 \ + OR (r.run_id IS NULL AND mc.cited_at >= ?1) \ + GROUP BY mc.memory_id", + )?; + let rows = stmt.query_map(params![ts], |r| { + Ok(Row { + memory_id: r.get(0)?, + count: r.get(1)?, + }) + })?; + rows.collect::, _>>()? + } + None => { + let mut stmt = conn.prepare( + "SELECT memory_id, COUNT(*) FROM memory_citations GROUP BY memory_id", + )?; + let rows = stmt.query_map([], |r| { + Ok(Row { + memory_id: r.get(0)?, + count: r.get(1)?, + }) + })?; + rows.collect::, _>>()? + } + }; + + // For each memory_id, resolve its kind from the memories table. + // Unknown / invalidated memories default to Fact (conservative). + let mut by_kind: std::collections::HashMap = + std::collections::HashMap::new(); + for row in &rows { + let kind: Option = conn + .query_row( + "SELECT kind FROM memories WHERE memory_id = ?1", + params![row.memory_id], + |r| r.get(0), + ) + .optional()?; + let mk = kind + .as_deref() + .and_then(|s| s.parse::().ok()) + .unwrap_or(MemoryKind::Fact); + *by_kind.entry(mk).or_insert(0) += row.count; + } + by_kind.into_iter().collect() + }; + + let total_citations: u64 = citations_by_kind.iter().map(|(_, c)| *c as u64).sum(); + let estimated_saved_tokens = estimate_savings(&citations_by_kind); + let net_tokens = estimated_saved_tokens as i64 - injected_tokens as i64; + + // --- USD --- + let price = resolve_price_per_mtok(model_name, price_per_mtok_override); + let usd = price.map(|p_per_mtok| { + let saved_usd = estimated_saved_tokens as f64 / 1_000_000.0 * p_per_mtok; + let spent_usd = injected_tokens as f64 / 1_000_000.0 * p_per_mtok; + RoiUsd { + saved: saved_usd, + spent: spent_usd, + net: saved_usd - spent_usd, + } + }); + + Ok(RoiReport { + window_days: window.days(), + injected_tokens, + served_events, + citations: total_citations, + estimated_saved_tokens, + net_tokens, + usd, + }) +} + +// --------------------------------------------------------------------------- +// Per-session mini-report (for the Stop hook) +// --------------------------------------------------------------------------- + +/// Compute a per-session ROI mini-report for the Stop hook. +/// +/// Attribution strategy (conservative): +/// 1. `context.served` events with matching `session_id` in payload → used +/// for `served_events` and `injected_tokens`. +/// 2. Citations: we cannot directly attribute `memory_citations` rows to a +/// session_id (citations are keyed by run_id, not session_id). Instead +/// we fall back to a time-window bounded by the earliest and latest +/// `context.served` event timestamps for this session. If session_id +/// is absent (old hook payload), the time window covers the last 24 hours +/// as a rough proxy. +/// 3. ZERO citations → returns `None` (silence; no savings line emitted). +/// +/// All errors are swallowed and `None` returned — the hook must never fail. +pub fn session_roi( + conn: &rusqlite::Connection, + session_id: Option<&str>, + model_name: &str, + price_per_mtok_override: Option, +) -> Option { + session_roi_inner(conn, session_id, model_name, price_per_mtok_override).unwrap_or(None) +} + +fn session_roi_inner( + conn: &rusqlite::Connection, + session_id: Option<&str>, + model_name: &str, + price_per_mtok_override: Option, +) -> KimetsuResult> { + // 1. Find context.served events for this session. + let (served_events, injected_tokens, earliest_ts, latest_ts) = + session_served_stats(conn, session_id)?; + + // 2. Determine citation time window. + let (ts_lo, ts_hi) = match (earliest_ts.as_deref(), latest_ts.as_deref()) { + (Some(lo), Some(hi)) => (lo.to_string(), hi.to_string()), + _ => { + // Fall back to last 24h. + let now = time::OffsetDateTime::now_utc(); + let fmt = time::format_description::well_known::Rfc3339; + let lo = (now - time::Duration::seconds(86_400)) + .format(&fmt) + .unwrap_or_default(); + let hi = now.format(&fmt).unwrap_or_default(); + (lo, hi) + } + }; + + // 3. Collect citations in the time window. + let citations_by_kind = citations_in_window(conn, &ts_lo, &ts_hi)?; + let total_citations: u64 = citations_by_kind.iter().map(|(_, c)| *c as u64).sum(); + + // Silence when nothing was cited this session. + if total_citations == 0 { + return Ok(None); + } + + let estimated_saved_tokens = estimate_savings(&citations_by_kind); + let net_tokens = estimated_saved_tokens as i64 - injected_tokens as i64; + + let price = resolve_price_per_mtok(model_name, price_per_mtok_override); + let usd = price.map(|p_per_mtok| { + let saved_usd = estimated_saved_tokens as f64 / 1_000_000.0 * p_per_mtok; + let spent_usd = injected_tokens as f64 / 1_000_000.0 * p_per_mtok; + RoiUsd { + saved: saved_usd, + spent: spent_usd, + net: saved_usd - spent_usd, + } + }); + + Ok(Some(SessionRoi { + served_events, + injected_tokens, + citations: total_citations, + estimated_saved_tokens, + net_tokens, + usd, + })) +} + +/// Lightweight per-session ROI summary used by the Stop hook. +#[derive(Debug, Clone)] +pub struct SessionRoi { + pub served_events: u64, + pub injected_tokens: u64, + pub citations: u64, + pub estimated_saved_tokens: u64, + pub net_tokens: i64, + pub usd: Option, +} + +impl SessionRoi { + /// Build a one-line savings sentence for the Stop hook `systemMessage`. + /// Returns a human-readable string like: + /// "[Kimetsu] Brain saved ~1 200 tokens (~$0.004) this session." + pub fn savings_sentence(&self) -> String { + match &self.usd { + Some(u) if u.net >= 0.0 => format!( + "[Kimetsu] Brain saved ~{} tokens (~${:.4}) this session.", + format_tokens(self.estimated_saved_tokens), + u.saved, + ), + Some(u) => format!( + "[Kimetsu] Brain used ~{} tokens (net −${:.4}) this session.", + format_tokens(self.injected_tokens), + u.spent - u.saved, + ), + None => format!( + "[Kimetsu] Brain saved ~{} tokens this session.", + format_tokens(self.estimated_saved_tokens), + ), + } + } +} + +fn format_tokens(n: u64) -> String { + // Human-friendly: thousands separator via simple manual formatting. + if n < 1_000 { + return n.to_string(); + } + let s = n.to_string(); + let mut out = String::new(); + let rem = s.len() % 3; + for (i, ch) in s.chars().enumerate() { + if i > 0 && (i % 3 == rem) { + out.push(' '); + } + out.push(ch); + } + out +} + +// --------------------------------------------------------------------------- +// Internal helpers +// --------------------------------------------------------------------------- + +/// Returns (served_events, injected_tokens, earliest_ts, latest_ts) for a +/// given session_id. When session_id is None the query returns all events. +fn session_served_stats( + conn: &rusqlite::Connection, + session_id: Option<&str>, +) -> KimetsuResult<(u64, u64, Option, Option)> { + // context.served events for this session. + let served_payloads: Vec = match session_id { + Some(sid) => { + let mut stmt = conn.prepare( + "SELECT payload_json FROM events \ + WHERE kind = 'context.served' \ + AND json_extract(payload_json, '$.session_id') = ?1", + )?; + let rows = stmt.query_map(params![sid], |r| r.get::<_, String>(0))?; + rows.collect::, _>>()? + } + None => { + // No session_id available — return empty; caller falls back to 24h window. + return Ok((0, 0, None, None)); + } + }; + + // Also collect context.injected in the same session window by time. + // Derive earliest/latest ts from the served events first. + let mut earliest: Option = None; + let mut latest: Option = None; + let served_count = served_payloads.len() as u64; + + // Parse timestamps from the events table for the served events. + if let Some(sid) = session_id { + if served_count > 0 { + let ts_row: (Option, Option) = conn.query_row( + "SELECT MIN(ts), MAX(ts) FROM events \ + WHERE kind = 'context.served' \ + AND json_extract(payload_json, '$.session_id') = ?1", + params![sid], + |r| Ok((r.get(0)?, r.get(1)?)), + )?; + earliest = ts_row.0; + latest = ts_row.1; + } + } + + // Injected tokens: sum from context.injected events in [earliest, latest]. + let injected_tokens: u64 = match (earliest.as_deref(), latest.as_deref()) { + (Some(lo), Some(hi)) => { + let payloads: Vec = { + let mut stmt = conn.prepare( + "SELECT payload_json FROM events \ + WHERE kind = 'context.injected' AND ts >= ?1 AND ts <= ?2", + )?; + let rows = stmt.query_map(params![lo, hi], |r| r.get::<_, String>(0))?; + rows.collect::, _>>()? + }; + let mut sum: u64 = 0; + for p in &payloads { + let v: serde_json::Value = serde_json::from_str(p)?; + if let Some(t) = v.get("used_tokens").and_then(|x| x.as_u64()) { + sum += t; + } + } + sum + } + _ => 0, + }; + + Ok((served_count, injected_tokens, earliest, latest)) +} + +/// Collect (MemoryKind, count) citation pairs for citations whose `cited_at` +/// falls in `[ts_lo, ts_hi]`. +fn citations_in_window( + conn: &rusqlite::Connection, + ts_lo: &str, + ts_hi: &str, +) -> KimetsuResult> { + struct Row { + memory_id: String, + count: u32, + } + let mut stmt = conn.prepare( + "SELECT memory_id, COUNT(*) FROM memory_citations \ + WHERE cited_at >= ?1 AND cited_at <= ?2 \ + GROUP BY memory_id", + )?; + let rows = stmt.query_map(params![ts_lo, ts_hi], |r| { + Ok(Row { + memory_id: r.get(0)?, + count: r.get(1)?, + }) + })?; + let rows: Vec = rows.collect::, _>>()?; + + let mut by_kind: std::collections::HashMap = std::collections::HashMap::new(); + for row in &rows { + let kind: Option = conn + .query_row( + "SELECT kind FROM memories WHERE memory_id = ?1", + params![row.memory_id], + |r| r.get(0), + ) + .optional()?; + let mk = kind + .as_deref() + .and_then(|s| s.parse::().ok()) + .unwrap_or(MemoryKind::Fact); + *by_kind.entry(mk).or_insert(0) += row.count; + } + Ok(by_kind.into_iter().collect()) +} + +// --------------------------------------------------------------------------- +// Tests +// --------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use kimetsu_core::memory::MemoryKind; + + // --- Pure function tests --- + + #[test] + fn estimate_savings_zero_when_empty() { + assert_eq!(estimate_savings(&[]), 0); + } + + #[test] + fn estimate_savings_single_kind() { + // 2 failure_pattern citations × 1500 = 3000 + assert_eq!(estimate_savings(&[(MemoryKind::FailurePattern, 2)]), 3_000); + } + + #[test] + fn estimate_savings_multi_kind() { + let citations = vec![ + (MemoryKind::FailurePattern, 1), // 1500 + (MemoryKind::Command, 2), // 800 + (MemoryKind::Convention, 1), // 300 + (MemoryKind::Fact, 1), // 500 + (MemoryKind::Preference, 3), // 600 + ]; + assert_eq!(estimate_savings(&citations), 1500 + 800 + 300 + 500 + 600); + } + + #[test] + fn estimate_savings_all_kinds_covered() { + // Every kind must appear in SAVED_TOKENS_PER_CITATION. + for kind in [ + MemoryKind::FailurePattern, + MemoryKind::Command, + MemoryKind::Convention, + MemoryKind::Fact, + MemoryKind::Preference, + ] { + let v = SAVED_TOKENS_PER_CITATION + .iter() + .find(|(k, _)| k == &kind) + .map(|(_, v)| *v); + assert!( + v.is_some(), + "kind {:?} missing from SAVED_TOKENS_PER_CITATION", + kind + ); + assert!(v.unwrap() > 0, "kind {:?} has zero constant", kind); + } + } + + #[test] + fn resolve_price_override_wins() { + assert_eq!( + resolve_price_per_mtok("claude-sonnet-4-7", Some(5.0)), + Some(5.0) + ); + } + + #[test] + fn resolve_price_known_model() { + let p = resolve_price_per_mtok("claude-sonnet-4-7", None); + assert!(p.is_some(), "claude-sonnet-4 should match"); + assert!((p.unwrap() - 3.0).abs() < 1e-9); + } + + #[test] + fn resolve_price_unknown_model_none() { + assert!(resolve_price_per_mtok("my-custom-llm-v9", None).is_none()); + } + + #[test] + fn resolve_price_longest_prefix_wins() { + // "claude-opus-4" and "claude-opus-4" — make sure haiku doesn't + // match opus prefix. + let opus_p = resolve_price_per_mtok("claude-opus-4-5", None).unwrap_or(0.0); + let haiku_p = resolve_price_per_mtok("claude-haiku-4-5", None).unwrap_or(0.0); + assert!(opus_p > haiku_p, "opus should be more expensive than haiku"); + } + + #[test] + fn roi_window_parse() { + assert_eq!(RoiWindow::parse("7d").unwrap(), RoiWindow::Days(7)); + assert_eq!(RoiWindow::parse("30d").unwrap(), RoiWindow::Days(30)); + assert_eq!(RoiWindow::parse("all").unwrap(), RoiWindow::All); + assert_eq!(RoiWindow::parse("ALL").unwrap(), RoiWindow::All); + assert!(RoiWindow::parse("bad").is_err()); + } + + #[test] + fn format_tokens_below_1000() { + assert_eq!(format_tokens(42), "42"); + assert_eq!(format_tokens(999), "999"); + } + + #[test] + fn format_tokens_thousands() { + assert_eq!(format_tokens(1_000), "1 000"); + assert_eq!(format_tokens(12_345), "12 345"); + assert_eq!(format_tokens(1_234_567), "1 234 567"); + } + + // --- DB-backed tests --- + + use crate::{ + project::{init_project, load_project}, + projector, + user_brain::with_user_brain_disabled, + }; + use kimetsu_core::{event::Event, ids::RunId, memory::MemoryScope}; + use ulid::Ulid; + + fn test_root() -> std::path::PathBuf { + let root = std::env::temp_dir().join(format!("kimetsu-roi-test-{}", Ulid::new())); + kimetsu_core::paths::git_init_boundary(&root); + root + } + + fn seed_memory(root: &std::path::Path, kind: MemoryKind, text: &str) -> String { + crate::project::add_memory(root, MemoryScope::Project, kind, text).expect("add_memory") + } + + fn seed_injected_event(conn: &rusqlite::Connection, run_id: RunId, used_tokens: u64) { + let ev = Event::new( + run_id, + "context.injected", + serde_json::json!({ + "stage": "localization", + "memory_ids": [], + "used_tokens": used_tokens, + "capsule_count": 1, + }), + ); + projector::apply_events(conn, &[ev]).expect("seed injected"); + } + + fn seed_citation(conn: &rusqlite::Connection, run_id: RunId, memory_id: &str, turn: i64) { + let ev = Event::new( + run_id, + "memory.cited", + serde_json::json!({ + "memory_id": memory_id, + "turn": turn, + }), + ); + projector::apply_events(conn, &[ev]).expect("seed citation"); + } + + #[test] + fn roi_report_empty_db_returns_zeros() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + let (_paths, config, conn) = load_project(&root).expect("load"); + let report = + roi_report(&conn, RoiWindow::All, &config.model.model, None).expect("roi_report"); + assert_eq!(report.injected_tokens, 0); + assert_eq!(report.citations, 0); + assert_eq!(report.estimated_saved_tokens, 0); + assert_eq!(report.net_tokens, 0); + }); + } + + #[test] + fn roi_report_with_citations_computes_savings() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + let m1 = seed_memory(&root, MemoryKind::FailurePattern, "fp1"); + let (_paths, config, conn) = load_project(&root).expect("load"); + let run_id = RunId::new(); + seed_injected_event(&conn, run_id, 300); + seed_citation(&conn, run_id, &m1, 1); + + let report = + roi_report(&conn, RoiWindow::All, &config.model.model, None).expect("roi_report"); + // 1 failure_pattern citation = 1500 saved tokens. + assert_eq!(report.estimated_saved_tokens, 1500); + assert_eq!(report.injected_tokens, 300); + assert_eq!(report.net_tokens, 1500 - 300); + assert_eq!(report.citations, 1); + }); + } + + #[test] + fn roi_report_negative_net_when_overhead_exceeds_savings() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + let m1 = seed_memory(&root, MemoryKind::Preference, "pref1"); + let (_paths, config, conn) = load_project(&root).expect("load"); + let run_id = RunId::new(); + // Inject 500 tokens but cite 1 preference (200 saved) → net = -300. + seed_injected_event(&conn, run_id, 500); + seed_citation(&conn, run_id, &m1, 1); + + let report = + roi_report(&conn, RoiWindow::All, &config.model.model, None).expect("roi_report"); + assert_eq!(report.estimated_saved_tokens, 200); + assert_eq!(report.net_tokens, 200 - 500); // −300 + }); + } + + #[test] + fn roi_report_usd_with_known_model() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + let m1 = seed_memory(&root, MemoryKind::Command, "cmd1"); + let (_paths, _config, conn) = load_project(&root).expect("load"); + let run_id = RunId::new(); + seed_injected_event(&conn, run_id, 200); + seed_citation(&conn, run_id, &m1, 1); + + // Use a known model directly. + let report = + roi_report(&conn, RoiWindow::All, "claude-sonnet-4-7", None).expect("roi_report"); + let usd = report.usd.expect("usd must be Some for known model"); + // 400 saved tokens @ $3/MTok = $0.0012 + assert!((usd.saved - 400.0 / 1_000_000.0 * 3.0).abs() < 1e-9); + // 200 injected tokens @ $3/MTok + assert!((usd.spent - 200.0 / 1_000_000.0 * 3.0).abs() < 1e-9); + assert!((usd.net - (usd.saved - usd.spent)).abs() < 1e-12); + }); + } + + #[test] + fn roi_report_usd_with_override() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + let m1 = seed_memory(&root, MemoryKind::Fact, "fact1"); + let (_paths, _config, conn) = load_project(&root).expect("load"); + let run_id = RunId::new(); + seed_injected_event(&conn, run_id, 0); + seed_citation(&conn, run_id, &m1, 1); + + let report = + roi_report(&conn, RoiWindow::All, "my-custom-llm", Some(10.0)).expect("roi_report"); + // 500 saved tokens @ $10/MTok = $0.005 + let usd = report.usd.expect("usd with override"); + assert!((usd.saved - 500.0 / 1_000_000.0 * 10.0).abs() < 1e-9); + }); + } + + #[test] + fn roi_report_unknown_model_no_usd() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + let (_paths, _config, conn) = load_project(&root).expect("load"); + + let report = roi_report(&conn, RoiWindow::All, "totally-unknown-llm-xyz", None) + .expect("roi_report"); + assert!(report.usd.is_none(), "usd must be None for unknown model"); + }); + } + + #[test] + fn session_roi_returns_none_when_no_citations() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + let (_paths, _config, conn) = load_project(&root).expect("load"); + + let result = session_roi(&conn, Some("sess-abc"), "claude-sonnet-4", None); + assert!(result.is_none(), "no citations → no session roi"); + }); + } + + #[test] + fn savings_sentence_positive_no_usd() { + let sr = SessionRoi { + served_events: 3, + injected_tokens: 100, + citations: 2, + estimated_saved_tokens: 1200, + net_tokens: 1100, + usd: None, + }; + let s = sr.savings_sentence(); + assert!(s.contains("1 200"), "expected formatted token count"); + assert!(s.contains("[Kimetsu]"), "must have brand prefix"); + } + + #[test] + fn savings_sentence_positive_with_usd() { + let sr = SessionRoi { + served_events: 3, + injected_tokens: 100, + citations: 2, + estimated_saved_tokens: 1500, + net_tokens: 1400, + usd: Some(RoiUsd { + saved: 0.0045, + spent: 0.0003, + net: 0.0042, + }), + }; + let s = sr.savings_sentence(); + assert!(s.contains("$"), "must include dollar sign when usd present"); + } +} diff --git a/crates/kimetsu-brain/src/schema.rs b/crates/kimetsu-brain/src/schema.rs index b8cc5e7..7e428dc 100644 --- a/crates/kimetsu-brain/src/schema.rs +++ b/crates/kimetsu-brain/src/schema.rs @@ -52,6 +52,14 @@ pub fn initialize(conn: &Connection) -> KimetsuResult<()> { Ok(()) } +/// Test seam: exposes `create_baseline` so integration tests in sibling +/// modules can build a v1 DB without going through the full `initialize` +/// (which would run all migrations and advance to the current version). +#[cfg(test)] +pub fn create_baseline_for_test(conn: &Connection) -> KimetsuResult<()> { + create_baseline(conn) +} + /// Create the baseline v1 schema (pragmas + all tables/indexes/FTS as of the /// original v1 shape). Seeds `schema_info` with version **1** so the migration /// runner knows where to start. On an existing DB every CREATE is a no-op @@ -320,6 +328,26 @@ pub(crate) fn migrate_v1_to_v2(conn: &Connection) -> KimetsuResult<()> { Ok(()) } +/// The v2→v3 migration: add `superseded_by` column to `memories`. +/// +/// Superseded rows point at their survivor via this column. Retrieval, +/// listing, and export already filter `invalidated_at IS NULL`; this +/// migration adds a companion `AND superseded_by IS NULL` guard to all +/// such queries (applied in `context.rs`, `user_brain.rs`, and +/// `project.rs`). The FTS index and ANN index keep the same "remove on +/// supersede" semantics as invalidation. +/// +/// NOTE: this function runs INSIDE a transaction owned by the migration +/// runner. Do NOT issue BEGIN/COMMIT here. +pub(crate) fn migrate_v2_to_v3(conn: &Connection) -> KimetsuResult<()> { + add_column_if_missing(conn, "memories", "superseded_by TEXT")?; + conn.execute_batch( + "CREATE INDEX IF NOT EXISTS idx_memories_superseded + ON memories (superseded_by);", + )?; + Ok(()) +} + pub fn validate(conn: &Connection) -> KimetsuResult<()> { // Apply performance pragmas on read-only connections too. The helper // skips pragmas that error (journal_mode/mmap_size on some read-only @@ -448,18 +476,18 @@ mod tests { } // ------------------------------------------------------------------ - // 1. Fresh init reaches v2 with full shape + // 1. Fresh init reaches v3 with full shape // ------------------------------------------------------------------ #[test] - fn fresh_init_reaches_v2_with_full_shape() { + fn fresh_init_reaches_v3_with_full_shape() { let conn = Connection::open_in_memory().expect("open_in_memory"); initialize(&conn).expect("initialize"); - // Version must be 2. + // Version must be 3. assert_eq!( migrate::current_version(&conn).expect("current_version"), - 2, - "fresh DB must be at schema version 2 after initialize" + 3, + "fresh DB must be at schema version 3 after initialize" ); // Post-migration columns exist on `memories`. @@ -476,6 +504,11 @@ mod tests { mem_cols.contains(&"last_useful_at".to_string()), "memories must have `last_useful_at` column" ); + // v3: superseded_by column + assert!( + mem_cols.contains(&"superseded_by".to_string()), + "memories must have `superseded_by` column after v3 migration" + ); // Tables added by the migration exist. assert!( @@ -519,8 +552,8 @@ mod tests { ); assert_eq!( migrate::current_version(&conn).expect("current_version"), - 2, - "version must still be 2" + 3, + "version must still be 3" ); // Data must be intact. @@ -535,7 +568,7 @@ mod tests { } // ------------------------------------------------------------------ - // 3. Idempotent initialize: calling initialize twice succeeds, version stays 2 + // 3. Idempotent initialize: calling initialize twice succeeds, version stays 3 // ------------------------------------------------------------------ #[test] fn idempotent_initialize_twice() { @@ -544,8 +577,8 @@ mod tests { initialize(&conn).expect("second initialize must not error"); assert_eq!( migrate::current_version(&conn).expect("current_version"), - 2, - "version must still be 2 after double initialize" + 3, + "version must still be 3 after double initialize" ); } @@ -633,6 +666,52 @@ mod tests { ); } + // ------------------------------------------------------------------ + // A5-v3. v2→v3 migration adds superseded_by + index + // ------------------------------------------------------------------ + #[test] + fn v2_to_v3_migration_adds_superseded_by() { + let conn = Connection::open_in_memory().expect("open_in_memory"); + // Seed a v2 DB manually (baseline + v1→v2 migration, no v2→v3). + create_baseline(&conn).expect("create_baseline"); + migrate_v1_to_v2(&conn).expect("migrate_v1_to_v2"); + conn.execute( + "UPDATE schema_info SET value = 2 WHERE key = 'kimetsu_schema_version'", + [], + ) + .expect("set v2"); + + // superseded_by must NOT exist yet. + let cols_before = column_names(&conn, "memories"); + assert!( + !cols_before.contains(&"superseded_by".to_string()), + "superseded_by must not exist before v3 migration" + ); + + // Run v2→v3. + migrate_v2_to_v3(&conn).expect("migrate_v2_to_v3"); + + // Now it must exist. + let cols_after = column_names(&conn, "memories"); + assert!( + cols_after.contains(&"superseded_by".to_string()), + "superseded_by must exist after v3 migration" + ); + + // Index must also exist. + let idx_count: i64 = conn + .query_row( + "SELECT COUNT(*) FROM sqlite_master WHERE type='index' AND name='idx_memories_superseded'", + [], + |r| r.get(0), + ) + .expect("query index"); + assert_eq!( + idx_count, 1, + "idx_memories_superseded must exist after v3 migration" + ); + } + // ------------------------------------------------------------------ // A5-3. validate hard-errors (non-SchemaNeedsMigration) for a newer DB // ------------------------------------------------------------------ diff --git a/crates/kimetsu-brain/src/tune.rs b/crates/kimetsu-brain/src/tune.rs new file mode 100644 index 0000000..30c8702 --- /dev/null +++ b/crates/kimetsu-brain/src/tune.rs @@ -0,0 +1,265 @@ +//! v1.5: Self-Tuning Brain sweep engine. +//! +//! Pure functions for objective scoring, holdout splitting, and history I/O. +//! The sweep itself is driven from the CLI (kimetsu-cli) which calls +//! `evaluate_combo` in-process with injected embedder + optional reranker. +//! +//! Sweep space (config-addressable only): +//! - min_lexical_coverage ∈ {0.3, 0.4, 0.5, 0.6} +//! - min_semantic_score ∈ {-1.0(auto), 0.0, 0.25, 0.35, 0.45} +//! - reranker id ∈ {off, ms-marco-tinybert-l-2-v2, jina-reranker-v1-tiny-en, +//! ms-marco-minilm-l-4-v2} +//! +//! NOT swept (compile-time or complex-deploy): +//! - RERANK_POOL (compile-time const in the daemon) — deferred. +//! +//! Objective: mean_MRR - cost_weight * mean_injected_tokens + +use serde::{Deserialize, Serialize}; + +// ─── Sweep parameter space ──────────────────────────────────────────────────── + +pub const LEXICAL_FLOORS: &[f32] = &[0.3, 0.4, 0.5, 0.6]; +pub const SEMANTIC_FLOORS: &[f32] = &[-1.0, 0.0, 0.25, 0.35, 0.45]; +pub const RERANKER_IDS: &[&str] = &[ + "off", + "ms-marco-tinybert-l-2-v2", + "jina-reranker-v1-tiny-en", + "ms-marco-minilm-l-4-v2", +]; + +// ─── Combo ──────────────────────────────────────────────────────────────────── + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct TuneCombo { + pub min_lexical_coverage: f32, + pub min_semantic_score: f32, + pub reranker_id: String, +} + +impl TuneCombo { + pub fn all_combos() -> Vec { + let mut out = Vec::new(); + for &lex in LEXICAL_FLOORS { + for &sem in SEMANTIC_FLOORS { + for &rr in RERANKER_IDS { + out.push(TuneCombo { + min_lexical_coverage: lex, + min_semantic_score: sem, + reranker_id: rr.to_string(), + }); + } + } + } + out + } +} + +// ─── Per-combo result ───────────────────────────────────────────────────────── + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct ComboResult { + pub combo: TuneCombo, + pub mean_mrr: f64, + pub mean_tokens: f64, + /// mean_mrr − cost_weight * mean_tokens + pub objective: f64, +} + +// ─── Tune history entry ─────────────────────────────────────────────────────── + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct TuneHistoryEntry { + pub timestamp: String, + pub before: TuneCombo, + pub after: TuneCombo, + pub train_objective: f64, + pub holdout_objective: f64, + pub holdout_mrr: f64, + pub baseline_holdout_objective: f64, +} + +// ─── Pure functions ─────────────────────────────────────────────────────────── + +/// Compute the tuning objective for a combo result. +/// +/// `objective = mean_mrr - cost_weight * mean_tokens` +pub fn compute_objective(mean_mrr: f64, mean_tokens: f64, cost_weight: f64) -> f64 { + mean_mrr - cost_weight * mean_tokens +} + +/// Split cases into (train, holdout) using a deterministic seed derived from +/// `case_count`. 80 % train, 20 % holdout. Indices into `cases` are returned. +/// +/// The split is stable: the same set of N cases always produces the same +/// train/holdout partition regardless of case order. +pub fn train_holdout_split(case_count: usize) -> (Vec, Vec) { + if case_count == 0 { + return (Vec::new(), Vec::new()); + } + let holdout_size = (case_count / 5).max(1); // ≥1 holdout + // Deterministic: pick every 5th index as holdout. + let holdout: Vec = (0..case_count).filter(|i| i % 5 == 0).collect(); + let train: Vec = (0..case_count).filter(|i| i % 5 != 0).collect(); + let _ = holdout_size; // used via filter logic above + (train, holdout) +} + +/// Select the best combo from a slice of `ComboResult` by objective score. +/// Returns `None` when the slice is empty. +pub fn select_winner(results: &[ComboResult]) -> Option<&ComboResult> { + results.iter().max_by(|a, b| { + a.objective + .partial_cmp(&b.objective) + .unwrap_or(std::cmp::Ordering::Equal) + }) +} + +// ─── Tune history I/O ───────────────────────────────────────────────────────── + +/// Append a `TuneHistoryEntry` to `.kimetsu/tune-history.json`. +pub fn append_tune_history( + kimetsu_dir: &std::path::Path, + entry: TuneHistoryEntry, +) -> kimetsu_core::KimetsuResult<()> { + let path = kimetsu_dir.join("tune-history.json"); + let mut entries: Vec = if path.exists() { + let text = std::fs::read_to_string(&path)?; + serde_json::from_str(&text).unwrap_or_default() + } else { + Vec::new() + }; + entries.push(entry); + let json = serde_json::to_string_pretty(&entries)?; + std::fs::write(&path, json)?; + Ok(()) +} + +/// Read the latest entry from `.kimetsu/tune-history.json`, if any. +pub fn latest_tune_history( + kimetsu_dir: &std::path::Path, +) -> kimetsu_core::KimetsuResult> { + let path = kimetsu_dir.join("tune-history.json"); + if !path.exists() { + return Ok(None); + } + let text = std::fs::read_to_string(&path)?; + let entries: Vec = serde_json::from_str(&text).unwrap_or_default(); + Ok(entries.into_iter().last()) +} + +// ─── Unit tests ─────────────────────────────────────────────────────────────── + +#[cfg(test)] +mod tests { + use super::*; + use ulid::Ulid; + + #[test] + fn all_combos_count_is_80() { + let combos = TuneCombo::all_combos(); + assert_eq!( + combos.len(), + 4 * 5 * 4, + "expected 4×5×4=80 combos, got {}", + combos.len() + ); + } + + #[test] + fn compute_objective_formula() { + let obj = compute_objective(0.75, 1000.0, 0.005); + // 0.75 - 0.005 * 1000 = 0.75 - 5.0 = -4.25 + assert!((obj - (-4.25)).abs() < 1e-9, "objective: {obj}"); + } + + #[test] + fn compute_objective_zero_cost_weight_is_just_mrr() { + let obj = compute_objective(0.85, 500.0, 0.0); + assert!((obj - 0.85).abs() < 1e-9, "objective with 0 cost: {obj}"); + } + + #[test] + fn train_holdout_split_80_20() { + let (train, holdout) = train_holdout_split(10); + // Indices 0..10, every 5th (0,5) → holdout, rest → train. + assert_eq!(holdout, vec![0, 5]); + assert_eq!(train, vec![1, 2, 3, 4, 6, 7, 8, 9]); + assert_eq!(train.len() + holdout.len(), 10); + } + + #[test] + fn train_holdout_split_empty() { + let (train, holdout) = train_holdout_split(0); + assert!(train.is_empty()); + assert!(holdout.is_empty()); + } + + #[test] + fn select_winner_picks_highest_objective() { + let combos = vec![ + ComboResult { + combo: TuneCombo { + min_lexical_coverage: 0.3, + min_semantic_score: 0.0, + reranker_id: "off".to_string(), + }, + mean_mrr: 0.7, + mean_tokens: 100.0, + objective: 0.2, + }, + ComboResult { + combo: TuneCombo { + min_lexical_coverage: 0.4, + min_semantic_score: 0.25, + reranker_id: "off".to_string(), + }, + mean_mrr: 0.9, + mean_tokens: 80.0, + objective: 0.5, + }, + ]; + let winner = select_winner(&combos).expect("winner"); + assert!((winner.objective - 0.5).abs() < 1e-9); + } + + #[test] + fn tune_history_roundtrip() { + let tmp = std::env::temp_dir().join(format!("kimetsu-tune-hist-{}", Ulid::new())); + std::fs::create_dir_all(&tmp).unwrap(); + + let entry = TuneHistoryEntry { + timestamp: "2026-06-11T00:00:00Z".to_string(), + before: TuneCombo { + min_lexical_coverage: 0.5, + min_semantic_score: -1.0, + reranker_id: "off".to_string(), + }, + after: TuneCombo { + min_lexical_coverage: 0.4, + min_semantic_score: 0.25, + reranker_id: "ms-marco-tinybert-l-2-v2".to_string(), + }, + train_objective: 0.55, + holdout_objective: 0.50, + holdout_mrr: 0.70, + baseline_holdout_objective: 0.45, + }; + + append_tune_history(&tmp, entry.clone()).unwrap(); + let latest = latest_tune_history(&tmp).unwrap().unwrap(); + assert!((latest.holdout_objective - 0.50).abs() < 1e-9); + assert_eq!(latest.after.reranker_id, "ms-marco-tinybert-l-2-v2"); + + std::fs::remove_dir_all(&tmp).ok(); + } + + #[test] + fn tune_history_empty_when_no_file() { + let tmp = std::env::temp_dir().join(format!("kimetsu-tune-empty-{}", Ulid::new())); + std::fs::create_dir_all(&tmp).unwrap(); + let latest = latest_tune_history(&tmp).unwrap(); + assert!(latest.is_none(), "no history file → None"); + std::fs::remove_dir_all(&tmp).ok(); + } +} diff --git a/crates/kimetsu-brain/src/tuneset.rs b/crates/kimetsu-brain/src/tuneset.rs new file mode 100644 index 0000000..6ef2b47 --- /dev/null +++ b/crates/kimetsu-brain/src/tuneset.rs @@ -0,0 +1,338 @@ +//! v1.5: personal eval-set builder for `kimetsu brain tune`. +//! +//! Walks `context.served` events that carry a raw `query` field (present when +//! `store_queries = true` in `project.toml`). Joins to `memory_citations` via +//! `session_id` (exact match) or, when session_id is absent, a ±30-minute +//! time window. A served event becomes a POSITIVE eval case when ≥1 citation +//! occurred in that window. Zero-citation served events are counted as noise +//! (used only for cost statistics; they do NOT appear in the `cases` vec). +//! +//! Deduplication: when the same query text appears multiple times (the same +//! task is worked on across sessions), only the latest served event is kept. + +use rusqlite::Connection; +use time::OffsetDateTime; +use time::format_description::well_known::Rfc3339; + +use crate::eval::EvalCase; +use kimetsu_core::KimetsuResult; + +/// Output of [`build_personal_eval`]. +#[derive(Debug, Clone, Default)] +pub struct PersonalEval { + /// Positive eval cases (query + relevant memory ids). + pub cases: Vec, + /// Number of served events with zero subsequent citations (noise pool). + pub noise_count: usize, + /// RFC-3339 timestamp of the oldest positive served event, if any. + pub oldest: Option, + /// RFC-3339 timestamp of the newest positive served event, if any. + pub newest: Option, +} + +/// Build a personal eval set from the events already in `conn`. +/// +/// Parameters: +/// - `window_secs`: maximum seconds between a served event and a citation for +/// them to be considered linked when no `session_id` is available (default 1800 = 30 min). +pub fn build_personal_eval(conn: &Connection, window_secs: i64) -> KimetsuResult { + // 1. Collect served events that carry a query. + let mut stmt = conn.prepare( + "SELECT payload_json, ts FROM events + WHERE kind = 'context.served' + ORDER BY ts DESC", + )?; + let rows = stmt.query_map([], |row| { + Ok((row.get::<_, String>(0)?, row.get::<_, String>(1)?)) + })?; + + // Dedup by query text: keep latest ts for each unique query. + let mut seen_queries: std::collections::HashMap = + std::collections::HashMap::new(); + for row in rows { + let (payload_json, ts) = row?; + let payload: serde_json::Value = + serde_json::from_str(&payload_json).unwrap_or(serde_json::Value::Null); + let Some(query) = payload.get("query").and_then(|v| v.as_str()) else { + continue; // no raw query stored → skip + }; + if !seen_queries.contains_key(query) { + seen_queries.insert(query.to_string(), (ts, payload)); + } + } + + if seen_queries.is_empty() { + return Ok(PersonalEval::default()); + } + + // 2. For each unique served event, find citations in-window. + let mut cases: Vec = Vec::new(); + let mut noise_count = 0usize; + let mut oldest: Option = None; + let mut newest: Option = None; + + for (query, (ts, payload)) in &seen_queries { + let session_id = payload.get("session_id").and_then(|v| v.as_str()); + + // Find citation memory_ids linked to this served event. + let relevant = citations_for_served(conn, ts, session_id, window_secs)?; + + if relevant.is_empty() { + noise_count += 1; + } else { + // Track oldest/newest timestamps. + if oldest.as_deref().map(|o| ts.as_str() < o).unwrap_or(true) { + oldest = Some(ts.clone()); + } + if newest.as_deref().map(|n| ts.as_str() > n).unwrap_or(true) { + newest = Some(ts.clone()); + } + cases.push(EvalCase { + query: query.clone(), + relevant, + }); + } + } + + Ok(PersonalEval { + cases, + noise_count, + oldest, + newest, + }) +} + +/// Collect distinct memory_ids cited in-window relative to a served event. +/// +/// Strategy: +/// 1. If session_id is present: find all `memory.cited` events whose +/// payload `session_id` matches. (Citation events emitted by the MCP +/// `kimetsu_brain_cite` tool carry `session_id` in their payload.) +/// → NOT currently stored there; fall through to time-window. +/// 2. Time-window fallback: find `memory_citations` rows whose `cited_at` +/// falls within [ts − window_secs, ts + window_secs]. +/// +/// Note on design: `memory_citations` has `cited_at` (RFC-3339 text) and +/// `run_id`. We cannot reliably join on `run_id` for MCP cites (sentinel +/// run_id shared by all). So the primary join key is `session_id` from the +/// event payload when available; otherwise the time window is used. +fn citations_for_served( + conn: &Connection, + served_ts: &str, + session_id: Option<&str>, + window_secs: i64, +) -> KimetsuResult> { + // Try session_id join first (citations from the same Claude Code session). + if let Some(sid) = session_id { + let mut stmt = conn.prepare( + "SELECT DISTINCT mc.memory_id + FROM memory_citations mc + JOIN events e ON e.run_id = mc.run_id + AND e.kind = 'memory.cited' + WHERE json_extract(e.payload_json, '$.session_id') = ?1", + )?; + let ids: Vec = stmt + .query_map([sid], |row| row.get(0))? + .filter_map(|r| r.ok()) + .collect(); + if !ids.is_empty() { + return Ok(ids); + } + // Fall through: session_id join found nothing (MCP cites without session_id + // in payload, or old agent cites) — use time window below. + } + + // Time-window fallback: parse the served ts, compute bounds. + let served_dt = OffsetDateTime::parse(served_ts, &Rfc3339) + .map_err(|e| format!("parse served_ts {served_ts:?}: {e}"))?; + let lo = (served_dt - time::Duration::seconds(window_secs)) + .format(&Rfc3339) + .map_err(|e| format!("format lo: {e}"))?; + let hi = (served_dt + time::Duration::seconds(window_secs)) + .format(&Rfc3339) + .map_err(|e| format!("format hi: {e}"))?; + + let mut stmt = conn.prepare( + "SELECT DISTINCT memory_id FROM memory_citations + WHERE cited_at >= ?1 AND cited_at <= ?2", + )?; + let ids: Vec = stmt + .query_map([&lo, &hi], |row| row.get(0))? + .filter_map(|r| r.ok()) + .collect(); + Ok(ids) +} + +// ─── Unit tests ─────────────────────────────────────────────────────────────── + +#[cfg(test)] +mod tests { + use super::*; + use crate::{ + project::{add_memory, init_project}, + projector, + user_brain::with_user_brain_disabled, + }; + use kimetsu_core::{ + event::Event, + ids::RunId, + memory::{MemoryKind, MemoryScope}, + }; + use ulid::Ulid; + + fn test_root() -> std::path::PathBuf { + let root = std::env::temp_dir().join(format!("kimetsu-tuneset-test-{}", Ulid::new())); + kimetsu_core::paths::git_init_boundary(&root); + root + } + + fn seed_context_served_with_query( + conn: &Connection, + query: &str, + session_id: Option<&str>, + ts_offset_secs: i64, + ) -> String { + // Build a timestamp slightly offset from now for ordering tests. + let now = OffsetDateTime::now_utc() + time::Duration::seconds(ts_offset_secs); + let ts = now.format(&Rfc3339).unwrap(); + + let run_id = RunId::new(); + let mut payload = serde_json::json!({ + "query_hash": format!("{:016x}", query.len()), + "query": query, + "capsule_count": 2, + "top_score": 0.8, + "skipped": false, + "stage": "localization", + "retrieval_path": "fts", + }); + if let Some(sid) = session_id { + payload["session_id"] = serde_json::json!(sid); + } + // Insert directly into events with the given ts. + let event = Event { + event_id: kimetsu_core::ids::EventId(Ulid::new()), + run_id, + ts: now, + parent_event_id: None, + kind: "context.served".to_string(), + schema_version: 1, + payload, + }; + projector::apply_events(conn, &[event]).expect("seed served"); + ts + } + + fn seed_memory_cited(conn: &Connection, memory_id: &str, ts_offset_secs: i64) { + let now = OffsetDateTime::now_utc() + time::Duration::seconds(ts_offset_secs); + let run_id = RunId::new(); + let event = Event { + event_id: kimetsu_core::ids::EventId(Ulid::new()), + run_id, + ts: now, + parent_event_id: None, + kind: "memory.cited".to_string(), + schema_version: 1, + payload: serde_json::json!({ + "memory_id": memory_id, + "turn": 1, + }), + }; + projector::apply_events(conn, &[event]).expect("seed cited"); + } + + #[test] + fn build_personal_eval_empty_when_no_queries_stored() { + with_user_brain_disabled(|| { + let root = test_root(); + std::fs::create_dir_all(&root).expect("create"); + init_project(&root, false).expect("init"); + let (_, _, conn) = crate::project::load_project(&root).expect("load"); + let eval = build_personal_eval(&conn, 1800).expect("build"); + assert!(eval.cases.is_empty(), "no served events → empty cases"); + assert_eq!(eval.noise_count, 0); + std::fs::remove_dir_all(&root).ok(); + }); + } + + #[test] + fn build_personal_eval_positive_case_from_time_window() { + with_user_brain_disabled(|| { + let root = test_root(); + std::fs::create_dir_all(&root).expect("create"); + init_project(&root, false).expect("init"); + + let mid = add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "tuneset window test memory", + ) + .expect("add memory"); + + let (_, _, conn) = crate::project::load_project(&root).expect("load"); + + // Served event at t=0, citation at t=+5min → within 30min window. + seed_context_served_with_query(&conn, "find files fast", None, 0); + seed_memory_cited(&conn, &mid, 5 * 60); + + let eval = build_personal_eval(&conn, 1800).expect("build"); + assert_eq!(eval.cases.len(), 1, "should have 1 positive case"); + assert_eq!(eval.cases[0].query, "find files fast"); + assert!( + eval.cases[0].relevant.contains(&mid), + "memory_id must be in relevant" + ); + assert_eq!(eval.noise_count, 0); + std::fs::remove_dir_all(&root).ok(); + }); + } + + #[test] + fn build_personal_eval_noise_when_no_citation_in_window() { + with_user_brain_disabled(|| { + let root = test_root(); + std::fs::create_dir_all(&root).expect("create"); + init_project(&root, false).expect("init"); + let (_, _, conn) = crate::project::load_project(&root).expect("load"); + + // Served event but citation far outside window (3 hours later). + let mid = add_memory(&root, MemoryScope::Project, MemoryKind::Fact, "noise test") + .expect("add memory"); + seed_context_served_with_query(&conn, "some query", None, 0); + seed_memory_cited(&conn, &mid, 3 * 3600); + + let eval = build_personal_eval(&conn, 1800).expect("build"); + assert_eq!( + eval.cases.len(), + 0, + "out-of-window citation → no positive case" + ); + assert_eq!(eval.noise_count, 1, "should count 1 noise entry"); + std::fs::remove_dir_all(&root).ok(); + }); + } + + #[test] + fn build_personal_eval_deduplicates_same_query() { + with_user_brain_disabled(|| { + let root = test_root(); + std::fs::create_dir_all(&root).expect("create"); + init_project(&root, false).expect("init"); + + let mid = add_memory(&root, MemoryScope::Project, MemoryKind::Fact, "dedup test") + .expect("add"); + let (_, _, conn) = crate::project::load_project(&root).expect("load"); + + // Same query twice (different sessions/times), both with a citation. + seed_context_served_with_query(&conn, "duplicate query", None, -100); + seed_context_served_with_query(&conn, "duplicate query", None, 0); + seed_memory_cited(&conn, &mid, 60); + + let eval = build_personal_eval(&conn, 1800).expect("build"); + // Dedup: both map to same query string → one case (the latest ts kept). + assert_eq!(eval.cases.len(), 1, "dedup must produce exactly 1 case"); + std::fs::remove_dir_all(&root).ok(); + }); + } +} diff --git a/crates/kimetsu-brain/src/user_brain.rs b/crates/kimetsu-brain/src/user_brain.rs index cc500df..0f7bc82 100644 --- a/crates/kimetsu-brain/src/user_brain.rs +++ b/crates/kimetsu-brain/src/user_brain.rs @@ -189,6 +189,7 @@ pub fn add_user_memory( SELECT memory_id FROM memories WHERE scope = ?1 AND kind = ?2 AND normalized_text = ?3 AND invalidated_at IS NULL + AND superseded_by IS NULL LIMIT 1 ", rusqlite::params!["global_user".to_string(), kind.to_string(), &normalized], @@ -282,6 +283,7 @@ pub fn list_user_memories(conn: &Connection) -> KimetsuResult> { SELECT memory_id, scope, kind, text, confidence, use_count, usefulness_score FROM memories WHERE invalidated_at IS NULL + AND superseded_by IS NULL ORDER BY created_at DESC LIMIT 100 ", @@ -554,26 +556,28 @@ mod tests { // → run_migrations → migrates v1→v2 with a backup. let conn = open_user_brain().expect("re-open ok").expect("enabled"); - // Assert 1: version is back at 2. + // Assert 1: version is at 3 (v1→v2→v3 migration chain). let ver = crate::migrate::current_version(&conn).expect("current_version after re-open"); - assert_eq!(ver, 2, "user brain must be at v2 after re-open"); + assert_eq!(ver, 3, "user brain must be at v3 after re-open"); - // Assert 2: backup sidecar brain.db.bak-1-2-* exists next to brain.db. + // Assert 2: backup sidecar brain.db.bak-1-3-* exists next to brain.db + // (migrating from v1 to target v3 produces one backup named with the + // full from-to span: brain.db.bak---). let bak_files: Vec<_> = std::fs::read_dir(&tmp) .expect("read tmp dir") .filter_map(|e| e.ok()) .filter(|e| { e.file_name() .to_str() - .map(|n| n.starts_with("brain.db.bak-1-2-")) + .map(|n| n.starts_with("brain.db.bak-1-3-")) .unwrap_or(false) }) .collect(); assert_eq!( bak_files.len(), 1, - "exactly one user-brain backup sidecar brain.db.bak-1-2-* must exist; found: {:?}", + "exactly one user-brain backup sidecar brain.db.bak-1-3-* must exist; found: {:?}", bak_files.iter().map(|e| e.file_name()).collect::>() ); @@ -700,4 +704,51 @@ mod tests { assert!(tmp.join("brain.db").exists()); }); } + + // ------------------------------------------------------------------ + // Fix 5: add_user_memory dedup must not collapse onto superseded rows + // ------------------------------------------------------------------ + #[test] + fn fix5_dedup_does_not_collapse_onto_superseded_row() { + let tmp = tempdir_in_test("kimetsu-user-brain-fix5"); + with_user_brain_at(&tmp, || { + let conn = open_user_brain().expect("open").expect("enabled"); + + // Insert a memory and immediately stamp it as superseded. + let original = add_user_memory(&conn, MemoryKind::Preference, "use anyhow", 1.0) + .expect("original"); + conn.execute( + "UPDATE memories SET superseded_by = 'fake-survivor' WHERE memory_id = ?1", + rusqlite::params![&original], + ) + .expect("stamp superseded"); + + // Adding the same normalized text again must create a NEW row, + // not return the superseded one. + let second = + add_user_memory(&conn, MemoryKind::Preference, "use anyhow", 1.0).expect("second"); + assert_ne!( + original, second, + "adding a text that matches only a superseded row must produce a new memory_id" + ); + + // Both rows exist; original is superseded, second is active. + let count: i64 = conn + .query_row("SELECT COUNT(*) FROM memories", [], |r| r.get(0)) + .expect("count"); + assert_eq!(count, 2, "two rows: original (superseded) + new active"); + + let second_superseded: Option = conn + .query_row( + "SELECT superseded_by FROM memories WHERE memory_id = ?1", + rusqlite::params![&second], + |r| r.get(0), + ) + .expect("query second"); + assert!( + second_superseded.is_none(), + "newly created row must not be superseded" + ); + }); + } } diff --git a/crates/kimetsu-chat/src/bridge.rs b/crates/kimetsu-chat/src/bridge.rs index f0393aa..9565356 100644 --- a/crates/kimetsu-chat/src/bridge.rs +++ b/crates/kimetsu-chat/src/bridge.rs @@ -13,6 +13,8 @@ pub enum BridgeTarget { ClaudeCode, Codex, Kimetsu, + Cursor, + GeminiCli, #[cfg(feature = "openclaw")] OpenClaw, #[cfg(feature = "pi")] @@ -25,6 +27,8 @@ impl BridgeTarget { "claude" | "claude-code" | "cc" => Ok(Self::ClaudeCode), "codex" => Ok(Self::Codex), "kimetsu" => Ok(Self::Kimetsu), + "cursor" => Ok(Self::Cursor), + "gemini" | "gemini-cli" => Ok(Self::GeminiCli), #[cfg(feature = "openclaw")] "openclaw" | "claw" => Ok(Self::OpenClaw), #[cfg(not(feature = "openclaw"))] @@ -48,6 +52,8 @@ impl BridgeTarget { Self::ClaudeCode => "claude-code", Self::Codex => "codex", Self::Kimetsu => "kimetsu", + Self::Cursor => "cursor", + Self::GeminiCli => "gemini-cli", #[cfg(feature = "openclaw")] Self::OpenClaw => "openclaw", #[cfg(feature = "pi")] @@ -467,6 +473,63 @@ Optional mode: Kimetsu brain context is a preferred first step for non-trivial work. If the MCP server is unavailable, note the absence and continue normally. "#; +/// Kimetsu brain guidance installed in `.cursor/rules/kimetsu-brain/rule.md`. +/// +/// Cursor reads rules from `.cursor/rules//rule.md` (or `.cursor/rules/` +/// directly as `.mdc` files in older versions). The `alwaysApply: true` front- +/// matter ensures the guidance is active in every chat session without requiring +/// the user to invoke it manually. +/// +/// Cursor has NO `UserPromptSubmit`-style hooks: MCP tools plus this always-on +/// rule are the only integration surfaces. +/// +/// Source: https://cursor.com/docs/mcp and https://cursor.com/docs/rules +const CURSOR_RULES_MD: &str = r#"--- +description: Use Kimetsu persistent brain MCP tools as a sidecar for this workspace. +alwaysApply: true +--- +# Kimetsu brain + +You have a persistent memory brain attached via MCP (tools prefixed `kimetsu_`). + +- **Before non-trivial tasks**: call `kimetsu_brain_context` with a short query. If the brain + has relevant prior knowledge it will return it. If not (`skipped: true`), proceed as normal — + this is zero overhead. +- **After solving a non-obvious problem**: call `kimetsu_brain_record` with what you learned + and 2-5 domain tags. Keep lessons concrete and actionable, not platitudes. + +Do not call either tool on simple/one-liner tasks. The brain is for things that required real +effort or that you would want to remember next session. +"#; + +/// GEMINI.md guidance installed in the project root (workspace install) or +/// `~/.gemini/GEMINI.md` (global install). +/// +/// Gemini CLI discovers GEMINI.md files from the project root and parent dirs +/// up to the git root, as well as `~/.gemini/GEMINI.md` for global context. +/// All discovered files are concatenated and sent with every prompt. +/// +/// Gemini CLI has no hook system — MCP + GEMINI.md is the complete integration +/// surface. +/// +/// Source: https://google-gemini.github.io/gemini-cli/docs/cli/gemini-md.html +const GEMINI_MD_CONTENT: &str = r#"# Kimetsu brain + +You have a persistent memory brain attached via MCP (tools prefixed `kimetsu_`). + +- **Before non-trivial tasks**: call `kimetsu_brain_context` with a short query. If the brain + has relevant prior knowledge it will return it. If not (`skipped: true`), proceed as normal — + this is zero overhead. +- **After solving a non-obvious problem**: call `kimetsu_brain_record` with what you learned + and 2-5 domain tags. Keep lessons concrete and actionable, not platitudes. + +Do not call either tool on simple/one-liner tasks. The brain is for things that required real +effort or that you would want to remember next session. +"#; + +const GEMINI_MD_BEGIN: &str = ""; +const GEMINI_MD_END: &str = ""; + pub fn bridge_scan(workspace: &Path, config: &SkillConfig) -> Result { let workspace = normalize_path(workspace); let registry = SkillRegistry::discover(&workspace, config)?; @@ -575,6 +638,8 @@ pub fn bridge_export_skill( BridgeTarget::ClaudeCode => workspace.join(".claude").join("skills").join(&name), BridgeTarget::Codex => workspace.join(".codex").join("skills").join(&name), BridgeTarget::Kimetsu => workspace.join(".kimetsu").join("skills").join(&name), + BridgeTarget::Cursor => workspace.join(".cursor").join("skills").join(&name), + BridgeTarget::GeminiCli => workspace.join(".gemini").join("skills").join(&name), #[cfg(feature = "openclaw")] BridgeTarget::OpenClaw => workspace .join(".openclaw") @@ -909,6 +974,66 @@ fn plugin_install_inner( files.push(normalize_path(&dir)); } + BridgeTarget::Cursor => { + // Cursor: MCP config in .cursor/mcp.json (workspace) or + // ~/.cursor/mcp.json (global), key `mcpServers`. + // No hooks available in Cursor — wire MCP + always-on rules file only. + // + // Config schema verified from https://cursor.com/docs/mcp (June 2026): + // mcpServers..type = "stdio" + // mcpServers..command = "kimetsu" + // mcpServers..args = ["mcp", "serve", "--workspace", "."] + let cursor_dir = match home { + Some(h) => h.join(".cursor"), + None => workspace.join(".cursor"), + }; + fs::create_dir_all(&cursor_dir) + .map_err(|err| format!("create {}: {err}", cursor_dir.display()))?; + let mcp = cursor_dir.join("mcp.json"); + write_cursor_mcp_config(&mcp)?; + files.push(normalize_path(&mcp)); + + // Rules file: .cursor/rules/kimetsu-brain/rule.md + // (workspace install only; global Cursor config does not support rules files) + if home.is_none() { + let rule = workspace + .join(".cursor") + .join("rules") + .join("kimetsu-brain") + .join("rule.md"); + write_text_file(&rule, CURSOR_RULES_MD, true)?; + files.push(normalize_path(&rule)); + } + } + + BridgeTarget::GeminiCli => { + // Gemini CLI: MCP config in .gemini/settings.json (workspace) or + // ~/.gemini/settings.json (global), key `mcpServers`. + // No hook system — wire MCP + GEMINI.md context file only. + // + // Config schema verified from google-gemini/gemini-cli docs (June 2026): + // mcpServers..command = "kimetsu" + // mcpServers..args = ["mcp", "serve", "--workspace", "."] + // GEMINI.md: project root (workspace) or ~/.gemini/GEMINI.md (global) + let gemini_dir = match home { + Some(h) => h.join(".gemini"), + None => workspace.join(".gemini"), + }; + fs::create_dir_all(&gemini_dir) + .map_err(|err| format!("create {}: {err}", gemini_dir.display()))?; + let settings = gemini_dir.join("settings.json"); + write_gemini_settings(&settings)?; + files.push(normalize_path(&settings)); + + // GEMINI.md: merge into project root (workspace) or ~/.gemini/GEMINI.md (global) + let gemini_md_path = match home { + Some(h) => h.join(".gemini").join("GEMINI.md"), + None => workspace.join("GEMINI.md"), + }; + merge_gemini_md(&gemini_md_path)?; + files.push(normalize_path(&gemini_md_path)); + } + #[cfg(feature = "openclaw")] BridgeTarget::OpenClaw => { // OpenClaw supports MCP natively. @@ -1154,6 +1279,75 @@ fn detect_codex_agent(codex_dir: &Path) -> bool { .is_file() } +/// Returns true if `.cursor/mcp.json` has `mcpServers.kimetsu`. +fn detect_cursor_mcp(cursor_dir: &Path) -> bool { + let mcp = cursor_dir.join("mcp.json"); + if !mcp.is_file() { + return false; + } + let Ok(text) = fs::read_to_string(&mcp) else { + return false; + }; + let Ok(root) = serde_json::from_str::(strip_bom(&text)) else { + return false; + }; + root.get("mcpServers") + .and_then(|v| v.as_object()) + .map(|m| m.contains_key("kimetsu")) + .unwrap_or(false) +} + +/// Returns true if `.cursor/rules/kimetsu-brain/` directory exists in `workspace`. +fn detect_cursor_rules(workspace: &Path) -> bool { + workspace + .join(".cursor") + .join("rules") + .join("kimetsu-brain") + .is_dir() +} + +/// Returns true if `.gemini/settings.json` has `mcpServers.kimetsu`. +fn detect_gemini_mcp(gemini_dir: &Path) -> bool { + let settings = gemini_dir.join("settings.json"); + if !settings.is_file() { + return false; + } + let Ok(text) = fs::read_to_string(&settings) else { + return false; + }; + let Ok(root) = serde_json::from_str::(strip_bom(&text)) else { + return false; + }; + root.get("mcpServers") + .and_then(|v| v.as_object()) + .map(|m| m.contains_key("kimetsu")) + .unwrap_or(false) +} + +/// Returns true if `GEMINI.md` in the workspace root has the Kimetsu begin marker. +fn detect_gemini_md(workspace: &Path) -> bool { + let md = workspace.join("GEMINI.md"); + if !md.is_file() { + return false; + } + let Ok(text) = fs::read_to_string(&md) else { + return false; + }; + text.contains(GEMINI_MD_BEGIN) +} + +/// Returns true if `~/.gemini/GEMINI.md` (global) has the Kimetsu begin marker. +fn detect_gemini_global_md(gemini_dir: &Path) -> bool { + let md = gemini_dir.join("GEMINI.md"); + if !md.is_file() { + return false; + } + let Ok(text) = fs::read_to_string(&md) else { + return false; + }; + text.contains(GEMINI_MD_BEGIN) +} + #[cfg(feature = "pi")] /// Returns true if Pi's `settings.json` registers the kimetsu extension AND /// `extensions/kimetsu.ts` exists in `pi_dir`. @@ -1264,7 +1458,12 @@ fn plugin_status_inner(workspace: &Path) -> Vec { let home_opt = resolve_home().ok(); #[allow(unused_mut)] - let mut scan_targets = vec![BridgeTarget::ClaudeCode, BridgeTarget::Codex]; + let mut scan_targets = vec![ + BridgeTarget::ClaudeCode, + BridgeTarget::Codex, + BridgeTarget::Cursor, + BridgeTarget::GeminiCli, + ]; #[cfg(feature = "openclaw")] scan_targets.push(BridgeTarget::OpenClaw); #[cfg(feature = "pi")] @@ -1390,6 +1589,88 @@ fn plugin_status_inner(workspace: &Path) -> Vec { // Not a user-installable host; skip. } + BridgeTarget::Cursor => { + let cursor_dir = match home { + Some(h) => h.join(".cursor"), + None => workspace.join(".cursor"), + }; + + let mcp_ok = detect_cursor_mcp(&cursor_dir); + // Rules are workspace-only; only check when scope == Workspace. + let rules_ok = if home.is_none() { + detect_cursor_rules(&workspace) + } else { + true // global install doesn't write rules — not missing + }; + + let mut present = Vec::new(); + let mut missing = Vec::new(); + + if mcp_ok { + present.push("mcp".to_string()); + } else { + missing.push("mcp".to_string()); + } + if !rules_ok { + missing.push("rules".to_string()); + } else if home.is_none() { + present.push("rules".to_string()); + } + + let state = aggregate_state( + &present.iter().map(|s| s.as_str()).collect::>(), + &missing.iter().map(|s| s.as_str()).collect::>(), + ); + + results.push(PluginScopeStatus { + host: target.as_str().to_string(), + scope: scope.as_str().to_string(), + state, + present, + missing, + config_path: cursor_dir.to_string_lossy().to_string(), + }); + } + + BridgeTarget::GeminiCli => { + let gemini_dir = match home { + Some(h) => h.join(".gemini"), + None => workspace.join(".gemini"), + }; + + let mcp_ok = detect_gemini_mcp(&gemini_dir); + let gemini_md_ok = if home.is_none() { + detect_gemini_md(&workspace) + } else { + detect_gemini_global_md(&gemini_dir) + }; + + let mut present = Vec::new(); + let mut missing = Vec::new(); + + for (name, ok) in [("mcp", mcp_ok), ("GEMINI.md", gemini_md_ok)] { + if ok { + present.push(name.to_string()); + } else { + missing.push(name.to_string()); + } + } + + let state = aggregate_state( + &present.iter().map(|s| s.as_str()).collect::>(), + &missing.iter().map(|s| s.as_str()).collect::>(), + ); + + results.push(PluginScopeStatus { + host: target.as_str().to_string(), + scope: scope.as_str().to_string(), + state, + present, + missing, + config_path: gemini_dir.to_string_lossy().to_string(), + }); + } + #[cfg(feature = "openclaw")] BridgeTarget::OpenClaw => { // OpenClaw: global → ~/.openclaw/; workspace → .openclaw/ @@ -1595,6 +1876,53 @@ fn plugin_uninstall_inner( // Extensions are user data; uninstall is a no-op for this target. } + BridgeTarget::Cursor => { + let cursor_dir = match home { + Some(h) => h.join(".cursor"), + None => workspace.join(".cursor"), + }; + + // mcp.json — remove mcpServers.kimetsu. + let mcp = cursor_dir.join("mcp.json"); + if uninstall_cursor_mcp(&mcp)? { + report.modified.push(normalize_path(&mcp)); + } + + // Delete .cursor/rules/kimetsu-brain/ directory (workspace only). + if home.is_none() { + let rules_dir = workspace + .join(".cursor") + .join("rules") + .join("kimetsu-brain"); + if remove_path_if_exists(&rules_dir)? { + report.removed.push(normalize_path(&rules_dir)); + } + } + } + + BridgeTarget::GeminiCli => { + let gemini_dir = match home { + Some(h) => h.join(".gemini"), + None => workspace.join(".gemini"), + }; + + // settings.json — remove mcpServers.kimetsu. + let settings = gemini_dir.join("settings.json"); + if uninstall_gemini_settings(&settings)? { + report.modified.push(normalize_path(&settings)); + } + + // GEMINI.md — remove the kimetsu block (workspace = project root; + // global = ~/.gemini/GEMINI.md). + let gemini_md = match home { + Some(h) => h.join(".gemini").join("GEMINI.md"), + None => workspace.join("GEMINI.md"), + }; + if uninstall_gemini_md(&gemini_md)? { + report.modified.push(normalize_path(&gemini_md)); + } + } + #[cfg(feature = "openclaw")] BridgeTarget::OpenClaw => { let oc_dir = match home { @@ -1853,93 +2181,301 @@ fn uninstall_codex_config(path: &Path) -> Result { Ok(true) } -#[cfg(feature = "pi")] -/// Strip the `"./extensions/kimetsu.ts"` entry from Pi's `settings.json`. -/// Returns `true` if the file was changed. Missing file or absent entry → Ok(false). -fn uninstall_pi_settings(path: &Path) -> Result { +/// Upsert `mcpServers.kimetsu` into Cursor's `mcp.json`. +/// +/// Schema verified from https://cursor.com/docs/mcp (June 2026): +/// - STDIO server uses `type: "stdio"`, `command`, and `args` fields. +/// - Both workspace (`.cursor/mcp.json`) and global (`~/.cursor/mcp.json`) +/// use the same `mcpServers` key — only `mcpServers`, no `servers` twin key. +fn write_cursor_mcp_config(path: &Path) -> Result<(), String> { + let mut root = if path.is_file() { + let text = + fs::read_to_string(path).map_err(|err| format!("read {}: {err}", path.display()))?; + serde_json::from_str::(strip_bom(&text)) + .map_err(|err| format!("parse {}: {err}", path.display()))? + } else { + serde_json::json!({}) + }; + let root_obj = root + .as_object_mut() + .ok_or_else(|| format!("{} must be a JSON object", path.display()))?; + let servers = root_obj + .entry("mcpServers".to_string()) + .or_insert_with(|| serde_json::json!({})); + let servers_obj = servers + .as_object_mut() + .ok_or_else(|| format!("{} `mcpServers` must be a JSON object", path.display()))?; + servers_obj.insert( + "kimetsu".to_string(), + serde_json::json!({ + "type": "stdio", + "command": "kimetsu", + "args": ["mcp", "serve", "--workspace", "."] + }), + ); + let text = serde_json::to_string_pretty(&root) + .map_err(|err| format!("serialize {}: {err}", path.display()))?; + write_text_file(path, &text, true) +} + +/// Remove `mcpServers.kimetsu` from Cursor's `mcp.json`. +/// Returns `true` if the file was changed. +fn uninstall_cursor_mcp(path: &Path) -> Result { if !path.is_file() { return Ok(false); } let text = fs::read_to_string(path).map_err(|err| format!("read {}: {err}", path.display()))?; let mut root: serde_json::Value = serde_json::from_str(strip_bom(&text)) .map_err(|err| format!("parse {}: {err}", path.display()))?; - let Some(root_obj) = root.as_object_mut() else { return Ok(false); }; - let Some(extensions_value) = root_obj.get_mut("extensions") else { + let Some(servers) = root_obj + .get_mut("mcpServers") + .and_then(|v| v.as_object_mut()) + else { return Ok(false); }; - let Some(arr) = extensions_value.as_array_mut() else { + if servers.remove("kimetsu").is_none() { return Ok(false); - }; - - const EXT_PATH: &str = "./extensions/kimetsu.ts"; - let before = arr.len(); - arr.retain(|v| v.as_str() != Some(EXT_PATH)); - - if arr.len() == before { - return Ok(false); // nothing removed } - - // If the extensions array is now empty, remove it entirely. - if arr.is_empty() { - root_obj.remove("extensions"); - } - let out = serde_json::to_string_pretty(&root) .map_err(|err| format!("serialize {}: {err}", path.display()))?; write_text_file(path, &out, true)?; Ok(true) } -#[cfg(feature = "openclaw")] -/// Upsert the `kimetsu` MCP server and plugin entry into `openclaw.json`. +/// Upsert `mcpServers.kimetsu` into Gemini CLI's `settings.json`. /// -/// `openclaw.json` is JSON5 (supports comments and trailing commas). We parse -/// it with `json5` to tolerate the source format, then write it back with -/// `serde_json::to_string_pretty`. **Comments in the original file are lost** -/// after the first Kimetsu install — we push a note so the caller can surface -/// this to the user. Idempotent: re-running just refreshes the same entries, -/// preserving all other keys. -fn write_openclaw_config(path: &Path, notes: &mut Vec) -> Result<(), String> { - let had_file = path.is_file(); - let mut root = if had_file { +/// Schema verified from google-gemini/gemini-cli docs (June 2026): +/// - STDIO server uses `command` + `args` fields under the `mcpServers` key. +/// - Both workspace (`.gemini/settings.json`) and global (`~/.gemini/settings.json`) +/// use `mcpServers` — same as Gemini's own documented examples. +fn write_gemini_settings(path: &Path) -> Result<(), String> { + let mut root = if path.is_file() { let text = fs::read_to_string(path).map_err(|err| format!("read {}: {err}", path.display()))?; - json5::from_str::(&text) + serde_json::from_str::(strip_bom(&text)) .map_err(|err| format!("parse {}: {err}", path.display()))? } else { serde_json::json!({}) }; - let root_obj = root .as_object_mut() .ok_or_else(|| format!("{} must be a JSON object", path.display()))?; + let servers = root_obj + .entry("mcpServers".to_string()) + .or_insert_with(|| serde_json::json!({})); + let servers_obj = servers + .as_object_mut() + .ok_or_else(|| format!("{} `mcpServers` must be a JSON object", path.display()))?; + servers_obj.insert( + "kimetsu".to_string(), + serde_json::json!({ + "command": "kimetsu", + "args": ["mcp", "serve", "--workspace", "."] + }), + ); + let text = serde_json::to_string_pretty(&root) + .map_err(|err| format!("serialize {}: {err}", path.display()))?; + write_text_file(path, &text, true) +} - // Upsert mcp.servers.kimetsu - { - let mcp = root_obj - .entry("mcp".to_string()) - .or_insert_with(|| serde_json::json!({})); - let mcp_obj = mcp - .as_object_mut() - .ok_or_else(|| format!("{} `mcp` must be a JSON object", path.display()))?; - let servers = mcp_obj - .entry("servers".to_string()) - .or_insert_with(|| serde_json::json!({})); - let servers_obj = servers - .as_object_mut() - .ok_or_else(|| format!("{} `mcp.servers` must be a JSON object", path.display()))?; - servers_obj.insert( - "kimetsu".to_string(), - serde_json::json!({ - "command": "kimetsu", - "args": ["mcp", "serve", "--workspace", "."] - }), - ); +/// Remove `mcpServers.kimetsu` from Gemini CLI's `settings.json`. +/// Returns `true` if the file was changed. +fn uninstall_gemini_settings(path: &Path) -> Result { + if !path.is_file() { + return Ok(false); } - + let text = fs::read_to_string(path).map_err(|err| format!("read {}: {err}", path.display()))?; + let mut root: serde_json::Value = serde_json::from_str(strip_bom(&text)) + .map_err(|err| format!("parse {}: {err}", path.display()))?; + let Some(root_obj) = root.as_object_mut() else { + return Ok(false); + }; + let Some(servers) = root_obj + .get_mut("mcpServers") + .and_then(|v| v.as_object_mut()) + else { + return Ok(false); + }; + if servers.remove("kimetsu").is_none() { + return Ok(false); + } + let out = serde_json::to_string_pretty(&root) + .map_err(|err| format!("serialize {}: {err}", path.display()))?; + write_text_file(path, &out, true)?; + Ok(true) +} + +/// Merge the Kimetsu brain guidance block into a `GEMINI.md` file. +/// +/// Uses the same `` marker idiom as `merge_claude_md` +/// so the block can be found and updated idempotently. Missing file → create. +/// Existing user content is never clobbered. +fn merge_gemini_md(path: &Path) -> Result<(), String> { + let block = format!("{GEMINI_MD_BEGIN}\n{GEMINI_MD_CONTENT}{GEMINI_MD_END}\n"); + let raw = if path.is_file() { + fs::read_to_string(path).map_err(|err| format!("read {}: {err}", path.display()))? + } else { + String::new() + }; + let existing = strip_bom(&raw); + let merged = match (existing.find(GEMINI_MD_BEGIN), existing.find(GEMINI_MD_END)) { + (Some(start), Some(end_start)) if end_start >= start => { + let end = end_start + GEMINI_MD_END.len(); + let after = existing[end..] + .strip_prefix('\n') + .unwrap_or(&existing[end..]); + format!("{}{block}{after}", &existing[..start]) + } + (Some(start), _) => { + // BEGIN present but END missing — corrupt; replace from BEGIN onward. + let before = existing[..start].trim_end_matches('\n'); + if before.is_empty() { + block + } else { + format!("{before}\n\n{block}") + } + } + _ => { + let mut out = existing.to_string(); + if !out.is_empty() { + if !out.ends_with('\n') { + out.push('\n'); + } + out.push('\n'); + } + out.push_str(&block); + out + } + }; + write_text_file(path, &merged, true) +} + +/// Remove the ` … ` block from +/// `GEMINI.md`. Returns `true` if the file was changed. +fn uninstall_gemini_md(path: &Path) -> Result { + if !path.is_file() { + return Ok(false); + } + let raw = fs::read_to_string(path).map_err(|err| format!("read {}: {err}", path.display()))?; + let text = strip_bom(&raw); + + let (begin_pos, end_pos) = match (text.find(GEMINI_MD_BEGIN), text.find(GEMINI_MD_END)) { + (Some(b), Some(e)) if e >= b => (b, e), + _ => return Ok(false), + }; + + let end_of_block = end_pos + GEMINI_MD_END.len(); + let after_start = if text[end_of_block..].starts_with('\n') { + end_of_block + 1 + } else { + end_of_block + }; + + let before = &text[..begin_pos]; + let after = &text[after_start..]; + let before_trimmed = before.trim_end_matches('\n'); + let merged = if before_trimmed.is_empty() { + after.to_string() + } else if after.is_empty() || after.trim().is_empty() { + format!("{before_trimmed}\n") + } else { + format!("{before_trimmed}\n\n{after}") + }; + + write_text_file(path, &merged, true)?; + Ok(true) +} + +#[cfg(feature = "pi")] +/// Strip the `"./extensions/kimetsu.ts"` entry from Pi's `settings.json`. +/// Returns `true` if the file was changed. Missing file or absent entry → Ok(false). +fn uninstall_pi_settings(path: &Path) -> Result { + if !path.is_file() { + return Ok(false); + } + let text = fs::read_to_string(path).map_err(|err| format!("read {}: {err}", path.display()))?; + let mut root: serde_json::Value = serde_json::from_str(strip_bom(&text)) + .map_err(|err| format!("parse {}: {err}", path.display()))?; + + let Some(root_obj) = root.as_object_mut() else { + return Ok(false); + }; + let Some(extensions_value) = root_obj.get_mut("extensions") else { + return Ok(false); + }; + let Some(arr) = extensions_value.as_array_mut() else { + return Ok(false); + }; + + const EXT_PATH: &str = "./extensions/kimetsu.ts"; + let before = arr.len(); + arr.retain(|v| v.as_str() != Some(EXT_PATH)); + + if arr.len() == before { + return Ok(false); // nothing removed + } + + // If the extensions array is now empty, remove it entirely. + if arr.is_empty() { + root_obj.remove("extensions"); + } + + let out = serde_json::to_string_pretty(&root) + .map_err(|err| format!("serialize {}: {err}", path.display()))?; + write_text_file(path, &out, true)?; + Ok(true) +} + +#[cfg(feature = "openclaw")] +/// Upsert the `kimetsu` MCP server and plugin entry into `openclaw.json`. +/// +/// `openclaw.json` is JSON5 (supports comments and trailing commas). We parse +/// it with `json5` to tolerate the source format, then write it back with +/// `serde_json::to_string_pretty`. **Comments in the original file are lost** +/// after the first Kimetsu install — we push a note so the caller can surface +/// this to the user. Idempotent: re-running just refreshes the same entries, +/// preserving all other keys. +fn write_openclaw_config(path: &Path, notes: &mut Vec) -> Result<(), String> { + let had_file = path.is_file(); + let mut root = if had_file { + let text = + fs::read_to_string(path).map_err(|err| format!("read {}: {err}", path.display()))?; + json5::from_str::(&text) + .map_err(|err| format!("parse {}: {err}", path.display()))? + } else { + serde_json::json!({}) + }; + + let root_obj = root + .as_object_mut() + .ok_or_else(|| format!("{} must be a JSON object", path.display()))?; + + // Upsert mcp.servers.kimetsu + { + let mcp = root_obj + .entry("mcp".to_string()) + .or_insert_with(|| serde_json::json!({})); + let mcp_obj = mcp + .as_object_mut() + .ok_or_else(|| format!("{} `mcp` must be a JSON object", path.display()))?; + let servers = mcp_obj + .entry("servers".to_string()) + .or_insert_with(|| serde_json::json!({})); + let servers_obj = servers + .as_object_mut() + .ok_or_else(|| format!("{} `mcp.servers` must be a JSON object", path.display()))?; + servers_obj.insert( + "kimetsu".to_string(), + serde_json::json!({ + "command": "kimetsu", + "args": ["mcp", "serve", "--workspace", "."] + }), + ); + } + // Upsert plugins.entries.kimetsu (activation config) { let plugins = root_obj @@ -5876,4 +6412,620 @@ mod tests { fs::remove_dir_all(ws).ok(); } + + // ------------------------------------------------------------------------- + // Cursor — workspace + global install/uninstall/status tests + // ------------------------------------------------------------------------- + + /// Cursor workspace install writes `.cursor/mcp.json` with `type: "stdio"` + /// and a rules file at `.cursor/rules/kimetsu-brain/rule.md`. + #[test] + fn cursor_workspace_install_writes_mcp_and_rules() { + let ws = temp_root("cursor_ws_install"); + + plugin_install_inner( + &ws, + BridgeTarget::Cursor, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("Cursor workspace install"); + + let mcp_path = ws.join(".cursor/mcp.json"); + assert!(mcp_path.is_file(), ".cursor/mcp.json must exist"); + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&mcp_path).unwrap()).unwrap(); + assert_eq!(v["mcpServers"]["kimetsu"]["type"], "stdio"); + assert_eq!(v["mcpServers"]["kimetsu"]["command"], "kimetsu"); + let args = v["mcpServers"]["kimetsu"]["args"].as_array().unwrap(); + assert!( + args.iter().any(|a| a == "serve"), + "args must include 'serve'" + ); + + let rule_path = ws.join(".cursor/rules/kimetsu-brain/rule.md"); + assert!(rule_path.is_file(), "Cursor rule file must exist"); + let rule_text = fs::read_to_string(&rule_path).unwrap(); + assert!( + rule_text.contains("alwaysApply: true"), + "rule must have alwaysApply frontmatter" + ); + assert!(rule_text.contains("Kimetsu"), "rule must mention Kimetsu"); + + fs::remove_dir_all(ws).ok(); + } + + /// Cursor workspace install is idempotent: a second run must not error and + /// must not duplicate entries in `.cursor/mcp.json`. + #[test] + fn cursor_workspace_install_is_idempotent() { + let ws = temp_root("cursor_ws_idem"); + + for _ in 0..2 { + plugin_install_inner( + &ws, + BridgeTarget::Cursor, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("Cursor install must be idempotent"); + } + + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(ws.join(".cursor/mcp.json")).unwrap()) + .unwrap(); + assert_eq!( + v["mcpServers"].as_object().unwrap().len(), + 1, + "exactly one entry in mcpServers — no duplicates" + ); + + fs::remove_dir_all(ws).ok(); + } + + /// Cursor workspace install preserves a pre-existing user MCP server. + #[test] + fn cursor_workspace_install_preserves_user_server() { + let ws = temp_root("cursor_ws_preserve"); + let cursor_dir = ws.join(".cursor"); + fs::create_dir_all(&cursor_dir).unwrap(); + fs::write( + cursor_dir.join("mcp.json"), + serde_json::to_string_pretty(&json!({ + "mcpServers": { + "my-server": { "type": "stdio", "command": "my-cmd", "args": [] } + } + })) + .unwrap(), + ) + .unwrap(); + + plugin_install_inner( + &ws, + BridgeTarget::Cursor, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("Cursor install with pre-seeded mcp.json"); + + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(cursor_dir.join("mcp.json")).unwrap()) + .unwrap(); + assert_eq!( + v["mcpServers"]["my-server"]["command"], "my-cmd", + "user server must survive" + ); + assert_eq!(v["mcpServers"]["kimetsu"]["command"], "kimetsu"); + + fs::remove_dir_all(ws).ok(); + } + + /// Cursor global install writes into `~/.cursor/mcp.json` (injected home). + #[test] + fn cursor_global_install_writes_to_home() { + let ws = temp_root("cursor_global_ws"); + let home = temp_root("cursor_global_home"); + + plugin_install_inner( + &ws, + BridgeTarget::Cursor, + InstallScope::Global, + PluginMode::Optional, + false, + false, + Some(home.as_path()), + ) + .expect("Cursor global install"); + + let mcp_path = home.join(".cursor/mcp.json"); + assert!( + mcp_path.is_file(), + "~/.cursor/mcp.json must exist for global install" + ); + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&mcp_path).unwrap()).unwrap(); + assert_eq!(v["mcpServers"]["kimetsu"]["command"], "kimetsu"); + // Workspace directory must remain untouched. + assert!( + !ws.join(".cursor").exists(), + "workspace .cursor must not exist for global install" + ); + + fs::remove_dir_all(ws).ok(); + fs::remove_dir_all(home).ok(); + } + + /// Cursor uninstall removes `mcpServers.kimetsu` from `.cursor/mcp.json`. + #[test] + fn cursor_uninstall_removes_mcp_entry() { + let ws = temp_root("cursor_uninstall"); + + // Install first. + plugin_install_inner( + &ws, + BridgeTarget::Cursor, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("install"); + + let cursor_dir = ws.join(".cursor"); + assert!(detect_cursor_mcp(&cursor_dir)); + + // Uninstall. + let report = plugin_uninstall(&ws, BridgeTarget::Cursor, InstallScope::Workspace) + .expect("Cursor uninstall"); + + assert!( + !detect_cursor_mcp(&cursor_dir), + "mcp entry must be gone after uninstall" + ); + assert!( + !report.modified.is_empty(), + "report must list modified files" + ); + + fs::remove_dir_all(ws).ok(); + } + + /// `plugin_status` detects an installed Cursor workspace entry. + #[test] + fn cursor_status_detects_installed_workspace() { + let ws = temp_root("cursor_status_ws"); + + plugin_install_inner( + &ws, + BridgeTarget::Cursor, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("install"); + + let statuses = plugin_status_inner(&ws); + let entry = statuses + .iter() + .find(|s| s.host == "cursor" && s.scope == "workspace"); + assert!(entry.is_some(), "cursor/workspace status entry must exist"); + let entry = entry.unwrap(); + assert!( + matches!(entry.state, WiringState::Installed), + "state must be Installed, got {:?}", + entry.state + ); + + fs::remove_dir_all(ws).ok(); + } + + // ------------------------------------------------------------------------- + // Gemini CLI — workspace + global install/uninstall/status tests + // ------------------------------------------------------------------------- + + /// Gemini CLI workspace install writes `.gemini/settings.json` (mcpServers) + /// and merges a `GEMINI.md` block at the project root. + #[test] + fn gemini_workspace_install_writes_settings_and_md() { + let ws = temp_root("gemini_ws_install"); + + plugin_install_inner( + &ws, + BridgeTarget::GeminiCli, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("Gemini CLI workspace install"); + + let settings_path = ws.join(".gemini/settings.json"); + assert!(settings_path.is_file(), ".gemini/settings.json must exist"); + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&settings_path).unwrap()).unwrap(); + assert_eq!(v["mcpServers"]["kimetsu"]["command"], "kimetsu"); + let args = v["mcpServers"]["kimetsu"]["args"].as_array().unwrap(); + assert!( + args.iter().any(|a| a == "serve"), + "args must include 'serve'" + ); + + let gemini_md_path = ws.join("GEMINI.md"); + assert!( + gemini_md_path.is_file(), + "GEMINI.md must exist at workspace root" + ); + let md_text = fs::read_to_string(&gemini_md_path).unwrap(); + assert!( + md_text.contains(GEMINI_MD_BEGIN), + "GEMINI.md must have begin marker" + ); + assert!( + md_text.contains("Kimetsu"), + "GEMINI.md must mention Kimetsu" + ); + + fs::remove_dir_all(ws).ok(); + } + + /// Gemini CLI workspace install is idempotent. + #[test] + fn gemini_workspace_install_is_idempotent() { + let ws = temp_root("gemini_ws_idem"); + + for _ in 0..2 { + plugin_install_inner( + &ws, + BridgeTarget::GeminiCli, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("Gemini install must be idempotent"); + } + + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(ws.join(".gemini/settings.json")).unwrap()) + .unwrap(); + assert_eq!( + v["mcpServers"].as_object().unwrap().len(), + 1, + "exactly one entry in mcpServers" + ); + + let md_text = fs::read_to_string(ws.join("GEMINI.md")).unwrap(); + assert_eq!( + md_text.matches(GEMINI_MD_BEGIN).count(), + 1, + "GEMINI.md must have exactly one Kimetsu block after two installs" + ); + + fs::remove_dir_all(ws).ok(); + } + + /// Gemini CLI workspace install preserves a pre-existing user MCP server. + #[test] + fn gemini_workspace_install_preserves_user_server() { + let ws = temp_root("gemini_ws_preserve"); + let gemini_dir = ws.join(".gemini"); + fs::create_dir_all(&gemini_dir).unwrap(); + fs::write( + gemini_dir.join("settings.json"), + serde_json::to_string_pretty(&json!({ + "mcpServers": { + "my-tool": { "command": "my-tool-cmd", "args": [] } + } + })) + .unwrap(), + ) + .unwrap(); + + plugin_install_inner( + &ws, + BridgeTarget::GeminiCli, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("Gemini install with pre-seeded settings.json"); + + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(gemini_dir.join("settings.json")).unwrap()) + .unwrap(); + assert_eq!( + v["mcpServers"]["my-tool"]["command"], "my-tool-cmd", + "user tool must survive" + ); + assert_eq!(v["mcpServers"]["kimetsu"]["command"], "kimetsu"); + + fs::remove_dir_all(ws).ok(); + } + + /// Gemini CLI workspace install preserves pre-existing user content in GEMINI.md. + #[test] + fn gemini_workspace_install_preserves_gemini_md_user_content() { + let ws = temp_root("gemini_ws_md_preserve"); + // Pre-seed a GEMINI.md with user instructions. + fs::write(ws.join("GEMINI.md"), "# Project rules\nAlways test.\n").unwrap(); + + plugin_install_inner( + &ws, + BridgeTarget::GeminiCli, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("Gemini install with pre-seeded GEMINI.md"); + + let text = fs::read_to_string(ws.join("GEMINI.md")).unwrap(); + assert!( + text.contains("# Project rules"), + "user content must survive" + ); + assert!(text.contains("Always test."), "user detail must survive"); + assert!(text.contains("Kimetsu"), "kimetsu block appended"); + assert!( + text.find("# Project rules").unwrap() < text.find(GEMINI_MD_BEGIN).unwrap(), + "user content must precede kimetsu block" + ); + + fs::remove_dir_all(ws).ok(); + } + + /// Gemini CLI global install writes to `~/.gemini/settings.json` and + /// `~/.gemini/GEMINI.md` (injected home). + #[test] + fn gemini_global_install_writes_to_home() { + let ws = temp_root("gemini_global_ws"); + let home = temp_root("gemini_global_home"); + + plugin_install_inner( + &ws, + BridgeTarget::GeminiCli, + InstallScope::Global, + PluginMode::Optional, + false, + false, + Some(home.as_path()), + ) + .expect("Gemini CLI global install"); + + let settings_path = home.join(".gemini/settings.json"); + assert!( + settings_path.is_file(), + "~/.gemini/settings.json must exist" + ); + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&settings_path).unwrap()).unwrap(); + assert_eq!(v["mcpServers"]["kimetsu"]["command"], "kimetsu"); + + let gemini_md_path = home.join(".gemini/GEMINI.md"); + assert!( + gemini_md_path.is_file(), + "~/.gemini/GEMINI.md must exist for global install" + ); + let md_text = fs::read_to_string(&gemini_md_path).unwrap(); + assert!(md_text.contains(GEMINI_MD_BEGIN)); + + // Workspace directory must remain untouched. + assert!( + !ws.join(".gemini").exists(), + "workspace .gemini must not exist for global install" + ); + assert!( + !ws.join("GEMINI.md").exists(), + "workspace GEMINI.md must not exist for global install" + ); + + fs::remove_dir_all(ws).ok(); + fs::remove_dir_all(home).ok(); + } + + /// Gemini CLI uninstall removes `mcpServers.kimetsu` from + /// `.gemini/settings.json` and strips the kimetsu block from `GEMINI.md`. + #[test] + fn gemini_uninstall_removes_settings_and_md() { + let ws = temp_root("gemini_uninstall"); + + // Install first. + plugin_install_inner( + &ws, + BridgeTarget::GeminiCli, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("install"); + + assert!(detect_gemini_mcp(&ws.join(".gemini"))); + assert!(detect_gemini_md(&ws)); + + // Uninstall. + let report = plugin_uninstall(&ws, BridgeTarget::GeminiCli, InstallScope::Workspace) + .expect("Gemini CLI uninstall"); + + assert!( + !detect_gemini_mcp(&ws.join(".gemini")), + "mcp entry must be gone after uninstall" + ); + assert!( + !detect_gemini_md(&ws), + "GEMINI.md block must be gone after uninstall" + ); + assert!( + !report.modified.is_empty(), + "report must list modified files" + ); + + fs::remove_dir_all(ws).ok(); + } + + /// `plugin_status` detects an installed Gemini CLI workspace entry. + #[test] + fn gemini_status_detects_installed_workspace() { + let ws = temp_root("gemini_status_ws"); + + plugin_install_inner( + &ws, + BridgeTarget::GeminiCli, + InstallScope::Workspace, + PluginMode::Optional, + false, + false, + None, + ) + .expect("install"); + + let statuses = plugin_status_inner(&ws); + let entry = statuses + .iter() + .find(|s| s.host == "gemini-cli" && s.scope == "workspace"); + assert!( + entry.is_some(), + "gemini-cli/workspace status entry must exist" + ); + let entry = entry.unwrap(); + assert!( + matches!(entry.state, WiringState::Installed), + "state must be Installed, got {:?}", + entry.state + ); + + fs::remove_dir_all(ws).ok(); + } + + // ------------------------------------------------------------------------- + // merge_gemini_md — unit tests (mirrors merge_claude_md suite) + // ------------------------------------------------------------------------- + + #[test] + fn merge_gemini_md_fresh_file() { + let root = temp_root("gemini_md_fresh"); + let p = root.join("GEMINI.md"); + merge_gemini_md(&p).unwrap(); + let text = fs::read_to_string(&p).unwrap(); + assert!(text.contains(GEMINI_MD_BEGIN)); + assert!(text.contains("Kimetsu")); + assert!(text.contains(GEMINI_MD_END)); + fs::remove_dir_all(root).ok(); + } + + #[test] + fn merge_gemini_md_preserves_user_content() { + let root = temp_root("gemini_md_preserve"); + let p = root.join("GEMINI.md"); + fs::write(&p, "# My rules\nAlways use tabs.\n").unwrap(); + merge_gemini_md(&p).unwrap(); + let text = fs::read_to_string(&p).unwrap(); + assert!(text.contains("# My rules")); + assert!(text.contains("Always use tabs.")); + assert!(text.contains("Kimetsu")); + assert!( + text.find("My rules").unwrap() < text.find(GEMINI_MD_BEGIN).unwrap(), + "user content precedes the kimetsu block" + ); + fs::remove_dir_all(root).ok(); + } + + #[test] + fn merge_gemini_md_idempotent() { + let root = temp_root("gemini_md_idem"); + let p = root.join("GEMINI.md"); + fs::write(&p, "# Mine\n").unwrap(); + merge_gemini_md(&p).unwrap(); + merge_gemini_md(&p).unwrap(); + let text = fs::read_to_string(&p).unwrap(); + assert_eq!( + text.matches(GEMINI_MD_BEGIN).count(), + 1, + "no duplicate block" + ); + assert_eq!(text.matches(GEMINI_MD_END).count(), 1); + assert!(text.contains("# Mine")); + fs::remove_dir_all(root).ok(); + } + + #[test] + fn merge_gemini_md_upgrades_in_place() { + let root = temp_root("gemini_md_upgrade"); + let p = root.join("GEMINI.md"); + fs::write( + &p, + format!("# Top\n\n{GEMINI_MD_BEGIN}\nOLD STALE\n{GEMINI_MD_END}\n\n# Bottom\n"), + ) + .unwrap(); + merge_gemini_md(&p).unwrap(); + let text = fs::read_to_string(&p).unwrap(); + assert!(!text.contains("OLD STALE"), "stale block replaced"); + assert!(text.contains("Kimetsu")); + assert!(text.contains("# Top")); + assert!(text.contains("# Bottom")); + assert_eq!(text.matches(GEMINI_MD_BEGIN).count(), 1); + fs::remove_dir_all(root).ok(); + } + + // ------------------------------------------------------------------------- + // write_cursor_mcp_config / write_gemini_settings — unit tests + // ------------------------------------------------------------------------- + + #[test] + fn write_cursor_mcp_config_fresh_and_idempotent() { + let root = temp_root("cursor_mcp_unit"); + let path = root.join("mcp.json"); + write_cursor_mcp_config(&path).unwrap(); + write_cursor_mcp_config(&path).unwrap(); // idempotent + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap(); + assert_eq!(v["mcpServers"]["kimetsu"]["type"], "stdio"); + assert_eq!(v["mcpServers"]["kimetsu"]["command"], "kimetsu"); + assert_eq!( + v["mcpServers"].as_object().unwrap().len(), + 1, + "no duplicate entries" + ); + fs::remove_dir_all(root).ok(); + } + + #[test] + fn write_gemini_settings_fresh_and_idempotent() { + let root = temp_root("gemini_settings_unit"); + let path = root.join("settings.json"); + write_gemini_settings(&path).unwrap(); + write_gemini_settings(&path).unwrap(); // idempotent + let v: serde_json::Value = + serde_json::from_str(&fs::read_to_string(&path).unwrap()).unwrap(); + assert_eq!(v["mcpServers"]["kimetsu"]["command"], "kimetsu"); + assert_eq!( + v["mcpServers"].as_object().unwrap().len(), + 1, + "no duplicate entries" + ); + // Gemini CLI does NOT use `type: "stdio"` — just command + args. + assert!( + v["mcpServers"]["kimetsu"].get("type").is_none(), + "Gemini CLI settings must not have a 'type' field" + ); + fs::remove_dir_all(root).ok(); + } } diff --git a/crates/kimetsu-chat/src/mcp_server.rs b/crates/kimetsu-chat/src/mcp_server.rs index 4de1557..ac137b2 100644 --- a/crates/kimetsu-chat/src/mcp_server.rs +++ b/crates/kimetsu-chat/src/mcp_server.rs @@ -12,7 +12,7 @@ use crate::bridge::{ }; use crate::skills::{SkillConfig, SkillRegistry, skill_origin_label}; -const KIMETSU_MCP_INSTRUCTIONS: &str = "Kimetsu is a persistent brain sidecar for Claude Code and Codex. It accumulates generalizable knowledge across sessions and retrieves it on demand. Recommended workflow: (1) Call kimetsu_brain_context early on non-trivial tasks — if skipped:true is returned, the brain has nothing relevant and you paid zero overhead. (2) After solving a non-obvious problem that took real effort, call kimetsu_brain_record with a concrete lesson and 2-5 domain tags. Do NOT call for trivial or well-known knowledge. (3) For Terminal-Bench tasks, call kimetsu_benchmark_context instead — it prioritizes semantic_operator and anti_pattern memories over episodic summaries. Use kimetsu_bridge_status and kimetsu_skills_search when portable skills may help. Brain tools retrieve and curate durable context; bridge tools discover capabilities."; +const KIMETSU_MCP_INSTRUCTIONS: &str = "Kimetsu is a persistent brain sidecar for Claude Code and Codex. It accumulates generalizable knowledge across sessions and retrieves it on demand. Recommended workflow: (1) Call kimetsu_brain_context early on non-trivial tasks — if skipped:true is returned, the brain has nothing relevant and you paid zero overhead. (2) After solving a non-obvious problem that took real effort, call kimetsu_brain_record with a concrete lesson and 2-5 domain tags. Do NOT call for trivial or well-known knowledge. (3) When a retrieved memory materially helped, call kimetsu_brain_cite with its memory_id — this closes the ground-truth loop and powers self-tuning. (4) For Terminal-Bench tasks, call kimetsu_benchmark_context instead — it prioritizes semantic_operator and anti_pattern memories over episodic summaries. Use kimetsu_bridge_status and kimetsu_skills_search when portable skills may help. Brain tools retrieve and curate durable context; bridge tools discover capabilities."; const BRAIN_STATUS_DESCRIPTION: &str = "Inspect the Kimetsu brain for this workspace. Use this to see whether brain.db is initialized, how many memories/runs/proposals exist, and which memories have positive outcome usefulness. Call before relying on memory if you need to know whether the brain has signal."; @@ -62,6 +62,8 @@ const BRIDGE_SYNC_DESCRIPTION: &str = "Bulk-import all discovered non-Kimetsu sk const PLUGIN_INSTALL_DESCRIPTION: &str = "Install Kimetsu MCP/plugin wiring for a target harness in this workspace. For codex, writes .codex/config.toml, .codex/hooks.json, the kimetsu-bridge skill, and the kimetsu-memory-harvester custom agent; for claude-code, writes .mcp.json, command docs, and .claude/settings.json hooks. Set mode=optional to recommend brain-first usage, or mode=required to tell the host harness that non-trivial work must load Kimetsu brain context. Installed guidance tells benchmark agents to prefer kimetsu_benchmark_context and record outcomes through kimetsu_benchmark_record_outcome. Set scope=workspace (default) to install into this workspace, or scope=global to install into the user's home (~/.claude, ~/.claude.json, ~/.codex) for all sessions. Existing user hooks are preserved (merged, not replaced)."; +const BRAIN_CITE_DESCRIPTION: &str = "Call when a retrieved Kimetsu memory materially helped you solve the current task. This records a ground-truth citation that powers Kimetsu's self-tuning: the brain learns which memories actually earn their keep. ROI: each citation trains the retrieval objective so future queries surface that memory sooner. Pass memory_id (from the capsule's provenance or kimetsu_brain_memory_list) and an optional note describing how it helped."; + #[derive(Debug, Clone)] pub struct McpServeConfig { pub workspace: PathBuf, @@ -293,6 +295,7 @@ fn call_tool( "kimetsu_brain_insights" => Ok(kimetsu_brain_insights(workspace, &arguments)), "kimetsu_brain_context" => Ok(kimetsu_brain_context(workspace, &arguments)), "kimetsu_brain_record" => Ok(kimetsu_brain_record(workspace, &arguments)), + "kimetsu_brain_cite" => kimetsu_brain_cite(workspace, &arguments), "kimetsu_benchmark_context" => Ok(kimetsu_benchmark_context(workspace, &arguments)), "kimetsu_benchmark_record_outcome" => { kimetsu_benchmark_record_outcome(workspace, &arguments) @@ -447,6 +450,7 @@ fn is_privileged_write_tool(name: &str) -> bool { matches!( name, "kimetsu_brain_record" + | "kimetsu_brain_cite" | "kimetsu_benchmark_record_outcome" | "kimetsu_brain_memory_add" | "kimetsu_brain_memory_accept" @@ -784,6 +788,22 @@ pub fn brain_context_tool( cap, ); } + + // v1.5 (Story 2.1): render-time compression. Load compress_capsules + // best-effort — any config error means no compression (safe default). + // Ranking is NEVER affected; this runs after retrieval + reranking. + let compress = kimetsu_core::paths::ProjectPaths::discover(workspace) + .ok() + .and_then(|paths| project::load_config(&paths).ok()) + .map(|cfg| cfg.broker.compress_capsules) + .unwrap_or(false); + if compress { + use kimetsu_brain::context::compress_for_render; + for capsule in &mut bundle.capsules { + capsule.summary = compress_for_render(&capsule.summary, 3); + } + } + Ok(json!({ "ok": true, "skipped": false, @@ -919,6 +939,31 @@ fn kimetsu_brain_record(workspace: &Path, arguments: &Value) -> Value { } } +/// Record a ground-truth citation for a memory that materially helped. +/// Writes a `memory.cited` event with the all-zero sentinel run_id so +/// it links to no active run — this is the MCP path (primary Claude Code usage) +/// where there is no agent run in progress. +fn kimetsu_brain_cite(workspace: &Path, arguments: &Value) -> Result { + let memory_id = match arguments.get("memory_id").and_then(Value::as_str) { + Some(s) if !s.trim().is_empty() => s.trim(), + _ => { + return Ok( + json!({ "ok": false, "error": "memory_id is required and must be non-empty" }), + ); + } + }; + let note = arguments.get("note").and_then(Value::as_str); + match project::record_mcp_citation(workspace, memory_id, note) { + Ok(()) => Ok(json!({ + "ok": true, + "memory_id": memory_id, + "recorded": "memory.cited", + "usage": "Citation recorded. This closes the ground-truth loop and trains Kimetsu's self-tuning objective." + })), + Err(err) => Ok(json!({ "ok": false, "error": err.to_string() })), + } +} + /// W3.2: load `broker.ambient` from the project config, best-effort. /// Returns `true` (the default) if the config is missing or unreadable /// so existing behavior is preserved when the project hasn't been @@ -1768,6 +1813,18 @@ fn tool_definitions() -> Value { "required": ["query"] } }, + { + "name": "kimetsu_brain_cite", + "description": BRAIN_CITE_DESCRIPTION, + "inputSchema": { + "type": "object", + "properties": { + "memory_id": { "type": "string", "description": "The memory_id of the retrieved memory that helped (from capsule provenance or kimetsu_brain_memory_list)." }, + "note": { "type": "string", "description": "Optional short description of how the memory helped." } + }, + "required": ["memory_id"] + } + }, { "name": "kimetsu_brain_record", "description": "Record a concrete, reusable lesson into the brain. Call after solving a non-obvious problem that required real effort. Do NOT call for trivial or well-known knowledge. High-confidence lessons (≥0.7) are accepted immediately; low-confidence go to pending proposals.", @@ -2848,4 +2905,83 @@ mod tests { ); fs::remove_dir_all(root).expect("remove temp root"); } + + // v1.5: kimetsu_brain_cite tests + #[test] + fn cite_tool_listed_and_write_gated() { + let result = handle_mcp_method( + "tools/list", + json!({}), + Path::new("."), + &SkillConfig::default(), + ) + .expect("tools/list"); + let tools = result["tools"].as_array().unwrap(); + let cite = tools + .iter() + .find(|t| t["name"].as_str() == Some("kimetsu_brain_cite")) + .expect("kimetsu_brain_cite must appear in tools/list"); + assert!( + cite["description"] + .as_str() + .unwrap_or("") + .contains("self-tuning"), + "description must mention self-tuning" + ); + } + + #[test] + fn cite_tool_is_write_gated() { + assert!( + is_privileged_write_tool("kimetsu_brain_cite"), + "kimetsu_brain_cite must be privileged" + ); + } + + #[test] + fn cite_tool_writes_memory_citations_row() { + kimetsu_brain::user_brain::with_user_brain_disabled(|| { + let root = temp_root("kimetsu-mcp-cite-test"); + fs::create_dir_all(&root).expect("create root"); + project::init_project(&root, false).expect("init"); + let memory_id = project::add_memory( + &root, + kimetsu_core::memory::MemoryScope::Project, + kimetsu_core::memory::MemoryKind::Fact, + "cite MCP test memory", + ) + .expect("add memory"); + + unsafe { + std::env::set_var("KIMETSU_MCP_ENABLE_WRITE_TOOLS", "1"); + } + let result = call_tool( + "kimetsu_brain_cite", + json!({ "memory_id": memory_id, "note": "it helped" }), + &root, + &SkillConfig::default(), + ) + .expect("call kimetsu_brain_cite"); + unsafe { + std::env::remove_var("KIMETSU_MCP_ENABLE_WRITE_TOOLS"); + } + + assert_eq!( + result["ok"].as_bool(), + Some(true), + "cite must return ok:true" + ); + + let (_paths, _config, conn) = project::load_project(&root).expect("load"); + let count: i64 = conn + .query_row( + "SELECT COUNT(*) FROM memory_citations WHERE memory_id = ?1", + [memory_id.as_str()], + |r| r.get(0), + ) + .expect("count"); + assert_eq!(count, 1, "memory_citations row must exist"); + fs::remove_dir_all(root).ok(); + }); + } } diff --git a/crates/kimetsu-cli/Cargo.toml b/crates/kimetsu-cli/Cargo.toml index d03d91c..7615ee1 100644 --- a/crates/kimetsu-cli/Cargo.toml +++ b/crates/kimetsu-cli/Cargo.toml @@ -50,7 +50,9 @@ kimetsu-core = { path = "../kimetsu-core", version = "1.0.0" } # v0.4.6: `kimetsu doctor` serializes its report struct so --json # output can be piped into CI / hooks. reqwest.workspace = true +rusqlite.workspace = true serde.workspace = true +time.workspace = true serde_json.workspace = true sha2.workspace = true toml.workspace = true diff --git a/crates/kimetsu-cli/src/distiller.rs b/crates/kimetsu-cli/src/distiller.rs index 13db08f..4e111fc 100644 --- a/crates/kimetsu-cli/src/distiller.rs +++ b/crates/kimetsu-cli/src/distiller.rs @@ -396,6 +396,47 @@ pub fn run_session_end_hook(workspace: &Path) { run_distiller_for_transcript(workspace, transcript_path); } +/// Construct a boxed `ModelProvider` from a `ResolvedDistiller`, consuming +/// the credential fields. Returns `None` when the provider variant is unknown +/// or credential construction fails. +pub fn make_provider_for_resolved(resolved: &ResolvedDistiller) -> Option> { + match resolved.provider.as_str() { + "anthropic" => AnthropicProvider::for_distiller( + &resolved.model, + resolved.key.clone(), + resolved.base_url.clone(), + resolved.timeout_secs, + ) + .ok() + .map(|p| Box::new(p) as Box), + "openai" => OpenAiProvider::for_distiller( + &resolved.model, + resolved.key.clone(), + resolved.base_url.clone(), + resolved.timeout_secs, + ) + .ok() + .map(|p| Box::new(p) as Box), + "bedrock" => { + let region = resolved.region.clone()?; + let secret_key = resolved.secret_key.clone()?; + BedrockProvider::for_distiller( + &resolved.model, + region, + resolved.key.clone(), + secret_key, + resolved.session_token.clone(), + 1024, + 0.2, + resolved.timeout_secs, + ) + .ok() + .map(|p| Box::new(p) as Box) + } + _ => None, + } +} + /// Run the configured distiller against a known transcript path. Used by /// SessionEnd hooks where available, and by Codex Stop hooks because current /// Codex releases expose Stop but not SessionEnd. diff --git a/crates/kimetsu-cli/src/main.rs b/crates/kimetsu-cli/src/main.rs index f66dde6..98426e6 100644 --- a/crates/kimetsu-cli/src/main.rs +++ b/crates/kimetsu-cli/src/main.rs @@ -717,6 +717,52 @@ enum BrainCommand { /// Results are written to --out as JSON files + a summary.md table. /// Requires `--features embeddings`. Bench(BrainBenchArgs), + /// ROI ledger — did kimetsu pay for itself? + /// + /// Estimates token savings from cited memories (conservative per-kind + /// calibration), subtracts brain-injection overhead, and shows a + /// net-positive / net-negative verdict. Honest negatives are shown as + /// such. Use `--json` for stable machine-readable output. + Roi(RoiArgs), + /// Self-tuning sweep — optimize retrieval config from personal eval data. + /// + /// --status: show accumulated eval cases and readiness. + /// --apply: write the winning config to project.toml (dry-run by default). + /// --revert: restore the previous tune-history entry. + Tune(TuneArgs), + /// Merge near-duplicate memories and optionally distil loose clusters. + /// + /// Story 3.1 (--merge, default): brute-force cosine scan over stored embeddings; + /// memories with cosine ≥ THRESHOLD (default 0.92) are merged — survivor keeps + /// its text/id; members get `superseded_by` set and are removed from retrieval. + /// Citations are reassigned to the survivor so `memory blame` stays accurate. + /// + /// Story 3.2 (--distill): looser clusters (0.75–0.85 cosine band) of ≥ 3 + /// memories sharing ≥ 1 domain tag are fed to the configured distiller (same + /// model the SessionEnd hook uses). Result lands as a memory proposal for human + /// review. If no distiller is configured, prints the clusters and exits 0. + /// + /// Examples: + /// kimetsu brain consolidate --dry-run + /// kimetsu brain consolidate --yes + /// kimetsu brain consolidate --threshold 0.88 --yes + /// kimetsu brain consolidate --distill --dry-run + /// kimetsu brain consolidate --distill --yes + Consolidate(ConsolidateArgs), + /// List fading memories and prune them interactively. + /// + /// Shows memories with usefulness_score < SCORE_FLOOR (default 0.2) AND + /// last_useful_at / created_at older than AGE_DAYS (default 30 days), + /// with id / kind / age / usefulness / text-head. + /// + /// Interactive per-item [k]eep / [p]rune / [s]kip (requires a TTY). + /// Use --prune-all --yes for batch non-interactive pruning. + /// + /// Examples: + /// kimetsu brain triage + /// kimetsu brain triage --score-floor 0.1 --age-days 60 + /// kimetsu brain triage --prune-all --yes + Triage(TriageArgs), } #[derive(Debug, Subcommand)] @@ -878,6 +924,84 @@ struct SessionEndHookArgs { workspace: Option, } +/// Args for `kimetsu brain tune`. +#[derive(Debug, Args)] +struct TuneArgs { + /// Show personal eval-set statistics without running the sweep. + #[arg(long)] + status: bool, + /// Cost penalty weight per estimated token injected per query. + /// Default 0.005 ≈ one MRR rank position ≈ 200 tokens. + #[arg(long, default_value_t = 0.005f64)] + cost_weight: f64, + /// Apply the winning config to project.toml (without this flag, dry-run only). + #[arg(long)] + apply: bool, + /// Revert the most recent tune-history entry. + #[arg(long)] + revert: bool, + /// Override the brain workspace path (defaults to current directory). + #[arg(long)] + workspace: Option, +} + +/// Args for `kimetsu brain consolidate` (Stories 3.1 + 3.2). +#[derive(Debug, Args)] +struct ConsolidateArgs { + /// Print merge plan without writing to the DB. + #[arg(long)] + dry_run: bool, + /// Cosine similarity threshold for near-duplicate clustering (Story 3.1). + /// Memories with cosine ≥ threshold are merged. Default: 0.92. + #[arg(long, default_value_t = 0.92f32)] + threshold: f32, + /// Skip the interactive confirmation prompt (required when stdin is not a TTY). + #[arg(long)] + yes: bool, + /// Also run Story 3.2 distillation of loose clusters (0.75–0.85 band). + /// Result lands as a memory proposal for human review. + /// Requires a configured distiller; prints clusters and exits 0 otherwise. + #[arg(long)] + distill: bool, + /// Override the brain workspace path (defaults to current directory). + #[arg(long)] + workspace: Option, +} + +/// Args for `kimetsu brain triage` (Story 3.3). +#[derive(Debug, Args)] +struct TriageArgs { + /// Usefulness score floor: memories below this threshold are candidates. + #[arg(long, default_value_t = 0.2f32)] + score_floor: f32, + /// Age threshold in days: memories last useful (or created) before this are candidates. + #[arg(long, default_value_t = 30u32)] + age_days: u32, + /// Prune all candidates non-interactively (requires --yes). + #[arg(long)] + prune_all: bool, + /// Skip the confirmation prompt for --prune-all. + #[arg(long)] + yes: bool, + /// Override the brain workspace path (defaults to current directory). + #[arg(long)] + workspace: Option, +} + +/// Args for `kimetsu brain roi`. +#[derive(Debug, Args)] +struct RoiArgs { + /// Time window: "7d", "30d", or "all". Default: 30d. + #[arg(long, default_value = "30d")] + window: String, + /// Emit machine-readable JSON (stable RoiReport schema). + #[arg(long)] + json: bool, + /// Override the brain workspace path (defaults to current directory). + #[arg(long)] + workspace: Option, +} + #[derive(Debug, Args)] struct ReindexArgs { /// Which DB(s) to reindex: `project`, `user`, or `all`. @@ -932,6 +1056,16 @@ struct BrainExportArgs { /// Override the brain workspace path (defaults to current directory). #[arg(long)] workspace: Option, + /// Strip the trailing `(context: …)` segment from each exported memory text. + /// Useful when sharing memories between projects: the context annotation is + /// project-specific and not meaningful elsewhere. + #[arg(long)] + redact: bool, + /// Strip the leading `[tags: …]` prefix from each exported memory text. + /// Usable on its own (tags only) or with `--redact` for a fully clean + /// lesson body with no metadata. + #[arg(long)] + redact_tags: bool, } #[derive(Debug, Args)] @@ -1641,10 +1775,13 @@ fn restart_cmd(args: RestartArgs) -> KimetsuResult<()> { /// 4. None present + TTY → prompt with the provided `reader`. /// /// Factored as a pure-ish function so it can be unit-tested without real installs. +#[allow(clippy::too_many_arguments)] pub fn resolve_setup_hosts( arg: Option<&str>, present_claude: bool, present_codex: bool, + present_cursor: bool, + present_gemini: bool, present_openclaw: bool, present_pi: bool, is_tty: bool, @@ -1668,6 +1805,12 @@ pub fn resolve_setup_hosts( if present_codex { detected.push(BridgeTarget::Codex); } + if present_cursor { + detected.push(BridgeTarget::Cursor); + } + if present_gemini { + detected.push(BridgeTarget::GeminiCli); + } #[cfg(feature = "openclaw")] if present_openclaw { detected.push(BridgeTarget::OpenClaw); @@ -1694,13 +1837,15 @@ pub fn resolve_setup_hosts( Ok(vec![BridgeTarget::ClaudeCode]) } else { #[cfg(all(feature = "pi", feature = "openclaw"))] - let prompt = "Which host agent do you use? [claude-code/codex/openclaw/pi/both]: "; + let prompt = + "Which host agent do you use? [claude-code/codex/cursor/gemini-cli/openclaw/pi/both]: "; #[cfg(all(feature = "pi", not(feature = "openclaw")))] - let prompt = "Which host agent do you use? [claude-code/codex/pi/both]: "; + let prompt = "Which host agent do you use? [claude-code/codex/cursor/gemini-cli/pi/both]: "; #[cfg(all(not(feature = "pi"), feature = "openclaw"))] - let prompt = "Which host agent do you use? [claude-code/codex/openclaw/both]: "; + let prompt = + "Which host agent do you use? [claude-code/codex/cursor/gemini-cli/openclaw/both]: "; #[cfg(all(not(feature = "pi"), not(feature = "openclaw")))] - let prompt = "Which host agent do you use? [claude-code/codex/both]: "; + let prompt = "Which host agent do you use? [claude-code/codex/cursor/gemini-cli/both]: "; print!("{prompt}"); io::stdout().flush().ok(); let mut line = String::new(); @@ -1720,9 +1865,10 @@ pub fn resolve_setup_hosts( } } -/// Detect whether the home config directories for Claude Code, Codex, OpenClaw, and Pi exist. -/// Returns `(claude_present, codex_present, openclaw_present, pi_present)`. -fn detect_present_hosts() -> (bool, bool, bool, bool) { +/// Detect whether the home config directories for Claude Code, Codex, Cursor, Gemini CLI, +/// OpenClaw, and Pi exist. +/// Returns `(claude_present, codex_present, cursor_present, gemini_present, openclaw_present, pi_present)`. +fn detect_present_hosts() -> (bool, bool, bool, bool, bool, bool) { let home = std::env::var_os("USERPROFILE") .filter(|v| !v.is_empty()) .or_else(|| std::env::var_os("HOME").filter(|v| !v.is_empty())) @@ -1730,11 +1876,15 @@ fn detect_present_hosts() -> (bool, bool, bool, bool) { let home = match home { Some(h) => h, - None => return (false, false, false, false), + None => return (false, false, false, false, false, false), }; let claude_present = home.join(".claude").is_dir(); let codex_present = home.join(".codex").is_dir(); + // Cursor: global config lives in ~/.cursor + let cursor_present = home.join(".cursor").is_dir(); + // Gemini CLI: global config lives in ~/.gemini + let gemini_present = home.join(".gemini").is_dir(); #[cfg(feature = "openclaw")] let openclaw_present = home.join(".openclaw").is_dir(); #[cfg(not(feature = "openclaw"))] @@ -1743,7 +1893,14 @@ fn detect_present_hosts() -> (bool, bool, bool, bool) { let pi_present = home.join(".pi").is_dir(); #[cfg(not(feature = "pi"))] let pi_present = false; - (claude_present, codex_present, openclaw_present, pi_present) + ( + claude_present, + codex_present, + cursor_present, + gemini_present, + openclaw_present, + pi_present, + ) } /// `kimetsu setup` — one-command onboarding. @@ -1797,13 +1954,22 @@ fn setup_cmd(args: SetupArgs) -> KimetsuResult<()> { // ── Step 2: Choose host(s) ──────────────────────────────────────────────── println!(); println!("[2/4] Selecting host(s)..."); - let (present_claude, present_codex, present_openclaw, present_pi) = detect_present_hosts(); + let ( + present_claude, + present_codex, + present_cursor, + present_gemini, + present_openclaw, + present_pi, + ) = detect_present_hosts(); let is_tty = io::stdin().is_terminal(); let stdin = io::stdin(); let hosts = resolve_setup_hosts( args.host.as_deref(), present_claude, present_codex, + present_cursor, + present_gemini, present_openclaw, present_pi, is_tty, @@ -1845,6 +2011,8 @@ fn setup_cmd(args: SetupArgs) -> KimetsuResult<()> { BridgeTarget::ClaudeCode => "Claude Code", BridgeTarget::Codex => "Codex", BridgeTarget::Kimetsu => "Kimetsu", + BridgeTarget::Cursor => "Cursor", + BridgeTarget::GeminiCli => "Gemini CLI", #[cfg(feature = "openclaw")] BridgeTarget::OpenClaw => "OpenClaw", #[cfg(feature = "pi")] @@ -2002,6 +2170,8 @@ fn setup_cmd(args: SetupArgs) -> KimetsuResult<()> { BridgeTarget::ClaudeCode => "Claude Code", BridgeTarget::Codex => "Codex", BridgeTarget::Kimetsu => "Kimetsu", + BridgeTarget::Cursor => "Cursor", + BridgeTarget::GeminiCli => "Gemini CLI", #[cfg(feature = "openclaw")] BridgeTarget::OpenClaw => "OpenClaw", #[cfg(feature = "pi")] @@ -2396,6 +2566,8 @@ fn plugin(command: PluginCommand) -> KimetsuResult<()> { BridgeTarget::ClaudeCode => "Claude Code", BridgeTarget::Codex => "Codex", BridgeTarget::Kimetsu => "Kimetsu", + BridgeTarget::Cursor => "Cursor", + BridgeTarget::GeminiCli => "Gemini CLI", #[cfg(feature = "openclaw")] BridgeTarget::OpenClaw => "OpenClaw", #[cfg(feature = "pi")] @@ -3346,6 +3518,10 @@ fn brain(command: BrainCommand) -> KimetsuResult<()> { BrainCommand::Daemon(args) => brain_daemon(args), BrainCommand::Eval(args) => brain_eval(args), BrainCommand::Bench(args) => brain_bench(args), + BrainCommand::Roi(args) => brain_roi(args), + BrainCommand::Tune(args) => brain_tune(args), + BrainCommand::Consolidate(args) => brain_consolidate(args), + BrainCommand::Triage(args) => brain_triage(args), } } @@ -3715,7 +3891,8 @@ fn brain_export(args: BrainExportArgs) -> KimetsuResult<()> { }) .transpose()?; - let memories = project::export_memories(&workspace, scope, kind)?; + let memories = + project::export_memories(&workspace, scope, kind, args.redact, args.redact_tags)?; let json = serde_json::to_string_pretty(&memories) .map_err(|e| format!("brain export: failed to serialize: {e}"))?; @@ -4300,6 +4477,922 @@ fn brain_insights( Ok(()) } +/// v1.5: `kimetsu brain roi` — ROI ledger. +fn brain_roi(args: RoiArgs) -> KimetsuResult<()> { + use kimetsu_brain::roi::{RoiWindow, roi_report}; + + let workspace = args + .workspace + .unwrap_or_else(|| env::current_dir().unwrap_or_default()); + + let window = RoiWindow::parse(&args.window) + .map_err(|e| -> Box { e.into() })?; + + let (_paths, config, conn) = kimetsu_brain::project::load_project_readonly(&workspace)?; + let report = roi_report( + &conn, + window, + &config.model.model, + config.model.price_per_mtok, + )?; + + if args.json { + println!("{}", serde_json::to_string_pretty(&report)?); + return Ok(()); + } + + // Human output. + let window_label = match report.window_days { + Some(d) => format!("last {d} days"), + None => "all time".to_string(), + }; + println!("── ROI Ledger ({window_label}) ────────────────────────"); + println!(" served events: {}", report.served_events); + println!(" citations: {}", report.citations); + println!( + " injected tokens: {}", + format_token_count(report.injected_tokens) + ); + println!( + " est. saved tokens: {}", + format_token_count(report.estimated_saved_tokens) + ); + let net_sign = if report.net_tokens >= 0 { "+" } else { "" }; + println!(" net tokens: {net_sign}{}", report.net_tokens); + + if let Some(ref usd) = report.usd { + println!( + "── USD ({} $/MTok) ─────────────────────────────", + { + // Reverse-lookup the price to show it. + kimetsu_brain::roi::resolve_price_per_mtok( + &config.model.model, + config.model.price_per_mtok, + ) + .map(|p| format!("{p:.2}")) + .unwrap_or_else(|| "?".to_string()) + } + ); + println!(" saved: ${:.4}", usd.saved); + println!(" spent: ${:.4}", usd.spent); + let net_usd_sign = if usd.net >= 0.0 { "+" } else { "" }; + println!(" net: {net_usd_sign}${:.4}", usd.net); + } + + // Verdict line. + println!("──────────────────────────────────────────────"); + if report.citations == 0 { + println!( + " No retrieval activity recorded yet — the ledger starts \ + counting as you work." + ); + } else if report.net_tokens >= 0 { + match &report.usd { + Some(u) if u.net >= 0.0 => println!( + " Net positive: kimetsu saved you ~{} tokens (~${:.4}) this window.", + format_token_count(report.estimated_saved_tokens), + u.net, + ), + _ => println!( + " Net positive: kimetsu saved you ~{} tokens this window.", + format_token_count(report.estimated_saved_tokens), + ), + } + } else { + // Honest negative. + match &report.usd { + Some(u) => println!( + " Net negative: brain overhead exceeded savings by ~{} tokens (~${:.4}) this window.", + format_token_count( + report + .injected_tokens + .saturating_sub(report.estimated_saved_tokens) + ), + (u.spent - u.saved).abs(), + ), + None => println!( + " Net negative: brain overhead exceeded savings by ~{} tokens this window.", + format_token_count( + report + .injected_tokens + .saturating_sub(report.estimated_saved_tokens) + ), + ), + } + } + + Ok(()) +} + +/// v1.5: `kimetsu brain tune` — personal eval readiness + optional sweep. +fn brain_tune(args: TuneArgs) -> KimetsuResult<()> { + use kimetsu_brain::tuneset::build_personal_eval; + + let workspace = args + .workspace + .clone() + .unwrap_or_else(|| env::current_dir().unwrap_or_default()); + let paths = kimetsu_core::paths::ProjectPaths::discover(&workspace)?; + let (_paths2, _config, conn) = kimetsu_brain::project::load_project_readonly(&workspace)?; + + if args.revert { + return brain_tune_revert(&workspace); + } + + let eval = build_personal_eval(&conn, 1800).map_err(|e| format!("build_personal_eval: {e}"))?; + + let positive_count = eval.cases.len(); + let noise_count = eval.noise_count; + + let readiness = if positive_count >= 30 { + "READY — enough cases for a meaningful sweep." + } else { + "accumulating — synthetic fixture will be used for the sweep (< 30 positive cases)." + }; + + // Coverage by memory kind (from relevant memory ids). + let kind_coverage = kind_coverage_from_eval(&conn, &eval.cases); + + println!("=== kimetsu brain tune --status ==="); + println!("Positive cases (query + ≥1 cited memory): {positive_count}"); + println!("Noise entries (served, no citation): {noise_count}"); + if let Some(o) = &eval.oldest { + println!("Oldest positive case: {o}"); + } + if let Some(n) = &eval.newest { + println!("Newest positive case: {n}"); + } + println!(); + println!("Coverage by memory kind:"); + for (kind, count) in &kind_coverage { + println!(" {kind:<22} {count}"); + } + println!(); + println!("Readiness: {readiness}"); + + if args.status { + return Ok(()); + } + + // Sweep (or dry-run report). + brain_tune_sweep(&workspace, &paths, args, eval) +} + +fn kind_coverage_from_eval( + conn: &rusqlite::Connection, + cases: &[kimetsu_brain::eval::EvalCase], +) -> Vec<(String, usize)> { + use std::collections::HashMap; + let mut counts: HashMap = HashMap::new(); + for case in cases { + for mid in &case.relevant { + let kind: Option = conn + .query_row( + "SELECT kind FROM memories WHERE memory_id = ?1", + rusqlite::params![mid], + |r| r.get(0), + ) + .ok(); + let kind = kind.unwrap_or_else(|| "unknown".to_string()); + *counts.entry(kind).or_default() += 1; + } + } + let mut vec: Vec<(String, usize)> = counts.into_iter().collect(); + vec.sort_by_key(|a| std::cmp::Reverse(a.1)); + vec +} + +fn brain_tune_sweep( + workspace: &std::path::Path, + paths: &kimetsu_core::paths::ProjectPaths, + args: TuneArgs, + eval: kimetsu_brain::tuneset::PersonalEval, +) -> KimetsuResult<()> { + use kimetsu_brain::context::{ContextRequest, rerank_capsules}; + use kimetsu_brain::embeddings::{open_embedder_for, open_reranker_for_model}; + use kimetsu_brain::eval::{mean, mrr}; + use kimetsu_brain::project::BrainSession; + use kimetsu_brain::tune::{ + ComboResult, TuneCombo, TuneHistoryEntry, append_tune_history, compute_objective, + select_winner, train_holdout_split, + }; + use std::collections::HashMap; + use time::format_description::well_known::Rfc3339; + + let config = project::load_config(paths)?; + // Tune against the PRODUCTION retrieval pipeline: the same embedder + // resolution as retrieve_context_with_request. On embeddings builds this + // loads the real model (semantic floors only discriminate with real + // cosines); lean builds degrade to Noop and sweep FTS-only — the status + // output should make that visible to the user. + let embedder = open_embedder_for(config.embedder.enabled); + if embedder.is_noop() { + println!( + "note: lean build/embedder disabled — sweeping FTS-only retrieval \ + (semantic floor values will not differentiate)" + ); + } + let current_combo = TuneCombo { + min_lexical_coverage: config.broker.min_lexical_coverage, + min_semantic_score: config.broker.min_semantic_score, + reranker_id: config.embedder.reranker.clone(), + }; + + // Choose eval cases: personal if READY, else fall back to fixture. + let fallback_fixture_path = std::path::Path::new("fixtures/eval-retrieval.json"); + let (cases, using_personal) = if eval.cases.len() >= 30 { + (eval.cases.clone(), true) + } else { + // Load the committed fixture. + if !fallback_fixture_path.exists() { + println!( + "note: fewer than 30 personal eval cases ({}) and no fixture at {}. \ + Sweep skipped. Accumulate more sessions with store_queries=true.", + eval.cases.len(), + fallback_fixture_path.display() + ); + return Ok(()); + } + let text = std::fs::read_to_string(fallback_fixture_path) + .map_err(|e| format!("read fixture: {e}"))?; + let fixture: kimetsu_brain::eval::EvalFixture = + serde_json::from_str(&text).map_err(|e| format!("parse fixture: {e}"))?; + // Fixture uses key-based relevance, not memory_ids. For the sweep + // we need memory_ids. We cannot map them here (fixture is hermetic). + // Instead: use fixture cases as-is for MRR calculation but note that + // relevant ids won't match real DB memories → MRR will be 0. + // The sweep is still meaningful for comparing COMBOS relatively. + let eval_cases: Vec = fixture + .cases + .into_iter() + .map(|c| kimetsu_brain::eval::EvalCase { + query: c.query, + relevant: c.relevant, + }) + .collect(); + (eval_cases, false) + }; + + if !using_personal { + println!( + "note: fewer than 30 personal eval cases ({}). Using fixture file for relative sweep.", + eval.cases.len() + ); + // Fix 3: guard --apply behind personal data. + // In fixture mode MRR≡0 for every combo (fixture IDs don't match real + // memories), so the objective degenerates to pure token-minimisation. + // Applying the resulting floors would optimise for fewer tokens at the + // cost of recall. Refuse --apply until the user has ≥30 cited cases. + if args.apply { + println!( + "note: fixture mode is relative-only — --apply refused. \ + Accumulate ≥30 cited cases first (see `kimetsu brain tune --status`)." + ); + return Ok(()); + } + } + + let n = cases.len(); + if n == 0 { + println!("No eval cases available. Run more sessions with store_queries=true."); + return Ok(()); + } + + let (train_idx, holdout_idx) = train_holdout_split(n); + let train_cases: Vec<&kimetsu_brain::eval::EvalCase> = + train_idx.iter().map(|&i| &cases[i]).collect(); + let holdout_cases: Vec<&kimetsu_brain::eval::EvalCase> = + holdout_idx.iter().map(|&i| &cases[i]).collect(); + + println!( + "Sweep: {} combos × {} train / {} holdout cases", + kimetsu_brain::tune::TuneCombo::all_combos().len(), + train_cases.len(), + holdout_cases.len() + ); + + // Cache reranker handles (load once, reuse). + let mut reranker_cache: HashMap>> = + HashMap::new(); + for rr_id in kimetsu_brain::tune::RERANKER_IDS { + let rr: Option> = if *rr_id == "off" { + None + } else { + open_reranker_for_model(rr_id) + }; + reranker_cache.insert(rr_id.to_string(), rr); + } + + // Helper: evaluate one combo over a slice of cases. + let evaluate_cases = + |combo: &TuneCombo, case_slice: &[&kimetsu_brain::eval::EvalCase]| -> (f64, f64) { + let session = match BrainSession::open_readonly(workspace) { + Ok(s) => s, + Err(_) => return (0.0, 0.0), + }; + let rr_ref = reranker_cache + .get(&combo.reranker_id) + .and_then(|r| r.as_deref()); + let rerank_floor = 0.30f32; + let rerank_cap = 4usize; + let pool = 8usize; + + let mut mrr_vals: Vec = Vec::new(); + let mut token_vals: Vec = Vec::new(); + + for case in case_slice { + let request = ContextRequest { + stage: "localization".to_string(), + query: case.query.clone(), + budget_tokens: 6000, + max_capsules: pool, + min_semantic_score: combo.min_semantic_score, + min_lexical_coverage: combo.min_lexical_coverage, + ..Default::default() + }; + let mut bundle = + match session.retrieve_context_with_injected_embedder(request, embedder) { + Ok(b) => b, + Err(_) => continue, + }; + if let Some(rr) = rr_ref { + bundle.capsules = + rerank_capsules(&case.query, bundle.capsules, rr, rerank_floor, rerank_cap); + } + + let ranked_ids: Vec = bundle + .capsules + .iter() + .filter_map(|c| { + c.expansion_handle + .strip_prefix("memory:") + .map(str::to_string) + }) + .collect(); + + let mrr_val = mrr(&ranked_ids, &case.relevant); + mrr_vals.push(mrr_val); + + let tokens: f64 = bundle + .capsules + .iter() + .map(|c| c.token_estimate as f64) + .sum(); + token_vals.push(tokens); + } + + (mean(&mrr_vals), mean(&token_vals)) + }; + + // Evaluate current config on holdout for baseline. + let (baseline_holdout_mrr, baseline_holdout_tokens) = + evaluate_cases(¤t_combo, &holdout_cases); + let baseline_holdout_obj = compute_objective( + baseline_holdout_mrr, + baseline_holdout_tokens, + args.cost_weight, + ); + + // Sweep all combos on TRAIN set. + let all_combos = TuneCombo::all_combos(); + let mut combo_results: Vec = Vec::new(); + + for (i, combo) in all_combos.iter().enumerate() { + if i % 10 == 0 { + print!("\r sweeping combo {}/{} ...", i + 1, all_combos.len()); + use std::io::Write; + let _ = std::io::stdout().flush(); + } + let (mmrr, mtok) = evaluate_cases(combo, &train_cases); + let obj = compute_objective(mmrr, mtok, args.cost_weight); + combo_results.push(ComboResult { + combo: combo.clone(), + mean_mrr: mmrr, + mean_tokens: mtok, + objective: obj, + }); + } + println!(); + + let winner = match select_winner(&combo_results) { + Some(w) => w, + None => { + println!("No combos evaluated. Nothing to tune."); + return Ok(()); + } + }; + + // Evaluate winner on HOLDOUT. + let (holdout_mrr, holdout_tokens) = evaluate_cases(&winner.combo, &holdout_cases); + let holdout_obj = compute_objective(holdout_mrr, holdout_tokens, args.cost_weight); + let improvement = holdout_obj - baseline_holdout_obj; + + println!(); + println!("=== Tune Sweep Results ==="); + println!( + "Current config: lex={:.2} sem={:.3} rr={}", + current_combo.min_lexical_coverage, + current_combo.min_semantic_score, + current_combo.reranker_id + ); + println!( + "Best combo: lex={:.2} sem={:.3} rr={}", + winner.combo.min_lexical_coverage, + winner.combo.min_semantic_score, + winner.combo.reranker_id + ); + println!( + "Train objective: {:.4} (MRR {:.4}, avg_tokens {:.1})", + winner.objective, winner.mean_mrr, winner.mean_tokens + ); + println!( + "Holdout objective: {:.4} vs baseline {:.4} (improvement: {:+.4})", + holdout_obj, baseline_holdout_obj, improvement + ); + + if improvement < 0.01 { + println!(); + println!( + "verdict: no change recommended (holdout improvement {improvement:+.4} < 0.01 threshold)" + ); + return Ok(()); + } + + println!(); + // Reranker change recommendation (never auto-applied). + if winner.combo.reranker_id != current_combo.reranker_id { + println!( + "note: reranker change recommended ({} → {}) — apply manually after \ + downloading the model and restarting the MCP daemon.", + current_combo.reranker_id, winner.combo.reranker_id + ); + } + + if !args.apply { + if !using_personal { + println!( + "note: fixture mode — results are relative only; \ + --apply is disabled until you have ≥30 cited cases." + ); + } + println!( + "DRY RUN — to apply floor changes: kimetsu brain tune --apply\n\ + (floor changes: lex {:.2}→{:.2}, sem {:.3}→{:.3})", + current_combo.min_lexical_coverage, + winner.combo.min_lexical_coverage, + current_combo.min_semantic_score, + winner.combo.min_semantic_score, + ); + return Ok(()); + } + + // --apply: write floors to project.toml (reranker change only recommended). + let mut new_config = project::load_config(paths)?; + new_config.broker.min_lexical_coverage = winner.combo.min_lexical_coverage; + new_config.broker.min_semantic_score = winner.combo.min_semantic_score; + std::fs::write(&paths.project_toml, new_config.to_toml()?)?; + + // Snapshot to tune-history. + let now_str = time::OffsetDateTime::now_utc() + .format(&Rfc3339) + .unwrap_or_else(|_| "unknown".to_string()); + let history_entry = TuneHistoryEntry { + timestamp: now_str, + before: current_combo, + after: winner.combo.clone(), + train_objective: winner.objective, + holdout_objective: holdout_obj, + holdout_mrr, + baseline_holdout_objective: baseline_holdout_obj, + }; + append_tune_history(&paths.kimetsu_dir, history_entry)?; + + println!( + "Applied: lex_coverage={:.2}, sem_score={:.3} → project.toml updated.", + winner.combo.min_lexical_coverage, winner.combo.min_semantic_score + ); + println!("Snaphotted to .kimetsu/tune-history.json"); + + Ok(()) +} + +fn brain_tune_revert(workspace: &std::path::Path) -> KimetsuResult<()> { + use kimetsu_brain::tune::latest_tune_history; + + let paths = kimetsu_core::paths::ProjectPaths::discover(workspace)?; + let Some(entry) = latest_tune_history(&paths.kimetsu_dir)? else { + println!("No tune history found — nothing to revert."); + return Ok(()); + }; + + let mut config = project::load_config(&paths)?; + config.broker.min_lexical_coverage = entry.before.min_lexical_coverage; + config.broker.min_semantic_score = entry.before.min_semantic_score; + std::fs::write(&paths.project_toml, config.to_toml()?)?; + + println!( + "Reverted: lex_coverage={:.2}, sem_score={:.3} (from tune at {})", + entry.before.min_lexical_coverage, entry.before.min_semantic_score, entry.timestamp + ); + Ok(()) +} + +// --------------------------------------------------------------------------- +// Story 3.1 + 3.2: kimetsu brain consolidate +// --------------------------------------------------------------------------- + +fn brain_consolidate(args: ConsolidateArgs) -> KimetsuResult<()> { + use kimetsu_brain::consolidate::{ + ConsolidateOptions, DistillOptions, find_distill_clusters, load_embeddable_rows, + run_consolidation, + }; + + let workspace = args + .workspace + .unwrap_or_else(|| env::current_dir().unwrap_or_default()); + + let (paths, _config, conn) = kimetsu_brain::project::load_project(&workspace)?; + + let stdout = io::stdout(); + let mut out = stdout.lock(); + + // --- Story 3.1: near-duplicate merge --- + // --distill is additive; 3.1 merge always runs alongside it. + { + let opts = ConsolidateOptions { + threshold: args.threshold, + dry_run: args.dry_run, + }; + + if !args.dry_run && !args.yes { + // Check TTY requirement. + if !io::stdin().is_terminal() { + return Err( + "stdin is not a TTY; pass --yes to confirm consolidation non-interactively" + .into(), + ); + } + // Interactive prompt. + write!( + out, + "Consolidate near-duplicate memories (threshold={:.2})? [y/N] ", + args.threshold + )?; + out.flush()?; + let mut line = String::new(); + io::stdin().lock().read_line(&mut line)?; + let answer = line.trim().to_ascii_lowercase(); + if answer != "y" && answer != "yes" { + writeln!(out, "Aborted.")?; + return Ok(()); + } + } + + run_consolidation(&conn, &opts, &mut out)?; + } + + // --- Story 3.2: cluster distillation (--distill flag) --- + if args.distill { + let dopts = DistillOptions::default(); + let by_model = load_embeddable_rows(&conn)?; + let all_rows: Vec<_> = by_model.into_values().flatten().collect(); + let clusters = find_distill_clusters(&all_rows, &dopts); + + if clusters.is_empty() { + writeln!( + out, + "\nNo distillable clusters found (lo={:.2} hi={:.2}, min_size={}).", + dopts.lo, dopts.hi, dopts.min_cluster_size + )?; + return Ok(()); + } + + // Try to resolve a distiller. + let resolved = distiller::resolve_distiller(&workspace); + + if resolved.is_none() || args.dry_run { + writeln!( + out, + "\nDistillable clusters ({} found — lo={:.2} hi={:.2}):", + clusters.len(), + dopts.lo, + dopts.hi + )?; + for (i, cluster) in clusters.iter().enumerate() { + writeln!( + out, + "\nCluster {} [tags: {}]:", + i + 1, + cluster.shared_tags.join(", ") + )?; + for m in &cluster.memories { + writeln!( + out, + " [{}] {}", + m.memory_id, + &m.text[..m.text.len().min(80)] + )?; + } + } + if resolved.is_none() { + writeln!( + out, + "\nNo distiller configured — printed clusters above. Configure [learning.distiller] to auto-distil." + )?; + } + return Ok(()); + } + + // Distiller is available — generate proposals. + let distiller_resolved = resolved.unwrap(); + let mut proposals_created = 0usize; + for cluster in &clusters { + let cluster_text = cluster + .memories + .iter() + .enumerate() + .map(|(i, m)| format!("{}. {}", i + 1, m.text)) + .collect::>() + .join("\n"); + let prompt = format!( + "Distill these {} related lessons into ONE general principle \ + (2-4 sentences, imperative, no project-specific context):\n\n{cluster_text}", + cluster.memories.len() + ); + let mut provider = distiller::make_provider_for_resolved(&distiller_resolved); + if let Some(ref mut p) = provider { + let lessons = distiller::distill_lessons(&prompt, p.as_mut()); + for lesson in lessons { + let result = kimetsu_brain::project::propose_memory( + &distiller_resolved.record_start, + distiller_resolved.scope, + MemoryKind::Convention, + &lesson.lesson, + lesson.confidence.clamp(0.0, 1.0), + &format!( + "distilled from cluster [tags: {}]", + cluster.shared_tags.join(", ") + ), + ); + if result.is_ok() { + proposals_created += 1; + } + } + } + } + + writeln!( + out, + "\nCreated {proposals_created} distillation proposal(s). Review with: kimetsu brain memory proposals" + )?; + drop(paths); + } + + Ok(()) +} + +// --------------------------------------------------------------------------- +// Story 3.3: kimetsu brain triage +// --------------------------------------------------------------------------- + +fn brain_triage(args: TriageArgs) -> KimetsuResult<()> { + let workspace = args + .workspace + .unwrap_or_else(|| env::current_dir().unwrap_or_default()); + + let (_paths, _config, conn) = kimetsu_brain::project::load_project_readonly(&workspace)?; + + let stdout = io::stdout(); + let mut out = stdout.lock(); + let stdin = io::stdin(); + let mut sin = stdin.lock(); + + let candidates = triage_candidates(&conn, args.score_floor, args.age_days)?; + + if candidates.is_empty() { + writeln!( + out, + "No fading memories found (score_floor={:.2}, age_days={}).", + args.score_floor, args.age_days + )?; + return Ok(()); + } + + writeln!( + out, + "{} fading memor{} (score < {:.2}, age > {}d):", + candidates.len(), + if candidates.len() == 1 { "y" } else { "ies" }, + args.score_floor, + args.age_days + )?; + + if args.prune_all { + if !args.yes { + if !io::stdin().is_terminal() { + return Err( + "stdin is not a TTY; pass --yes to confirm --prune-all non-interactively" + .into(), + ); + } + write!(out, "Prune all {} candidates? [y/N] ", candidates.len())?; + out.flush()?; + let mut line = String::new(); + sin.read_line(&mut line)?; + let answer = line.trim().to_ascii_lowercase(); + if answer != "y" && answer != "yes" { + writeln!(out, "Aborted.")?; + return Ok(()); + } + } + let mut pruned = 0usize; + for c in &candidates { + let reason = format!( + "triage_prune score={:.2} age_days={}", + c.usefulness_score, c.age_days + ); + if kimetsu_brain::project::invalidate_memory(&workspace, &c.memory_id, Some(&reason)) + .is_ok() + { + pruned += 1; + } + } + writeln!( + out, + "Pruned {pruned} memor{}.", + if pruned == 1 { "y" } else { "ies" } + )?; + return Ok(()); + } + + // Interactive per-item loop. + if !io::stdin().is_terminal() { + // Non-TTY with no --prune-all: just print the list. + for c in &candidates { + writeln!( + out, + "[{}] {}/{} age={}d score={:.2} — {}", + c.memory_id, + c.scope, + c.kind, + c.age_days, + c.usefulness_score, + &c.text[..c.text.len().min(80)] + )?; + } + writeln!(out, "\nPass --prune-all --yes to prune non-interactively.")?; + return Ok(()); + } + + triage_interactive_loop(&workspace, &candidates, &mut sin, &mut out) +} + +/// A fading memory candidate for triage. +#[derive(Debug)] +struct TriageCandidate { + memory_id: String, + scope: String, + kind: String, + text: String, + age_days: i64, + usefulness_score: f32, +} + +/// Query the DB for triage candidates. +fn triage_candidates( + conn: &rusqlite::Connection, + score_floor: f32, + age_days: u32, +) -> KimetsuResult> { + use rusqlite::params; + use time::OffsetDateTime; + + // Compute the cutoff timestamp. + let now = OffsetDateTime::now_utc(); + let cutoff = now - time::Duration::days(i64::from(age_days)); + let cutoff_str = cutoff + .format(&time::format_description::well_known::Rfc3339) + .unwrap_or_default(); + + let mut stmt = conn.prepare( + "SELECT memory_id, scope, kind, text, usefulness_score, + COALESCE(last_useful_at, created_at) AS ref_ts + FROM memories + WHERE invalidated_at IS NULL + AND superseded_by IS NULL + AND usefulness_score < ?1 + AND COALESCE(last_useful_at, created_at) < ?2 + ORDER BY usefulness_score ASC, COALESCE(last_useful_at, created_at) ASC + LIMIT 200", + )?; + + let rows = stmt.query_map(params![score_floor as f64, cutoff_str], |row| { + Ok(( + row.get::<_, String>(0)?, + row.get::<_, String>(1)?, + row.get::<_, String>(2)?, + row.get::<_, String>(3)?, + row.get::<_, f64>(4)?, + row.get::<_, String>(5)?, + )) + })?; + + let mut candidates = Vec::new(); + for row in rows { + let (memory_id, scope, kind, text, score, ref_ts) = row?; + let age = { + use time::format_description::well_known::Rfc3339; + OffsetDateTime::parse(&ref_ts, &Rfc3339) + .map(|t| (now - t).whole_days().max(0)) + .unwrap_or(0) + }; + candidates.push(TriageCandidate { + memory_id, + scope, + kind, + text, + age_days: age, + usefulness_score: score as f32, + }); + } + Ok(candidates) +} + +/// Interactive decision loop — mirrors the `decide_preflight_action` pattern +/// in update.rs. Generic over BufRead + Write for testability. +fn triage_interactive_loop( + workspace: &std::path::Path, + candidates: &[TriageCandidate], + reader: &mut R, + writer: &mut W, +) -> KimetsuResult<()> { + let mut pruned = 0usize; + let mut kept = 0usize; + let mut skipped = 0usize; + + for c in candidates { + writeln!( + writer, + "\n[{}] {}/{} age={}d score={:.2}", + c.memory_id, c.scope, c.kind, c.age_days, c.usefulness_score + )?; + writeln!(writer, " {}", &c.text[..c.text.len().min(120)])?; + write!(writer, " [k]eep / [p]rune / [s]kip: ")?; + writer.flush()?; + + let mut line = String::new(); + reader.read_line(&mut line)?; + match line.trim().to_ascii_lowercase().as_str() { + "p" | "prune" => { + let reason = format!( + "triage_prune score={:.2} age_days={}", + c.usefulness_score, c.age_days + ); + if kimetsu_brain::project::invalidate_memory(workspace, &c.memory_id, Some(&reason)) + .is_ok() + { + pruned += 1; + writeln!(writer, " → pruned.")?; + } else { + writeln!(writer, " → prune failed.")?; + } + } + "k" | "keep" => { + kept += 1; + writeln!(writer, " → kept.")?; + } + _ => { + skipped += 1; + writeln!(writer, " → skipped.")?; + } + } + } + + writeln!( + writer, + "\nTriage complete: {} pruned, {} kept, {} skipped.", + pruned, kept, skipped + )?; + Ok(()) +} + +/// Format a token count with thousands separator (space). +fn format_token_count(n: u64) -> String { + if n < 1_000 { + return n.to_string(); + } + let s = n.to_string(); + let mut out = String::new(); + let rem = s.len() % 3; + for (i, ch) in s.chars().enumerate() { + if i > 0 && (i % 3 == rem) { + out.push(' '); + } + out.push(ch); + } + out +} + /// v0.6: `kimetsu brain context-hook` — UserPromptSubmit hook. /// Reads `{"prompt":"..."}` JSON from stdin, retrieves relevant capsules, /// prints Codex/Claude-compatible hook JSON to stdout for injection. @@ -4316,17 +5409,32 @@ fn brain_context_hook(args: ContextHookArgs) -> KimetsuResult<()> { let mut input = String::new(); std::io::stdin().read_to_string(&mut input).unwrap_or(0); + // Parse the full hook payload once so we can extract both `prompt` + // and `session_id` (Change A + Change B). + let hook_payload: Option = if input.trim().is_empty() { + None + } else { + serde_json::from_str(input.trim()).ok() + }; + + // Change B: extract session_id — present in Claude Code's + // UserPromptSubmit payload; absent in Codex / plain-text fallbacks. + let session_id: Option = hook_payload + .as_ref() + .and_then(|v| v.get("session_id")) + .and_then(serde_json::Value::as_str) + .filter(|s| !s.trim().is_empty()) + .map(str::to_string); + // Extract the prompt text from the hook payload - let prompt = if input.trim().is_empty() { - String::new() - } else if let Ok(v) = serde_json::from_str::(&input) { - v.get("prompt") + let prompt = match &hook_payload { + Some(v) => v + .get("prompt") .and_then(serde_json::Value::as_str) .unwrap_or("") - .to_string() - } else { - // Plain text fallback - input.trim().to_string() + .to_string(), + None if !input.trim().is_empty() => input.trim().to_string(), // plain-text fallback + None => String::new(), }; // Too short to be meaningful @@ -4357,48 +5465,197 @@ fn brain_context_hook(args: ContextHookArgs) -> KimetsuResult<()> { // logged. Best-effort (let _ =) — telemetry must never break the hook. // Gate behind KIMETSU_BRAIN_LOG_RETRIEVAL=0 opt-out (default ON). if std::env::var("KIMETSU_BRAIN_LOG_RETRIEVAL").as_deref() != Ok("0") { - let top_score = bundle.top_score; - let query_hash = { - use std::collections::hash_map::DefaultHasher; - use std::hash::{Hash, Hasher}; - let mut h = DefaultHasher::new(); - request.query.hash(&mut h); - format!("{:016x}", h.finish()) - }; - let _ = project::log_telemetry_event( - &workspace, - "context.served", - serde_json::json!({ - "query_hash": query_hash, - "capsule_count": bundle.capsules.len(), - "top_score": top_score, - "skipped": bundle.skipped, - "stage": &request.stage, - "retrieval_path": retrieval_path, - }), - ); + // Change A: load store_queries from project config best-effort. + // Telemetry must never break the hook, so any config error just + // falls through to the safe default (true = store the query). + let store_queries = kimetsu_core::paths::ProjectPaths::discover(&workspace) + .ok() + .and_then(|paths| project::load_config(&paths).ok()) + .map(|cfg| cfg.learning.store_queries) + .unwrap_or(true); + + let payload = build_served_event_payload(ServedEventArgs { + query: &request.query, + capsule_count: bundle.capsules.len(), + top_score: bundle.top_score, + skipped: bundle.skipped, + stage: &request.stage, + retrieval_path, + store_queries, + session_id: session_id.as_deref(), + }); + let _ = project::log_telemetry_event(&workspace, "context.served", payload); + } + + // Change C1: capture top-10 dropped MEMORY capsules to the rolling + // sidecar. Best-effort — telemetry must never break the hook. + // We capture AFTER the telemetry event so a slow sidecar write + // doesn't block the event. Only capsules whose expansion_handle + // starts with "memory:" are interesting for regret detection. + { + use kimetsu_brain::dropped_capsule; + let cache_dir = kimetsu_core::paths::ProjectPaths::discover(&workspace) + .ok() + .map(|p| kimetsu_core::paths::user_cache_dir_for(&p.repo_root)); + if let Some(cache_dir) = cache_dir { + let dropped_ids = bundle + .excluded + .iter() + .filter(|c| c.expansion_handle.starts_with("memory:")) + .filter_map(|c| { + c.expansion_handle + .strip_prefix("memory:") + .map(str::to_string) + }) + .take(10); + let now = dropped_capsule::now_secs(); + dropped_capsule::append_dropped(&cache_dir, dropped_ids, now); + } } if bundle.skipped || bundle.capsules.is_empty() { return Ok(()); // Nothing relevant — zero output } + // v1.5: load broker.compress_capsules + broker.session_dedupe best-effort. + // The hook must never fail on config errors — fallback to defaults (both ON). + let (compress_capsules, session_dedupe) = + kimetsu_core::paths::ProjectPaths::discover(&workspace) + .ok() + .and_then(|paths| project::load_config(&paths).ok()) + .map(|cfg| (cfg.broker.compress_capsules, cfg.broker.session_dedupe)) + .unwrap_or((true, true)); + + // v1.5 (Story 2.3): session-scoped cross-turn dedupe. + // Load the proactive-state sidecar (already used by proactive hooks) to + // track which capsule handles were injected earlier this session. + // The context hook has session_id from the hook payload (Change B). + let state_path = kimetsu_core::paths::ProjectPaths::discover(&workspace) + .ok() + .map(|p| { + let cache_dir = kimetsu_core::paths::user_cache_dir_for(&p.repo_root); + proactive_state::session_path(&cache_dir, session_id.as_deref()) + }); + let mut state = state_path + .as_deref() + .map(proactive_state::load) + .unwrap_or_default(); + + // Apply soft dedupe: filter already-surfaced handles, but fall back to the + // full set if filtering would leave nothing (a repeated top memory may still + // be the right context). Uses the pure `dedupe_filter` function. + let capsules_to_render: Vec<_> = if session_dedupe { + let handles: Vec<&str> = bundle + .capsules + .iter() + .map(|c| c.expansion_handle.as_str()) + .collect(); + let indices = proactive_state::dedupe_filter(&handles, &state); + indices.into_iter().map(|i| &bundle.capsules[i]).collect() + } else { + bundle.capsules.iter().collect() + }; + let mut additional_context = String::from("Kimetsu brain relevant knowledge for this task:"); - for capsule in &bundle.capsules { + for capsule in &capsules_to_render { + // v1.5 (Story 2.1): render-time compression — runs AFTER retrieval and + // reranking, purely on the injected text. Full summary untouched in DB. + let rendered: String = if compress_capsules { + kimetsu_brain::context::compress_for_render(&capsule.summary, 3) + } else { + capsule.summary.clone() + }; // Strip the "scope:kind - " prefix from the summary for readability - let text = capsule - .summary + let text = rendered .split(" - ") .nth(1) - .unwrap_or(&capsule.summary); + .map(str::to_string) + .unwrap_or(rendered); additional_context.push('\n'); - additional_context.push_str(text); + additional_context.push_str(&text); } print_user_prompt_submit_context(&additional_context)?; + + // v1.5 (Story 2.3): persist newly surfaced handles so subsequent prompts + // in the same session skip them. Best-effort — state write must never + // break the hook's primary output. + if session_dedupe { + for capsule in &capsules_to_render { + if !capsule.expansion_handle.is_empty() { + state.mark_surfaced(&capsule.expansion_handle); + } + } + if let Some(ref path) = state_path { + proactive_state::save(path, &state); + } + } + Ok(()) } +/// v1.5: inputs for the `context.served` telemetry payload builder. +/// +/// Grouped into a struct to keep [`build_served_event_payload`] under +/// the clippy `too_many_arguments` threshold and to make call-sites +/// self-documenting. +pub struct ServedEventArgs<'a> { + /// Raw retrieval query text. + pub query: &'a str, + /// How many capsules were included in the bundle. + pub capsule_count: usize, + /// Best composite score before the skip check. + pub top_score: f32, + /// True when the top score was below `min_score` (no injection). + pub skipped: bool, + /// Retrieval stage tag (e.g. `"localization"`). + pub stage: &'a str, + /// `"daemon"` or `"fts_fallback"`. + pub retrieval_path: &'a str, + /// When true, include the raw query text in the payload. + /// When false, only the hash is stored (pre-v1.5 behavior). + pub store_queries: bool, + /// Claude Code session id from the hook payload, when available. + /// Codex and plain-text fallbacks may omit it. + pub session_id: Option<&'a str>, +} + +/// v1.5: pure builder for the `context.served` telemetry payload. +/// +/// Extracted so the logic can be unit-tested without hitting the FS or +/// spawning hooks. Always emits `query_hash` for backward compatibility; +/// adds `query` only when `args.store_queries` is true; adds `session_id` +/// only when present. +pub fn build_served_event_payload(args: ServedEventArgs<'_>) -> serde_json::Value { + use std::collections::hash_map::DefaultHasher; + use std::hash::{Hash, Hasher}; + + let mut h = DefaultHasher::new(); + args.query.hash(&mut h); + let query_hash = format!("{:016x}", h.finish()); + + let mut map = serde_json::Map::new(); + map.insert("query_hash".into(), serde_json::json!(query_hash)); + if args.store_queries { + map.insert("query".into(), serde_json::json!(args.query)); + } + map.insert( + "capsule_count".into(), + serde_json::json!(args.capsule_count), + ); + map.insert("top_score".into(), serde_json::json!(args.top_score)); + map.insert("skipped".into(), serde_json::json!(args.skipped)); + map.insert("stage".into(), serde_json::json!(args.stage)); + map.insert( + "retrieval_path".into(), + serde_json::json!(args.retrieval_path), + ); + if let Some(sid) = args.session_id { + map.insert("session_id".into(), serde_json::json!(sid)); + } + serde_json::Value::Object(map) +} + fn print_user_prompt_submit_context(additional_context: &str) -> KimetsuResult<()> { let output = user_prompt_submit_context_output(additional_context); println!("{}", serde_json::to_string(&output)?); @@ -4421,6 +5678,8 @@ fn user_prompt_submit_context_output(additional_context: &str) -> serde_json::Va /// file Claude Code writes) instead of a non-existent inline array, and /// — when nothing was recorded in a non-trivial session — points at the /// memory-harvester subagent. Silent exit for short sessions. +/// v1.5: when the session had ≥1 citation, appends a savings sentence to +/// the `systemMessage` banner. fn brain_stop_hook(args: StopHookArgs) -> KimetsuResult<()> { use std::io::Read; @@ -4457,8 +5716,15 @@ fn brain_stop_hook(args: StopHookArgs) -> KimetsuResult<()> { } }; + // v1.5: compute per-session ROI (best-effort; errors are silently ignored). + let sid = session.get("session_id").and_then(|v| v.as_str()); + let session_savings = compute_stop_hook_savings(&workspace, sid); + if recorded > 0 { - return emit_stop_hook_json(stop_lessons_recorded_json(recorded)); + return emit_stop_hook_json(stop_lessons_recorded_json_with_savings( + recorded, + session_savings.as_deref(), + )); } // Short sessions exit silently — no nagging for quick lookups. The // count is transcript *lines* (user/assistant/tool messages), so the @@ -4484,7 +5750,6 @@ fn brain_stop_hook(args: StopHookArgs) -> KimetsuResult<()> { .map(|c| c.learning.auto_harvest) .unwrap_or(true); let distiller_enabled = distiller::resolve_distiller(&workspace).is_some(); - let sid = session.get("session_id").and_then(|v| v.as_str()); let state_path = paths.as_ref().map(|p| { let cache_dir = kimetsu_core::paths::user_cache_dir_for(&p.repo_root); proactive_state::session_path(&cache_dir, sid) @@ -4526,7 +5791,30 @@ fn brain_stop_hook(args: StopHookArgs) -> KimetsuResult<()> { } } - emit_stop_hook_json(stop_no_lessons_json()) + emit_stop_hook_json(stop_no_lessons_json_with_savings( + session_savings.as_deref(), + )) +} + +/// v1.5: Compute a per-session savings sentence for the Stop hook. +/// +/// Best-effort: returns `None` on any error (DB not found, no data, etc.) +/// so the hook never fails due to ROI computation. +/// +/// Returns `None` also when there are zero citations this session (silence +/// is the correct behavior — we don't dilute the harvest cue). +fn compute_stop_hook_savings(workspace: &Path, session_id: Option<&str>) -> Option { + use kimetsu_brain::roi::session_roi; + + let (paths, config, conn) = kimetsu_brain::project::load_project_readonly(workspace).ok()?; + let _ = paths; // suppress unused warning + let sr = session_roi( + &conn, + session_id, + &config.model.model, + config.model.price_per_mtok, + )?; + Some(sr.savings_sentence()) } /// Emit a Claude Code `Stop`-hook result on stdout. Claude Code validates a @@ -4542,13 +5830,29 @@ fn emit_stop_hook_json(value: serde_json::Value) -> KimetsuResult<()> { /// User-facing banner confirming how many lessons were recorded. Surfaced via /// `systemMessage` (shown to the user; it does not re-enter the model). +/// Kept for test compatibility; production code uses `_with_savings` directly. +#[cfg_attr(not(test), allow(dead_code))] fn stop_lessons_recorded_json(recorded: usize) -> serde_json::Value { - serde_json::json!({ - "systemMessage": format!( - "[Kimetsu] {recorded} lesson{} recorded this session.", - if recorded == 1 { "" } else { "s" } - ), - }) + stop_lessons_recorded_json_with_savings(recorded, None) +} + +/// v1.5: lessons-recorded banner with optional savings sentence appended. +/// When `savings` is `Some`, it is appended after the lessons line. +/// The original `stop_lessons_recorded_json` delegates here so existing tests +/// continue to pass unchanged. +fn stop_lessons_recorded_json_with_savings( + recorded: usize, + savings: Option<&str>, +) -> serde_json::Value { + let base = format!( + "[Kimetsu] {recorded} lesson{} recorded this session.", + if recorded == 1 { "" } else { "s" } + ); + let msg = match savings { + Some(s) => format!("{base} {s}"), + None => base, + }; + serde_json::json!({ "systemMessage": msg }) } /// The end-of-session harvest cue. Uses `decision: "block"` so the cue text @@ -4567,11 +5871,21 @@ fn stop_harvest_cue_json() -> serde_json::Value { /// User-facing fallback nudge when nothing was recorded and the harvest cue /// path did not fire. Informational only, so it uses `systemMessage`. +/// Kept for test compatibility; production code uses `_with_savings` directly. +#[cfg_attr(not(test), allow(dead_code))] fn stop_no_lessons_json() -> serde_json::Value { - serde_json::json!({ - "systemMessage": - "[Kimetsu] No lessons recorded. After non-trivial solutions, call kimetsu_brain_record.", - }) + stop_no_lessons_json_with_savings(None) +} + +/// v1.5: no-lessons nudge with optional savings sentence appended. +fn stop_no_lessons_json_with_savings(savings: Option<&str>) -> serde_json::Value { + let base = + "[Kimetsu] No lessons recorded. After non-trivial solutions, call kimetsu_brain_record."; + let msg = match savings { + Some(s) => format!("{base} {s}"), + None => base.to_string(), + }; + serde_json::json!({ "systemMessage": msg }) } /// The end-of-session harvest cue fires only when auto-harvest is on AND @@ -4714,13 +6028,16 @@ fn proactive_hook(event: ProactiveEvent, args: ProactiveHookArgs) -> KimetsuResu }; // Honor the configured embedder id for consistency (proactive // retrieval is lexical-only, but this keeps labels coherent). Also - // capture the auto-harvest toggle for the resolution cue below. - let auto_harvest = match project::load_config(&paths) { + // capture the auto-harvest toggle and render flags. + let (auto_harvest, compress_capsules) = match project::load_config(&paths) { Ok(config) => { kimetsu_brain::embeddings::apply_embedder_selection(Some(&config.embedder.model)); - config.learning.auto_harvest + ( + config.learning.auto_harvest, + config.broker.compress_capsules, + ) } - Err(_) => true, + Err(_) => (true, true), }; let mut input = String::new(); @@ -4844,11 +6161,18 @@ fn proactive_hook(event: ProactiveEvent, args: ProactiveHookArgs) -> KimetsuResu return Ok(()); }; - let body = capsule - .summary + // v1.5 (Story 2.1): render-time compression for the proactive hook. + // Runs AFTER retrieval — ranking and stored text are unaffected. + let rendered: String = if compress_capsules { + kimetsu_brain::context::compress_for_render(&capsule.summary, 3) + } else { + capsule.summary.clone() + }; + let body = rendered .split(" - ") .nth(1) - .unwrap_or(&capsule.summary); + .map(str::to_string) + .unwrap_or(rendered); let header = proactive_header(event, loop_mode); let additional_context = format!("{header}\n{body}"); @@ -6629,6 +7953,12 @@ fn brain_bench_orchestrate(args: BrainBenchArgs) -> KimetsuResult<()> { mean_latency_ms: f64, p95_latency_ms: f64, noise_capsules: f64, + /// v1.5 (Story 2.1): mean rendered tokens per capsule after compression. + #[serde(default)] + rendered_tokens_mean: f64, + /// v1.5 (Story 2.1): mean raw (uncompressed) tokens per capsule. + #[serde(default)] + raw_tokens_mean: f64, } #[derive(serde::Deserialize)] struct ComboResult { @@ -6667,7 +7997,7 @@ fn brain_bench_orchestrate(args: BrainBenchArgs) -> KimetsuResult<()> { // Build summary table. let header = format!( - "| {:<25} | {:<35} | {:>8} | {:>8} | {:>7} | {:>8} | {:>7} | {:>10} | {:>15} | {:>11} |", + "| {:<25} | {:<35} | {:>8} | {:>8} | {:>7} | {:>8} | {:>7} | {:>10} | {:>15} | {:>11} | {:>12} | {:>14} |", "embedder", "reranker", "recall@2", @@ -6677,11 +8007,13 @@ fn brain_bench_orchestrate(args: BrainBenchArgs) -> KimetsuResult<()> { "p95 ms", "noise_caps", "load ms (emb+rr)", - "peak RSS MB" + "peak RSS MB", + "raw_tok_mean", + "rend_tok_mean", ); let sep = format!( - "| {:-<25} | {:-<35} | {:-<8} | {:-<8} | {:-<7} | {:-<8} | {:-<7} | {:-<10} | {:-<15} | {:-<11} |", - "", "", "", "", "", "", "", "", "", "" + "| {:-<25} | {:-<35} | {:-<8} | {:-<8} | {:-<7} | {:-<8} | {:-<7} | {:-<10} | {:-<15} | {:-<11} | {:-<12} | {:-<14} |", + "", "", "", "", "", "", "", "", "", "", "", "" ); let mut table_lines: Vec = vec![header, sep]; @@ -6692,7 +8024,7 @@ fn brain_bench_orchestrate(args: BrainBenchArgs) -> KimetsuResult<()> { .map(|v| format!("{v:.0}")) .unwrap_or_else(|| "n/a".to_string()); table_lines.push(format!( - "| {:<25} | {:<35} | {:>8.3} | {:>8.3} | {:>7.3} | {:>8.1} | {:>7.1} | {:>10.1} | {:>15} | {:>11} |", + "| {:<25} | {:<35} | {:>8.3} | {:>8.3} | {:>7.3} | {:>8.1} | {:>7.1} | {:>10.1} | {:>15} | {:>11} | {:>12.1} | {:>14.1} |", row.embedder, row.reranker, row.summary.recall_at_2, @@ -6703,6 +8035,8 @@ fn brain_bench_orchestrate(args: BrainBenchArgs) -> KimetsuResult<()> { row.summary.noise_capsules, load_ms, rss_str, + row.summary.raw_tokens_mean, + row.summary.rendered_tokens_mean, )); } @@ -7544,6 +8878,10 @@ fn brain_bench_single(args: BrainBenchArgs) -> KimetsuResult<()> { hit_at_4: bool, mrr: f64, latency_ms: u128, + /// v1.5 (Story 2.1): mean rendered tokens across the returned capsules + /// after compress_for_render(3) vs raw token estimates. + raw_tokens_mean: f64, + rendered_tokens_mean: f64, } let mut case_results: Vec = Vec::new(); @@ -7615,6 +8953,28 @@ fn brain_bench_single(args: BrainBenchArgs) -> KimetsuResult<()> { let mrr_val = kimetsu_brain::eval::mrr(&obtained_keys, &case.relevant); + // v1.5 (Story 2.1): token estimates — raw vs compressed — for the + // rendered capsule set. Computed per-case, averaged in the summary. + let (raw_tokens_mean, rendered_tokens_mean) = { + use kimetsu_brain::context::{compress_for_render, estimate_tokens}; + let n = bundle.capsules.len(); + if n == 0 { + (0.0, 0.0) + } else { + let raw: u32 = bundle + .capsules + .iter() + .map(|c| estimate_tokens(&c.summary)) + .sum(); + let rendered: u32 = bundle + .capsules + .iter() + .map(|c| estimate_tokens(&compress_for_render(&c.summary, 3))) + .sum(); + (raw as f64 / n as f64, rendered as f64 / n as f64) + } + }; + case_results.push(CaseResult { query: case.query.clone(), expected: case.relevant.clone(), @@ -7623,6 +8983,8 @@ fn brain_bench_single(args: BrainBenchArgs) -> KimetsuResult<()> { hit_at_4, mrr: mrr_val, latency_ms, + raw_tokens_mean, + rendered_tokens_mean, }); } @@ -7690,6 +9052,18 @@ fn brain_bench_single(args: BrainBenchArgs) -> KimetsuResult<()> { let peak = peak_rss_mb(); + // v1.5 (Story 2.1): aggregate rendered-token means across all cases. + let (agg_raw_tokens_mean, agg_rendered_tokens_mean) = { + let n = case_results.len(); + if n == 0 { + (0.0, 0.0) + } else { + let raw_sum: f64 = case_results.iter().map(|r| r.raw_tokens_mean).sum(); + let rend_sum: f64 = case_results.iter().map(|r| r.rendered_tokens_mean).sum(); + (raw_sum / n as f64, rend_sum / n as f64) + } + }; + // ── 7. Write combo JSON ─────────────────────────────────────────────────── let combo_json = serde_json::json!({ "embedder": embedder_id, @@ -7710,6 +9084,9 @@ fn brain_bench_single(args: BrainBenchArgs) -> KimetsuResult<()> { "mean_latency_ms": mean_latency_ms, "p95_latency_ms": p95_latency_ms, "noise_capsules": noise_capsules, + // v1.5 (Story 2.1): token-budget intelligence + "raw_tokens_mean": agg_raw_tokens_mean, + "rendered_tokens_mean": agg_rendered_tokens_mean, } }); @@ -8110,6 +9487,64 @@ mod tests { ); } + // ── v1.5 Stop-hook savings sentence tests ──────────────────────────────── + + #[test] + fn stop_lessons_recorded_with_savings_appends_sentence() { + let v = + stop_lessons_recorded_json_with_savings(2, Some("[Kimetsu] Brain saved ~500 tokens.")); + let msg = v["systemMessage"].as_str().unwrap(); + assert!(msg.contains("2 lessons recorded"), "{msg}"); + assert!(msg.contains("Brain saved"), "{msg}"); + } + + #[test] + fn stop_lessons_recorded_without_savings_unchanged() { + let with = stop_lessons_recorded_json_with_savings(1, None); + let without = stop_lessons_recorded_json(1); + assert_eq!( + with["systemMessage"].as_str().unwrap(), + without["systemMessage"].as_str().unwrap(), + "None savings must produce identical output" + ); + } + + #[test] + fn stop_no_lessons_with_savings_appends_sentence() { + let v = stop_no_lessons_json_with_savings(Some("[Kimetsu] Brain saved ~200 tokens.")); + let msg = v["systemMessage"].as_str().unwrap(); + assert!(msg.contains("No lessons recorded"), "{msg}"); + assert!(msg.contains("Brain saved"), "{msg}"); + } + + #[test] + fn stop_no_lessons_without_savings_unchanged() { + let with = stop_no_lessons_json_with_savings(None); + let without = stop_no_lessons_json(); + assert_eq!( + with["systemMessage"].as_str().unwrap(), + without["systemMessage"].as_str().unwrap(), + "None savings must produce identical output" + ); + } + + #[test] + fn stop_hook_with_savings_outputs_are_valid_json_objects() { + for value in [ + stop_lessons_recorded_json_with_savings(1, Some("savings.")), + stop_no_lessons_json_with_savings(Some("savings.")), + ] { + let serialized = serde_json::to_string(&value).expect("serializes"); + let reparsed: serde_json::Value = + serde_json::from_str(&serialized).expect("round-trips"); + assert!(reparsed.is_object(), "must be a JSON object"); + assert!( + reparsed["systemMessage"].is_string(), + "must have systemMessage string" + ); + } + } + // ── D2a: config_edit_with ───────────────────────────────────────────────── fn test_project_root(label: &str) -> std::path::PathBuf { @@ -8971,6 +10406,8 @@ ambient = false false, false, false, + false, + false, Cursor::new(b""), ) .unwrap(); @@ -8987,6 +10424,8 @@ ambient = false false, false, false, + false, + false, Cursor::new(b""), ) .unwrap(); @@ -8997,32 +10436,72 @@ ambient = false fn resolve_setup_hosts_auto_only_claude_present() { use kimetsu_chat::BridgeTarget; // Only Claude present → Claude. - let hosts = - resolve_setup_hosts(None, true, false, false, false, false, Cursor::new(b"")).unwrap(); + let hosts = resolve_setup_hosts( + None, + true, + false, + false, + false, + false, + false, + false, + Cursor::new(b""), + ) + .unwrap(); assert_eq!(hosts, vec![BridgeTarget::ClaudeCode]); } #[test] fn resolve_setup_hosts_auto_only_codex_present() { use kimetsu_chat::BridgeTarget; - let hosts = - resolve_setup_hosts(None, false, true, false, false, false, Cursor::new(b"")).unwrap(); + let hosts = resolve_setup_hosts( + None, + false, + true, + false, + false, + false, + false, + false, + Cursor::new(b""), + ) + .unwrap(); assert_eq!(hosts, vec![BridgeTarget::Codex]); } #[test] fn resolve_setup_hosts_auto_both_present() { use kimetsu_chat::BridgeTarget; - let hosts = - resolve_setup_hosts(None, true, true, false, false, false, Cursor::new(b"")).unwrap(); + let hosts = resolve_setup_hosts( + None, + true, + true, + false, + false, + false, + false, + false, + Cursor::new(b""), + ) + .unwrap(); assert_eq!(hosts, vec![BridgeTarget::ClaudeCode, BridgeTarget::Codex]); } #[test] fn resolve_setup_hosts_neither_present_non_tty_defaults_claude() { use kimetsu_chat::BridgeTarget; - let hosts = - resolve_setup_hosts(None, false, false, false, false, false, Cursor::new(b"")).unwrap(); + let hosts = resolve_setup_hosts( + None, + false, + false, + false, + false, + false, + false, + false, + Cursor::new(b""), + ) + .unwrap(); assert_eq!(hosts, vec![BridgeTarget::ClaudeCode]); } @@ -9036,6 +10515,8 @@ ambient = false false, false, false, + false, + false, true, Cursor::new(b"codex\n"), ) @@ -9052,6 +10533,8 @@ ambient = false false, false, false, + false, + false, Cursor::new(b""), ); assert!(result.is_err(), "bad --host should return Err"); @@ -9100,6 +10583,8 @@ ambient = false false, false, false, + false, + false, true, Cursor::new(b"both\n"), ) @@ -9111,8 +10596,18 @@ ambient = false #[test] fn resolve_setup_hosts_auto_only_pi_present() { use kimetsu_chat::BridgeTarget; - let hosts = - resolve_setup_hosts(None, false, false, false, true, false, Cursor::new(b"")).unwrap(); + let hosts = resolve_setup_hosts( + None, + false, + false, + false, + false, + false, + true, + false, + Cursor::new(b""), + ) + .unwrap(); assert_eq!(hosts, vec![BridgeTarget::Pi]); } @@ -9127,6 +10622,8 @@ ambient = false false, false, false, + false, + false, Cursor::new(b""), ) .unwrap(); @@ -9137,9 +10634,18 @@ ambient = false #[test] fn resolve_setup_hosts_tty_scripted_pi() { use kimetsu_chat::BridgeTarget; - let hosts = - resolve_setup_hosts(None, false, false, false, false, true, Cursor::new(b"pi\n")) - .unwrap(); + let hosts = resolve_setup_hosts( + None, + false, + false, + false, + false, + false, + false, + true, + Cursor::new(b"pi\n"), + ) + .unwrap(); assert_eq!(hosts, vec![BridgeTarget::Pi]); } @@ -9148,8 +10654,18 @@ ambient = false fn resolve_setup_hosts_auto_only_openclaw_present() { use kimetsu_chat::BridgeTarget; // Only OpenClaw present → OpenClaw detected. - let hosts = - resolve_setup_hosts(None, false, false, true, false, false, Cursor::new(b"")).unwrap(); + let hosts = resolve_setup_hosts( + None, + false, + false, + false, + false, + true, + false, + false, + Cursor::new(b""), + ) + .unwrap(); assert_eq!(hosts, vec![BridgeTarget::OpenClaw]); } @@ -9164,6 +10680,8 @@ ambient = false false, false, false, + false, + false, Cursor::new(b""), ) .unwrap(); @@ -9181,6 +10699,8 @@ ambient = false false, false, false, + false, + false, Cursor::new(b""), ) .unwrap(); @@ -9216,6 +10736,8 @@ ambient = false false, false, false, + false, + false, true, Cursor::new(b"openclaw\n"), ) @@ -9369,4 +10891,267 @@ ambient = false assert!(!bundle.skipped); assert!((bundle.top_score - 0.9).abs() < 1e-6); } + + // ── build_served_event_payload unit tests (Changes A + B) ─────────────── + + fn make_payload(store_queries: bool, session_id: Option<&str>) -> serde_json::Value { + build_served_event_payload(ServedEventArgs { + query: "what is the answer to life", + capsule_count: 3, + top_score: 0.72, + skipped: false, + stage: "localization", + retrieval_path: "daemon", + store_queries, + session_id, + }) + } + + #[test] + fn served_event_payload_always_includes_query_hash() { + let payload = make_payload(false, None); + assert!( + payload.get("query_hash").is_some(), + "query_hash must always be present" + ); + // When store_queries is false, raw query must be absent. + assert!( + payload.get("query").is_none(), + "query must be absent when store_queries=false" + ); + } + + #[test] + fn served_event_payload_includes_raw_query_when_store_queries_true() { + let query = "how does the embedding daemon flush its cache"; + let payload = build_served_event_payload(ServedEventArgs { + query, + capsule_count: 5, + top_score: 0.85, + skipped: false, + stage: "implementation", + retrieval_path: "daemon", + store_queries: true, + session_id: None, + }); + assert_eq!( + payload.get("query").and_then(|v| v.as_str()), + Some(query), + "raw query must be present when store_queries=true" + ); + } + + #[test] + fn served_event_payload_includes_session_id_when_present() { + let payload = make_payload(true, Some("ses-abc-123")); + assert_eq!( + payload.get("session_id").and_then(|v| v.as_str()), + Some("ses-abc-123"), + "session_id must appear when provided" + ); + } + + #[test] + fn served_event_payload_omits_session_id_when_absent() { + let payload = make_payload(false, None); + assert!( + payload.get("session_id").is_none(), + "session_id must be absent when not provided" + ); + } + + #[test] + fn served_event_payload_hash_is_stable_for_same_query() { + let p1 = build_served_event_payload(ServedEventArgs { + query: "stable hash test", + capsule_count: 1, + top_score: 0.5, + skipped: false, + stage: "loc", + retrieval_path: "daemon", + store_queries: false, + session_id: None, + }); + let p2 = build_served_event_payload(ServedEventArgs { + query: "stable hash test", + capsule_count: 1, + top_score: 0.5, + skipped: false, + stage: "loc", + retrieval_path: "daemon", + store_queries: false, + session_id: None, + }); + assert_eq!( + p1.get("query_hash").and_then(|v| v.as_str()), + p2.get("query_hash").and_then(|v| v.as_str()), + "query_hash must be deterministic for the same query" + ); + } + + #[test] + fn served_event_payload_has_required_fields() { + let payload = build_served_event_payload(ServedEventArgs { + query: "check fields", + capsule_count: 2, + top_score: 0.6, + skipped: false, + stage: "localization", + retrieval_path: "fts_fallback", + store_queries: true, + session_id: Some("s1"), + }); + for field in &[ + "query_hash", + "query", + "capsule_count", + "top_score", + "skipped", + "stage", + "retrieval_path", + "session_id", + ] { + assert!( + payload.get(field).is_some(), + "missing required field: {field}" + ); + } + } +} + +#[cfg(test)] +mod tune_tests { + use super::*; + use kimetsu_brain::user_brain::with_user_brain_disabled; + use kimetsu_core::paths::git_init_boundary; + use std::fs; + + fn tune_test_root(label: &str) -> std::path::PathBuf { + let root = + std::env::temp_dir().join(format!("kimetsu-tune-cli-{label}-{}", ulid::Ulid::new())); + git_init_boundary(&root); + root + } + + /// End-to-end: dry-run sweep over a temp brain with synthetic eval data. + /// Verifies that --status prints case count and --apply (when >0 cases) + /// writes tune-history.json. Uses the lean (Noop) embedder so it works + /// in non-embeddings builds (floors still sweep on FTS). + #[test] + fn brain_tune_dry_run_does_not_modify_config() { + with_user_brain_disabled(|| { + let root = tune_test_root("dryrun"); + fs::create_dir_all(&root).expect("create root"); + project::init_project(&root, false).expect("init"); + + let args = TuneArgs { + status: false, + cost_weight: 0.005, + apply: false, // DRY RUN + revert: false, + workspace: Some(root.clone()), + }; + + // Read config before. + let paths = kimetsu_core::paths::ProjectPaths::discover(&root).expect("paths"); + let config_before = project::load_config(&paths).expect("config before"); + + brain_tune(args).expect("tune dry-run"); + + // Config must NOT have changed. + let config_after = project::load_config(&paths).expect("config after"); + assert_eq!( + config_before.broker.min_lexical_coverage, config_after.broker.min_lexical_coverage, + "dry-run must not change min_lexical_coverage" + ); + + fs::remove_dir_all(&root).ok(); + }); + } + + #[test] + fn brain_tune_status_shows_zero_cases_when_empty() { + with_user_brain_disabled(|| { + let root = tune_test_root("status"); + fs::create_dir_all(&root).expect("create root"); + project::init_project(&root, false).expect("init"); + + let args = TuneArgs { + status: true, + cost_weight: 0.005, + apply: false, + revert: false, + workspace: Some(root.clone()), + }; + // Should not panic; prints 0 cases. + brain_tune(args).expect("tune --status on empty brain"); + + fs::remove_dir_all(&root).ok(); + }); + } + + // ------------------------------------------------------------------ + // Fix 3: --apply in fixture-fallback mode must leave project.toml + // and tune-history.json untouched. + // ------------------------------------------------------------------ + #[test] + fn fix3_apply_in_fixture_mode_leaves_config_untouched() { + with_user_brain_disabled(|| { + let root = tune_test_root("fix3"); + fs::create_dir_all(&root).expect("create root"); + project::init_project(&root, false).expect("init"); + + // Ensure no fixture file exists, so the sweep falls back AND + // exits early (no eval cases). Either way --apply must not + // write anything. + let paths = kimetsu_core::paths::ProjectPaths::discover(&root).expect("paths"); + let config_before = project::load_config(&paths).expect("config before"); + let toml_mtime_before = fs::metadata(&paths.project_toml) + .expect("project.toml must exist") + .modified() + .expect("mtime"); + + let history_path = paths.kimetsu_dir.join("tune-history.json"); + assert!( + !history_path.exists(), + "tune-history.json must not exist before test" + ); + + let args = TuneArgs { + status: false, + cost_weight: 0.005, + apply: true, // --apply with < 30 personal cases → fixture mode + revert: false, + workspace: Some(root.clone()), + }; + brain_tune(args).expect("brain_tune must not error in fixture mode"); + + // project.toml must not have been modified. + let config_after = project::load_config(&paths).expect("config after"); + assert_eq!( + config_before.broker.min_lexical_coverage, config_after.broker.min_lexical_coverage, + "fix3: --apply in fixture mode must not change min_lexical_coverage" + ); + assert_eq!( + config_before.broker.min_semantic_score, config_after.broker.min_semantic_score, + "fix3: --apply in fixture mode must not change min_semantic_score" + ); + let toml_mtime_after = fs::metadata(&paths.project_toml) + .expect("project.toml must still exist") + .modified() + .expect("mtime after"); + assert_eq!( + toml_mtime_before, toml_mtime_after, + "fix3: project.toml mtime must not change in fixture mode with --apply" + ); + + // tune-history.json must not have been written. + assert!( + !history_path.exists(), + "fix3: tune-history.json must not be created when --apply runs in fixture mode" + ); + + fs::remove_dir_all(&root).ok(); + }); + } } diff --git a/crates/kimetsu-cli/src/proactive_state.rs b/crates/kimetsu-cli/src/proactive_state.rs index 8821ed9..5b24df5 100644 --- a/crates/kimetsu-cli/src/proactive_state.rs +++ b/crates/kimetsu-cli/src/proactive_state.rs @@ -194,6 +194,43 @@ pub fn looks_like_failure(tool_response: &str) -> bool { FAILURE_MARKERS.iter().any(|m| lc.contains(m)) } +// ----------------------------------------------------------------------- +// v1.5 (Story 2.3): session-scoped cross-turn capsule dedupe +// ----------------------------------------------------------------------- + +/// Pure dedupe-decision function. Given a slice of capsules and the current +/// session state, returns the indices of capsules that should be injected: +/// +/// * Capsules whose `expansion_handle` is already surfaced this session +/// are skipped. +/// * **Soft policy**: if the filter would leave zero capsules (every +/// candidate is already surfaced), the full input slice is returned +/// — a repeated top memory may still be the right context. +/// * Capsules with an empty `expansion_handle` are never tracked and +/// always pass through. +/// +/// Returns a `Vec` of indices into `handles` that survive the filter. +/// Callers iterate these to collect the actual capsule objects and later +/// call `state.mark_surfaced` on the included handles. +/// +/// This is a pure function of (handles, state) with no I/O — easy to unit +/// test without touching the file system. +pub fn dedupe_filter(handles: &[&str], state: &SessionState) -> Vec { + let new_indices: Vec = handles + .iter() + .enumerate() + .filter(|(_, h)| h.is_empty() || !state.is_surfaced(h)) + .map(|(i, _)| i) + .collect(); + + // Soft policy: never empty the injection. + if new_indices.is_empty() { + (0..handles.len()).collect() + } else { + new_indices + } +} + impl SessionState { pub fn is_surfaced(&self, memory_id: &str) -> bool { self.surfaced_memory_ids.iter().any(|id| id == memory_id) @@ -420,4 +457,98 @@ mod tests { assert!(sig.to_ascii_lowercase().contains("linker")); assert!(error_signature("").is_none()); } + + // ── v1.5 Story 2.3: dedupe_filter unit tests ───────────────────────────── + + /// DD-1: a handle not in the session state passes through. + #[test] + fn dedupe_filter_passes_unsurfaced_handles() { + let state = SessionState::default(); + let handles = ["memory:aaa", "memory:bbb"]; + let indices = dedupe_filter(&handles, &state); + assert_eq!(indices, vec![0, 1], "unsurfaced capsules must all pass"); + } + + /// DD-2: a handle already surfaced this session is filtered out. + #[test] + fn dedupe_filter_removes_surfaced_handles() { + let mut state = SessionState::default(); + state.mark_surfaced("memory:aaa"); + + let handles = ["memory:aaa", "memory:bbb"]; + let indices = dedupe_filter(&handles, &state); + // Only index 1 ("memory:bbb") should pass. + assert_eq!(indices, vec![1], "surfaced capsule must be filtered"); + } + + /// DD-3: soft policy — if ALL handles are already surfaced, the full + /// set is returned (never empty the injection). + #[test] + fn dedupe_filter_never_empties_injection() { + let mut state = SessionState::default(); + state.mark_surfaced("memory:aaa"); + state.mark_surfaced("memory:bbb"); + + let handles = ["memory:aaa", "memory:bbb"]; + let indices = dedupe_filter(&handles, &state); + // Soft policy: return all indices rather than an empty vec. + assert_eq!(indices, vec![0, 1], "must not empty the injection set"); + } + + /// DD-4: empty expansion_handle is never tracked and always passes through. + #[test] + fn dedupe_filter_passes_empty_handle_always() { + let mut state = SessionState::default(); + // Even if we tried to surface an empty string, it should always pass. + state.mark_surfaced(""); + + let handles = ["", "memory:real"]; + let indices = dedupe_filter(&handles, &state); + // Both pass: "" is always a pass-through, "memory:real" is new. + assert!(indices.contains(&0), "empty handle must always pass"); + assert!(indices.contains(&1), "new real handle must pass"); + } + + /// DD-5: mix of surfaced and new — only new handles pass, + /// provided at least one is new (soft-policy not triggered). + #[test] + fn dedupe_filter_mixed_keeps_new_only() { + let mut state = SessionState::default(); + state.mark_surfaced("memory:old1"); + state.mark_surfaced("memory:old2"); + + let handles = ["memory:old1", "memory:new", "memory:old2"]; + let indices = dedupe_filter(&handles, &state); + assert_eq!(indices, vec![1], "only the new handle must pass"); + } + + /// DD-6: proactive-state roundtrip — save surfaced handles, reload, + /// and verify dedupe_filter still sees them as surfaced. + #[test] + fn dedupe_filter_survives_state_roundtrip() { + let tmp = std::env::temp_dir(); + let state_dir = tmp.join(format!( + "kimetsu-dedupe-test-{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.subsec_nanos()) + .unwrap_or(0) + )); + let path = session_path(&state_dir, Some("test-session-dedupe")); + + let mut state = SessionState::default(); + state.mark_surfaced("memory:persisted"); + save(&path, &state); + + let reloaded = load(&path); + let handles = ["memory:persisted", "memory:fresh"]; + let indices = dedupe_filter(&handles, &reloaded); + assert_eq!( + indices, + vec![1], + "persisted handle must still be deduplicated after reload" + ); + + let _ = std::fs::remove_dir_all(&state_dir); + } } diff --git a/crates/kimetsu-core/src/config.rs b/crates/kimetsu-core/src/config.rs index 97897a8..bf41783 100644 --- a/crates/kimetsu-core/src/config.rs +++ b/crates/kimetsu-core/src/config.rs @@ -169,6 +169,15 @@ impl Default for EmbedderSection { pub struct LearningSection { #[serde(default = "default_auto_harvest")] pub auto_harvest: bool, + /// v1.5: store the raw retrieval query in local `context.served` + /// telemetry so the self-tuning loop can build a personal eval set. + /// Data never leaves the machine and is never exported. Set false to + /// keep only the query hash (the pre-v1.5 behavior). Default true so + /// new installs gain the eval-set signal immediately on upgrade; + /// `#[serde(default = "default_true")]` keeps pre-v1.5 project.toml + /// files loading cleanly (they get store_queries = true). + #[serde(default = "default_true")] + pub store_queries: bool, /// Opt-in credentialed SessionEnd distiller (configured by the install /// wizard). Disabled by default; `#[serde(default)]` keeps older /// project.toml files loading. @@ -184,6 +193,7 @@ impl Default for LearningSection { fn default() -> Self { Self { auto_harvest: default_auto_harvest(), + store_queries: default_true(), distiller: DistillerSection::default(), } } @@ -265,6 +275,13 @@ pub struct ModelSection { /// loading cleanly. #[serde(default = "default_region_env")] pub region_env: String, + /// v1.5: override the built-in $/MTok price table for the ROI ledger. + /// When set, this value is used for USD conversion instead of the + /// approximate built-in table. Useful for private-endpoint pricing or + /// non-standard model deployments. `#[serde(default)]` keeps all + /// pre-v1.5 project.toml files loading cleanly (they get `None`). + #[serde(default)] + pub price_per_mtok: Option, } fn default_region_env() -> String { @@ -282,6 +299,7 @@ impl Default for ModelSection { request_timeout_secs: 120, region: None, region_env: default_region_env(), + price_per_mtok: None, } } } @@ -367,6 +385,31 @@ pub struct BrokerSection { /// files loading unchanged. #[serde(default = "default_true")] pub ambient: bool, + /// v1.5 (Story 2.1): render-time capsule compression. When true (default), + /// capsule summaries are compressed with [`compress_for_render`] before + /// being injected into hook stdout or MCP tool responses. Compression + /// strips `[tags: ...]` / `(context: ...)` annotations and caps at 3 + /// sentences. Ranking is NEVER affected — compression runs only after + /// retrieval and reranking. Set false to inject full memory text (useful + /// for debugging or when summaries are already concise). + /// + /// `#[serde(default = "default_true")]` keeps pre-v1.5 project.toml files + /// loading cleanly (they get compression ON). + #[serde(default = "default_true")] + pub compress_capsules: bool, + /// v1.5 (Story 2.3): session-scoped cross-turn capsule dedupe. When true + /// (default), the `UserPromptSubmit` context hook skips capsules whose + /// `expansion_handle` was already injected earlier in the same session + /// (tracked via the proactive-state sidecar). A soft policy: skipping only + /// happens when at least one NEW capsule remains — if dedupe would empty + /// the injection entirely, all capsules are injected anyway (a repeated + /// top memory may still be the right context). Set false to disable + /// session dedupe and always inject the full ranked set. + /// + /// `#[serde(default = "default_true")]` keeps pre-v1.5 project.toml files + /// loading cleanly (they get session dedupe ON). + #[serde(default = "default_true")] + pub session_dedupe: bool, } fn default_max_capsules() -> usize { @@ -400,6 +443,8 @@ impl Default for BrokerSection { budget_floor_tokens: default_budget_floor_tokens(), budget_run_cap_tokens: default_budget_run_cap_tokens(), ambient: default_true(), + compress_capsules: default_true(), + session_dedupe: default_true(), } } } @@ -697,6 +742,25 @@ max_total_cost_usd = 250.0 config.embedder.reranker, "ms-marco-tinybert-l-2-v2", "embedder.reranker must default to ms-marco-tinybert-l-2-v2" ); + // v1.5: a pre-v1.5 project.toml has no learning.store_queries — + // defaults to true so existing installs gain the eval-set signal + // on upgrade without any config change. + assert!( + config.learning.store_queries, + "learning.store_queries must default to true" + ); + // v1.5 (Story 2.1): a pre-v1.5 project.toml without broker.compress_capsules + // must load cleanly and default to true (compression ON). + assert!( + config.broker.compress_capsules, + "broker.compress_capsules must default to true" + ); + // v1.5 (Story 2.3): a pre-v1.5 project.toml without broker.session_dedupe + // must load cleanly and default to true (dedupe ON). + assert!( + config.broker.session_dedupe, + "broker.session_dedupe must default to true" + ); } /// A1: default_for_project must use KIMETSU_CONFIG_VERSION (the @@ -838,4 +902,87 @@ max_total_cost_usd = 250.0 ); } } + + /// v1.5: a pre-v1.5 project.toml without `model.price_per_mtok` must + /// load cleanly and default to `None` (backward compatibility). + #[test] + fn pre_v1_5_config_without_price_per_mtok_loads_with_none() { + let toml = r#" +[kimetsu] +project_id = "demo" +schema_version = 7 + +[model] +provider = "anthropic" +model = "claude-sonnet-4-7" +api_key_env = "ANTHROPIC_API_KEY" +max_output_tokens = 8192 +temperature = 0.2 +request_timeout_secs = 120 + +[broker] +default_budget_tokens = 6000 + +[broker.weights] +relevance = 0.5 +confidence = 0.2 +freshness = 0.2 +scope = 0.1 + +[shell] +default_timeout_secs = 60 +max_timeout_secs = 600 +env_allowlist_extra = [] +redact_secrets = true + +[ingestion] +max_file_bytes = 524288 +extra_skip_dirs = [] +max_total_files = 50000 + +[run] +max_total_tool_calls = 60 +max_total_model_turns = 30 +max_total_cost_usd = 250.0 +"#; + let config = ProjectConfig::from_toml(toml).expect("pre-v1.5 toml must load"); + assert!( + config.model.price_per_mtok.is_none(), + "price_per_mtok must default to None when absent from project.toml" + ); + } + + /// v1.5 (Story 2.1+2.3): compress_capsules and session_dedupe survive a + /// round-trip through serialize → deserialize when set to false. + #[test] + fn broker_v1_5_fields_round_trip_as_false() { + let mut config = ProjectConfig::default_for_project("demo"); + config.broker.compress_capsules = false; + config.broker.session_dedupe = false; + let serialized = config.to_toml().expect("serialize"); + let reloaded = ProjectConfig::from_toml(&serialized).expect("reload"); + assert!( + !reloaded.broker.compress_capsules, + "compress_capsules must survive as false" + ); + assert!( + !reloaded.broker.session_dedupe, + "session_dedupe must survive as false" + ); + } + + /// v1.5: when `model.price_per_mtok` is set in project.toml it must + /// round-trip cleanly through serialize → deserialize. + #[test] + fn price_per_mtok_round_trips() { + let mut config = ProjectConfig::default_for_project("demo"); + config.model.price_per_mtok = Some(7.5); + let serialized = config.to_toml().expect("serialize"); + let reloaded = ProjectConfig::from_toml(&serialized).expect("reload"); + assert_eq!( + reloaded.model.price_per_mtok, + Some(7.5), + "price_per_mtok must round-trip" + ); + } } diff --git a/crates/kimetsu-core/src/lib.rs b/crates/kimetsu-core/src/lib.rs index 138264f..948344b 100644 --- a/crates/kimetsu-core/src/lib.rs +++ b/crates/kimetsu-core/src/lib.rs @@ -6,7 +6,7 @@ pub mod memory; pub mod paths; pub mod secret; -pub const KIMETSU_SCHEMA_VERSION: i64 = 2; +pub const KIMETSU_SCHEMA_VERSION: i64 = 3; /// The `project.toml` config-file format version. Deliberately decoupled /// from `KIMETSU_SCHEMA_VERSION` (the brain.db schema): the DB schema can /// advance via migrations without forcing every project.toml to be rewritten. diff --git a/crates/kimetsu-e2e/tests/migration.rs b/crates/kimetsu-e2e/tests/migration.rs index b6609a4..bbe9bb9 100644 --- a/crates/kimetsu-e2e/tests/migration.rs +++ b/crates/kimetsu-e2e/tests/migration.rs @@ -1,13 +1,15 @@ -//! A7 e2e: v1→v2 schema migration end-to-end — project brain. +//! A7 e2e: v1→v3 schema migration end-to-end — project brain. //! -//! Verifies that an EXISTING populated project brain upgrades from v1 to v2 -//! with a backup sidecar and data preserved, using the normal open path. +//! Verifies that an EXISTING populated project brain upgrades from v1 to the +//! current target version (v3) with a backup sidecar and data preserved, +//! using the normal open path. //! //! Technique: (a) init a real project and record a memory (brain.db lands at -//! v2); (b) stamp the schema_info version back to 1 to simulate a pre-upgrade -//! DB; (c) re-open via `load_project` (which calls `schema::initialize` → -//! `run_migrations`); (d) assert current_version == 2, backup sidecar exists, -//! and the seeded memory survives. +//! current target); (b) stamp the schema_info version back to 1 to simulate a +//! pre-upgrade DB; (c) re-open via `load_project` (which calls +//! `schema::initialize` → `run_migrations`); (d) assert +//! current_version == target, backup sidecar exists, and the seeded memory +//! survives. //! //! The user-brain migration is covered by a parallel unit test in //! `kimetsu-brain/src/user_brain.rs` (see `migration_upgrades_user_brain`). @@ -18,7 +20,7 @@ use kimetsu_brain::user_brain::with_user_brain_disabled; use kimetsu_core::memory::{MemoryKind, MemoryScope}; use kimetsu_e2e::prelude::*; -/// Project brain: v1→v2 migration with a populated DB creates a backup +/// Project brain: v1→current migration with a populated DB creates a backup /// sidecar and preserves the seeded memory. #[test] fn project_brain_v1_to_v2_migration_creates_backup_and_preserves_data() { @@ -26,7 +28,7 @@ fn project_brain_v1_to_v2_migration_creates_backup_and_preserves_data() { let project = TempProject::init("migration_project"); // (a) Seed a memory — this creates brain.db and runs initialize() - // which leaves it at v2. + // which leaves it at the current target version. let mem_id = project::add_memory( project.root(), MemoryScope::Repo, @@ -43,9 +45,9 @@ fn project_brain_v1_to_v2_migration_creates_backup_and_preserves_data() { ); // (b) Stamp the version back to 1 to simulate a pre-upgrade DB. - // The table structure is already v2 (idempotent migration), - // so re-running the migration is a safe no-op on the DDL side - // but WILL write a new backup (row count > 0). + // The table structure is already at the current target (idempotent + // migrations), so re-running the migration is a safe no-op on the + // DDL side but WILL write a new backup (row count > 0). { let conn = rusqlite::Connection::open(project.brain_db()).expect("open for stamp-down"); conn.execute( @@ -67,11 +69,16 @@ fn project_brain_v1_to_v2_migration_creates_backup_and_preserves_data() { // schema::initialize which calls run_migrations. let (_, _, conn) = project::load_project(project.root()).expect("load_project after stamp"); - // Assert 1: version is back at 2. + // Assert 1: version is at current target. + let target = migrate::target_version(); let ver = migrate::current_version(&conn).expect("current_version"); - assert_eq!(ver, 2, "project brain must be at v2 after re-open"); + assert_eq!( + ver, target, + "project brain must be at v{target} after re-open" + ); // Assert 2: backup sidecar exists next to brain.db. + // The sidecar is named brain.db.bak--- where from=1, to=target. let brain_dir = project .brain_db() .parent() @@ -83,7 +90,7 @@ fn project_brain_v1_to_v2_migration_creates_backup_and_preserves_data() { .and_then(|n| n.to_str()) .unwrap_or("brain.db") .to_string(); - let bak_prefix = format!("{stem}.bak-1-2-"); + let bak_prefix = format!("{stem}.bak-1-{target}-"); let bak_files: Vec<_> = std::fs::read_dir(&brain_dir) .expect("read brain dir") .filter_map(|e| e.ok()) @@ -97,7 +104,7 @@ fn project_brain_v1_to_v2_migration_creates_backup_and_preserves_data() { assert_eq!( bak_files.len(), 1, - "exactly one backup sidecar brain.db.bak-1-2-* should exist; found: {:?}", + "exactly one backup sidecar {bak_prefix}* should exist; found: {:?}", bak_files.iter().map(|e| e.file_name()).collect::>() ); @@ -105,7 +112,7 @@ fn project_brain_v1_to_v2_migration_creates_backup_and_preserves_data() { let memories_after = project::list_memories(project.root()).expect("list after migration"); assert!( memories_after.iter().any(|m| m.memory_id == mem_id), - "seeded memory must survive the v1→v2 migration; mem_id={mem_id}" + "seeded memory must survive the v1→v{target} migration; mem_id={mem_id}" ); }); } diff --git a/docs/HOW-KIMETSU-WORKS.md b/docs/HOW-KIMETSU-WORKS.md index 7ced28b..c919348 100644 --- a/docs/HOW-KIMETSU-WORKS.md +++ b/docs/HOW-KIMETSU-WORKS.md @@ -81,7 +81,7 @@ recovery). The materialized tables: ### Durable upgrades: schema migrations -brain.db carries a schema version (`KIMETSU_SCHEMA_VERSION`, currently **2**) +brain.db carries a schema version (`KIMETSU_SCHEMA_VERSION`, currently **3**) in its `schema_info` table. On every read-write open, a versioned, forward-only migration runner brings the DB up to the binary's target. Each migration runs inside **one transaction** (the DDL and the version bump commit @@ -178,7 +178,7 @@ selection is unchanged. Tunable knobs in `[broker]`: - `max_capsules` (default **8**) — hard cap on capsules rendered into a prompt. -- `min_semantic_score` (default `0.0` = off) — the embeddings-only relevance +- `min_semantic_score` (default `-1.0` = AUTO: 0.35 on bge-family, off otherwise) — the embeddings-only relevance floor described above. - `budget_floor_tokens` (default `1500`) and `budget_run_cap_tokens` (default `8000`) — bounds for the adaptive per-run budget (see the agent @@ -390,6 +390,43 @@ These are backed by two event additions: a **`context.served`** event logs injected-token count — so the hit-rate and token-economy numbers are real counts, not estimates. +### ROI ledger + +`kimetsu brain roi [--window 7d|30d|all] [--json]` estimates how much the +brain saved you. It sums conservative per-kind token credits for every cited +memory (failure_pattern=1500, command=400, convention=300, fact=500, +preference=200 tokens), subtracts injected-token overhead, and shows a +net-positive / net-negative verdict. Dollar values are shown when the active +model is in the built-in price table or when `[model] price_per_mtok` is set. +Honest negatives are displayed as-is. The Stop hook appends a per-session +savings line when ≥1 citation occurred. Methodology: `docs/ROI-METHODOLOGY.md`. + +### Consolidation and triage + +`kimetsu brain consolidate` merges near-duplicate memories that accumulated +different phrasings of the same lesson. Default threshold: cosine ≥ 0.92 +within the same embedding model. The survivor keeps its id and text; members +get `superseded_by` set (schema v3) and a `memory.superseded` event so +`brain rebuild` reproduces the merge. Citations are reassigned to the survivor. +`--distill` adds a second pass: loose clusters (0.75–0.85 cosine, ≥3 memories, +≥1 shared tag) are fed to the configured distiller and the result lands as a +memory proposal for review. `--dry-run` prints the plan without writing. + +`kimetsu brain triage` lists memories below a usefulness + age threshold +(defaults: score < 0.2 and last-useful > 30 days) and prompts keep / prune / +skip interactively. `--prune-all --yes` for non-interactive batch pruning. + +### Self-tuning + +`kimetsu brain tune --status` shows how many positive eval cases the brain has +accumulated (from `context.served` + citation joins). `kimetsu brain tune` +sweeps `broker.min_lexical_coverage` × `broker.min_semantic_score` against the +production embedder and picks the combo that maximises the objective on the +training split. A holdout guardrail (deterministic 20% split) prevents +writing a config that regresses holdout quality. `--apply` writes only the +floor parameters to `project.toml`; `--revert` restores the previous entry. +Dry-run by default. + --- ## 7. The MCP surface @@ -421,6 +458,7 @@ Run `kimetsu mcp serve` and the host harness gets ~28 | `kimetsu_benchmark_record_outcome` | Record run outcome → proposal | | `kimetsu_bridge_status` / `_export` / `_import` / `_sync` | Cross-harness skill registry + install/sync | | `kimetsu_skills_search` / `kimetsu_skill` | Find / invoke a portable skill | +| `kimetsu_brain_cite` | (write-gated) Record that a retrieved memory materially helped — closes the ground-truth loop for self-tuning | | `cite_memory` | (in-run) Mark a memory as cited | | `expand_capsule` | (in-run) Expand a lazily-injected capsule headline to full detail by resolving its `memory:` / `file:` handle (§3a) | @@ -715,9 +753,12 @@ model = "bge-small-en-v1.5" # or "bge-m3", "jina-v2-base-code" default_budget_tokens = 6000 # flat fallback; the adaptive budget supersedes it ambient = true # false → don't append workspace context to queries max_capsules = 8 # hard cap on capsules rendered into a prompt -min_semantic_score = 0.0 # >0 sets the embeddings-only relevance floor +min_semantic_score = -1.0 # AUTO (bge: 0.35, others: off); >0 sets an explicit floor budget_floor_tokens = 1500 # adaptive-budget floor (small tasks not starved) budget_run_cap_tokens = 8000 # per-run ceiling on brain-injected tokens +compress_capsules = true # v1.5: compress rendered capsule text (strips tags/context + # annotations, caps at 3 sentences); ranking unaffected +session_dedupe = true # v1.5: skip capsules already injected this session [broker.weights] relevance = 0.50 @@ -753,6 +794,9 @@ max_total_cost_usd = 250.0 # advisory under subscription providers [learning] auto_harvest = true +store_queries = true # v1.5: include raw query text in context.served telemetry + # (on-machine only; powers the personal eval set for brain tune) + # set false to revert to query-hash-only (pre-v1.5 behavior) [learning.distiller] enabled = false diff --git a/docs/INSTALL.md b/docs/INSTALL.md index ceeb5a7..1b3c935 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -5,7 +5,7 @@ install time — **lean vs semantic (embeddings)** — because that's the only p baked into the binary. *Which host agents you use* (Claude Code, Codex, Pi, OpenClaw) is a **runtime** choice you change anytime with `kimetsu plugin install`/`uninstall` — no reinstall. The official prebuilt + npm binaries -include all four host integrations; a bare source `cargo install` is minimal and +include all six host integrations; a bare source `cargo install` is minimal and adds them with `--features pi,openclaw`. ## cargo @@ -51,7 +51,7 @@ For **Linux / macOS / Windows** on every [GitHub Release](https://github.com/RodCor/kimetsu/releases). Extract the archive and put `kimetsu` / `kimetsu.exe` somewhere on `PATH` (`~/.local/bin`, `/usr/local/bin`, or `%USERPROFILE%\.cargo\bin`). Every prebuilt archive — lean and embeddings — -bundles all four host integrations, so switching hosts never needs a reinstall. +bundles all six host integrations, so switching hosts never needs a reinstall. Lean archives are published for Linux, macOS Intel, macOS Apple Silicon, and Windows. Embeddings archives are published where ONNX Runtime prebuilts are available: Linux x86_64, @@ -89,13 +89,15 @@ are only needed for benchmark runs. ## Wiring host agents Wire Kimetsu into any supported host. The built-in installers cover Claude Code, -Codex, Pi, and OpenClaw: +Codex, Pi, OpenClaw, Cursor, and Gemini CLI: ```bash -kimetsu plugin install claude --workspace . # writes .mcp.json + .claude/settings.json -kimetsu plugin install codex --workspace . # writes .codex/config.toml + .codex/hooks.json + skill + agent -kimetsu plugin install openclaw --workspace . # MCP server + hooks plugin + skill in .openclaw/ (requires --features openclaw on source builds) -kimetsu plugin install pi --workspace . # TS extension (Pi has no MCP) + skill in .pi/ (requires --features pi on source builds) +kimetsu plugin install claude --workspace . # writes .mcp.json + .claude/settings.json +kimetsu plugin install codex --workspace . # writes .codex/config.toml + .codex/hooks.json + skill + agent +kimetsu plugin install openclaw --workspace . # MCP server + hooks plugin + skill in .openclaw/ (requires --features openclaw on source builds) +kimetsu plugin install pi --workspace . # TS extension (Pi has no MCP) + skill in .pi/ (requires --features pi on source builds) +kimetsu plugin install cursor --workspace . # writes .cursor/mcp.json + .cursor/rules/kimetsu-brain/rule.md +kimetsu plugin install gemini-cli --workspace . # writes .gemini/settings.json + merges GEMINI.md into project root # Install globally for every project (writes to the host's home config dir): kimetsu plugin install claude --scope global @@ -113,6 +115,15 @@ kimetsu plugin uninstall claude-code --yes # drop the old host's wiring kimetsu plugin install pi # wire the new one ``` +**Cursor and Gemini CLI:** neither host has a `UserPromptSubmit`-style hook +system, so MCP + an always-on guidance file are the complete integration +surface for those hosts (no automatic prompt-time context injection; the +model must call `kimetsu_brain_context` manually). **The config schemas for +Cursor (`.cursor/mcp.json`) and Gemini CLI (`.gemini/settings.json`) were +inferred from their public documentation as of June 2026 and have not been +verified against live host builds — review the generated files before +committing them.** + `--scope` defaults to `workspace`. The installer **merges** into existing config: if you already have hooks — even on the same events Kimetsu uses (`UserPromptSubmit`, `PreToolUse`, …) — your hooks are kept and Kimetsu's are diff --git a/docs/ROI-METHODOLOGY.md b/docs/ROI-METHODOLOGY.md new file mode 100644 index 0000000..6850386 --- /dev/null +++ b/docs/ROI-METHODOLOGY.md @@ -0,0 +1,114 @@ +# ROI Methodology — `kimetsu brain roi` + +> **Source of truth for the per-kind constants:** +> `crates/kimetsu-brain/src/roi.rs` — `SAVED_TOKENS_PER_CITATION` + +--- + +## Formula + +``` +estimated_saved_tokens = Σ (citations_of_kind × saved_tokens_per_kind) +net_tokens = estimated_saved_tokens − injected_tokens +net_usd = net_tokens / 1_000_000 × price_per_mtok +``` + +- **`citations`**: rows in `memory_citations` for the selected window (attributed + to runs via `runs.started_at` or to the hook sentinel via `memory_citations.cited_at`). +- **`injected_tokens`**: sum of the `used_tokens` field across `context.injected` + events in the window. This is the actual token cost the brain added to each + prompt — the "spent" side of the ledger. +- **`price_per_mtok`**: resolved from `[model] price_per_mtok` in `project.toml` + (user override), falling back to the built-in approximate table in `roi.rs`. + Unknown models produce `usd: null` in the JSON output rather than a + potentially-wrong number. + +--- + +## Per-kind constants + +| Kind | Saved tokens / citation | Reasoning | +|-------------------|------------------------:|-----------| +| `failure_pattern` | 1 500 | Avoids the "try → fail → diagnose → fix" loop. Estimated at ~3 tool calls × ~500 tokens/call. | +| `fact` | 500 | Avoids a docs lookup or user question. ~1 exchange. | +| `command` | 400 | Avoids a `--help` trial or web search. ~1–2 tool calls. | +| `convention` | 300 | Avoids a code-search to find the project pattern. ~1–2 searches. | +| `preference` | 200 | Avoids one clarifying question. ~1 exchange. | + +**These are deliberate under-estimates.** The goal is for every "net positive" +result to be _credible_, not just flattering. A user who sees a positive +number should be able to believe it. + +--- + +## Why under-claiming is deliberate + +Kimetsu's ROI story is built on honesty: + +1. **Net CAN be negative** — and is shown as such. If the brain injected more + tokens than it saved (e.g. every retrieved memory was a silent passenger), the + ledger says so. +2. **Constants are calibrated low** — the actual avoided cost of re-discovering + a `failure_pattern` is often much higher (the model might run the same broken + command five times before giving up). We use a conservative floor. +3. **Citations are sparse in MCP hosts** — Claude Code currently has no + `kimetsu_brain_cite` tool, so the `memory_citations` table is populated only + when the pipeline (`run coding`) is in use. The ledger will under-count + savings in pure MCP-host sessions until the cite tool ships. That's fine: + under-counting keeps the results trustworthy. + +--- + +## Sanity anchor: Terminal-Bench calibration evidence + +The 16-task slice recorded in internal bench runs shows: + +| Condition | Cost/win | +|-----------|----------:| +| Without brain context | $2.47 | +| With brain context | $0.19 | + +This ~13× difference is the _observed_ cost difference, not a model. +The per-kind constants were back-derived from this data: if a typical winning +run uses ~2 citations of mixed kinds, the implied saving is roughly +($2.47 − $0.19) / 2 ≈ $1.14/citation at Claude Sonnet 3.5 pricing +(≈$3/MTok → ~380k tokens / citation). Our constants sum to far less than +that for a 2-citation session — confirming that we are well inside the +conservative zone. + +--- + +## Limitations + +1. **Citations are sparse in MCP-host sessions.** Until `kimetsu_brain_cite` is + deployed, the ledger only counts citations generated by the autonomous + pipeline. The `net_tokens` figure may appear falsely negative for users + who work exclusively via the Claude Code plugin. +2. **Counterfactuals are estimates, not measurements.** We cannot observe what + the model *would* have done without the brain context. The constants are + conservative estimates based on tool-call patterns, not A/B experiments. +3. **Price table is approximate.** The built-in $/MTok figures are rounded + retail/API-key prices as of June 2026. Set `[model] price_per_mtok` in + `project.toml` for accurate billing data. +4. **Injection cost uses input-token pricing.** The model also produces output + tokens in response to the injected context; those are not counted here + (they depend on the model's response length, which varies). + +--- + +## Config reference + +```toml +# project.toml — override the built-in price table for this project +[model] +model = "claude-sonnet-4-7" +price_per_mtok = 3.0 # optional; defaults to built-in table +``` + +``` +# CLI +kimetsu brain roi # last 30 days (default) +kimetsu brain roi --window 7d # last 7 days +kimetsu brain roi --window all # all time +kimetsu brain roi --json # machine-readable RoiReport +```