diff --git a/CHANGELOG.md b/CHANGELOG.md index 4a09bc6..5b3c649 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,53 @@ follow [SemVer](https://semver.org/spec/v2.0.0.html) with the caveat that pre-1.0 minor bumps may include breaking changes (documented in the release notes). +## v0.8.0 — proactive recall, selectable embedding model, full MCP control + +The release that makes the brain **proactive** and gives the agent (and user) +full control over it from inside Claude Code / Codex. + +ADDED + * **Proactive recall (mid-work).** New `PreToolUse` / `PostToolUse` Bash hooks + surface a relevant memory *while the agent works*, not just on prompt: + - after a **failed** Bash command, surface a matching `failure_pattern` / + `command` fix; + - **before** a risky Bash command, warn from a matching `failure_pattern`; + - on a **repeated** failing command (loop), lower the score floor and bypass + the throttle so help arrives sooner. + Retrieval is lexical-FTS-only (no embedding-model load), gated by a high + score floor (0.45; loop 0.35), capped at one capsule, with per-session + dedupe + a refractory throttle. Token cost stays ~0 (silent when nothing + qualifies). Wired into both Claude Code (`.claude/settings.json`) and Codex + (`.codex/hooks.json`) with a `matcher: "Bash"`; opt out with + `kimetsu plugin install --no-proactive` (or `proactive:false` over MCP). + Per-session state lives in `/.kimetsu/proactive/.json` + and is garbage-collected after 7 days. + * **Selectable embedding model.** New `[embedder]` table in `project.toml` + (precedence: `KIMETSU_BRAIN_EMBEDDER` env > config > default). Inspect and + change it with `kimetsu brain model list` / `kimetsu brain model set ` + (the latter writes the config and re-embeds the corpus with the new model). + Curated built-ins: `bge-small-en-v1.5` (384d, default), `bge-m3` (1024d), + `jina-v2-base-code` (768d). + * **Full MCP control surface.** New tools so an agent can manage the brain + without leaving Claude Code / Codex: `kimetsu_brain_model_list` / + `kimetsu_brain_model_set` (re-embeds in-process with the new model), + `kimetsu_brain_reindex`, `kimetsu_brain_memory_search` (FTS over memory + text), `kimetsu_brain_conflict_resolve`, `kimetsu_brain_prune`, and + `kimetsu_brain_config_show`. `kimetsu_brain_memory_list` and + `..._memory_proposals` gained `limit`/`offset` pagination. + +CHANGED + * `reindex` can now run against an explicit embedder + (`reindex_all_with_embedder`), so a model switch re-embeds with the chosen + model regardless of the process's cached default embedder. + +FIXED + * **Test isolation.** Tests created project roots under the system temp dir; + on a machine whose `$HOME` is itself a git repo, `ProjectPaths::discover` + climbed to `$HOME` and made parallel tests share one `brain.db` + + `project.lock`. Each test root now gets its own git boundary, so plain + `cargo test` is hermetic. + ## v0.7.2 — remove kimetsu-harbor-rs; first crates.io publish of the v0.7 line Maintenance + distribution release, layered on top of the v0.7.1 security diff --git a/Cargo.lock b/Cargo.lock index 087bc14..89101d2 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1483,7 +1483,7 @@ dependencies = [ [[package]] name = "kimetsu-agent" -version = "0.7.3" +version = "0.8.0" dependencies = [ "blake3", "kimetsu-brain", @@ -1499,7 +1499,7 @@ dependencies = [ [[package]] name = "kimetsu-brain" -version = "0.7.3" +version = "0.8.0" dependencies = [ "blake3", "fastembed", @@ -1515,7 +1515,7 @@ dependencies = [ [[package]] name = "kimetsu-chat" -version = "0.7.3" +version = "0.8.0" dependencies = [ "base64 0.22.1", "crossterm", @@ -1530,7 +1530,7 @@ dependencies = [ [[package]] name = "kimetsu-cli" -version = "0.7.3" +version = "0.8.0" dependencies = [ "clap", "kimetsu-agent", @@ -1546,7 +1546,7 @@ dependencies = [ [[package]] name = "kimetsu-core" -version = "0.7.3" +version = "0.8.0" dependencies = [ "serde", "serde_json", @@ -1557,7 +1557,7 @@ dependencies = [ [[package]] name = "kimetsu-e2e" -version = "0.7.3" +version = "0.8.0" dependencies = [ "kimetsu-agent", "kimetsu-brain", diff --git a/Cargo.toml b/Cargo.toml index fa125ff..5ccfc9b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -14,7 +14,7 @@ resolver = "3" # everything below except the per-crate `description`, `keywords`, # and `categories` which live in each `[package]` block since # crates.io enforces them per crate. -version = "0.7.3" +version = "0.8.0" edition = "2024" # Rust ecosystem dual-license (matches tokio, serde, fastembed-rs, # etc.). UNLICENSED would block crates.io entirely. diff --git a/crates/kimetsu-agent/Cargo.toml b/crates/kimetsu-agent/Cargo.toml index c5a3fd3..ec230d8 100644 --- a/crates/kimetsu-agent/Cargo.toml +++ b/crates/kimetsu-agent/Cargo.toml @@ -23,8 +23,8 @@ embeddings = ["kimetsu-brain/embeddings"] [dependencies] blake3.workspace = true -kimetsu-brain = { path = "../kimetsu-brain", version = "0.7.3" } -kimetsu-core = { path = "../kimetsu-core", version = "0.7.3" } +kimetsu-brain = { path = "../kimetsu-brain", version = "0.8.0" } +kimetsu-core = { path = "../kimetsu-core", version = "0.8.0" } regex.workspace = true reqwest.workspace = true rusqlite.workspace = true diff --git a/crates/kimetsu-agent/src/agent_loop.rs b/crates/kimetsu-agent/src/agent_loop.rs index e86a622..a1e2a4e 100644 --- a/crates/kimetsu-agent/src/agent_loop.rs +++ b/crates/kimetsu-agent/src/agent_loop.rs @@ -386,6 +386,9 @@ mod tests { fn temp_project() -> std::path::PathBuf { let root = std::env::temp_dir().join(format!("kimetsu-agent-loop-test-{}", new_id())); fs::create_dir_all(&root).expect("create temp root"); + // Isolate from any enclosing git repo so discover() resolves + // here, not a shared ancestor (see git_init_boundary). + kimetsu_core::paths::git_init_boundary(&root); root } } diff --git a/crates/kimetsu-agent/src/bench.rs b/crates/kimetsu-agent/src/bench.rs index e821485..9b26628 100644 --- a/crates/kimetsu-agent/src/bench.rs +++ b/crates/kimetsu-agent/src/bench.rs @@ -366,6 +366,10 @@ fn run_task_mode( ) -> KimetsuResult { let repo = fixture_root.join(format!("{}-{}", task.id, mode.as_str())); write_fixture_repo(&repo, task)?; + // Make each fixture its own git boundary so its brain initializes + // inside the fixture, never climbing to an enclosing repo (e.g. a + // dev's $HOME repo) and leaking fixture memories there. + kimetsu_core::paths::git_init_boundary(&repo); project::init_project(&repo, false)?; if let Some(model_config) = model_config { configure_fixture_model(&repo, model_config, remaining_cost_usd)?; @@ -1881,35 +1885,42 @@ mod tests { #[test] fn benchmark_reports_warm_memory_reuse() { - let repo = std::env::temp_dir().join(format!("kimetsu-bench-report-{}", new_id())); - fs::create_dir_all(&repo).expect("create repo"); - - let report = run_benchmark(BenchOptions { - repo: repo.clone(), - keep_fixtures: false, - model_backed: false, - limit: None, - max_cost_usd: 1.0, - }) - .expect("run benchmark"); - - assert_eq!(report.task_count, 16); - assert!(report.report_path.exists()); - assert!(report.results_path.exists()); - let warm = report - .summaries - .iter() - .find(|summary| summary.mode == "brain_on_warm") - .expect("warm summary"); - assert!(warm.accepted_memories_used >= report.task_count as u32); - assert!(!warm.stage_profiles.is_empty()); - assert!( - warm.stage_profiles + // Isolate from the developer's real user brain + // (`~/.kimetsu/brain.db`): the multi-task run opens many + // connections, and sharing the one real file serializes them + // into SQLITE_BUSY. Fixture repos are already git-isolated (see + // run_task_mode), so this only scopes the user brain. + kimetsu_brain::user_brain::with_user_brain_disabled(|| { + let repo = std::env::temp_dir().join(format!("kimetsu-bench-report-{}", new_id())); + fs::create_dir_all(&repo).expect("create repo"); + + let report = run_benchmark(BenchOptions { + repo: repo.clone(), + keep_fixtures: false, + model_backed: false, + limit: None, + max_cost_usd: 1.0, + }) + .expect("run benchmark"); + + assert_eq!(report.task_count, 16); + assert!(report.report_path.exists()); + assert!(report.results_path.exists()); + let warm = report + .summaries .iter() - .any(|profile| profile.stage == "repo_scan") - ); + .find(|summary| summary.mode == "brain_on_warm") + .expect("warm summary"); + assert!(warm.accepted_memories_used >= report.task_count as u32); + assert!(!warm.stage_profiles.is_empty()); + assert!( + warm.stage_profiles + .iter() + .any(|profile| profile.stage == "repo_scan") + ); - fs::remove_dir_all(repo).expect("cleanup"); + fs::remove_dir_all(repo).expect("cleanup"); + }); } #[test] diff --git a/crates/kimetsu-agent/src/pipeline.rs b/crates/kimetsu-agent/src/pipeline.rs index 36abf7a..14ed209 100644 --- a/crates/kimetsu-agent/src/pipeline.rs +++ b/crates/kimetsu-agent/src/pipeline.rs @@ -2422,6 +2422,8 @@ mod tests { fn dry_run_pipeline_writes_trace_patch_plan_and_report() { let root = std::env::temp_dir().join(format!("kimetsu-pipeline-test-{}", new_id())); fs::create_dir_all(root.join("src")).expect("create src"); + // Isolate from any enclosing git repo (see git_init_boundary). + kimetsu_core::paths::git_init_boundary(&root); fs::write( root.join("Cargo.toml"), "[package]\nname = \"fixture\"\nversion = \"0.1.0\"\n", diff --git a/crates/kimetsu-agent/src/tools.rs b/crates/kimetsu-agent/src/tools.rs index c6be1aa..d4c86a5 100644 --- a/crates/kimetsu-agent/src/tools.rs +++ b/crates/kimetsu-agent/src/tools.rs @@ -1569,6 +1569,8 @@ mod tests { fn temp_project() -> PathBuf { let root = std::env::temp_dir().join(format!("kimetsu-tools-test-{}", new_id())); fs::create_dir_all(&root).expect("create temp root"); + // Isolate from any enclosing git repo (see git_init_boundary). + kimetsu_core::paths::git_init_boundary(&root); root } } diff --git a/crates/kimetsu-brain/Cargo.toml b/crates/kimetsu-brain/Cargo.toml index 096ba49..8c1a14f 100644 --- a/crates/kimetsu-brain/Cargo.toml +++ b/crates/kimetsu-brain/Cargo.toml @@ -31,7 +31,7 @@ blake3.workspace = true # supports BGE-M3 + Jina-v2-base-code alongside BGE-small. fastembed = { version = "5", optional = true } ignore.workspace = true -kimetsu-core = { path = "../kimetsu-core", version = "0.7.3" } +kimetsu-core = { path = "../kimetsu-core", version = "0.8.0" } # v0.4.5: regex backs the secret-redaction patterns in # `kimetsu_brain::redact`. The crate is already a workspace pin # elsewhere; we just opt this crate into it now. diff --git a/crates/kimetsu-brain/src/context.rs b/crates/kimetsu-brain/src/context.rs index ec7e5c7..70cfeb2 100644 --- a/crates/kimetsu-brain/src/context.rs +++ b/crates/kimetsu-brain/src/context.rs @@ -85,6 +85,13 @@ pub struct ContextRequest { /// of these strings receive an additional 1.3× multiplier after the /// tag boost (e.g. `["semantic_operator", "anti_pattern"]` for bench). pub prefer_roles: Vec, + /// v0.8: hard kind filter applied BEFORE scoring + capping. When + /// non-empty, only candidates whose capsule `kind` is in this list + /// survive — so a higher-ranked repo file or off-kind memory can't + /// consume a (often single) slot. Used by the proactive engine to + /// restrict recall to actionable kinds (failure_pattern, command, + /// convention). Empty (default) keeps all kinds, prior behaviour. + pub kinds: Vec, } #[derive(Debug, Clone)] @@ -185,6 +192,21 @@ pub fn retrieve_context_with_embedder( candidates.extend(repo_file_candidates(conn, repo_root, &request.query, 30)?); candidates.extend(manifest_candidates(conn, repo_root, &request.query)?); + // v0.8: proactive kind filter — restrict to actionable kinds BEFORE + // scoring + capping so a higher-ranked repo file or off-kind memory + // can't take the proactive slot and get filtered out afterwards. + // Memory capsules carry the generic `kind: "memory"` and encode the + // real memory kind in the summary prefix ("scope:kind - text"), so + // match against that for memories. + if !request.kinds.is_empty() { + candidates.retain(|c| { + request + .kinds + .iter() + .any(|k| capsule_matches_kind(&c.capsule, k)) + }); + } + normalize_and_score(&mut candidates, weights_for_stage(weights, &request.stage)); // v0.6: apply tag boost (1.4×) and role-preference boost (1.3×) after @@ -1025,7 +1047,24 @@ const CLASS_HINTS: &[(&[&str], &[&str])] = &[ (&["rename", "move file", "mv "], &["move_file"]), ]; -fn fts_query(query: &str) -> Option { +/// v0.8: does a capsule satisfy a requested (memory) kind? Repo/manifest +/// capsules match only by their literal `kind`; memory capsules +/// (`kind == "memory"`) match against the real kind embedded in their +/// `"scope:kind - text"` summary prefix. +fn capsule_matches_kind(capsule: &ContextCapsule, wanted: &str) -> bool { + if capsule.kind == wanted { + return true; + } + if capsule.kind == "memory" + && let Some((prefix, _)) = capsule.summary.split_once(" - ") + && let Some((_scope, mkind)) = prefix.split_once(':') + { + return mkind == wanted; + } + false +} + +pub(crate) fn fts_query(query: &str) -> Option { let tokens = query_tokens(query); if tokens.is_empty() { return None; @@ -1149,6 +1188,34 @@ fn one_line(text: &str) -> String { mod tests { use super::*; + fn capsule(kind: &str, summary: &str) -> ContextCapsule { + ContextCapsule { + id: "c".into(), + kind: kind.into(), + summary: summary.into(), + token_estimate: 1, + expansion_handle: "memory:x".into(), + provenance: vec![], + confidence: 1.0, + freshness: 1.0, + relevance: 1.0, + scope_weight: 1.0, + score: 1.0, + } + } + + #[test] + fn capsule_matches_kind_reads_memory_summary_prefix() { + // Memory capsule: real kind lives in the "scope:kind - text" prefix. + let mem = capsule("memory", "project:failure_pattern - linker not found"); + assert!(capsule_matches_kind(&mem, "failure_pattern")); + assert!(!capsule_matches_kind(&mem, "command")); + // Non-memory capsules match only by literal kind, never via prefix. + let repo = capsule("repo_file", "src/lib.rs:command - run build"); + assert!(capsule_matches_kind(&repo, "repo_file")); + assert!(!capsule_matches_kind(&repo, "command")); + } + /// MP-17e: zero-use rows are neutral (no data); use_count >= 1 starts /// blending toward the full multiplier (Bayesian smoothing). #[test] diff --git a/crates/kimetsu-brain/src/embeddings.rs b/crates/kimetsu-brain/src/embeddings.rs index 55512e2..df32bde 100644 --- a/crates/kimetsu-brain/src/embeddings.rs +++ b/crates/kimetsu-brain/src/embeddings.rs @@ -267,20 +267,132 @@ fn build_default_embedder() -> Box { Box::new(NoopEmbedder) } +/// v0.8: open a FRESH (uncached) embedder for an explicit built-in +/// model id. Unlike [`open_default_embedder`], this bypasses the +/// process-static cache AND the env/override resolution — the caller +/// asked for a *specific* model (e.g. `kimetsu brain model set` and the +/// MCP `model_set` reindex, which must re-embed with the newly-chosen +/// model even though the running process may have a different default +/// embedder cached). Returns [`NoopEmbedder`] on the lean build or if +/// the model fails to load. +pub fn open_embedder_for_model(model_id: &str) -> Box { + #[cfg(feature = "embeddings")] + { + match fastembed_backend::FastembedEmbedder::try_open(model_id) { + Ok(engine) => return Box::new(engine), + Err(err) => { + eprintln!( + "kimetsu-brain: failed to open embedder `{model_id}` ({err}); \ + using NoopEmbedder (no vectors produced)." + ); + } + } + } + #[cfg(not(feature = "embeddings"))] + { + let _ = model_id; + } + Box::new(NoopEmbedder) +} + /// v0.4.3: env-driven kill switch. Truthy values (1/true/yes/on) /// force-disable the embedder for this process; "noop", "off", /// "none" do the same. Anything else (or unset) leaves the /// `embeddings` feature in control. fn env_disables_embedder() -> bool { match std::env::var("KIMETSU_BRAIN_EMBEDDER") { - Ok(value) => { - let v = value.trim().to_ascii_lowercase(); - matches!(v.as_str(), "noop" | "off" | "none" | "0" | "false" | "no") - } + Ok(value) => is_disable_value(&value.trim().to_ascii_lowercase()), Err(_) => false, } } +fn is_disable_value(v: &str) -> bool { + matches!(v, "noop" | "off" | "none" | "0" | "false" | "no") +} + +/// v0.8: curated built-in embedding models, surfaced by +/// `kimetsu brain model list` and `kimetsu_brain_model_list`. Tuple +/// = (stable id, vector dimension, human blurb). This is the single +/// source of truth for the selectable set; the fastembed backend +/// maps these ids → `EmbeddingModel` in `try_open`. +pub const BUILTIN_MODELS: &[(&str, usize, &str)] = &[ + ("bge-small-en-v1.5", 384, "English, default, ~67 MB int8"), + ("bge-m3", 1024, "Multilingual, ~600 MB int8"), + ( + "jina-v2-base-code", + 768, + "English + code-tuned, ~165 MB int8", + ), +]; + +/// v0.8: process-global embedder override recorded by +/// [`apply_embedder_selection`]. Brain-internal callers +/// ([`pick_builtin_model_from_env`], the fastembed backend, reindex) +/// have no `ProjectConfig` in hand, so the CLI/MCP layer stashes the +/// config-selected id here once, early, before any embed happens. +static EMBEDDER_OVERRIDE: std::sync::OnceLock = std::sync::OnceLock::new(); + +/// v0.8: record the config-provided embedder id so brain-internal +/// callers resolve it when `KIMETSU_BRAIN_EMBEDDER` is unset (the env +/// var always wins). Call once, early, before the first retrieval or +/// embed — after the embedder `OnceLock` initializes this has no +/// effect. No-op for `None`/empty, and only the first call sticks. +pub fn apply_embedder_selection(config_embedder: Option<&str>) { + if let Some(id) = config_embedder { + let id = id.trim(); + if !id.is_empty() { + let _ = EMBEDDER_OVERRIDE.set(id.to_string()); + } + } +} + +/// v0.8: map any accepted alias (env value, config value, built-in +/// id) to a stable built-in id. Unknown values warn and fall back to +/// the lean English default. Disable values map to the default too; +/// the *actual* disable is handled separately by +/// [`env_disables_embedder`]. +fn map_builtin_id(v: &str) -> &'static str { + match v { + "" | "default" | "bge-small" | "bge-small-en-v1.5" => "bge-small-en-v1.5", + "bge-m3" | "m3" => "bge-m3", + "jina-code" | "jina-v2-base-code" | "jina-embeddings-v2-base-code" => "jina-v2-base-code", + "noop" | "off" | "none" | "0" | "false" | "no" => "bge-small-en-v1.5", + other => { + eprintln!( + "kimetsu-brain: unknown embedder {other:?}, \ + falling back to bge-small-en-v1.5" + ); + "bge-small-en-v1.5" + } + } +} + +/// v0.8: resolve the active built-in model id. Precedence: +/// 1. `KIMETSU_BRAIN_EMBEDDER` env (unless it's a disable value) +/// 2. the explicit `config_embedder` arg, else the override set by +/// [`apply_embedder_selection`] +/// 3. `bge-small-en-v1.5` default +pub fn resolve_embedder_id(config_embedder: Option<&str>) -> &'static str { + if let Ok(raw) = std::env::var("KIMETSU_BRAIN_EMBEDDER") { + let v = raw.trim().to_ascii_lowercase(); + if !v.is_empty() && !is_disable_value(&v) { + return map_builtin_id(&v); + } + // empty / disable values fall through: the model *id* still + // resolves from config/default even when retrieval is off. + } + let cfg = config_embedder + .map(str::to_string) + .or_else(|| EMBEDDER_OVERRIDE.get().cloned()); + if let Some(c) = cfg { + let v = c.trim().to_ascii_lowercase(); + if !v.is_empty() { + return map_builtin_id(&v); + } + } + "bge-small-en-v1.5" +} + /// v0.4.3: pick which builtin model to load from the env, returning /// a stable identifier. Used both by the fastembed backend (to map /// id → `EmbeddingModel`) and by `kimetsu brain reindex` (to label @@ -297,27 +409,10 @@ fn env_disables_embedder() -> bool { /// code-tuned) /// * anything else falls back to bge-small with a warning. pub fn pick_builtin_model_from_env() -> &'static str { - let raw = std::env::var("KIMETSU_BRAIN_EMBEDDER").ok(); - let v = raw - .as_deref() - .map(|s| s.trim().to_ascii_lowercase()) - .unwrap_or_default(); - match v.as_str() { - "" | "default" | "bge-small" | "bge-small-en-v1.5" => "bge-small-en-v1.5", - "bge-m3" | "m3" => "bge-m3", - "jina-code" | "jina-v2-base-code" | "jina-embeddings-v2-base-code" => "jina-v2-base-code", - // "noop"/etc. are handled by env_disables_embedder; this - // function shouldn't be called in those cases but defensively - // pick the lean default. - "noop" | "off" | "none" | "0" | "false" | "no" => "bge-small-en-v1.5", - other => { - eprintln!( - "kimetsu-brain: unknown KIMETSU_BRAIN_EMBEDDER={other:?}, \ - falling back to bge-small-en-v1.5" - ); - "bge-small-en-v1.5" - } - } + // v0.8: env > config-override (set via `apply_embedder_selection`) + // > default. Kept as a named entry point for the fastembed backend + // and `reindex`, which have no `ProjectConfig` to pass. + resolve_embedder_id(None) } // v0.4.3: real fastembed-backed embedder. Lives behind the @@ -537,6 +632,49 @@ pub fn decode_embedding(bytes: &[u8], expected_dim: Option) -> KimetsuRes mod tests { use super::*; + #[test] + fn map_builtin_id_maps_aliases_and_defaults_unknown() { + assert_eq!(map_builtin_id("bge-small-en-v1.5"), "bge-small-en-v1.5"); + assert_eq!(map_builtin_id("default"), "bge-small-en-v1.5"); + assert_eq!(map_builtin_id("m3"), "bge-m3"); + assert_eq!(map_builtin_id("bge-m3"), "bge-m3"); + assert_eq!(map_builtin_id("jina-code"), "jina-v2-base-code"); + assert_eq!( + map_builtin_id("jina-embeddings-v2-base-code"), + "jina-v2-base-code" + ); + // disable values resolve to the lean default (the kill-switch + // is handled separately by env_disables_embedder). + assert_eq!(map_builtin_id("noop"), "bge-small-en-v1.5"); + // unknown -> warn + default. + assert_eq!(map_builtin_id("totally-made-up"), "bge-small-en-v1.5"); + } + + #[test] + fn builtin_models_table_is_consistent() { + // Every advertised id must map back to itself. + for (id, _dim, _blurb) in BUILTIN_MODELS { + assert_eq!(map_builtin_id(id), *id, "id {id} must be stable"); + } + } + + #[test] + fn resolve_embedder_id_uses_config_when_env_unset() { + // Env mutation in parallel tests is racy + unsafe in edition + // 2024, so we only assert the config/default paths when the env + // var is genuinely absent. The env-wins path is exercised + // manually (see the plan's verification section). + if std::env::var_os("KIMETSU_BRAIN_EMBEDDER").is_some() { + return; + } + assert_eq!(resolve_embedder_id(Some("bge-m3")), "bge-m3"); + assert_eq!(resolve_embedder_id(Some("jina-code")), "jina-v2-base-code"); + // unknown config value -> default. + assert_eq!(resolve_embedder_id(Some("nope")), "bge-small-en-v1.5"); + // None + no override stored in this test binary -> default. + assert_eq!(resolve_embedder_id(None), "bge-small-en-v1.5"); + } + #[test] fn noop_embedder_returns_not_implemented_and_is_noop() { let e = NoopEmbedder; diff --git a/crates/kimetsu-brain/src/project.rs b/crates/kimetsu-brain/src/project.rs index 05160ee..ba42897 100644 --- a/crates/kimetsu-brain/src/project.rs +++ b/crates/kimetsu-brain/src/project.rs @@ -58,6 +58,18 @@ pub struct MemoryRow { pub usefulness_score: f32, } +/// v0.8: a full-text search hit over memory text, returned by +/// [`search_memories`] and the `kimetsu_brain_memory_search` MCP tool. +/// `rank` is the BM25-derived relevance (higher = more relevant). +#[derive(Debug, Clone)] +pub struct MemorySearchHit { + pub memory_id: String, + pub scope: String, + pub kind: String, + pub text: String, + pub rank: f32, +} + #[derive(Debug, Clone)] pub struct ProposalRow { pub proposal_id: String, @@ -79,6 +91,9 @@ pub struct ProposalFilter { pub min_confidence: Option, pub status: Option, pub limit: u32, + /// v0.8: row offset for paginated navigation from the MCP surface. + /// 0 = first page (prior behaviour). + pub offset: u32, } #[derive(Debug, Clone, Default)] @@ -289,6 +304,24 @@ impl BrainSession { ) } + /// v0.8: proactive (mid-work) retrieval. Pins [`NoopEmbedder`] so + /// it stays lexical-FTS-only — NO embedding model is loaded even in + /// `--features embeddings` builds, keeping the per-tool-call hook + /// cheap. `request.kinds` should restrict to actionable kinds; the + /// caller sets a high `min_score` and `max_capsules: 1` so recall is + /// rare and confident (the human-brain "it comes to you" model). + pub fn retrieve_proactive(&self, request: ContextRequest) -> KimetsuResult { + let extras: Vec<&Connection> = self.user_conn.as_ref().into_iter().collect(); + context::retrieve_context_with_embedder( + &self.conn, + &self.repo_root, + &self.config.broker.weights, + request, + &extras, + &embeddings::NoopEmbedder, + ) + } + pub fn repo_root(&self) -> &Path { &self.paths.repo_root } @@ -645,13 +678,43 @@ pub fn propose_or_merge_memory( } } +/// v0.8: pagination + scope filter for `list_memories_with`, surfaced +/// by the `kimetsu_brain_memory_list` MCP tool so an agent can page +/// through the corpus from inside Claude/Codex. +#[derive(Debug, Clone)] +pub struct ListOptions { + /// Max project rows to return. 0 → 100 (the prior default). + pub limit: u32, + /// Project-row offset (for paging). 0 → first page. + pub offset: u32, + /// Optional scope filter (global_user / project / repo / run). + pub scope: Option, +} + +impl Default for ListOptions { + fn default() -> Self { + Self { + limit: 100, + offset: 0, + scope: None, + } + } +} + pub fn list_memories(start: &Path) -> KimetsuResult> { + list_memories_with(start, ListOptions::default()) +} + +/// v0.8: paginated/scoped memory listing. The project page is bounded +/// by `limit`/`offset`; the user brain's portable rows are appended +/// only on the first page (`offset == 0`) so they appear exactly once +/// during navigation rather than on every page. +pub fn list_memories_with(start: &Path, opts: ListOptions) -> KimetsuResult> { let (_paths, _config, conn) = load_project(start)?; - let mut memories = list_memories_from_conn(&conn)?; - // v0.4.1: merge user-brain rows so the user sees their portable - // capsules alongside the per-repo ones. We read user brain via - // the read-only path (no file-create side-effect on list). - if let Some(user_conn) = user_brain::open_user_brain_readonly()? { + let mut memories = list_memories_from_conn(&conn, &opts)?; + if opts.offset == 0 + && let Some(user_conn) = user_brain::open_user_brain_readonly()? + { memories.extend(user_brain::list_user_memories(&user_conn)?); } Ok(memories) @@ -853,33 +916,42 @@ fn text_preview(text: &str, max_chars: usize) -> String { } } -fn list_memories_from_conn(conn: &Connection) -> KimetsuResult> { - let mut stmt = conn.prepare( - " - SELECT memory_id, scope, kind, text, confidence, use_count, usefulness_score - FROM memories - ORDER BY created_at DESC - LIMIT 100 - ", - )?; +fn list_memories_from_conn(conn: &Connection, opts: &ListOptions) -> KimetsuResult> { + let limit = if opts.limit == 0 { 100 } else { opts.limit } as i64; + let offset = opts.offset as i64; - let rows = stmt.query_map([], |row| { - Ok(MemoryRow { - memory_id: row.get(0)?, - scope: row.get(1)?, - kind: row.get(2)?, - text: row.get(3)?, - confidence: row.get(4)?, - use_count: row.get(5)?, - usefulness_score: row.get::<_, f64>(6)? as f32, - }) - })?; + let (sql, scope_param): (&str, Option) = if let Some(scope) = opts.scope.as_deref() { + ( + " + SELECT memory_id, scope, kind, text, confidence, use_count, usefulness_score + FROM memories + WHERE lower(scope) = lower(?1) + ORDER BY created_at DESC + LIMIT ?2 OFFSET ?3 + ", + Some(scope.to_string()), + ) + } else { + ( + " + SELECT memory_id, scope, kind, text, confidence, use_count, usefulness_score + FROM memories + ORDER BY created_at DESC + LIMIT ?1 OFFSET ?2 + ", + None, + ) + }; - let mut memories = Vec::new(); - for row in rows { - memories.push(row?); - } - Ok(memories) + let mut stmt = conn.prepare(sql)?; + let rows = if let Some(scope) = scope_param { + stmt.query_map(params![scope, limit, offset], map_memory_row)? + .collect::, _>>()? + } else { + stmt.query_map(params![limit, offset], map_memory_row)? + .collect::, _>>()? + }; + Ok(rows) } /// MP-6: ranked list of memories sorted by the same usefulness ratio the @@ -1140,7 +1212,10 @@ pub fn list_proposals(start: &Path, filter: ProposalFilter) -> KimetsuResult = params.iter().map(|p| p.as_ref()).collect(); @@ -1236,6 +1311,92 @@ pub fn retrieve_context_readonly_with_request( BrainSession::open_readonly(start)?.retrieve_context_with_request(request) } +/// v0.8: read-only proactive retrieval (lexical-FTS-only, no model +/// load). The caller builds a `ContextRequest` with `kinds` set to the +/// actionable set, a high `min_score`, and `max_capsules: 1`. +pub fn retrieve_proactive_readonly( + start: &Path, + request: ContextRequest, +) -> KimetsuResult { + BrainSession::open_readonly(start)?.retrieve_proactive(request) +} + +/// v0.8: full-text search over memory text, for navigating the corpus +/// from the MCP surface. Project rows are paged by `limit`/`offset`; +/// user-brain rows are appended only on the first page so they appear +/// once. Returns empty when the query yields no FTS tokens. +pub fn search_memories( + start: &Path, + query: &str, + limit: u32, + offset: u32, + kind: Option<&str>, + scope: Option<&str>, +) -> KimetsuResult> { + let Some(fts) = context::fts_query(query) else { + return Ok(Vec::new()); + }; + let (_paths, _config, conn) = load_project(start)?; + let mut hits = search_memories_in_conn(&conn, &fts, limit, offset, kind, scope)?; + if offset == 0 + && let Some(user_conn) = user_brain::open_user_brain_readonly()? + { + hits.extend(search_memories_in_conn( + &user_conn, &fts, limit, 0, kind, scope, + )?); + } + Ok(hits) +} + +fn search_memories_in_conn( + conn: &Connection, + fts_query: &str, + limit: u32, + offset: u32, + kind: Option<&str>, + scope: Option<&str>, +) -> KimetsuResult> { + let limit = if limit == 0 { 20 } else { limit } as i64; + let offset = offset as i64; + let mut sql = String::from( + " + SELECT m.memory_id, m.scope, m.kind, m.text, bm25(memories_fts) AS rank + FROM memories_fts + JOIN memories m ON m.memory_id = memories_fts.memory_id + WHERE m.invalidated_at IS NULL + AND memories_fts MATCH ? + ", + ); + let mut bind: Vec> = vec![Box::new(fts_query.to_string())]; + if let Some(k) = kind { + sql.push_str(" AND m.kind = ?"); + bind.push(Box::new(k.to_string())); + } + if let Some(s) = scope { + sql.push_str(" AND lower(m.scope) = lower(?)"); + bind.push(Box::new(s.to_string())); + } + // bm25() is more-negative = more-relevant, so ascending rank is best. + sql.push_str(" ORDER BY rank LIMIT ? OFFSET ?"); + bind.push(Box::new(limit)); + bind.push(Box::new(offset)); + + let mut stmt = conn.prepare(&sql)?; + let refs: Vec<&dyn rusqlite::ToSql> = bind.iter().map(|b| b.as_ref()).collect(); + let rows = stmt.query_map(refs.as_slice(), |row| { + let raw_rank = row.get::<_, f64>(4)? as f32; + Ok(MemorySearchHit { + memory_id: row.get(0)?, + scope: row.get(1)?, + kind: row.get(2)?, + text: row.get(3)?, + // surface a positive relevance (higher = better) for callers. + rank: (-raw_rank).max(0.0), + }) + })?; + rows.collect::, _>>().map_err(Into::into) +} + #[allow(clippy::too_many_arguments)] pub fn retrieve_benchmark_context_readonly( start: &Path, @@ -1710,10 +1871,159 @@ mod tests { // `user_brain::tests` and opt-in via `with_user_brain_at`. use crate::user_brain::with_user_brain_disabled; + /// v0.8: create an isolated temp project root. A minimal `git init` + /// gives the dir its own git toplevel so `ProjectPaths::discover` + /// resolves to THIS dir instead of climbing to an enclosing repo + /// (e.g. a developer's `$HOME` git repo) — which would otherwise + /// make parallel tests share one brain.db + project.lock. Without + /// this, tests pass only when `TMP` points outside any git repo. + fn test_root() -> std::path::PathBuf { + let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + kimetsu_core::paths::git_init_boundary(&root); + root + } + + #[test] + fn search_memories_paginates_and_filters_by_kind() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + add_memory( + &root, + MemoryScope::Project, + MemoryKind::FailurePattern, + "linker link.exe not found on windows", + ) + .expect("add fp"); + add_memory( + &root, + MemoryScope::Project, + MemoryKind::Command, + "run cargo build with the link.exe linker on PATH", + ) + .expect("add cmd"); + add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "the office plant needs watering on tuesdays", + ) + .expect("add fact"); + + // "linker" matches the two link.exe memories, not the plant fact. + let hits = search_memories(&root, "linker", 10, 0, None, None).expect("search"); + assert!(hits.len() >= 2, "expected >=2 hits, got {}", hits.len()); + assert!( + hits.iter() + .all(|h| h.text.to_ascii_lowercase().contains("link")) + ); + + // Pagination: two single-row pages return distinct rows. + let p1 = search_memories(&root, "linker", 1, 0, None, None).expect("p1"); + let p2 = search_memories(&root, "linker", 1, 1, None, None).expect("p2"); + assert_eq!(p1.len(), 1); + assert_eq!(p2.len(), 1); + assert_ne!(p1[0].memory_id, p2[0].memory_id, "offset must advance"); + + // Kind filter narrows to failure_pattern only. + let fp = + search_memories(&root, "linker", 10, 0, Some("failure_pattern"), None).expect("fp"); + assert!(!fp.is_empty()); + assert!(fp.iter().all(|h| h.kind == "failure_pattern")); + + // A query with no FTS tokens returns empty, not an error. + assert!( + search_memories(&root, " ", 10, 0, None, None) + .unwrap() + .is_empty() + ); + }); + } + + #[test] + fn reindex_with_explicit_embedder_uses_that_model() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "alpha beta gamma", + ) + .expect("add"); + // The explicit-embedder path (used by `model set`) must + // re-embed with the GIVEN embedder, regardless of the + // process default. + use crate::embeddings::Embedder as _; + let stub = crate::embeddings::StubEmbedder::new(); + let report = crate::reindex::reindex_all_with_embedder( + &root, + crate::reindex::ReindexOptions { + scope: crate::reindex::ReindexScope::Project, + dry_run: false, + force: false, + limit: None, + }, + &stub, + ) + .expect("reindex"); + assert_eq!(report.embedder_model_id, stub.model_id()); + assert!( + report.project.updated >= 1, + "the row should be re-embedded with the stub model" + ); + }); + } + + #[test] + fn retrieve_proactive_returns_actionable_kind_and_excludes_others() { + with_user_brain_disabled(|| { + let root = test_root(); + init_project(&root, false).expect("init"); + add_memory( + &root, + MemoryScope::Project, + MemoryKind::FailurePattern, + "linker link.exe not found -> run from x64 Native Tools prompt", + ) + .expect("add fp"); + // A high-overlap FACT that would outrank lexically but is NOT an + // actionable kind — the kinds filter must drop it. + add_memory( + &root, + MemoryScope::Project, + MemoryKind::Fact, + "linker link.exe trivia: link.exe ships with MSVC", + ) + .expect("add fact"); + + let request = ContextRequest { + stage: "localization".to_string(), + query: "error: linker `link.exe` not found".to_string(), + budget_tokens: 600, + min_score: 0.2, + max_capsules: 1, + kinds: vec!["failure_pattern".to_string(), "command".to_string()], + ..Default::default() + }; + let bundle = retrieve_proactive_readonly(&root, request).expect("proactive"); + assert!(!bundle.skipped, "should surface the failure_pattern"); + assert_eq!(bundle.capsules.len(), 1); + // The single capsule must be the failure_pattern, not the fact. + assert!( + bundle.capsules[0].summary.contains("failure_pattern"), + "got summary: {}", + bundle.capsules[0].summary + ); + assert!(!bundle.capsules[0].summary.contains("trivia")); + }); + } + #[test] fn memory_add_survives_projection_rebuild_from_trace() { with_user_brain_disabled(|| { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); @@ -1750,7 +2060,7 @@ mod tests { #[test] fn add_memory_redacts_secrets_before_persist() { with_user_brain_disabled(|| { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); @@ -1785,7 +2095,7 @@ mod tests { #[test] fn repo_ingest_indexes_searchable_files_and_context_capsules() { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(root.join("src")).expect("create src"); fs::create_dir_all(root.join("target")).expect("create target"); fs::write( @@ -1866,7 +2176,7 @@ mod tests { // // Per-run counting: the same memory injected into two // stages of one run still counts once. - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); let memory_id = add_memory( @@ -1952,7 +2262,7 @@ mod tests { #[test] fn run_finished_gives_weak_signal_to_silent_passenger_memories() { with_user_brain_disabled(|| { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); let memory_id = add_memory( @@ -2015,7 +2325,7 @@ mod tests { #[test] fn blame_run_separates_cited_from_silent_passengers() { with_user_brain_disabled(|| { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); let cited_id = add_memory( @@ -2098,7 +2408,7 @@ mod tests { // run.failed with category != "Gate" decrements; category == "Gate" // is a graceful early-exit (e.g. the plan-create existence guard) // and must not blame memories that happened to be in context. - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); let memory_id = add_memory( @@ -2209,7 +2519,7 @@ mod tests { #[test] fn run_aborted_does_not_update_usefulness() { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); let memory_id = add_memory( @@ -2267,7 +2577,7 @@ mod tests { #[test] fn list_proposals_filters_and_reject_records_reason() { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); @@ -2390,7 +2700,7 @@ mod tests { /// projection rebuild (event is canonical). #[test] fn invalidate_memory_persists_invalidated_metadata_and_survives_rebuild() { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); @@ -2446,7 +2756,7 @@ mod tests { #[test] fn invalidated_memory_is_excluded_from_broker_retrieval() { with_user_brain_disabled(|| { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); @@ -2495,7 +2805,7 @@ mod tests { /// only shows entries the broker bias actually applies to. #[test] fn list_memories_top_sorts_by_usefulness_ratio_and_drops_small_samples() { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); @@ -2574,7 +2884,7 @@ mod tests { } fn prune_low_usefulness_dry_run_then_apply_body() { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); @@ -2728,7 +3038,7 @@ mod tests { } fn batch_review_accepts_filtered_subset_and_rejects_remainder_body() { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); @@ -2889,7 +3199,7 @@ mod tests { #[test] fn add_memory_distinct_texts_no_conflicts() { with_user_brain_disabled(|| { - let root = std::env::temp_dir().join(format!("kimetsu-test-{}", Ulid::new())); + let root = test_root(); fs::create_dir_all(&root).expect("create temp project"); init_project(&root, false).expect("init project"); diff --git a/crates/kimetsu-brain/src/reindex.rs b/crates/kimetsu-brain/src/reindex.rs index f69537b..758b672 100644 --- a/crates/kimetsu-brain/src/reindex.rs +++ b/crates/kimetsu-brain/src/reindex.rs @@ -119,7 +119,19 @@ impl Default for ReindexOptions { /// rows whose stored `embedding_model` doesn't match the active /// embedder. pub fn reindex_all(repo_start: &Path, opts: ReindexOptions) -> KimetsuResult { - let embedder = embeddings::open_default_embedder(); + reindex_all_with_embedder(repo_start, opts, embeddings::open_default_embedder()) +} + +/// v0.8: reindex against an EXPLICIT embedder rather than the process +/// default. Used by `model set` (CLI + MCP) so the corpus is re-embedded +/// with the freshly-chosen model — see +/// [`embeddings::open_embedder_for_model`] — regardless of which model +/// the running process loaded into its static cache. +pub fn reindex_all_with_embedder( + repo_start: &Path, + opts: ReindexOptions, + embedder: &(dyn Embedder + Send + Sync), +) -> KimetsuResult { let model_id = embedder.model_id().to_string(); let noop = embedder.is_noop(); diff --git a/crates/kimetsu-chat/Cargo.toml b/crates/kimetsu-chat/Cargo.toml index 8d90620..c71e254 100644 --- a/crates/kimetsu-chat/Cargo.toml +++ b/crates/kimetsu-chat/Cargo.toml @@ -36,9 +36,9 @@ embeddings = ["kimetsu-brain/embeddings"] # surface, not a benchmark harness. [dependencies] -kimetsu-agent = { path = "../kimetsu-agent", version = "0.7.3" } -kimetsu-brain = { path = "../kimetsu-brain", version = "0.7.3" } -kimetsu-core = { path = "../kimetsu-core", version = "0.7.3" } +kimetsu-agent = { path = "../kimetsu-agent", version = "0.8.0" } +kimetsu-brain = { path = "../kimetsu-brain", version = "0.8.0" } +kimetsu-core = { path = "../kimetsu-core", version = "0.8.0" } base64.workspace = true crossterm.workspace = true reqwest.workspace = true diff --git a/crates/kimetsu-chat/src/bridge.rs b/crates/kimetsu-chat/src/bridge.rs index 565bad3..c5645bd 100644 --- a/crates/kimetsu-chat/src/bridge.rs +++ b/crates/kimetsu-chat/src/bridge.rs @@ -336,6 +336,7 @@ pub fn plugin_install( target: BridgeTarget, mode: PluginMode, force: bool, + proactive: bool, ) -> Result { let workspace = normalize_path(workspace); let mut files = Vec::new(); @@ -373,7 +374,7 @@ pub fn plugin_install( // Claude Code hooks live in `.claude/settings.json` under the // `hooks` key (keyed by real events, fed via stdin JSON) — not // standalone `.ps1`/`.sh` files driven by env vars. - write_claude_settings(&workspace, force, &mut files)?; + write_claude_settings(&workspace, force, proactive, &mut files)?; } BridgeTarget::Codex => { let config = workspace.join(".codex").join("config.toml"); @@ -394,7 +395,7 @@ pub fn plugin_install( force, )?; files.push(normalize_path(&skill)); - write_codex_hooks(&workspace, force, &mut files)?; + write_codex_hooks(&workspace, force, proactive, &mut files)?; } BridgeTarget::Kimetsu => { let dir = workspace.join(".kimetsu").join("extensions"); @@ -416,6 +417,7 @@ pub fn extensions_root(workspace: &Path) -> PathBuf { fn write_codex_hooks( workspace: &Path, force: bool, + proactive: bool, files: &mut Vec, ) -> Result<(), String> { let hooks = workspace.join(".codex").join("hooks.json"); @@ -454,6 +456,35 @@ fn write_codex_hooks( }] }]), ); + if proactive { + // v0.8: Codex supports PreToolUse/PostToolUse with a regex + // matcher on tool name (Bash). Both surface a relevant memory + // via hookSpecificOutput.additionalContext without blocking. + hooks_obj.insert( + "PreToolUse".to_string(), + serde_json::json!([{ + "matcher": "Bash", + "hooks": [{ + "type": "command", + "command": "kimetsu brain pretool-hook --workspace .", + "statusMessage": "Kimetsu proactive check", + "timeout": 15 + }] + }]), + ); + hooks_obj.insert( + "PostToolUse".to_string(), + serde_json::json!([{ + "matcher": "Bash", + "hooks": [{ + "type": "command", + "command": "kimetsu brain posttool-hook --workspace .", + "statusMessage": "Kimetsu proactive check", + "timeout": 15 + }] + }]), + ); + } let text = serde_json::to_string_pretty(&root) .map_err(|err| format!("serialize Codex hooks: {err}"))?; write_text_file(&hooks, &text, true)?; @@ -629,6 +660,7 @@ effort or that you would want to remember next session. fn write_claude_settings( workspace: &Path, force: bool, + proactive: bool, files: &mut Vec, ) -> Result<(), String> { let claude_dir = workspace.join(".claude"); @@ -643,14 +675,15 @@ fn write_claude_settings( files.push(normalize_path(&claude_md)); let settings = claude_dir.join("settings.json"); - write_claude_hooks(&settings, force)?; + write_claude_hooks(&settings, force, proactive)?; files.push(normalize_path(&settings)); Ok(()) } -/// Merge Kimetsu's `UserPromptSubmit`/`Stop` hooks into `.claude/settings.json`, -/// preserving any other settings already present. -fn write_claude_hooks(path: &Path, force: bool) -> Result<(), String> { +/// Merge Kimetsu's `UserPromptSubmit`/`Stop` hooks (plus the v0.8 +/// proactive `PreToolUse`/`PostToolUse` Bash hooks when `proactive`) +/// into `.claude/settings.json`, preserving any other settings. +fn write_claude_hooks(path: &Path, force: bool, proactive: bool) -> Result<(), String> { let mut root = if path.is_file() { let text = fs::read_to_string(path).map_err(|err| format!("read {}: {err}", path.display()))?; @@ -668,19 +701,33 @@ fn write_claude_hooks(path: &Path, force: bool) -> Result<(), String> { path.display() )); } - root_obj.insert( - "hooks".to_string(), - serde_json::json!({ - "UserPromptSubmit": [{ - "matcher": "", - "hooks": [{ "type": "command", "command": "kimetsu brain context-hook" }] - }], - "Stop": [{ - "matcher": "", - "hooks": [{ "type": "command", "command": "kimetsu brain stop-hook" }] - }] - }), - ); + let mut hooks = serde_json::json!({ + "UserPromptSubmit": [{ + "matcher": "", + "hooks": [{ "type": "command", "command": "kimetsu brain context-hook" }] + }], + "Stop": [{ + "matcher": "", + "hooks": [{ "type": "command", "command": "kimetsu brain stop-hook" }] + }] + }); + if proactive && let Some(map) = hooks.as_object_mut() { + map.insert( + "PreToolUse".to_string(), + serde_json::json!([{ + "matcher": "Bash", + "hooks": [{ "type": "command", "command": "kimetsu brain pretool-hook" }] + }]), + ); + map.insert( + "PostToolUse".to_string(), + serde_json::json!([{ + "matcher": "Bash", + "hooks": [{ "type": "command", "command": "kimetsu brain posttool-hook" }] + }]), + ); + } + root_obj.insert("hooks".to_string(), hooks); let text = serde_json::to_string_pretty(&root) .map_err(|err| format!("serialize Claude settings: {err}"))?; write_text_file(path, &text, true) @@ -840,8 +887,14 @@ mod tests { fn plugin_install_writes_optional_and_required_modes() { let root = temp_root("plugin_install_modes"); - let optional = plugin_install(&root, BridgeTarget::Codex, PluginMode::Optional, false) - .expect("optional install"); + let optional = plugin_install( + &root, + BridgeTarget::Codex, + PluginMode::Optional, + false, + true, + ) + .expect("optional install"); assert_eq!(optional.mode, PluginMode::Optional); let skill_path = root.join(".codex/skills/kimetsu-bridge/SKILL.md"); let optional_text = fs::read_to_string(&skill_path).expect("optional skill"); @@ -863,10 +916,19 @@ mod tests { hooks_json["hooks"]["UserPromptSubmit"][0]["hooks"][0]["command"].as_str(), Some("kimetsu brain context-hook --workspace .") ); + // v0.8: proactive on by default wires the Bash PreToolUse/PostToolUse hooks. + assert_eq!( + hooks_json["hooks"]["PostToolUse"][0]["matcher"].as_str(), + Some("Bash") + ); + assert_eq!( + hooks_json["hooks"]["PreToolUse"][0]["hooks"][0]["command"].as_str(), + Some("kimetsu brain pretool-hook --workspace .") + ); assert!(!root.join(".codex/mcp.json").exists()); assert!(!root.join(".codex/hooks/pre-turn.ps1").exists()); - let required = plugin_install(&root, BridgeTarget::Codex, PluginMode::Required, true) + let required = plugin_install(&root, BridgeTarget::Codex, PluginMode::Required, true, true) .expect("required install"); assert_eq!(required.mode, PluginMode::Required); let required_text = fs::read_to_string(&skill_path).expect("required skill"); @@ -885,6 +947,28 @@ mod tests { fs::remove_dir_all(root).ok(); } + #[test] + fn plugin_install_no_proactive_skips_tool_hooks() { + let root = temp_root("plugin_install_no_proactive"); + plugin_install( + &root, + BridgeTarget::Codex, + PluginMode::Optional, + false, + false, + ) + .expect("install without proactive"); + let hooks_text = fs::read_to_string(root.join(".codex/hooks.json")).expect("codex hooks"); + let hooks_json: serde_json::Value = serde_json::from_str(&hooks_text).expect("hooks json"); + assert!(hooks_json["hooks"]["UserPromptSubmit"].is_array()); + assert!( + hooks_json["hooks"]["PreToolUse"].is_null(), + "proactive disabled must not write PreToolUse" + ); + assert!(hooks_json["hooks"]["PostToolUse"].is_null()); + fs::remove_dir_all(root).ok(); + } + fn temp_root(label: &str) -> PathBuf { let nanos = SystemTime::now() .duration_since(UNIX_EPOCH) diff --git a/crates/kimetsu-chat/src/mcp_server.rs b/crates/kimetsu-chat/src/mcp_server.rs index ea1cf3f..642bb47 100644 --- a/crates/kimetsu-chat/src/mcp_server.rs +++ b/crates/kimetsu-chat/src/mcp_server.rs @@ -87,6 +87,14 @@ pub fn serve_mcp( .workspace .canonicalize() .unwrap_or_else(|_| config.workspace.clone()); + // v0.8: honor the [embedder] config (env still wins) before any + // retrieval initializes the process-static embedder. A server + // started after `model set` therefore loads the configured model. + if let Ok(paths) = kimetsu_core::paths::ProjectPaths::discover(&workspace) + && let Ok(project_config) = project::load_config(&paths) + { + kimetsu_brain::embeddings::apply_embedder_selection(Some(&project_config.embedder.model)); + } for line in reader.lines() { let line = line.map_err(|err| format!("read MCP stdin: {err}"))?; let line = line.trim_start_matches('\u{feff}'); @@ -338,13 +346,26 @@ fn call_tool( .get("force") .and_then(Value::as_bool) .unwrap_or(false); - let report = plugin_install(workspace, target, mode, force)?; + // v0.8: proactive defaults on; pass proactive:false to skip + // the PreToolUse/PostToolUse Bash hooks. + let proactive = arguments + .get("proactive") + .and_then(Value::as_bool) + .unwrap_or(true); + let report = plugin_install(workspace, target, mode, force, proactive)?; Ok(json!({ "target": report.target.as_str(), "mode": report.mode.as_str(), "files": report.files, })) } + "kimetsu_brain_model_list" => kimetsu_brain_model_list(workspace), + "kimetsu_brain_model_set" => kimetsu_brain_model_set(workspace, &arguments), + "kimetsu_brain_reindex" => kimetsu_brain_reindex(workspace, &arguments), + "kimetsu_brain_memory_search" => kimetsu_brain_memory_search(workspace, &arguments), + "kimetsu_brain_conflict_resolve" => kimetsu_brain_conflict_resolve(workspace, &arguments), + "kimetsu_brain_prune" => kimetsu_brain_prune(workspace, &arguments), + "kimetsu_brain_config_show" => kimetsu_brain_config_show(workspace), other => Err(format!("unknown Kimetsu MCP tool `{other}`")), } } @@ -508,6 +529,7 @@ fn kimetsu_brain_context(workspace: &Path, arguments: &Value) -> Value { min_score, max_capsules, prefer_roles, + ..Default::default() }; match project::retrieve_context_readonly_with_request(workspace, request) { @@ -824,12 +846,23 @@ fn kimetsu_benchmark_record_outcome(workspace: &Path, arguments: &Value) -> Resu } fn kimetsu_brain_memory_list(workspace: &Path, arguments: &Value) -> Result { - let limit = u32_arg(arguments, "limit", 50, 1, 100) as usize; - let memories = project::list_memories(workspace) - .map_err(|err| format!("kimetsu brain memory list: {err}"))?; + let limit = u32_arg(arguments, "limit", 50, 1, 100); + let offset = arguments.get("offset").and_then(Value::as_u64).unwrap_or(0) as u32; + let memories = project::list_memories_with( + workspace, + project::ListOptions { + limit, + offset, + scope: optional_string_arg(arguments, "scope"), + }, + ) + .map_err(|err| format!("kimetsu brain memory list: {err}"))?; Ok(json!({ - "memories": memories.iter().take(limit).map(json_memory_row).collect::>(), - "usage": "Use kimetsu_brain_memory_top for outcome-ranked trust signals; use memory_id with kimetsu_brain_memory_invalidate to retire stale memories." + "limit": limit, + "offset": offset, + "count": memories.len(), + "memories": memories.iter().map(json_memory_row).collect::>(), + "usage": "Page with limit+offset. Use kimetsu_brain_memory_search to find by text, kimetsu_brain_memory_top for outcome-ranked trust signals, and kimetsu_brain_memory_invalidate to retire stale memories." })) } @@ -880,6 +913,7 @@ fn kimetsu_brain_memory_proposals(workspace: &Path, arguments: &Value) -> Result status: optional_string_arg(arguments, "status") .or_else(|| Some("pending".to_string())), limit: u32_arg(arguments, "limit", 50, 1, 200), + offset: arguments.get("offset").and_then(Value::as_u64).unwrap_or(0) as u32, }, ) .map_err(|err| format!("kimetsu brain memory proposals: {err}"))?; @@ -1009,6 +1043,252 @@ fn kimetsu_brain_ingest_repo(workspace: &Path, arguments: &Value) -> Result Result { + use kimetsu_brain::embeddings::{BUILTIN_MODELS, resolve_embedder_id}; + let config_model = project::load_config( + &kimetsu_core::paths::ProjectPaths::discover(workspace) + .map_err(|err| format!("discover workspace: {err}"))?, + ) + .ok() + .map(|cfg| cfg.embedder.model); + let active = resolve_embedder_id(config_model.as_deref()); + let models: Vec = BUILTIN_MODELS + .iter() + .map(|(id, dim, blurb)| { + json!({ "id": id, "dim": dim, "description": blurb, "active": *id == active }) + }) + .collect(); + Ok(json!({ + "ok": true, + "active": active, + "configured": config_model, + "models": models, + "usage": "Call kimetsu_brain_model_set to change the model. Note: a switch only affects new embeddings; restart the MCP server and run kimetsu_brain_reindex (or `kimetsu brain reindex --force` from the CLI) to re-embed existing memories." + })) +} + +/// v0.8: change the embedding model. Records it in project.toml and +/// (unless `reindex:false`) re-embeds the corpus in-process with a +/// FRESH embedder for the new model — independent of the model this +/// server loaded at startup. Note: the server's *retrieval* query +/// embedder is a process-static singleton, so semantic retrieval in +/// THIS session keeps using the old model (cross-model rows safely fall +/// back to FTS) until the server restarts; the stored embeddings are +/// already migrated, so a restart fully activates the new model. +fn kimetsu_brain_model_set(workspace: &Path, arguments: &Value) -> Result { + use kimetsu_brain::embeddings::resolve_embedder_id; + let id = string_arg(arguments, "id")?; + // Validate against known aliases so a typo doesn't silently fall + // back to the default model. + if !is_known_embedder_alias(&id) { + return Err(format!( + "unknown embedder id `{id}`. Call kimetsu_brain_model_list for options." + )); + } + let canonical = resolve_embedder_id(Some(&id)); + let paths = kimetsu_core::paths::ProjectPaths::discover(workspace) + .map_err(|err| format!("discover workspace: {err}"))?; + let mut config = project::load_config(&paths).map_err(|err| format!("load config: {err}"))?; + let previous = config.embedder.model.clone(); + config.embedder.model = canonical.to_string(); + let toml = config + .to_toml() + .map_err(|err| format!("serialize config: {err}"))?; + std::fs::write(&paths.project_toml, toml) + .map_err(|err| format!("write project.toml: {err}"))?; + + let do_reindex = arguments + .get("reindex") + .and_then(Value::as_bool) + .unwrap_or(true); + if !do_reindex { + return Ok(json!({ + "ok": true, + "model": canonical, + "previous": previous, + "reindexed": false, + "note": "Recorded in project.toml. Passed reindex:false, so existing memories keep their old embeddings until you run kimetsu_brain_reindex or `kimetsu brain reindex --force`." + })); + } + + // Re-embed with a fresh embedder for the NEW model (not the server's + // cached default). The candidate predicate re-embeds every row whose + // embedding_model != the new model — i.e. all of them. + let embedder = kimetsu_brain::embeddings::open_embedder_for_model(canonical); + let report = kimetsu_brain::reindex::reindex_all_with_embedder( + workspace, + kimetsu_brain::reindex::ReindexOptions { + scope: kimetsu_brain::reindex::ReindexScope::All, + dry_run: false, + force: false, + limit: None, + }, + embedder.as_ref(), + ) + .map_err(|err| format!("reindex after model set: {err}"))?; + Ok(json!({ + "ok": true, + "model": canonical, + "previous": previous, + "reindexed": !report.embedder_noop, + "updated": report.updated_total(), + "embedder_noop": report.embedder_noop, + "note": if report.embedder_noop { + "Recorded, but this is a lean (no-embeddings) build so no vectors were produced. Reinstall with `--features embeddings` to enable semantic retrieval." + } else { + "Recorded and existing memories re-embedded with the new model. Restart the MCP server so its retrieval query embedder also switches (until then, retrieval falls back to FTS for the migrated rows)." + } + })) +} + +fn is_known_embedder_alias(id: &str) -> bool { + matches!( + id.trim().to_ascii_lowercase().as_str(), + "default" + | "bge-small" + | "bge-small-en-v1.5" + | "bge-m3" + | "m3" + | "jina-code" + | "jina-v2-base-code" + | "jina-embeddings-v2-base-code" + ) +} + +/// v0.8: backfill stale/missing embeddings using the server's CURRENT +/// embedder (the one loaded at startup). Useful after adding memories; +/// to switch models, see kimetsu_brain_model_set. +fn kimetsu_brain_reindex(workspace: &Path, arguments: &Value) -> Result { + let scope = kimetsu_brain::reindex::ReindexScope::parse( + &optional_string_arg(arguments, "scope").unwrap_or_else(|| "all".to_string()), + )?; + let report = kimetsu_brain::reindex::reindex_all( + workspace, + kimetsu_brain::reindex::ReindexOptions { + scope, + dry_run: arguments + .get("dry_run") + .and_then(Value::as_bool) + .unwrap_or(false), + force: arguments + .get("force") + .and_then(Value::as_bool) + .unwrap_or(false), + limit: arguments + .get("limit") + .and_then(Value::as_u64) + .map(|n| n as usize), + }, + ) + .map_err(|err| format!("kimetsu brain reindex: {err}"))?; + Ok(json!({ + "ok": true, + "model": report.embedder_model_id, + "embedder_noop": report.embedder_noop, + "candidates": report.candidates_total(), + "updated": report.updated_total(), + })) +} + +/// v0.8: full-text search over memory text for navigating the corpus. +fn kimetsu_brain_memory_search(workspace: &Path, arguments: &Value) -> Result { + let query = string_arg(arguments, "query")?; + let limit = u32_arg(arguments, "limit", 20, 1, 100); + let offset = arguments.get("offset").and_then(Value::as_u64).unwrap_or(0) as u32; + let hits = project::search_memories( + workspace, + &query, + limit, + offset, + optional_string_arg(arguments, "kind").as_deref(), + optional_string_arg(arguments, "scope").as_deref(), + ) + .map_err(|err| format!("kimetsu brain memory search: {err}"))?; + Ok(json!({ + "ok": true, + "query": query, + "limit": limit, + "offset": offset, + "count": hits.len(), + "results": hits.iter().map(|h| json!({ + "memory_id": h.memory_id, + "scope": h.scope, + "kind": h.kind, + "text": h.text, + "rank": h.rank, + })).collect::>(), + "usage": "Page with limit+offset. Filter by kind (failure_pattern/command/convention/preference/fact) or scope (global_user/project/repo/run)." + })) +} + +/// v0.8: settle an open memory conflict from inside the agent. +fn kimetsu_brain_conflict_resolve(workspace: &Path, arguments: &Value) -> Result { + let conflict_id = string_arg(arguments, "conflict_id")?; + let resolution = string_arg(arguments, "resolution")?; + if !matches!( + resolution.as_str(), + "kept_new" | "kept_existing" | "kept_both" + ) { + return Err("`resolution` must be one of: kept_new, kept_existing, kept_both".to_string()); + } + let resolved = project::resolve_conflict(workspace, &conflict_id, &resolution) + .map_err(|err| format!("kimetsu brain conflict resolve: {err}"))?; + Ok(json!({ + "ok": resolved, + "conflict_id": conflict_id, + "resolution": resolution, + "resolved": resolved, + "usage": if resolved { "Conflict settled. kept_new/kept_existing invalidates the losing memory; kept_both keeps both." } else { "No open conflict with that id (already resolved or unknown)." } + })) +} + +/// v0.8: prune net-negative memories. Defaults to a dry run (apply:false). +fn kimetsu_brain_prune(workspace: &Path, arguments: &Value) -> Result { + let apply = arguments + .get("apply") + .and_then(Value::as_bool) + .unwrap_or(false); + let summary = project::prune_low_usefulness( + workspace, + project::PruneOptions { + scope: optional_string_arg(arguments, "scope"), + min_uses: u32_arg(arguments, "min_uses", 3, 1, 1000), + max_ratio: optional_f32_arg(arguments, "max_ratio").unwrap_or(0.0), + apply, + }, + ) + .map_err(|err| format!("kimetsu brain prune: {err}"))?; + Ok(json!({ + "ok": true, + "apply": apply, + "candidate_count": summary.candidates.len(), + "invalidated": summary.invalidated, + "failed": summary.failed, + "candidates": summary.candidates.iter().map(|c| json!({ + "memory_id": c.memory_id, + "scope": c.scope, + "kind": c.kind, + "text": c.text, + "use_count": c.use_count, + "usefulness_score": c.usefulness_score, + })).collect::>(), + "usage": "This is a dry run unless apply:true. Candidates are memories with usefulness_score/use_count <= max_ratio and use_count >= min_uses." + })) +} + +/// v0.8: read-only view of the project.toml config. +fn kimetsu_brain_config_show(workspace: &Path) -> Result { + let raw = project::config_text(workspace) + .map_err(|err| format!("kimetsu brain config show: {err}"))?; + let parsed: Value = toml::from_str(&raw).unwrap_or(Value::Null); + Ok(json!({ + "ok": true, + "raw": raw, + "config": parsed, + })) +} + fn brain_unavailable_json(workspace: &Path, error: &str) -> Value { json!({ "initialized": false, @@ -1489,10 +1769,86 @@ fn tool_definitions() -> Value { "enum": ["optional", "required"], "description": "optional recommends Kimetsu brain first; required tells the host harness to block non-trivial work until Kimetsu context is available or explicitly waived. Benchmark guidance prefers kimetsu_benchmark_context." }, - "force": { "type": "boolean" } + "force": { "type": "boolean" }, + "proactive": { "type": "boolean", "description": "Default true. Set false to skip the proactive PreToolUse/PostToolUse Bash hooks (mid-work recall); UserPromptSubmit + Stop still install." } }, "required": ["target"] } + }, + { + "name": "kimetsu_brain_model_list", + "description": "List the curated built-in embedding models and the active one. The user can switch models from here (kimetsu_brain_model_set).", + "inputSchema": { "type": "object", "properties": {} } + }, + { + "name": "kimetsu_brain_model_set", + "description": "Set the brain's embedding model (a built-in id from kimetsu_brain_model_list). Records it in project.toml and (unless reindex:false) re-embeds the corpus with the new model in-process. The server's retrieval query embedder is fixed until restart, so semantic retrieval this session falls back to FTS for migrated rows; restart to fully activate.", + "inputSchema": { + "type": "object", + "properties": { + "id": { "type": "string", "description": "Built-in model id, e.g. bge-small-en-v1.5, bge-m3, jina-v2-base-code." }, + "reindex": { "type": "boolean", "description": "Default true. Re-embed existing memories with the new model now. Set false to record the id only." } + }, + "required": ["id"] + } + }, + { + "name": "kimetsu_brain_reindex", + "description": "Backfill stale/missing embeddings using the server's current embedder. Run after adding memories. To change models, use kimetsu_brain_model_set.", + "inputSchema": { + "type": "object", + "properties": { + "scope": { "type": "string", "enum": ["project", "user", "all"] }, + "dry_run": { "type": "boolean" }, + "force": { "type": "boolean" }, + "limit": { "type": "integer", "minimum": 1 } + } + } + }, + { + "name": "kimetsu_brain_memory_search", + "description": "Full-text search over memory text. Page with limit+offset; filter by kind or scope. Use this to navigate the memory corpus.", + "inputSchema": { + "type": "object", + "properties": { + "query": { "type": "string" }, + "limit": { "type": "integer", "minimum": 1, "maximum": 100 }, + "offset": { "type": "integer", "minimum": 0 }, + "kind": { "type": "string", "enum": ["preference", "convention", "command", "failure_pattern", "fact"] }, + "scope": { "type": "string", "enum": ["global_user", "project", "repo", "run"] } + }, + "required": ["query"] + } + }, + { + "name": "kimetsu_brain_conflict_resolve", + "description": "Settle an open memory conflict (from kimetsu_brain_memory_conflicts) by id. kept_new/kept_existing invalidates the losing memory; kept_both keeps both.", + "inputSchema": { + "type": "object", + "properties": { + "conflict_id": { "type": "string" }, + "resolution": { "type": "string", "enum": ["kept_new", "kept_existing", "kept_both"] } + }, + "required": ["conflict_id", "resolution"] + } + }, + { + "name": "kimetsu_brain_prune", + "description": "List (or with apply:true, invalidate) net-negative memories whose usefulness ratio is at or below max_ratio. Defaults to a dry run.", + "inputSchema": { + "type": "object", + "properties": { + "scope": { "type": "string", "enum": ["global_user", "project", "repo", "run"] }, + "min_uses": { "type": "integer", "minimum": 1 }, + "max_ratio": { "type": "number" }, + "apply": { "type": "boolean" } + } + } + }, + { + "name": "kimetsu_brain_config_show", + "description": "Read the project.toml config (raw + parsed), including the active embedder, broker weights, and run limits.", + "inputSchema": { "type": "object", "properties": {} } } ]) } @@ -1802,6 +2158,10 @@ mod tests { .duration_since(UNIX_EPOCH) .expect("time") .as_nanos(); - std::env::temp_dir().join(format!("{prefix}-{nanos}")) + let root = std::env::temp_dir().join(format!("{prefix}-{nanos}")); + // Isolate from any enclosing git repo (e.g. a dev's $HOME repo) + // so ProjectPaths::discover resolves here, not a shared ancestor. + kimetsu_core::paths::git_init_boundary(&root); + root } } diff --git a/crates/kimetsu-cli/Cargo.toml b/crates/kimetsu-cli/Cargo.toml index 67e9803..392cf4d 100644 --- a/crates/kimetsu-cli/Cargo.toml +++ b/crates/kimetsu-cli/Cargo.toml @@ -40,10 +40,10 @@ path = "src/main.rs" [dependencies] clap.workspace = true -kimetsu-agent = { path = "../kimetsu-agent", version = "0.7.3" } -kimetsu-brain = { path = "../kimetsu-brain", version = "0.7.3" } -kimetsu-chat = { path = "../kimetsu-chat", version = "0.7.3" } -kimetsu-core = { path = "../kimetsu-core", version = "0.7.3" } +kimetsu-agent = { path = "../kimetsu-agent", version = "0.8.0" } +kimetsu-brain = { path = "../kimetsu-brain", version = "0.8.0" } +kimetsu-chat = { path = "../kimetsu-chat", version = "0.8.0" } +kimetsu-core = { path = "../kimetsu-core", version = "0.8.0" } # v0.4.6: `kimetsu doctor` serializes its report struct so --json # output can be piped into CI / hooks. reqwest.workspace = true diff --git a/crates/kimetsu-cli/src/main.rs b/crates/kimetsu-cli/src/main.rs index e7a16c6..8c07003 100644 --- a/crates/kimetsu-cli/src/main.rs +++ b/crates/kimetsu-cli/src/main.rs @@ -4,6 +4,7 @@ use std::path::{Path, PathBuf}; use std::str::FromStr; mod doctor; +mod proactive_state; mod update; use clap::{Args, Parser, Subcommand}; @@ -278,6 +279,10 @@ struct PluginInstallArgs { mode: String, #[arg(long)] force: bool, + /// v0.8: skip wiring the proactive PreToolUse/PostToolUse Bash + /// hooks (mid-work recall). UserPromptSubmit + Stop still install. + #[arg(long)] + no_proactive: bool, } #[derive(Debug, Args)] @@ -331,6 +336,73 @@ enum BrainCommand { /// vectors) or after changing the embedder model via /// `KIMETSU_BRAIN_EMBEDDER=`. Reindex(ReindexArgs), + /// v0.8: inspect or change which built-in embedding model the + /// brain uses. `list` shows the curated set and the active id; + /// `set ` writes it to project.toml and re-embeds the corpus. + Model { + #[command(subcommand)] + command: ModelCommand, + }, + /// v0.8: proactive PreToolUse hook. Reads tool-call JSON from + /// stdin and, only for a high-confidence match against a stored + /// failure_pattern/convention, prints a one-line warning BEFORE a + /// risky Bash command runs. Exits 0 silently otherwise. + #[command(name = "pretool-hook")] + PreToolHook(ProactiveHookArgs), + /// v0.8: proactive PostToolUse hook. Reads tool-call JSON from + /// stdin and, when a Bash command failed and matches a stored + /// failure_pattern/command, surfaces the known fix. Exits 0 + /// silently otherwise. + #[command(name = "posttool-hook")] + PostToolHook(ProactiveHookArgs), +} + +#[derive(Debug, Subcommand)] +enum ModelCommand { + /// List the curated built-in embedding models and mark the active one. + List { + #[arg(long)] + json: bool, + }, + /// Set the active embedding model and re-embed the corpus. + Set(ModelSetArgs), +} + +#[derive(Debug, Args)] +struct ModelSetArgs { + /// Built-in model id (see `kimetsu brain model list`). + id: String, + /// Write the config but skip the (potentially slow) reindex. + #[arg(long)] + no_reindex: bool, + /// Override the brain workspace path (defaults to current directory). + #[arg(long)] + workspace: Option, + /// Emit machine-readable JSON. + #[arg(long)] + json: bool, +} + +#[derive(Debug, Args)] +struct ProactiveHookArgs { + /// Minimum relevance score for a proactive injection (FTS-only + /// scale; stricter than the reactive 0.20). Default 0.45. + #[arg(long, default_value_t = 0.45)] + min_score: f32, + /// Lower threshold used when a looping/repeated command is + /// detected (the agent is stuck — surface help more readily). + #[arg(long, default_value_t = 0.35)] + loop_min_score: f32, + /// Max capsules to inject. Default 1 (recall discipline). + #[arg(long, default_value_t = 1usize)] + max_capsules: usize, + /// Suppress further proactive injections for this many seconds + /// after one fires (refractory throttle). Default 90. + #[arg(long, default_value_t = 90u64)] + refractory_secs: u64, + /// Override the brain workspace path (defaults to current directory). + #[arg(long)] + workspace: Option, } #[derive(Debug, Args)] @@ -871,7 +943,7 @@ fn plugin(command: PluginCommand) -> KimetsuResult<()> { .map_err(|err| format!("kimetsu plugin install: {err}"))?; let mode = PluginMode::parse(&args.mode) .map_err(|err| format!("kimetsu plugin install: {err}"))?; - let report = plugin_install(&workspace, target, mode, args.force) + let report = plugin_install(&workspace, target, mode, args.force, !args.no_proactive) .map_err(|err| format!("kimetsu plugin install: {err}"))?; println!( "installed Kimetsu plugin surface for {} in {} mode", @@ -1101,6 +1173,28 @@ const CLAUDE_SETTINGS_CONTENT: &str = r#"{ ] } ], + "PreToolUse": [ + { + "matcher": "Bash", + "hooks": [ + { + "type": "command", + "command": "kimetsu brain pretool-hook" + } + ] + } + ], + "PostToolUse": [ + { + "matcher": "Bash", + "hooks": [ + { + "type": "command", + "command": "kimetsu brain posttool-hook" + } + ] + } + ], "Stop": [ { "matcher": "", @@ -1150,6 +1244,13 @@ fn config(command: ConfigCommand) -> KimetsuResult<()> { } fn brain(command: BrainCommand) -> KimetsuResult<()> { + // v0.8: honor the [embedder] config (env still wins) for every + // command except `model set`, which sets the new selection itself. + // The embedder is a process-static OnceLock, so this must run + // before any retrieval/reindex touches it — entry is the safe spot. + if !matches!(command, BrainCommand::Model { .. }) { + apply_embedder_from_cwd(); + } match command { BrainCommand::IngestRepo { path } => { let summary = project::ingest_repo(&path)?; @@ -1247,9 +1348,202 @@ fn brain(command: BrainCommand) -> KimetsuResult<()> { BrainCommand::ContextHook(args) => brain_context_hook(args), BrainCommand::StopHook(args) => brain_stop_hook(args), BrainCommand::Reindex(args) => reindex_brain(args), + BrainCommand::Model { command } => brain_model(command), + BrainCommand::PreToolHook(args) => proactive_hook(ProactiveEvent::PreTool, args), + BrainCommand::PostToolHook(args) => proactive_hook(ProactiveEvent::PostTool, args), + } +} + +/// v0.8: best-effort — load the project config from the current dir and +/// record its `[embedder] model` so brain-internal callers resolve it +/// (env still wins). Silently no-ops when the brain isn't initialized. +fn apply_embedder_from_cwd() { + if let Ok(cwd) = env::current_dir() + && let Ok(paths) = kimetsu_core::paths::ProjectPaths::discover(&cwd) + && let Ok(config) = project::load_config(&paths) + { + kimetsu_brain::embeddings::apply_embedder_selection(Some(&config.embedder.model)); + } +} + +/// v0.8: `kimetsu brain model list|set`. +fn brain_model(command: ModelCommand) -> KimetsuResult<()> { + match command { + ModelCommand::List { json } => brain_model_list(json), + ModelCommand::Set(args) => brain_model_set(args), } } +fn brain_model_list(json: bool) -> KimetsuResult<()> { + use kimetsu_brain::embeddings::{BUILTIN_MODELS, resolve_embedder_id}; + + // Resolve the active id + where it came from, best-effort. + let (config_model, source) = match env::current_dir() + .ok() + .and_then(|cwd| kimetsu_core::paths::ProjectPaths::discover(&cwd).ok()) + .and_then(|paths| project::load_config(&paths).ok()) + { + Some(cfg) => (Some(cfg.embedder.model.clone()), "config"), + None => (None, "default"), + }; + let env_set = env::var("KIMETSU_BRAIN_EMBEDDER") + .ok() + .filter(|v| !v.trim().is_empty()); + let source = if env_set.is_some() { "env" } else { source }; + let active = resolve_embedder_id(config_model.as_deref()); + + if json { + let models: Vec<_> = BUILTIN_MODELS + .iter() + .map(|(id, dim, blurb)| { + serde_json::json!({ + "id": id, + "dim": dim, + "description": blurb, + "active": *id == active, + }) + }) + .collect(); + println!( + "{}", + serde_json::to_string_pretty(&serde_json::json!({ + "ok": true, + "active": active, + "source": source, + "models": models, + }))? + ); + return Ok(()); + } + + println!("Embedding models (active resolved from {source}):"); + for (id, dim, blurb) in BUILTIN_MODELS { + let marker = if *id == active { "*" } else { " " }; + println!(" {marker} {id:<22} {dim:>5}d {blurb}"); + } + println!("\nChange with: kimetsu brain model set "); + println!("(env KIMETSU_BRAIN_EMBEDDER always overrides the config field)"); + Ok(()) +} + +fn brain_model_set(args: ModelSetArgs) -> KimetsuResult<()> { + use kimetsu_brain::embeddings::{apply_embedder_selection, resolve_embedder_id}; + + // Validate against the curated set so `set` never silently falls + // back to the default for a typo'd id. + if !is_known_alias(&args.id) { + return Err(format!( + "unknown embedder id `{}`. Run `kimetsu brain model list` for the options.", + args.id + ) + .into()); + } + let canonical = resolve_embedder_id(Some(&args.id)); + + let workspace = args.workspace.clone().unwrap_or(env::current_dir()?); + let paths = kimetsu_core::paths::ProjectPaths::discover(&workspace)?; + let mut config = project::load_config(&paths)?; + let previous = config.embedder.model.clone(); + let prev_dim = dim_for(resolve_embedder_id(Some(&previous))); + let new_dim = dim_for(canonical); + + config.embedder.model = canonical.to_string(); + std::fs::write(&paths.project_toml, config.to_toml()?)?; + + // Fresh CLI process: the embedder OnceLock is not yet initialized, + // so recording the override here means the reindex below loads the + // NEW model. + apply_embedder_selection(Some(canonical)); + + let dim_changed = prev_dim != new_dim; + + if args.no_reindex { + if args.json { + println!( + "{}", + serde_json::to_string_pretty(&serde_json::json!({ + "ok": true, "model": canonical, "previous": previous, + "reindexed": false, "dimension_changed": dim_changed, + }))? + ); + } else { + println!( + "Embedder set to `{canonical}` (was `{previous}`). Skipped reindex (--no-reindex)." + ); + if dim_changed { + println!( + "Dimension changed {prev_dim}d -> {new_dim}d: run `kimetsu brain reindex --force` so cosine retrieval uses the new model." + ); + } + } + return Ok(()); + } + + // Re-embed with a FRESH embedder for the new model (not whatever the + // default cache might resolve to), so the corpus is migrated to the + // chosen model deterministically. + let embedder = kimetsu_brain::embeddings::open_embedder_for_model(canonical); + let report = kimetsu_brain::reindex::reindex_all_with_embedder( + &workspace, + kimetsu_brain::reindex::ReindexOptions { + scope: kimetsu_brain::reindex::ReindexScope::All, + dry_run: false, + force: dim_changed, + limit: None, + }, + embedder.as_ref(), + )?; + + if args.json { + println!( + "{}", + serde_json::to_string_pretty(&serde_json::json!({ + "ok": true, "model": canonical, "previous": previous, + "reindexed": !report.embedder_noop, + "dimension_changed": dim_changed, + "updated": report.updated_total(), + "embedder_noop": report.embedder_noop, + }))? + ); + return Ok(()); + } + + println!("Embedder set to `{canonical}` (was `{previous}`)."); + if report.embedder_noop { + println!( + "Active embedder is `noop` (lean build or KIMETSU_BRAIN_EMBEDDER=noop): id recorded, but no vectors were produced. Build with `--features embeddings` then run `kimetsu brain reindex`." + ); + } else { + println!( + "Reindexed {} memories with the new model.", + report.updated_total() + ); + } + Ok(()) +} + +fn is_known_alias(id: &str) -> bool { + matches!( + id.trim().to_ascii_lowercase().as_str(), + "default" + | "bge-small" + | "bge-small-en-v1.5" + | "bge-m3" + | "m3" + | "jina-code" + | "jina-v2-base-code" + | "jina-embeddings-v2-base-code" + ) +} + +fn dim_for(canonical_id: &str) -> usize { + kimetsu_brain::embeddings::BUILTIN_MODELS + .iter() + .find(|(id, _, _)| *id == canonical_id) + .map(|(_, dim, _)| *dim) + .unwrap_or(0) +} + /// v0.4.3: `kimetsu brain reindex` — backfill missing / stale /// embeddings. The interesting cases: /// @@ -1554,6 +1848,216 @@ fn brain_stop_hook(args: StopHookArgs) -> KimetsuResult<()> { Ok(()) } +#[derive(Debug, Clone, Copy)] +enum ProactiveEvent { + PreTool, + PostTool, +} + +impl ProactiveEvent { + fn hook_event_name(self) -> &'static str { + match self { + ProactiveEvent::PreTool => "PreToolUse", + ProactiveEvent::PostTool => "PostToolUse", + } + } +} + +/// Harness-agnostic fields pulled from a PreToolUse/PostToolUse hook +/// payload. Both Claude Code and Codex send this superset; parse +/// defensively so a missing/odd field just disables the relevant path. +struct HookToolInput { + session_id: Option, + tool_name: Option, + command: Option, + tool_response: Option, +} + +fn parse_hook_tool_input(raw: &str) -> HookToolInput { + let v: serde_json::Value = serde_json::from_str(raw.trim()).unwrap_or(serde_json::Value::Null); + let str_field = |key: &str| { + v.get(key) + .and_then(serde_json::Value::as_str) + .map(str::to_string) + .filter(|s| !s.trim().is_empty()) + }; + let command = v + .get("tool_input") + .and_then(|ti| ti.get("command")) + .and_then(serde_json::Value::as_str) + .map(str::to_string) + .filter(|s| !s.trim().is_empty()); + // tool_response may be a string or a structured object; stringify + // objects so failure detection still has something to scan. + let tool_response = match v.get("tool_response") { + Some(serde_json::Value::String(s)) if !s.trim().is_empty() => Some(s.clone()), + Some(serde_json::Value::Null) | None => None, + Some(other) => Some(other.to_string()), + }; + HookToolInput { + session_id: str_field("session_id"), + tool_name: str_field("tool_name"), + command, + tool_response, + } +} + +/// v0.8: proactive PreToolUse / PostToolUse hook. Shared by both +/// events. Lexical-FTS-only retrieval, very high score floor, one +/// capsule, per-session dedupe + refractory + loop detection. Always +/// exits 0; emits hook JSON only on a confident, novel match. +fn proactive_hook(event: ProactiveEvent, args: ProactiveHookArgs) -> KimetsuResult<()> { + use kimetsu_brain::context::ContextRequest; + use std::io::Read; + + let workspace = args + .workspace + .clone() + .unwrap_or_else(|| env::current_dir().unwrap_or_default()); + + // Resolve the .kimetsu dir; if there's no brain here, stay silent. + let Ok(paths) = kimetsu_core::paths::ProjectPaths::discover(&workspace) else { + return Ok(()); + }; + // Honor the configured embedder id for consistency (proactive + // retrieval is lexical-only, but this keeps labels coherent). + if let Ok(config) = project::load_config(&paths) { + kimetsu_brain::embeddings::apply_embedder_selection(Some(&config.embedder.model)); + } + + let mut input = String::new(); + std::io::stdin().read_to_string(&mut input).unwrap_or(0); + if input.trim().is_empty() { + return Ok(()); + } + let hook = parse_hook_tool_input(&input); + + // Defensive tool-name gate (the hook matcher should already scope + // to Bash, but be safe across harness quirks). + if let Some(name) = hook.tool_name.as_deref() + && !name.eq_ignore_ascii_case("bash") + { + return Ok(()); + } + + let now = proactive_state::now_unix(); + proactive_state::gc(&paths.kimetsu_dir, now); + + // Build the retrieval query + actionable kinds per event. + let (query, kinds, error_sig): (String, &[&str], Option) = match event { + ProactiveEvent::PreTool => { + let Some(cmd) = hook.command.as_deref() else { + return Ok(()); + }; + (cmd.to_string(), &["failure_pattern", "convention"], None) + } + ProactiveEvent::PostTool => { + let resp = hook.tool_response.as_deref().unwrap_or(""); + if !proactive_state::looks_like_failure(resp) { + return Ok(()); // only react to failures + } + let cmd = hook.command.as_deref().unwrap_or(""); + ( + format!("{resp} {cmd}"), + &["failure_pattern", "command", "convention"], + proactive_state::error_signature(resp), + ) + } + }; + + // Load session state, record this command, decide loop mode. + let state_path = proactive_state::session_path(&paths.kimetsu_dir, hook.session_id.as_deref()); + let mut state = proactive_state::load(&state_path); + let norm = proactive_state::normalize_command(hook.command.as_deref().unwrap_or(&query)); + let seen_count = state.note_command(&norm, error_sig.as_deref(), now); + let loop_mode = seen_count >= proactive_state::LOOP_THRESHOLD; + + // Refractory throttle — unless the agent is clearly looping, stay + // quiet for a window after the last injection. Persist the loop + // counter increment even on a silent exit. + if !loop_mode && state.in_refractory(now, args.refractory_secs) { + proactive_state::save(&state_path, &state); + return Ok(()); + } + + let min_score = if loop_mode { + args.loop_min_score + } else { + args.min_score + }; + + let request = ContextRequest { + stage: "localization".to_string(), + query, + budget_tokens: 600, + min_score, + max_capsules: args.max_capsules.max(1), + kinds: kinds.iter().map(|k| k.to_string()).collect(), + ..Default::default() + }; + + let bundle = match project::retrieve_proactive_readonly(&workspace, request) { + Ok(b) => b, + Err(_) => { + proactive_state::save(&state_path, &state); + return Ok(()); + } + }; + + let Some(capsule) = bundle + .capsules + .iter() + .find(|c| !state.is_surfaced(&c.expansion_handle)) + else { + // Nothing relevant, or the only match already surfaced this + // session (it's already in working memory). + proactive_state::save(&state_path, &state); + return Ok(()); + }; + + let body = capsule + .summary + .splitn(3, " - ") + .nth(1) + .unwrap_or(&capsule.summary); + let header = proactive_header(event, loop_mode); + let additional_context = format!("{header}\n{body}"); + + print_tool_use_context(event, &additional_context)?; + + state.mark_surfaced(&capsule.expansion_handle); + state.record_injection(now); + proactive_state::save(&state_path, &state); + Ok(()) +} + +fn proactive_header(event: ProactiveEvent, loop_mode: bool) -> &'static str { + match (event, loop_mode) { + (_, true) => { + "You appear to be repeating a failing command. Kimetsu brain recalls a relevant lesson:" + } + (ProactiveEvent::PreTool, false) => { + "Kimetsu brain — a relevant prior failure for this command:" + } + (ProactiveEvent::PostTool, false) => "Kimetsu brain — a known fix for this failure:", + } +} + +fn print_tool_use_context(event: ProactiveEvent, additional_context: &str) -> KimetsuResult<()> { + // Non-blocking inject on both harnesses: hookSpecificOutput. + // additionalContext with the matching hookEventName. We never set + // permissionDecision / decision:block — proactive recall informs, + // it does not gate. + let output = serde_json::json!({ + "hookSpecificOutput": { + "hookEventName": event.hook_event_name(), + "additionalContext": additional_context, + }, + }); + println!("{}", serde_json::to_string(&output)?); + Ok(()) +} + fn stats() -> KimetsuResult<()> { let memories = project::list_memories(&env::current_dir()?)?; let runs = project::list_runs(&env::current_dir()?)?; @@ -1611,6 +2115,7 @@ fn memory(command: MemoryCommand) -> KimetsuResult<()> { min_confidence: args.min_confidence, status: Some(args.status), limit: args.limit, + offset: 0, }, )?; if proposals.is_empty() { @@ -1957,6 +2462,7 @@ fn review_proposals(args: ReviewArgs) -> KimetsuResult<()> { min_confidence: args.min_confidence, status: Some("pending".to_string()), limit: args.limit, + offset: 0, }, )?; @@ -2382,6 +2888,8 @@ mod tests { // ulid-named temp dir to avoid collisions when tests run concurrently. let root = std::env::temp_dir().join(format!("kimetsu-cli-test-{}", RunId::new())); fs::create_dir_all(&root).expect("create temp project"); + // Isolate from any enclosing git repo (see git_init_boundary). + kimetsu_core::paths::git_init_boundary(&root); project::init_project(&root, false).expect("init project"); // Inject 3 pending proposals via the brain's event-sourced path. @@ -2518,6 +3026,8 @@ mod tests { fn interactive_loop_quit_preserves_partial_decisions_body() { let root = std::env::temp_dir().join(format!("kimetsu-cli-test-{}", RunId::new())); fs::create_dir_all(&root).expect("create temp project"); + // Isolate from any enclosing git repo (see git_init_boundary). + kimetsu_core::paths::git_init_boundary(&root); project::init_project(&root, false).expect("init project"); let proposals: [(&str, &str, &str, f32, &str); 3] = [ diff --git a/crates/kimetsu-cli/src/proactive_state.rs b/crates/kimetsu-cli/src/proactive_state.rs new file mode 100644 index 0000000..6d7b28c --- /dev/null +++ b/crates/kimetsu-cli/src/proactive_state.rs @@ -0,0 +1,301 @@ +//! v0.8: per-session state for the proactive recall hooks. +//! +//! The proactive PreToolUse/PostToolUse hooks run as fresh CLI +//! processes per tool call (the "stateless now, daemon later" model), +//! so cross-call memory lives in a tiny JSON file under +//! `/.kimetsu/proactive/.json`. It powers three +//! brain-like behaviours: +//! +//! * **Dedupe** — a memory surfaces at most once per session +//! (`surfaced_memory_ids`). Once it's "in working memory" it +//! doesn't keep re-announcing itself. +//! * **Refractory throttle** — after an injection we stay quiet for +//! a short window (`last_injection_unix`) so the agent isn't +//! flooded with associations. +//! * **Loop detection** — repeated (command, error) pairs bump a +//! counter (`recent_commands`); once it crosses [`LOOP_THRESHOLD`] +//! the hook drops to a lower score floor and bypasses the +//! refractory once, because a stuck agent should get help sooner. +//! +//! Every read/write is best-effort: a corrupt or missing file just +//! resets to default. The file is intentionally small (a ring buffer +//! of recent commands plus a handful of ids), and stale session files +//! are garbage-collected opportunistically on each invocation. + +use std::fs; +use std::path::{Path, PathBuf}; +use std::time::{SystemTime, UNIX_EPOCH}; + +use serde::{Deserialize, Serialize}; + +/// Repeated (command, error) observations before loop mode kicks in. +pub const LOOP_THRESHOLD: u32 = 3; + +const MAX_RECENT_COMMANDS: usize = 12; +const SESSION_GC_MAX_AGE_SECS: u64 = 7 * 24 * 60 * 60; + +#[derive(Debug, Default, Serialize, Deserialize)] +pub struct SessionState { + #[serde(default)] + pub surfaced_memory_ids: Vec, + #[serde(default)] + pub last_injection_unix: u64, + #[serde(default)] + pub recent_commands: Vec, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct CmdSeen { + /// Normalized command text (lowercased, whitespace-collapsed). + pub norm: String, + /// Short signature of the failure, when this came from a failed + /// PostToolUse. `None` for pre-command observations. + #[serde(default)] + pub error_sig: Option, + pub count: u32, + pub last_unix: u64, +} + +pub fn now_unix() -> u64 { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0) +} + +/// Resolve the on-disk path for a session's state, sanitizing the +/// session id so it can't escape the `proactive/` directory. +pub fn session_path(kimetsu_dir: &Path, session_id: Option<&str>) -> PathBuf { + let raw = session_id.map(str::trim).filter(|s| !s.is_empty()); + let safe = match raw { + Some(id) => sanitize_id(id), + None => "_no_session".to_string(), + }; + kimetsu_dir.join("proactive").join(format!("{safe}.json")) +} + +fn sanitize_id(id: &str) -> String { + let cleaned: String = id + .chars() + .map(|c| { + if c.is_ascii_alphanumeric() || c == '-' || c == '_' { + c + } else { + '_' + } + }) + .take(128) + .collect(); + if cleaned.is_empty() { + "_no_session".to_string() + } else { + cleaned + } +} + +/// Best-effort load — any error (missing/corrupt) yields a fresh state. +pub fn load(path: &Path) -> SessionState { + fs::read_to_string(path) + .ok() + .and_then(|text| serde_json::from_str(&text).ok()) + .unwrap_or_default() +} + +/// Best-effort save — failures are swallowed (a hook must never break +/// the agent's turn just because it couldn't persist its own state). +pub fn save(path: &Path, state: &SessionState) { + if let Some(parent) = path.parent() { + let _ = fs::create_dir_all(parent); + } + if let Ok(text) = serde_json::to_string(state) { + let _ = fs::write(path, text); + } +} + +/// Delete session files older than the GC horizon. Best-effort and +/// cheap (the directory holds at most a handful of tiny files). +pub fn gc(kimetsu_dir: &Path, now: u64) { + let dir = kimetsu_dir.join("proactive"); + let Ok(entries) = fs::read_dir(&dir) else { + return; + }; + for entry in entries.flatten() { + let path = entry.path(); + if path.extension().and_then(|e| e.to_str()) != Some("json") { + continue; + } + let age_ok = entry + .metadata() + .and_then(|m| m.modified()) + .ok() + .and_then(|t| t.duration_since(UNIX_EPOCH).ok()) + .map(|d| now.saturating_sub(d.as_secs()) > SESSION_GC_MAX_AGE_SECS) + .unwrap_or(false); + if age_ok { + let _ = fs::remove_file(&path); + } + } +} + +/// Collapse a command to a stable comparison key: lowercase + single +/// spaces. Cheap and good enough to catch a literally-repeated command. +pub fn normalize_command(cmd: &str) -> String { + cmd.split_whitespace() + .collect::>() + .join(" ") + .to_ascii_lowercase() +} + +/// Derive a short, stable signature from failing tool output: the +/// first line that looks like an error, truncated. Used to tell apart +/// "same command, same failure" (a real loop) from incidental reruns. +pub fn error_signature(tool_response: &str) -> Option { + let line = tool_response + .lines() + .find(|l| { + let lc = l.to_ascii_lowercase(); + FAILURE_MARKERS.iter().any(|m| lc.contains(m)) + }) + .or_else(|| tool_response.lines().find(|l| !l.trim().is_empty()))?; + let sig: String = line.trim().chars().take(80).collect(); + if sig.is_empty() { None } else { Some(sig) } +} + +/// Substrings that mark tool output as a failure. Shared by +/// [`error_signature`] and the hook's failure gate. +pub const FAILURE_MARKERS: &[&str] = &[ + "error", + "failed", + "fatal", + "panic", + "exception", + "traceback", + "denied", + "not found", + "cannot", + "no such", +]; + +/// True when tool output looks like a failure (case-insensitive). +pub fn looks_like_failure(tool_response: &str) -> bool { + let lc = tool_response.to_ascii_lowercase(); + FAILURE_MARKERS.iter().any(|m| lc.contains(m)) +} + +impl SessionState { + pub fn is_surfaced(&self, memory_id: &str) -> bool { + self.surfaced_memory_ids.iter().any(|id| id == memory_id) + } + + pub fn mark_surfaced(&mut self, memory_id: &str) { + if !self.is_surfaced(memory_id) { + self.surfaced_memory_ids.push(memory_id.to_string()); + } + } + + /// True when a prior injection happened within `secs` of `now`. + pub fn in_refractory(&self, now: u64, secs: u64) -> bool { + self.last_injection_unix > 0 && now.saturating_sub(self.last_injection_unix) < secs + } + + pub fn record_injection(&mut self, now: u64) { + self.last_injection_unix = now; + } + + /// Record an observation of `norm`/`error_sig` and return the new + /// running count for that pair. Keeps a bounded ring buffer so the + /// state file stays tiny. + pub fn note_command(&mut self, norm: &str, error_sig: Option<&str>, now: u64) -> u32 { + if let Some(existing) = self + .recent_commands + .iter_mut() + .find(|c| c.norm == norm && c.error_sig.as_deref() == error_sig) + { + existing.count = existing.count.saturating_add(1); + existing.last_unix = now; + return existing.count; + } + self.recent_commands.push(CmdSeen { + norm: norm.to_string(), + error_sig: error_sig.map(str::to_string), + count: 1, + last_unix: now, + }); + if self.recent_commands.len() > MAX_RECENT_COMMANDS { + // Drop the oldest by last_unix. + if let Some((idx, _)) = self + .recent_commands + .iter() + .enumerate() + .min_by_key(|(_, c)| c.last_unix) + { + self.recent_commands.remove(idx); + } + } + 1 + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn dedupe_marks_once() { + let mut s = SessionState::default(); + assert!(!s.is_surfaced("m1")); + s.mark_surfaced("m1"); + s.mark_surfaced("m1"); + assert!(s.is_surfaced("m1")); + assert_eq!(s.surfaced_memory_ids.len(), 1); + } + + #[test] + fn refractory_window() { + let mut s = SessionState::default(); + assert!(!s.in_refractory(100, 90)); + s.record_injection(100); + assert!(s.in_refractory(150, 90)); + assert!(!s.in_refractory(200, 90)); + } + + #[test] + fn loop_counter_reaches_threshold() { + let mut s = SessionState::default(); + let norm = normalize_command("cargo BUILD"); + assert_eq!(norm, "cargo build"); + assert_eq!(s.note_command(&norm, Some("error: linker"), 1), 1); + assert_eq!(s.note_command(&norm, Some("error: linker"), 2), 2); + let third = s.note_command(&norm, Some("error: linker"), 3); + assert_eq!(third, LOOP_THRESHOLD); + // A different error signature tracks separately. + assert_eq!(s.note_command(&norm, Some("other failure"), 4), 1); + } + + #[test] + fn ring_buffer_is_bounded() { + let mut s = SessionState::default(); + for i in 0..(MAX_RECENT_COMMANDS + 5) { + s.note_command(&format!("cmd {i}"), None, i as u64); + } + assert!(s.recent_commands.len() <= MAX_RECENT_COMMANDS); + } + + #[test] + fn session_path_sanitizes() { + let dir = Path::new("/tmp/.kimetsu"); + let p = session_path(dir, Some("../../etc/passwd")); + assert!(p.starts_with("/tmp/.kimetsu/proactive")); + assert!(!p.to_string_lossy().contains("..")); + let none = session_path(dir, None); + assert!(none.to_string_lossy().contains("_no_session")); + } + + #[test] + fn error_signature_picks_error_line() { + let sig = + error_signature("compiling...\nerror: linker `link.exe` not found\nmore").unwrap(); + assert!(sig.to_ascii_lowercase().contains("linker")); + assert!(error_signature("").is_none()); + } +} diff --git a/crates/kimetsu-core/src/config.rs b/crates/kimetsu-core/src/config.rs index 96d75cd..0c2ca79 100644 --- a/crates/kimetsu-core/src/config.rs +++ b/crates/kimetsu-core/src/config.rs @@ -10,6 +10,13 @@ pub struct ProjectConfig { pub shell: ShellSection, pub ingestion: IngestionSection, pub run: RunSection, + /// v0.8: which built-in embedding model the brain uses. The + /// `#[serde(default)]` keeps every pre-v0.8 project.toml loading + /// cleanly (they get the lean English default). Resolution + /// precedence is `KIMETSU_BRAIN_EMBEDDER` env > this field > + /// default; see `kimetsu_brain::embeddings::resolve_embedder_id`. + #[serde(default)] + pub embedder: EmbedderSection, } impl ProjectConfig { @@ -24,6 +31,7 @@ impl ProjectConfig { shell: ShellSection::default(), ingestion: IngestionSection::default(), run: RunSection::default(), + embedder: EmbedderSection::default(), } } @@ -42,6 +50,29 @@ pub struct KimetsuSection { pub schema_version: i64, } +/// v0.8: embedding-model selection. `model` is one of the curated +/// built-in ids exposed by `kimetsu brain model list` +/// (`bge-small-en-v1.5`, `bge-m3`, `jina-v2-base-code`). Switching +/// changes the vector dimension, so a `kimetsu brain reindex` is +/// required for cosine retrieval to use the new model. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct EmbedderSection { + #[serde(default = "default_embedder_id")] + pub model: String, +} + +fn default_embedder_id() -> String { + "bge-small-en-v1.5".to_string() +} + +impl Default for EmbedderSection { + fn default() -> Self { + Self { + model: default_embedder_id(), + } + } +} + #[derive(Debug, Clone, Serialize, Deserialize)] pub struct ModelSection { pub provider: String, @@ -210,3 +241,68 @@ impl Default for RunSection { } } } + +#[cfg(test)] +mod tests { + use super::*; + + /// A pre-v0.8 project.toml has no `[embedder]` table. The + /// `#[serde(default)]` on the field must keep it loading cleanly, + /// defaulting to the lean English model. + #[test] + fn pre_v0_8_config_without_embedder_loads_with_default() { + let toml = r#" +[kimetsu] +project_id = "demo" +schema_version = 7 + +[model] +provider = "anthropic" +model = "claude-opus-4-7" +api_key_env = "ANTHROPIC_API_KEY" +max_output_tokens = 8192 +temperature = 0.2 +request_timeout_secs = 120 + +[broker] +default_budget_tokens = 6000 + +[broker.weights] +relevance = 0.5 +confidence = 0.2 +freshness = 0.2 +scope = 0.1 + +[shell] +default_timeout_secs = 60 +max_timeout_secs = 600 +env_allowlist_extra = [] +redact_secrets = true + +[ingestion] +max_file_bytes = 524288 +extra_skip_dirs = [] +max_total_files = 50000 + +[run] +max_total_tool_calls = 60 +max_total_model_turns = 30 +max_total_cost_usd = 250.0 +"#; + let config = ProjectConfig::from_toml(toml).expect("pre-v0.8 toml must load"); + assert_eq!(config.embedder.model, "bge-small-en-v1.5"); + } + + /// `model set` writes the whole config back via `to_toml`; a + /// round-trip must preserve the chosen embedder (and other sections). + #[test] + fn embedder_survives_toml_round_trip() { + let mut config = ProjectConfig::default_for_project("demo"); + config.embedder.model = "bge-m3".to_string(); + let serialized = config.to_toml().expect("serialize"); + let reloaded = ProjectConfig::from_toml(&serialized).expect("reload"); + assert_eq!(reloaded.embedder.model, "bge-m3"); + assert_eq!(reloaded.broker.default_budget_tokens, 6000); + assert_eq!(reloaded.kimetsu.project_id, "demo"); + } +} diff --git a/crates/kimetsu-core/src/paths.rs b/crates/kimetsu-core/src/paths.rs index 98b3211..90d4a94 100644 --- a/crates/kimetsu-core/src/paths.rs +++ b/crates/kimetsu-core/src/paths.rs @@ -49,6 +49,30 @@ pub fn discover_repo_root(start: &Path) -> KimetsuResult { } } +/// v0.8: make `dir` a standalone git repository (best-effort) so +/// [`discover_repo_root`] resolves to `dir` itself instead of climbing +/// to an enclosing repo. Two callers: +/// * the benchmark harness, for throwaway fixture repos — without +/// this, a fixture created under the system temp dir on a machine +/// whose `$HOME` (or any ancestor) is a git repo would init its +/// brain at that ancestor and leak fixture memories into it; +/// * tests that create isolated project roots under the temp dir. +/// +/// Creates `dir` if needed. Returns true when git reported success; a +/// failure (e.g. git not installed) just means the caller doesn't get +/// isolation, which is the prior behaviour. +pub fn git_init_boundary(dir: &Path) -> bool { + if std::fs::create_dir_all(dir).is_err() { + return false; + } + Command::new("git") + .args(["init", "--quiet"]) + .current_dir(dir) + .output() + .map(|o| o.status.success()) + .unwrap_or(false) +} + fn git_root(start: &Path) -> Option { let output = Command::new("git") .args(["rev-parse", "--show-toplevel"]) diff --git a/crates/kimetsu-e2e/Cargo.toml b/crates/kimetsu-e2e/Cargo.toml index d54a90f..4f09bc7 100644 --- a/crates/kimetsu-e2e/Cargo.toml +++ b/crates/kimetsu-e2e/Cargo.toml @@ -20,9 +20,9 @@ publish = false # Real (non-dev) deps so the test fixtures + scripted provider can be # re-exported from `kimetsu_e2e::prelude` to the integration tests in # `tests/`. Integration tests treat this crate as a normal library. -kimetsu-agent = { path = "../kimetsu-agent", version = "0.7.3" } -kimetsu-brain = { path = "../kimetsu-brain", version = "0.7.3" } -kimetsu-core = { path = "../kimetsu-core", version = "0.7.3" } +kimetsu-agent = { path = "../kimetsu-agent", version = "0.8.0" } +kimetsu-brain = { path = "../kimetsu-brain", version = "0.8.0" } +kimetsu-core = { path = "../kimetsu-core", version = "0.8.0" } rusqlite.workspace = true serde_json.workspace = true time.workspace = true diff --git a/crates/kimetsu-e2e/src/temp_project.rs b/crates/kimetsu-e2e/src/temp_project.rs index b82ba49..0fc7c6c 100644 --- a/crates/kimetsu-e2e/src/temp_project.rs +++ b/crates/kimetsu-e2e/src/temp_project.rs @@ -31,6 +31,9 @@ impl TempProject { pub fn init(label: &str) -> Self { let root = std::env::temp_dir().join(format!("kimetsu-e2e-{label}-{}", RunId::new())); fs::create_dir_all(&root).expect("create temp project root"); + // Give the root its own git boundary so init_project's discover + // resolves here, not an enclosing repo (e.g. a dev's $HOME repo). + kimetsu_core::paths::git_init_boundary(&root); kimetsu_brain::project::init_project(&root, false).expect("init_project"); Self { root, diff --git a/docs/HOW-KIMETSU-WORKS.md b/docs/HOW-KIMETSU-WORKS.md index cd4d0fb..1e936ff 100644 --- a/docs/HOW-KIMETSU-WORKS.md +++ b/docs/HOW-KIMETSU-WORKS.md @@ -137,11 +137,20 @@ budget. - **Embeddings** (default for the CLI): `cargo install kimetsu-cli` ships with `--features embeddings` on. Pulls fastembed-rs + ONNX runtime; needs the VS2022 C++ runtime on Windows (ort prebuilts). - Default model is BGE-small-en-v1.5; set - `KIMETSU_BRAIN_EMBEDDER=jina-v2-base-code` (or any fastembed-rs - model id) to swap. Cosine retrieval, semantic dedup, and conflict - detection all light up. The ~24 MB model downloads to - `~/.cache/huggingface/` on first embed call, then caches. + Default model is BGE-small-en-v1.5. Cosine retrieval, semantic + dedup, and conflict detection all light up. The ~24 MB model + downloads to `~/.cache/huggingface/` on first embed call, then + caches. +- **Choosing the model.** Three built-ins are curated: + `bge-small-en-v1.5` (384d, default), `bge-m3` (1024d, multilingual), + and `jina-v2-base-code` (768d, code-tuned). Resolution precedence is + `KIMETSU_BRAIN_EMBEDDER` env > the `[embedder]` table in + `project.toml` > default. Inspect/switch with `kimetsu brain model + list` / `kimetsu brain model set ` (or the `kimetsu_brain_model_list` + / `kimetsu_brain_model_set` MCP tools). Switching changes the vector + dimension, so `model set` re-embeds the corpus with the new model; + cross-model rows fall back to FTS until reindexed, so retrieval never + breaks mid-migration. - **Lean**: `cargo install kimetsu-cli --no-default-features`. No embedder binary, no model download. Retrieval is FTS-only via the `α=0` effective behavior. Semantic dedup and conflict detection at @@ -269,7 +278,7 @@ should have surfaced. ## 7. The MCP surface -Run `kimetsu chat --mcp-server` and the host harness gets ~24 +Run `kimetsu chat --mcp-server` and the host harness gets ~28 `kimetsu_*` tools. The ones you'll actually reach for: | Tool | What it does | @@ -280,11 +289,16 @@ Run `kimetsu chat --mcp-server` and the host harness gets ~24 | `kimetsu_brain_memory_add` | Persist a new memory directly | | `kimetsu_brain_memory_list` | List memories in scope, sorted by relevance | | `kimetsu_brain_memory_top` | Top memories by usefulness ratio | -| `kimetsu_brain_memory_proposals` | Pending proposals awaiting review | +| `kimetsu_brain_memory_proposals` | Pending proposals awaiting review (paginated: `limit`/`offset`) | | `kimetsu_brain_memory_accept` / `_reject` | Promote / reject a proposal | | `kimetsu_brain_memory_invalidate` | Retire a memory | +| `kimetsu_brain_memory_search` | Full-text search over memory text (paginated; filter by kind/scope) | | `kimetsu_brain_memory_blame` | Per-run citation attribution | -| `kimetsu_brain_memory_conflicts` | List open ingest conflicts | +| `kimetsu_brain_memory_conflicts` / `kimetsu_brain_conflict_resolve` | List / settle open ingest conflicts | +| `kimetsu_brain_prune` | List (or, with `apply`, invalidate) net-negative memories | +| `kimetsu_brain_model_list` / `kimetsu_brain_model_set` | Inspect / switch the embedding model (set re-embeds the corpus) | +| `kimetsu_brain_reindex` | Backfill stale/missing embeddings | +| `kimetsu_brain_config_show` | Read the parsed `project.toml` | | `kimetsu_brain_ingest_repo` | Index repo files + manifests | | `kimetsu_benchmark_context` | Retrieve a task-aware playbook (biases toward `semantic_operator` + `anti_pattern` roles) | | `kimetsu_benchmark_record_outcome` | Record run outcome → proposal | @@ -316,7 +330,39 @@ exists): un-captured session. Both are plain CLI subcommands, so the same hooks work under any -harness that can run a command on a prompt/stop event. +harness that can run a command on a prompt/stop event. `kimetsu plugin +install --target codex` writes the equivalent hooks into +`.codex/hooks.json`. + +### Proactive recall (mid-work) + +`UserPromptSubmit` only fires between turns. v0.8 adds two **tool-level** +hooks so the brain can surface a memory *while* the agent works — the +way a memory "comes to you" rather than you going to fetch it. Both use +a `matcher: "Bash"` so they only fire on shell commands (most tool calls +spawn nothing), and both emit `hookSpecificOutput.additionalContext` +without ever blocking: + +- **`PreToolUse` → `kimetsu brain pretool-hook`** runs *before* a Bash + command; if the command strongly matches a stored `failure_pattern` / + `convention`, it warns first — heading off a known mistake. +- **`PostToolUse` → `kimetsu brain posttool-hook`** runs *after* a Bash + command; when the output looks like a failure, it surfaces a matching + `failure_pattern` / `command` fix. + +Discipline keeps this near-zero-cost and non-spammy: retrieval is +**lexical-FTS-only** (no embedding-model load, so the per-call latency +stays low), gated by a **high score floor** (0.45; 0.35 once a +**repeated** failing command is detected), capped at **one capsule**, +**deduped per session** (a memory surfaces at most once), and +**throttled** by a refractory window between injections. When nothing +clears the bar the hook prints nothing — zero tokens. Per-session state +(surfaced ids, last injection, recent commands) lives in +`/.kimetsu/proactive/.json` and is GC'd after 7 days. + +Proactive hooks install by default with `kimetsu plugin install`; pass +`--no-proactive` (or `proactive:false` to `kimetsu_plugin_install`) to +wire only `UserPromptSubmit` + `Stop`. ---