From 0eda9cdda9405fbdf7e2c6ee0103314f5f68b2d1 Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 20:26:08 -0300 Subject: [PATCH 01/11] docs: spec for global distiller config in ~/.kimetsu Co-Authored-By: Claude Opus 4.8 --- .../2026-06-02-global-distiller-design.md | 168 ++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-02-global-distiller-design.md diff --git a/docs/superpowers/specs/2026-06-02-global-distiller-design.md b/docs/superpowers/specs/2026-06-02-global-distiller-design.md new file mode 100644 index 0000000..eaef9c8 --- /dev/null +++ b/docs/superpowers/specs/2026-06-02-global-distiller-design.md @@ -0,0 +1,168 @@ +# Global distiller config in `~/.kimetsu/` + +**Date:** 2026-06-02 +**Status:** Approved (pending spec review) +**Builds on:** the SessionEnd distiller + setup wizard (same v0.9.0 line) + +## Context + +The SessionEnd distiller is configured **per-workspace** today: the wizard writes +`[learning.distiller]` + the key into the install workspace, and the hook reads +`discover(cwd)`'s project config. A `--scope global` install puts the `SessionEnd` +**hook** in `~/.claude/settings.json` (it fires in every project), but the +distiller **config** still only exists in the one workspace where setup ran — so +the global hook silently no-ops everywhere else. + +This adds a **global distiller**: configure it once (`--scope global`) and have it +distill at the end of every session, recording into the always-available user +brain. `~/.kimetsu/` is already the user-brain home, so it becomes the home for +the global distiller config too — "everything global lives in `~/.kimetsu/`". + +## Decisions (settled in brainstorming) + +- **Recording scope (global):** globally-distilled lessons record with + `MemoryScope::GlobalUser` → `~/.kimetsu/brain.db` (the user brain), available in + every project. Reuses the existing GlobalUser routing. +- **Precedence:** workspace overrides global. If the current project's distiller + is enabled, use it (project key, Project-scoped recording); otherwise fall back + to the global distiller (global key, GlobalUser recording). +- **Config home:** `~/.kimetsu/project.toml` `[learning.distiller]` + `~/.kimetsu/.env`, + resolved via `kimetsu_core::paths::user_kimetsu_dir()` (respects + `KIMETSU_USER_BRAIN_DIR`; absent when the user brain is disabled). + +## Key facts grounding the design + +- `add_memory(start, MemoryScope::GlobalUser, kind, text)` routes to the user + brain via an early return **before** `load_project(start)` (project.rs:436-440), + so global recording works in any project — even one with no brain of its own. + The user brain has no proposal queue, so global recording is add-or-dedup (no + confidence gating); workspace recording keeps `propose_or_merge_memory`. +- `kimetsu_core::paths::{user_kimetsu_dir, user_brain_enabled}` already resolve the + `~/.kimetsu` dir and the enabled flag. +- `resolve_env_value(repo_root, name)` checks process env then `/.env`, + so a global key lookup is just `resolve_env_value(user_kimetsu_dir(), name)` + (process env → `~/.kimetsu/.env`) — no change to that function. + +## Components & files + +### 1. Wizard write target by scope (`crates/kimetsu-cli/src/harvest_setup.rs`, `main.rs`) +`run_harvest_setup` currently takes `&ProjectPaths`. The global config doesn't fit +that layout (its `project.toml` and `.env` both live directly in `~/.kimetsu/`, +not at `repo_root` + `repo_root/.kimetsu/`). Replace the parameter with an explicit +target: +```rust +pub struct SetupTarget { + pub project_toml: PathBuf, // where [learning.distiller] is written + pub env_path: PathBuf, // where the key/base URL are written + pub gitignore_dir: PathBuf,// dir whose .gitignore must ignore ".env" +} +``` +`apply_distiller_config(target.project_toml, model)` loads-or-defaults that toml, +sets the distiller fields, writes it back; `upsert_env_var(target.env_path, …)`; +`ensure_gitignored(target.gitignore_dir, ".env")`. No `discover`/`init_project` +(no climbing). The CLI gate builds the target from the install scope: +- **workspace:** `project_toml = /.kimetsu/project.toml`, `env = /.env`, + `gitignore_dir = ` (built from `ProjectPaths::at_root(&workspace)`). +- **global:** `dir = user_kimetsu_dir()` (skip with a message if `None`); + `project_toml = dir/project.toml`, `env = dir/.env`, `gitignore_dir = dir`. + +The wizard's success message names the scope ("global — applies to all projects" +vs "this workspace"). + +### 2. Distiller config resolution (`crates/kimetsu-cli/src/distiller.rs`) +A new resolver returns which distiller to run and how to record: +```rust +struct ResolvedDistiller { + section: DistillerSection, // model, api_key_env, base_url_env + key: String, + base_url: Option, + scope: MemoryScope, // Project or GlobalUser + record_start: PathBuf, // passed to the recorder +} + +fn resolve_distiller(workspace: &Path) -> Option +``` +Order: +1. **Workspace:** `discover(workspace)` + `load_config`; if `distiller.enabled` and + `resolve_env_value(repo_root, api_key_env)` is `Some` → `scope = Project`, + `record_start = repo_root`, base via `resolve_env_value(repo_root, base_url_env)`. +2. **Global:** if `user_brain_enabled()` and `user_kimetsu_dir()` is `Some(dir)`, + read `dir/project.toml` via `ProjectConfig::from_toml`; if `distiller.enabled` + and `resolve_env_value(dir, api_key_env)` is `Some` → `scope = GlobalUser`, + `record_start = workspace` (the cwd; `add_memory` ignores `start` on the + GlobalUser path, so this never depends on the cwd having a brain), base via + `resolve_env_value(dir, base_url_env)`. +3. Else `None`. + +Edge: a workspace with `distiller.enabled = false` (or no `[learning.distiller]`) +falls through to global. Explicitly opting out per-project while a global distiller +is configured isn't supported (documented). + +### 3. Recorder scope (`distill_and_record`) +Add a `scope: MemoryScope` parameter: +- `Project` → `project::propose_or_merge_memory(start, Project, kind, text, conf, …)` + (confidence-gated, semantic merge — current behavior). +- `GlobalUser` → `project::add_memory(start, GlobalUser, kind, text)` (routes to the + user brain; exact-text dedup; confidence not used by the user-brain write). + +### 4. Hook entry (`run_session_end_hook`) +Replace the inline workspace-only config load with `resolve_distiller(workspace)`. +If `Some(r)`: build `AnthropicProvider::for_distiller(r.section.model, r.key, +r.base_url, timeout)`, build the transcript view, `distill_and_record(&r.record_start, +&view, &mut provider, r.scope)`; banner `"[Kimetsu] distilled N lesson(s) at session +end."` Else silent no-op. Best-effort throughout. + +### 5. Stop-cue suppression (`brain_stop_hook`) +`should_emit_stop_harvest_cue` is unchanged; the call site now computes +`distiller_enabled = workspace.enabled || global.enabled`, where `global.enabled` +reads `~/.kimetsu/project.toml`'s distiller via a small `global_distiller_enabled()` +helper (false when the user brain is disabled / no `~/.kimetsu/project.toml`). So +either distiller suppresses the Stop end-of-session cue. + +## Data flow (global distiller, no workspace distiller) + +``` +session ends (any project) → global SessionEnd hook → kimetsu brain session-end-hook + → resolve_distiller(cwd): workspace disabled → read ~/.kimetsu/project.toml → + global distiller enabled + key in ~/.kimetsu/.env → scope = GlobalUser + → distill transcript with the global model → add_memory(_, GlobalUser, …) + → ~/.kimetsu/brain.db (available in every project) +``` + +## Error handling + +Best-effort and silent on failure everywhere (no `~/.kimetsu`, user brain disabled, +disabled distiller, missing key, unreadable transcript, model/HTTP error, malformed +config). The wizard skips global setup with a message if `user_kimetsu_dir()` is +`None`. Never breaks session shutdown or the install. + +## Testing + +- **Wizard target by scope:** `run_harvest_setup` with a `SetupTarget` pointing at a + temp "global dir" writes `project.toml` + `.env` there; with a workspace target + writes under the workspace. (No `discover`; tests are deterministic.) +- **`resolve_distiller` precedence:** with a temp `KIMETSU_USER_BRAIN_DIR` (under the + env lock) — workspace-enabled wins (Project); workspace-absent + global-enabled → + global (GlobalUser); neither → `None`. Key-absent disables each tier. +- **`distill_and_record` GlobalUser:** writes to a temp user brain (`KIMETSU_USER_BRAIN_DIR` + + `with_user_brain_disabled`-style lock); assert the lesson lands in the user brain + via `list_user_memories`. +- **Stop suppression:** `global_distiller_enabled()` true (global toml) suppresses the + cue even with no workspace distiller. + +## Risks / trade-offs + +- **`--scope global` writes `[learning.distiller]` into the user-brain + `~/.kimetsu/project.toml`** — intentional (the global config home); `brain.db` + untouched. +- **No per-project opt-out** when a global distiller is configured (disabled + workspace distiller falls through to global). Documented; a future explicit + `disabled` sentinel could opt out. +- **Global recording is exact-dedup only** (user brain has no proposal queue) — fine + for distilled lessons; cross-session duplicates collapse by normalized text. + +## Out of scope / follow-ups + +- OpenAI / Codex distiller providers (still deferred). +- A `kimetsu brain distiller test` command to validate the configured endpoint. +- Per-project explicit opt-out sentinel. From 18c12f607f678105797c1a369bc0b94626350c12 Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 20:30:38 -0300 Subject: [PATCH 02/11] docs: implementation plan for the global distiller config Co-Authored-By: Claude Opus 4.8 --- .../plans/2026-06-02-global-distiller.md | 755 ++++++++++++++++++ 1 file changed, 755 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-02-global-distiller.md diff --git a/docs/superpowers/plans/2026-06-02-global-distiller.md b/docs/superpowers/plans/2026-06-02-global-distiller.md new file mode 100644 index 0000000..4a80a92 --- /dev/null +++ b/docs/superpowers/plans/2026-06-02-global-distiller.md @@ -0,0 +1,755 @@ +# Global Distiller Config Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Let `kimetsu plugin install --scope global` configure a distiller once (in `~/.kimetsu/`) that distills every session and records into the always-available user brain, with workspace distillers taking precedence. + +**Architecture:** The distiller config gets a second home — `~/.kimetsu/project.toml` `[learning.distiller]` + `~/.kimetsu/.env`. A `resolve_distiller(cwd)` resolver picks the workspace distiller (record `Project`) if enabled, else the global one (record `GlobalUser` → `~/.kimetsu/brain.db`). The wizard writes to the workspace or `~/.kimetsu` per `--scope`. + +**Tech Stack:** Rust; `kimetsu_core::paths::{user_kimetsu_dir, user_brain_enabled}`; `kimetsu_brain::project::{add_memory, propose_or_merge_memory}` (GlobalUser already routes to the user brain). + +**Spec:** `docs/superpowers/specs/2026-06-02-global-distiller-design.md` + +## File Map +- **Modify** `crates/kimetsu-cli/src/distiller.rs` — `distill_and_record` gains a `scope`; new `ResolvedDistiller` + `resolve_distiller`/`resolve_distiller_with`; `global_distiller_enabled`/`global_distiller_enabled_in`; `run_session_end_hook` uses the resolver. +- **Modify** `crates/kimetsu-cli/src/harvest_setup.rs` — `SetupTarget` struct; `run_harvest_setup`/`apply_distiller_config` take it. +- **Modify** `crates/kimetsu-cli/src/main.rs` — CLI gate builds `SetupTarget` per `--scope`; `brain_stop_hook` ORs in the global distiller. + +--- + +## Task 1: `distill_and_record` records by scope + +**Files:** Modify `crates/kimetsu-cli/src/distiller.rs` (the `distill_and_record` fn + its caller in `run_session_end_hook` + the existing/new tests). + +- [ ] **Step 1: Write the failing test** + +Add a test helper + test to `distiller.rs` `mod tests` (the `text_response` helper already exists there): + +```rust + /// Run `f` with the user brain pointed at a temp dir (enabled), under + /// the process-wide env lock, restoring the previous env afterward. + fn with_user_brain_dir(dir: &std::path::Path, f: impl FnOnce() -> R) -> R { + let _g = kimetsu_brain::user_brain::test_env_lock() + .lock() + .unwrap_or_else(|p| p.into_inner()); + let prev_dir = std::env::var("KIMETSU_USER_BRAIN_DIR").ok(); + let prev_en = std::env::var("KIMETSU_USER_BRAIN").ok(); + // SAFETY: scoped by the shared lock. + unsafe { + std::env::set_var("KIMETSU_USER_BRAIN_DIR", dir); + std::env::remove_var("KIMETSU_USER_BRAIN"); + } + let out = f(); + unsafe { + match prev_dir { + Some(v) => std::env::set_var("KIMETSU_USER_BRAIN_DIR", v), + None => std::env::remove_var("KIMETSU_USER_BRAIN_DIR"), + } + match prev_en { + Some(v) => std::env::set_var("KIMETSU_USER_BRAIN", v), + None => std::env::remove_var("KIMETSU_USER_BRAIN"), + } + } + out + } + + #[test] + fn distill_and_record_global_writes_to_user_brain() { + let dir = std::env::temp_dir().join(format!( + "kimetsu_userbrain_{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + std::fs::create_dir_all(&dir).unwrap(); + with_user_brain_dir(&dir, || { + let mut provider = MockProvider::new([text_response( + "[{\"lesson\":\"Global lesson kept everywhere\",\"tags\":[\"x\"],\"confidence\":0.9}]", + )]); + // `start` is ignored on the GlobalUser path; pass the temp dir. + let n = distill_and_record(&dir, "user: a", &mut provider, MemoryScope::GlobalUser); + assert_eq!(n, 1); + let conn = kimetsu_brain::user_brain::open_user_brain_readonly() + .unwrap() + .expect("user brain exists"); + let mems = kimetsu_brain::user_brain::list_user_memories(&conn).unwrap(); + assert!(mems.iter().any(|m| m.text.contains("Global lesson kept everywhere"))); + }); + std::fs::remove_dir_all(dir).ok(); + } +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cargo test -p kimetsu-cli distill_and_record_global_writes_to_user_brain` +Expected: FAIL — `distill_and_record` takes 3 args, not 4 (arity mismatch). + +- [ ] **Step 3: Add the `scope` parameter** + +Replace `distill_and_record` with: + +```rust +/// Distill lessons from `view` and record each. `Project` scope uses the +/// confidence-gated `propose_or_merge_memory` (workspace brain); `GlobalUser` +/// uses `add_memory`, which routes to `~/.kimetsu/brain.db` (the user brain +/// has no proposal queue, so this is add-or-dedup). Returns the count recorded. +pub fn distill_and_record( + start: &Path, + view: &str, + provider: &mut dyn ModelProvider, + scope: MemoryScope, +) -> usize { + let mut recorded = 0; + for lesson in distill_lessons(view, provider) { + // Mirror kimetsu_brain_record's MCP kind mapping; semantic_operator + default store as Fact. + let kind = match lesson.kind.as_str() { + "anti_pattern" => MemoryKind::FailurePattern, + "convention" => MemoryKind::Convention, + _ => MemoryKind::Fact, + }; + let text = lesson.lesson.trim(); + let ok = match scope { + MemoryScope::GlobalUser => project::add_memory(start, MemoryScope::GlobalUser, kind, text).is_ok(), + _ => project::propose_or_merge_memory( + start, + scope, + kind, + text, + lesson.confidence.clamp(0.0, 1.0), + "auto-harvested at session end", + ) + .is_ok(), + }; + if ok { + recorded += 1; + } + } + recorded +} +``` + +- [ ] **Step 4: Update the existing caller + the existing temp-brain test** + +In `run_session_end_hook`, the current call is `distill_and_record(&paths.repo_root, &view, &mut provider)`. Change it to pass the project scope for now (Task 2 reworks this fn fully): + +```rust + let recorded = distill_and_record(&paths.repo_root, &view, &mut provider, MemoryScope::Project); +``` + +In the existing `distill_and_record_writes_to_a_temp_brain` test, add the scope arg to its call: + +```rust + let n = distill_and_record(&root, "user: a\nuser: b", &mut provider, MemoryScope::Project); +``` + +- [ ] **Step 5: Run tests** + +Run: `cargo test -p kimetsu-cli distiller` +Expected: PASS (existing distiller tests + the new GlobalUser test). + +If `list_user_memories` or `open_user_brain_readonly` aren't `pub` at `kimetsu_brain::user_brain::…`, confirm their path (they are used from `project.rs` as `user_brain::list_user_memories` / `open_user_brain_readonly`, so they are `pub`). `MemoryRow` (returned by `list_user_memories`) has a `.text` field. + +- [ ] **Step 6: Commit** + +```bash +git add crates/kimetsu-cli/src/distiller.rs +git commit -m "feat: distill_and_record records by MemoryScope (GlobalUser -> user brain)" +``` + +--- + +## Task 2: `resolve_distiller` — workspace-overrides-global resolution + +**Files:** Modify `crates/kimetsu-cli/src/distiller.rs` (new types/fns + rewrite `run_session_end_hook`; tests). + +- [ ] **Step 1: Write the failing tests** + +Add to `mod tests` (uses `with_user_brain_dir` from Task 1 only indirectly — here we inject the global dir directly, so no env mutation needed): + +```rust + fn write_distiller_toml(dir: &std::path::Path, enabled: bool, model: &str) { + std::fs::create_dir_all(dir).unwrap(); + let toml = format!( + "[learning.distiller]\nenabled = {enabled}\nprovider = \"anthropic\"\n\ + model = \"{model}\"\napi_key_env = \"ANTHROPIC_API_KEY\"\n\ + base_url_env = \"ANTHROPIC_BASE_URL\"\n" + ); + std::fs::write(dir.join("project.toml"), toml).unwrap(); + } + + #[test] + fn resolve_distiller_global_when_no_workspace() { + // No workspace config; a global dir with an enabled distiller + a key in + // its .env -> resolves to the global distiller, GlobalUser scope. + let ws = std::env::temp_dir().join(format!( + "km_rd_ws_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + std::fs::create_dir_all(&ws).unwrap(); + let gdir = std::env::temp_dir().join(format!( + "km_rd_g_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + write_distiller_toml(&gdir, true, "claude-haiku-4-5"); + std::fs::write(gdir.join(".env"), "ANTHROPIC_API_KEY=sk-global\n").unwrap(); + + let r = resolve_distiller_with(&ws, Some(gdir.clone())).expect("global resolved"); + assert_eq!(r.scope, MemoryScope::GlobalUser); + assert_eq!(r.model, "claude-haiku-4-5"); + assert_eq!(r.key, "sk-global"); + + // Disabled global -> None. + write_distiller_toml(&gdir, false, "claude-haiku-4-5"); + assert!(resolve_distiller_with(&ws, Some(gdir.clone())).is_none()); + + std::fs::remove_dir_all(ws).ok(); + std::fs::remove_dir_all(gdir).ok(); + } + + #[test] + fn resolve_distiller_workspace_wins() { + // Workspace dir (its own git toplevel) with an enabled distiller + .env + // key wins over a global dir. + let ws = std::env::temp_dir().join(format!( + "km_rd_wsw_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + std::fs::create_dir_all(ws.join(".kimetsu")).unwrap(); + kimetsu_core::paths::git_init_boundary(&ws); + write_distiller_toml(&ws.join(".kimetsu"), true, "ws-model"); + std::fs::write(ws.join(".env"), "ANTHROPIC_API_KEY=sk-ws\n").unwrap(); + + let gdir = std::env::temp_dir().join(format!( + "km_rd_gw_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + write_distiller_toml(&gdir, true, "g-model"); + std::fs::write(gdir.join(".env"), "ANTHROPIC_API_KEY=sk-global\n").unwrap(); + + let r = resolve_distiller_with(&ws, Some(gdir.clone())).expect("workspace resolved"); + assert_eq!(r.scope, MemoryScope::Project); + assert_eq!(r.model, "ws-model"); + assert_eq!(r.key, "sk-ws"); + + std::fs::remove_dir_all(ws).ok(); + std::fs::remove_dir_all(gdir).ok(); + } +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cargo test -p kimetsu-cli resolve_distiller` +Expected: FAIL — `resolve_distiller_with`/`ResolvedDistiller` not found. + +- [ ] **Step 3: Add the resolver** + +Add near the top of `distiller.rs` (after the imports), extending the existing `use` lines as needed: + +```rust +use kimetsu_core::config::ProjectConfig; +use kimetsu_core::paths::{user_brain_enabled, user_kimetsu_dir}; +``` + +Add the type + functions (place above `run_session_end_hook`): + +```rust +/// The distiller selected for this session: which model/key/endpoint to use, +/// and how to record (project vs the global user brain). +pub struct ResolvedDistiller { + pub model: String, + pub key: String, + pub base_url: Option, + pub timeout_secs: u64, + pub scope: MemoryScope, + pub record_start: std::path::PathBuf, +} + +/// Resolve the distiller for `workspace`, preferring the workspace distiller +/// over the global one (`~/.kimetsu`). `None` when neither is enabled + +/// credentialed. +pub fn resolve_distiller(workspace: &Path) -> Option { + let global_dir = if user_brain_enabled() { + user_kimetsu_dir() + } else { + None + }; + resolve_distiller_with(workspace, global_dir) +} + +/// Testable core: `global_dir` is injected (the `~/.kimetsu` dir, or `None`). +fn resolve_distiller_with( + workspace: &Path, + global_dir: Option, +) -> Option { + // 1. Workspace distiller (Project scope). + if let Ok(paths) = ProjectPaths::discover(workspace) + && let Ok(config) = project::load_config(&paths) + { + let d = &config.learning.distiller; + if d.enabled + && d.provider == "anthropic" + && let Some(key) = resolve_env_value(&paths.repo_root, &d.api_key_env) + { + return Some(ResolvedDistiller { + model: d.model.clone(), + key, + base_url: resolve_env_value(&paths.repo_root, &d.base_url_env), + timeout_secs: config.model.request_timeout_secs, + scope: MemoryScope::Project, + record_start: paths.repo_root.clone(), + }); + } + } + // 2. Global distiller (GlobalUser scope). + if let Some(dir) = global_dir + && let Ok(text) = std::fs::read_to_string(dir.join("project.toml")) + && let Ok(config) = ProjectConfig::from_toml(&text) + { + let d = &config.learning.distiller; + if d.enabled + && d.provider == "anthropic" + && let Some(key) = resolve_env_value(&dir, &d.api_key_env) + { + return Some(ResolvedDistiller { + model: d.model.clone(), + key, + base_url: resolve_env_value(&dir, &d.base_url_env), + timeout_secs: config.model.request_timeout_secs, + scope: MemoryScope::GlobalUser, + record_start: workspace.to_path_buf(), + }); + } + } + None +} +``` + +- [ ] **Step 4: Rewrite `run_session_end_hook` to use the resolver** + +Replace the whole `run_session_end_hook` body with: + +```rust +pub fn run_session_end_hook(workspace: &Path) { + let mut input = String::new(); + std::io::stdin().read_to_string(&mut input).ok(); + let payload: serde_json::Value = + serde_json::from_str(input.trim()).unwrap_or(serde_json::Value::Null); + + let Some(transcript_path) = payload + .get("transcript_path") + .and_then(|v| v.as_str()) + .filter(|p| !p.trim().is_empty()) + else { + return; + }; + let Some(resolved) = resolve_distiller(workspace) else { + return; + }; + let view = build_transcript_view(transcript_path, MAX_VIEW_CHARS); + if view.trim().is_empty() { + return; + } + let Ok(mut provider) = AnthropicProvider::for_distiller( + &resolved.model, + resolved.key, + resolved.base_url, + resolved.timeout_secs, + ) else { + return; + }; + let recorded = + distill_and_record(&resolved.record_start, &view, &mut provider, resolved.scope); + if recorded > 0 { + println!( + "[Kimetsu] distilled {recorded} lesson{} at session end.", + if recorded == 1 { "" } else { "s" } + ); + } +} +``` + +This removes the now-unused inline `config`/`distiller`/`api_key`/`base_url` locals. The imports `resolve_env_value`, `AnthropicProvider`, `project`, `ProjectPaths` are still used (by the resolver). Remove any import that becomes unused after this change (the compiler will warn). + +- [ ] **Step 5: Run tests + build** + +Run: `cargo test -p kimetsu-cli distiller` +Expected: PASS (the two new resolve tests + prior ones). + +Run: `cargo build -p kimetsu-cli` +Expected: clean (no unused-import warnings; fix any the compiler flags). + +- [ ] **Step 6: Commit** + +```bash +git add crates/kimetsu-cli/src/distiller.rs +git commit -m "feat: resolve_distiller (workspace overrides global ~/.kimetsu)" +``` + +--- + +## Task 3: `global_distiller_enabled` for Stop-cue suppression + +**Files:** Modify `crates/kimetsu-cli/src/distiller.rs` (new fns + test). + +- [ ] **Step 1: Write the failing test** + +Add to `mod tests`: + +```rust + #[test] + fn global_distiller_enabled_reads_global_toml() { + let gdir = std::env::temp_dir().join(format!( + "km_gde_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + write_distiller_toml(&gdir, true, "claude-haiku-4-5"); + assert!(global_distiller_enabled_in(Some(gdir.clone()))); + + write_distiller_toml(&gdir, false, "claude-haiku-4-5"); + assert!(!global_distiller_enabled_in(Some(gdir.clone()))); + + assert!(!global_distiller_enabled_in(None)); + std::fs::remove_dir_all(gdir).ok(); + } +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cargo test -p kimetsu-cli global_distiller_enabled_reads_global_toml` +Expected: FAIL — `global_distiller_enabled_in` not found. + +- [ ] **Step 3: Add the functions** + +In `distiller.rs`: + +```rust +/// True when a global distiller (`~/.kimetsu/project.toml`) is enabled. Used by +/// the Stop hook to suppress its end-of-session cue (the distiller owns it). +pub fn global_distiller_enabled() -> bool { + let dir = if user_brain_enabled() { + user_kimetsu_dir() + } else { + None + }; + global_distiller_enabled_in(dir) +} + +fn global_distiller_enabled_in(global_dir: Option) -> bool { + global_dir + .and_then(|dir| std::fs::read_to_string(dir.join("project.toml")).ok()) + .and_then(|text| ProjectConfig::from_toml(&text).ok()) + .map(|c| c.learning.distiller.enabled) + .unwrap_or(false) +} +``` + +- [ ] **Step 4: Run test** + +Run: `cargo test -p kimetsu-cli global_distiller_enabled` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add crates/kimetsu-cli/src/distiller.rs +git commit -m "feat: global_distiller_enabled helper for Stop-cue suppression" +``` + +--- + +## Task 4: `SetupTarget` — wizard writes to workspace or `~/.kimetsu` + +**Files:** Modify `crates/kimetsu-cli/src/harvest_setup.rs` (struct + signatures + tests). + +- [ ] **Step 1: Write the failing test** + +Replace the test module's `paths_for` helper and the two wizard tests with target-based versions. Add to `harvest_setup.rs` `mod tests`: + +```rust + fn target_at(dir: &Path) -> SetupTarget { + SetupTarget { + project_toml: dir.join(".kimetsu").join("project.toml"), + env_path: dir.join(".env"), + gitignore_dir: dir.to_path_buf(), + } + } +``` + +Replace the three wizard tests' bodies (`wizard_writes_env_and_config`, +`wizard_declined_writes_nothing`, `wizard_unrecognized_harness_aborts`) to use a +`SetupTarget` and read via `root` directly. The happy-path test in full: + +```rust + #[test] + fn wizard_writes_env_and_config() { + let root = std::env::temp_dir().join(format!( + "kimetsu_wizard_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + fs::create_dir_all(root.join(".kimetsu")).unwrap(); + let mut input = + Cursor::new("y\nclaude\nsk-litellm-123\nhttp://localhost:4000\n\n".as_bytes().to_vec()); + let mut output = Vec::new(); + let configured = + run_harvest_setup(&mut input, &mut output, &target_at(&root), "this workspace").unwrap(); + assert!(configured); + + let env = fs::read_to_string(root.join(".env")).unwrap(); + assert!(env.contains("ANTHROPIC_API_KEY=sk-litellm-123")); + assert!(env.contains("ANTHROPIC_BASE_URL=http://localhost:4000")); + let toml = fs::read_to_string(root.join(".kimetsu").join("project.toml")).unwrap(); + assert!(toml.contains("enabled = true")); + assert!(toml.contains("claude-haiku-4-5")); + assert!(fs::read_to_string(root.join(".gitignore")).unwrap().contains(".env")); + fs::remove_dir_all(root).ok(); + } +``` + +For `wizard_declined_writes_nothing` and `wizard_unrecognized_harness_aborts`: +same as the current versions, but (a) build the target with `let paths = target_at(&root);` +(after `fs::create_dir_all(&root).unwrap();`), (b) call +`run_harvest_setup(&mut input, &mut output, &paths, "this workspace")`, and (c) assert +`!root.join(".env").exists()`. Keep the `upsert_env_var_replaces_existing` test unchanged. + +- [ ] **Step 2: Run to verify failure** + +Run: `cargo test -p kimetsu-cli harvest_setup` +Expected: FAIL — `SetupTarget` not found / arity mismatch. + +- [ ] **Step 3: Add `SetupTarget` and re-thread the wizard** + +In `harvest_setup.rs`, change the imports and add the struct: + +```rust +use std::fs; +use std::io::{BufRead, Write}; +use std::path::{Path, PathBuf}; + +use kimetsu_core::config::ProjectConfig; + +/// Where the wizard writes the distiller config + secret. Built by the CLI gate +/// from the install scope (workspace dir, or `~/.kimetsu`). +pub struct SetupTarget { + pub project_toml: PathBuf, + pub env_path: PathBuf, + pub gitignore_dir: PathBuf, +} +``` + +(Remove the now-unused `use kimetsu_core::paths::ProjectPaths;` if nothing else references it.) + +Change `run_harvest_setup`'s signature + the write block: + +```rust +pub fn run_harvest_setup( + reader: &mut R, + writer: &mut W, + target: &SetupTarget, + scope_label: &str, +) -> std::io::Result { +``` + +(keep the prompt flow unchanged through the model prompt), then replace the write block at the end: + +```rust + apply_distiller_config(&target.project_toml, &model)?; + // Gitignore `.env` BEFORE writing the secret into it. + ensure_gitignored(&target.gitignore_dir, ".env")?; + upsert_env_var(&target.env_path, "ANTHROPIC_API_KEY", &key)?; + if !base_url.is_empty() { + upsert_env_var(&target.env_path, "ANTHROPIC_BASE_URL", &base_url)?; + } + + writeln!( + writer, + "\u{2713} Distiller configured for {scope_label} (model {model}). \ + Key stored in {} (gitignored). Note: the key was entered in plain text.", + target.env_path.display() + )?; + Ok(true) +} +``` + +Change `apply_distiller_config` to take the toml path directly: + +```rust +fn apply_distiller_config(project_toml: &Path, model: &str) -> std::io::Result<()> { + let io_err = + |e: Box| std::io::Error::other(e.to_string()); + let mut config = if project_toml.exists() { + ProjectConfig::from_toml(&fs::read_to_string(project_toml)?).map_err(io_err)? + } else { + // /.kimetsu/project.toml -> project_id from . + let project_id = project_toml + .parent() + .and_then(|p| p.parent()) + .and_then(|p| p.file_name()) + .and_then(|n| n.to_str()) + .unwrap_or("workspace") + .to_string(); + ProjectConfig::default_for_project(project_id) + }; + config.learning.distiller.enabled = true; + config.learning.distiller.provider = "anthropic".to_string(); + config.learning.distiller.model = model.to_string(); + config.learning.distiller.api_key_env = "ANTHROPIC_API_KEY".to_string(); + config.learning.distiller.base_url_env = "ANTHROPIC_BASE_URL".to_string(); + if let Some(parent) = project_toml.parent() { + fs::create_dir_all(parent)?; + } + fs::write(project_toml, config.to_toml().map_err(io_err)?) +} +``` + +- [ ] **Step 4: Run tests** + +Run: `cargo test -p kimetsu-cli harvest_setup` +Expected: PASS (4 tests). + +- [ ] **Step 5: Commit** + +```bash +git add crates/kimetsu-cli/src/harvest_setup.rs +git commit -m "feat: wizard writes to an explicit SetupTarget (workspace or ~/.kimetsu)" +``` + +--- + +## Task 5: CLI gate + Stop hook wiring + +**Files:** Modify `crates/kimetsu-cli/src/main.rs` (the `plugin` Install arm gate; `brain_stop_hook`). + +- [ ] **Step 1: Update the CLI gate to build a `SetupTarget` by scope** + +In the `plugin` fn's `Install` arm, replace the interactive block (the `if matches!(target, BridgeTarget::ClaudeCode) && !args.no_setup && interactive { … }` body) with: + +```rust + if matches!(target, BridgeTarget::ClaudeCode) && !args.no_setup && interactive { + let target_for_scope = match scope { + InstallScope::Global => match kimetsu_core::paths::user_kimetsu_dir() { + Some(dir) => Some(( + harvest_setup::SetupTarget { + project_toml: dir.join("project.toml"), + env_path: dir.join(".env"), + gitignore_dir: dir, + }, + "globally (all projects, ~/.kimetsu)", + )), + None => { + eprintln!( + "kimetsu plugin install: cannot resolve ~/.kimetsu; skipping distiller setup." + ); + None + } + }, + InstallScope::Workspace => { + let p = kimetsu_core::paths::ProjectPaths::at_root(&workspace); + Some(( + harvest_setup::SetupTarget { + project_toml: p.project_toml.clone(), + env_path: p.repo_root.join(".env"), + gitignore_dir: p.repo_root.clone(), + }, + "this workspace", + )) + } + }; + if let Some((setup_target, label)) = target_for_scope { + let stdin = std::io::stdin(); + let mut reader = stdin.lock(); + let mut stdout = std::io::stdout(); + if let Err(err) = harvest_setup::run_harvest_setup( + &mut reader, + &mut stdout, + &setup_target, + label, + ) { + eprintln!("kimetsu plugin install: distiller setup skipped: {err}"); + } + } + } +``` + +(`scope` and `workspace` are in scope in the `Install` arm; `InstallScope` is already imported in `fn plugin`.) + +- [ ] **Step 2: Update `brain_stop_hook` to OR in the global distiller** + +In `brain_stop_hook`, the config load is: + +```rust + let (auto_harvest, distiller_enabled) = paths + .as_ref() + .and_then(|p| project::load_config(p).ok()) + .map(|c| (c.learning.auto_harvest, c.learning.distiller.enabled)) + .unwrap_or((true, false)); +``` + +Change the `distiller_enabled` to also consider the global distiller: + +```rust + let (auto_harvest, workspace_distiller) = paths + .as_ref() + .and_then(|p| project::load_config(p).ok()) + .map(|c| (c.learning.auto_harvest, c.learning.distiller.enabled)) + .unwrap_or((true, false)); + let distiller_enabled = workspace_distiller || distiller::global_distiller_enabled(); +``` + +- [ ] **Step 3: Build + test** + +Run: `cargo build -p kimetsu-cli` +Expected: clean. + +Run: `cargo test -p kimetsu-cli` +Expected: PASS. + +- [ ] **Step 4: Commit** + +```bash +git add crates/kimetsu-cli/src/main.rs +git commit -m "feat: --scope global wizard target + global distiller Stop suppression" +``` + +--- + +## Task 6: Verification + docs + +- [ ] **Step 1: Full build/test/fmt** + +Run: `cargo build --workspace` → clean. +Run: `cargo test --workspace` → all green. +Run: `cargo fmt --all && cargo fmt --all --check` → clean. + +- [ ] **Step 2: Manual global smoke (sandbox `KIMETSU_USER_BRAIN_DIR`, non-home)** + +```bash +g=$(mktemp -d); ws=$(mktemp -d) +KIMETSU_USER_BRAIN_DIR="$(cygpath -m "$g")" \ + printf 'y\nclaude\nsk-global\n\n\n' | \ + KIMETSU_USER_BRAIN_DIR="$(cygpath -m "$g")" target/debug/kimetsu.exe \ + plugin install claude-code --scope global --workspace "$(cygpath -m "$ws")" --setup-harvest +cat "$g/project.toml" | grep -A2 distiller # enabled = true at ~/.kimetsu (sandbox) +cat "$g/.env" # ANTHROPIC_API_KEY=sk-global +ls "$ws/.env" 2>/dev/null && echo "WS POLLUTED" || echo "workspace clean (global only) ✓" +rm -rf "$g" "$ws" +``` + +Expected: the distiller config + key land in the sandbox global dir, not the workspace. + +- [ ] **Step 3: Update CHANGELOG + README** + +Extend the v0.9.0 distiller CHANGELOG bullet to mention `--scope global` configures a global distiller in `~/.kimetsu/` recording to the user brain (workspace overrides global). Add a sentence to the README distiller note. + +- [ ] **Step 4: Commit** + +```bash +git add CHANGELOG.md README.md +git commit -m "docs: document the global distiller (~/.kimetsu)" +``` + +--- + +## Done + +`--scope global` now configures a distiller in `~/.kimetsu/` that distills every session and records into the user brain (`MemoryScope::GlobalUser`), with per-workspace distillers taking precedence. The Stop hook suppresses its end-of-session cue when either a workspace or global distiller is enabled. From bf50798b05004a58f3347fe316d5f41f4aa5cd34 Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 20:33:43 -0300 Subject: [PATCH 03/11] feat: distill_and_record records by MemoryScope (GlobalUser -> user brain) --- crates/kimetsu-cli/src/distiller.rs | 97 ++++++++++++++++++++++++----- 1 file changed, 81 insertions(+), 16 deletions(-) diff --git a/crates/kimetsu-cli/src/distiller.rs b/crates/kimetsu-cli/src/distiller.rs index c43045b..0ff6282 100644 --- a/crates/kimetsu-cli/src/distiller.rs +++ b/crates/kimetsu-cli/src/distiller.rs @@ -191,9 +191,16 @@ fn tail_chars(s: &str, n: usize) -> String { } } -/// Distill lessons from `view` and record each via the confidence-gated -/// brain API. Returns the count recorded. -pub fn distill_and_record(start: &Path, view: &str, provider: &mut dyn ModelProvider) -> usize { +/// Distill lessons from `view` and record each. `Project` scope uses the +/// confidence-gated `propose_or_merge_memory` (workspace brain); `GlobalUser` +/// uses `add_memory`, which routes to `~/.kimetsu/brain.db` (the user brain +/// has no proposal queue, so this is add-or-dedup). Returns the count recorded. +pub fn distill_and_record( + start: &Path, + view: &str, + provider: &mut dyn ModelProvider, + scope: MemoryScope, +) -> usize { let mut recorded = 0; for lesson in distill_lessons(view, provider) { // Mirror kimetsu_brain_record's MCP kind mapping; semantic_operator + default store as Fact. @@ -202,17 +209,22 @@ pub fn distill_and_record(start: &Path, view: &str, provider: &mut dyn ModelProv "convention" => MemoryKind::Convention, _ => MemoryKind::Fact, }; - let confidence = lesson.confidence.clamp(0.0, 1.0); - if project::propose_or_merge_memory( - start, - MemoryScope::Project, - kind, - lesson.lesson.trim(), - confidence, - "auto-harvested at session end", - ) - .is_ok() - { + let text = lesson.lesson.trim(); + let ok = match scope { + MemoryScope::GlobalUser => { + project::add_memory(start, MemoryScope::GlobalUser, kind, text).is_ok() + } + _ => project::propose_or_merge_memory( + start, + scope, + kind, + text, + lesson.confidence.clamp(0.0, 1.0), + "auto-harvested at session end", + ) + .is_ok(), + }; + if ok { recorded += 1; } } @@ -265,7 +277,7 @@ pub fn run_session_end_hook(workspace: &Path) { ) else { return; }; - let recorded = distill_and_record(&paths.repo_root, &view, &mut provider); + let recorded = distill_and_record(&paths.repo_root, &view, &mut provider, MemoryScope::Project); if recorded > 0 { println!( "[Kimetsu] distilled {recorded} lesson{} at session end.", @@ -391,7 +403,7 @@ mod tests { let mut provider = MockProvider::new([text_response( "[{\"lesson\":\"Set USERPROFILE for global installs\",\"tags\":[\"cargo\",\"windows\"],\"confidence\":0.9}]", )]); - let n = distill_and_record(&root, "user: a\nuser: b", &mut provider); + let n = distill_and_record(&root, "user: a\nuser: b", &mut provider, MemoryScope::Project); assert_eq!(n, 1); let memories = kimetsu_brain::project::list_memories(&root).expect("list"); assert!( @@ -402,4 +414,57 @@ mod tests { std::fs::remove_dir_all(root).ok(); } + + /// Run `f` with the user brain pointed at a temp dir (enabled), under + /// the process-wide env lock, restoring the previous env afterward. + fn with_user_brain_dir(dir: &std::path::Path, f: impl FnOnce() -> R) -> R { + let _g = kimetsu_brain::user_brain::test_env_lock() + .lock() + .unwrap_or_else(|p| p.into_inner()); + let prev_dir = std::env::var("KIMETSU_USER_BRAIN_DIR").ok(); + let prev_en = std::env::var("KIMETSU_USER_BRAIN").ok(); + // SAFETY: scoped by the shared lock. + unsafe { + std::env::set_var("KIMETSU_USER_BRAIN_DIR", dir); + std::env::remove_var("KIMETSU_USER_BRAIN"); + } + let out = f(); + unsafe { + match prev_dir { + Some(v) => std::env::set_var("KIMETSU_USER_BRAIN_DIR", v), + None => std::env::remove_var("KIMETSU_USER_BRAIN_DIR"), + } + match prev_en { + Some(v) => std::env::set_var("KIMETSU_USER_BRAIN", v), + None => std::env::remove_var("KIMETSU_USER_BRAIN"), + } + } + out + } + + #[test] + fn distill_and_record_global_writes_to_user_brain() { + let dir = std::env::temp_dir().join(format!( + "kimetsu_userbrain_{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + std::fs::create_dir_all(&dir).unwrap(); + with_user_brain_dir(&dir, || { + let mut provider = MockProvider::new([text_response( + "[{\"lesson\":\"Global lesson kept everywhere\",\"tags\":[\"x\"],\"confidence\":0.9}]", + )]); + // `start` is ignored on the GlobalUser path; pass the temp dir. + let n = distill_and_record(&dir, "user: a", &mut provider, MemoryScope::GlobalUser); + assert_eq!(n, 1); + let conn = kimetsu_brain::user_brain::open_user_brain_readonly() + .unwrap() + .expect("user brain exists"); + let mems = kimetsu_brain::user_brain::list_user_memories(&conn).unwrap(); + assert!(mems.iter().any(|m| m.text.contains("Global lesson kept everywhere"))); + }); + std::fs::remove_dir_all(dir).ok(); + } } From 5b4599dd9fb172bc2eaac4507046a1198b484f1f Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 20:37:26 -0300 Subject: [PATCH 04/11] docs: note start is unused on the GlobalUser record path --- crates/kimetsu-cli/src/distiller.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/crates/kimetsu-cli/src/distiller.rs b/crates/kimetsu-cli/src/distiller.rs index 0ff6282..a7b60af 100644 --- a/crates/kimetsu-cli/src/distiller.rs +++ b/crates/kimetsu-cli/src/distiller.rs @@ -195,6 +195,7 @@ fn tail_chars(s: &str, n: usize) -> String { /// confidence-gated `propose_or_merge_memory` (workspace brain); `GlobalUser` /// uses `add_memory`, which routes to `~/.kimetsu/brain.db` (the user brain /// has no proposal queue, so this is add-or-dedup). Returns the count recorded. +/// For `GlobalUser`, `start` is ignored (the user brain is global). pub fn distill_and_record( start: &Path, view: &str, From bb0370febfb41cd1b0daf5cc162fd9507ddc287d Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 20:43:07 -0300 Subject: [PATCH 05/11] feat: resolve_distiller (workspace overrides global ~/.kimetsu) --- crates/kimetsu-cli/src/distiller.rs | 187 ++++++++++++++++++++++++---- 1 file changed, 163 insertions(+), 24 deletions(-) diff --git a/crates/kimetsu-cli/src/distiller.rs b/crates/kimetsu-cli/src/distiller.rs index a7b60af..ad4fc6f 100644 --- a/crates/kimetsu-cli/src/distiller.rs +++ b/crates/kimetsu-cli/src/distiller.rs @@ -11,9 +11,10 @@ use kimetsu_agent::model::{ MessageContent, MessageRole, ModelMessage, ModelProvider, ModelRequest, ToolChoice, }; use kimetsu_brain::project; +use kimetsu_core::config::ProjectConfig; use kimetsu_core::env_file::resolve_env_value; use kimetsu_core::memory::{MemoryKind, MemoryScope}; -use kimetsu_core::paths::ProjectPaths; +use kimetsu_core::paths::{user_brain_enabled, user_kimetsu_dir, ProjectPaths}; use serde::Deserialize; /// Max characters of transcript view fed to the distiller (keeps the @@ -232,6 +233,76 @@ pub fn distill_and_record( recorded } +/// The distiller selected for this session: which model/key/endpoint to use, +/// and how to record (project vs the global user brain). +pub struct ResolvedDistiller { + pub model: String, + pub key: String, + pub base_url: Option, + pub timeout_secs: u64, + pub scope: MemoryScope, + pub record_start: std::path::PathBuf, +} + +/// Resolve the distiller for `workspace`, preferring the workspace distiller +/// over the global one (`~/.kimetsu`). `None` when neither is enabled + +/// credentialed. +pub fn resolve_distiller(workspace: &Path) -> Option { + let global_dir = if user_brain_enabled() { + user_kimetsu_dir() + } else { + None + }; + resolve_distiller_with(workspace, global_dir) +} + +/// Testable core: `global_dir` is injected (the `~/.kimetsu` dir, or `None`). +fn resolve_distiller_with( + workspace: &Path, + global_dir: Option, +) -> Option { + // 1. Workspace distiller (Project scope). + if let Ok(paths) = ProjectPaths::discover(workspace) + && let Ok(config) = project::load_config(&paths) + { + let d = &config.learning.distiller; + if d.enabled + && d.provider == "anthropic" + && let Some(key) = resolve_env_value(&paths.repo_root, &d.api_key_env) + { + return Some(ResolvedDistiller { + model: d.model.clone(), + key, + base_url: resolve_env_value(&paths.repo_root, &d.base_url_env), + timeout_secs: config.model.request_timeout_secs, + scope: MemoryScope::Project, + record_start: paths.repo_root.clone(), + }); + } + } + // 2. Global distiller (GlobalUser scope). + if let Some(dir) = global_dir + && let Ok(text) = std::fs::read_to_string(dir.join("project.toml")) + && let Ok(config) = ProjectConfig::from_toml(&text) + { + let d = &config.learning.distiller; + if d.enabled + && d.provider == "anthropic" + && let Some(key) = resolve_env_value(&dir, &d.api_key_env) + { + return Some(ResolvedDistiller { + model: d.model.clone(), + key, + base_url: resolve_env_value(&dir, &d.base_url_env), + timeout_secs: config.model.request_timeout_secs, + scope: MemoryScope::GlobalUser, + record_start: workspace.to_path_buf(), + }); + } + } + None +} + /// `kimetsu brain session-end-hook` entry. Reads the SessionEnd payload /// from stdin, and if the distiller is enabled + credentialed, distills /// the transcript and records lessons. Silent no-op otherwise. @@ -241,24 +312,6 @@ pub fn run_session_end_hook(workspace: &Path) { let payload: serde_json::Value = serde_json::from_str(input.trim()).unwrap_or(serde_json::Value::Null); - // Resolve config at the project's git root (where the setup wizard writes - // when install is run at the repo root — the documented common case). A - // non-git-subdir install is the only mismatch, and it fails safe: the - // discovered root has no enabled distiller, so this is a silent no-op. - let Ok(paths) = ProjectPaths::discover(workspace) else { - return; - }; - let Ok(config) = project::load_config(&paths) else { - return; - }; - let distiller = &config.learning.distiller; - if !distiller.enabled || distiller.provider != "anthropic" { - return; - } - let Some(api_key) = resolve_env_value(&paths.repo_root, &distiller.api_key_env) else { - return; - }; - let base_url = resolve_env_value(&paths.repo_root, &distiller.base_url_env); let Some(transcript_path) = payload .get("transcript_path") .and_then(|v| v.as_str()) @@ -266,19 +319,23 @@ pub fn run_session_end_hook(workspace: &Path) { else { return; }; + let Some(resolved) = resolve_distiller(workspace) else { + return; + }; let view = build_transcript_view(transcript_path, MAX_VIEW_CHARS); if view.trim().is_empty() { return; } let Ok(mut provider) = AnthropicProvider::for_distiller( - &distiller.model, - api_key, - base_url, - config.model.request_timeout_secs, + &resolved.model, + resolved.key, + resolved.base_url, + resolved.timeout_secs, ) else { return; }; - let recorded = distill_and_record(&paths.repo_root, &view, &mut provider, MemoryScope::Project); + let recorded = + distill_and_record(&resolved.record_start, &view, &mut provider, resolved.scope); if recorded > 0 { println!( "[Kimetsu] distilled {recorded} lesson{} at session end.", @@ -443,6 +500,88 @@ mod tests { out } + /// Write a full `project.toml` to `dir` with the distiller section configured. + /// Uses `ProjectConfig::default_for_project` + `to_toml()` because a partial + /// TOML with only `[learning.distiller]` fails to parse — the `kimetsu` and + /// `model` sections are required by serde (no `#[serde(default)]` on those + /// `ProjectConfig` fields). + fn write_distiller_toml(dir: &std::path::Path, enabled: bool, model: &str) { + std::fs::create_dir_all(dir).unwrap(); + let mut config = ProjectConfig::default_for_project("test"); + config.learning.distiller.enabled = enabled; + config.learning.distiller.provider = "anthropic".to_string(); + config.learning.distiller.model = model.to_string(); + config.learning.distiller.api_key_env = "ANTHROPIC_API_KEY".to_string(); + config.learning.distiller.base_url_env = "ANTHROPIC_BASE_URL".to_string(); + let toml = config.to_toml().unwrap(); + std::fs::write(dir.join("project.toml"), toml).unwrap(); + } + + #[test] + fn resolve_distiller_global_when_no_workspace() { + let ws = std::env::temp_dir().join(format!( + "km_rd_ws_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + std::fs::create_dir_all(&ws).unwrap(); + let gdir = std::env::temp_dir().join(format!( + "km_rd_g_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + write_distiller_toml(&gdir, true, "claude-haiku-4-5"); + std::fs::write(gdir.join(".env"), "ANTHROPIC_API_KEY=sk-global\n").unwrap(); + + let r = resolve_distiller_with(&ws, Some(gdir.clone())).expect("global resolved"); + assert_eq!(r.scope, MemoryScope::GlobalUser); + assert_eq!(r.model, "claude-haiku-4-5"); + assert_eq!(r.key, "sk-global"); + + write_distiller_toml(&gdir, false, "claude-haiku-4-5"); + assert!(resolve_distiller_with(&ws, Some(gdir.clone())).is_none()); + + std::fs::remove_dir_all(ws).ok(); + std::fs::remove_dir_all(gdir).ok(); + } + + #[test] + fn resolve_distiller_workspace_wins() { + let ws = std::env::temp_dir().join(format!( + "km_rd_wsw_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + std::fs::create_dir_all(ws.join(".kimetsu")).unwrap(); + kimetsu_core::paths::git_init_boundary(&ws); + // Use default_for_project + set distiller fields + to_toml() because + // a partial toml with only [learning.distiller] fails to parse + // (kimetsu/model sections are required by serde). + { + let mut config = ProjectConfig::default_for_project("ws"); + config.learning.distiller.enabled = true; + config.learning.distiller.provider = "anthropic".to_string(); + config.learning.distiller.model = "ws-model".to_string(); + config.learning.distiller.api_key_env = "ANTHROPIC_API_KEY".to_string(); + config.learning.distiller.base_url_env = "ANTHROPIC_BASE_URL".to_string(); + let toml = config.to_toml().unwrap(); + std::fs::write(ws.join(".kimetsu").join("project.toml"), toml).unwrap(); + } + std::fs::write(ws.join(".env"), "ANTHROPIC_API_KEY=sk-ws\n").unwrap(); + + let gdir = std::env::temp_dir().join(format!( + "km_rd_gw_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + write_distiller_toml(&gdir, true, "g-model"); + std::fs::write(gdir.join(".env"), "ANTHROPIC_API_KEY=sk-global\n").unwrap(); + + let r = resolve_distiller_with(&ws, Some(gdir.clone())).expect("workspace resolved"); + assert_eq!(r.scope, MemoryScope::Project); + assert_eq!(r.model, "ws-model"); + assert_eq!(r.key, "sk-ws"); + + std::fs::remove_dir_all(ws).ok(); + std::fs::remove_dir_all(gdir).ok(); + } + #[test] fn distill_and_record_global_writes_to_user_brain() { let dir = std::env::temp_dir().join(format!( From a652b628f2316f2b66d77661150cdd33adf39003 Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 20:48:55 -0300 Subject: [PATCH 06/11] test: assert git_init_boundary in workspace-wins resolver test Assert the bool return of git_init_boundary in resolve_distiller_workspace_wins so a missing git binary produces a clear failure instead of a confusing false positive. Add a comment in resolve_distiller_global_when_no_workspace noting the non-git workspace is intentional. Co-Authored-By: Claude Sonnet 4.6 --- crates/kimetsu-cli/src/distiller.rs | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/crates/kimetsu-cli/src/distiller.rs b/crates/kimetsu-cli/src/distiller.rs index ad4fc6f..d304fb0 100644 --- a/crates/kimetsu-cli/src/distiller.rs +++ b/crates/kimetsu-cli/src/distiller.rs @@ -523,6 +523,8 @@ mod tests { "km_rd_ws_{}", std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() )); + // Intentionally NOT a git repo: exercises discover's non-repo fallback + // so the workspace tier finds no config and we fall through to global. std::fs::create_dir_all(&ws).unwrap(); let gdir = std::env::temp_dir().join(format!( "km_rd_g_{}", @@ -550,7 +552,10 @@ mod tests { std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() )); std::fs::create_dir_all(ws.join(".kimetsu")).unwrap(); - kimetsu_core::paths::git_init_boundary(&ws); + assert!( + kimetsu_core::paths::git_init_boundary(&ws), + "git_init_boundary failed — git needed for workspace isolation" + ); // Use default_for_project + set distiller fields + to_toml() because // a partial toml with only [learning.distiller] fails to parse // (kimetsu/model sections are required by serde). From 299e648fef1484efda0771073c57557f07506bfc Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 20:50:20 -0300 Subject: [PATCH 07/11] feat: global_distiller_enabled helper for Stop-cue suppression Co-Authored-By: Claude Sonnet 4.6 --- crates/kimetsu-cli/src/distiller.rs | 35 +++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/crates/kimetsu-cli/src/distiller.rs b/crates/kimetsu-cli/src/distiller.rs index d304fb0..32fcc85 100644 --- a/crates/kimetsu-cli/src/distiller.rs +++ b/crates/kimetsu-cli/src/distiller.rs @@ -256,6 +256,25 @@ pub fn resolve_distiller(workspace: &Path) -> Option { resolve_distiller_with(workspace, global_dir) } +/// True when a global distiller (`~/.kimetsu/project.toml`) is enabled. Used by +/// the Stop hook to suppress its end-of-session cue (the distiller owns it). +pub fn global_distiller_enabled() -> bool { + let dir = if user_brain_enabled() { + user_kimetsu_dir() + } else { + None + }; + global_distiller_enabled_in(dir) +} + +fn global_distiller_enabled_in(global_dir: Option) -> bool { + global_dir + .and_then(|dir| std::fs::read_to_string(dir.join("project.toml")).ok()) + .and_then(|text| ProjectConfig::from_toml(&text).ok()) + .map(|c| c.learning.distiller.enabled) + .unwrap_or(false) +} + /// Testable core: `global_dir` is injected (the `~/.kimetsu` dir, or `None`). fn resolve_distiller_with( workspace: &Path, @@ -517,6 +536,22 @@ mod tests { std::fs::write(dir.join("project.toml"), toml).unwrap(); } + #[test] + fn global_distiller_enabled_reads_global_toml() { + let gdir = std::env::temp_dir().join(format!( + "km_gde_{}", + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + )); + write_distiller_toml(&gdir, true, "claude-haiku-4-5"); + assert!(global_distiller_enabled_in(Some(gdir.clone()))); + + write_distiller_toml(&gdir, false, "claude-haiku-4-5"); + assert!(!global_distiller_enabled_in(Some(gdir.clone()))); + + assert!(!global_distiller_enabled_in(None)); + std::fs::remove_dir_all(gdir).ok(); + } + #[test] fn resolve_distiller_global_when_no_workspace() { let ws = std::env::temp_dir().join(format!( From a91743ce5c0d2a2b7cb5e1a80fff6d150f7dbdb1 Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 20:55:59 -0300 Subject: [PATCH 08/11] feat: wizard writes to an explicit SetupTarget (workspace or ~/.kimetsu) Co-Authored-By: Claude Sonnet 4.6 --- crates/kimetsu-cli/src/harvest_setup.rs | 118 ++++++++++++------------ crates/kimetsu-cli/src/main.rs | 10 +- 2 files changed, 65 insertions(+), 63 deletions(-) diff --git a/crates/kimetsu-cli/src/harvest_setup.rs b/crates/kimetsu-cli/src/harvest_setup.rs index 2f694f7..a145ea6 100644 --- a/crates/kimetsu-cli/src/harvest_setup.rs +++ b/crates/kimetsu-cli/src/harvest_setup.rs @@ -5,10 +5,17 @@ use std::fs; use std::io::{BufRead, Write}; -use std::path::Path; +use std::path::{Path, PathBuf}; use kimetsu_core::config::ProjectConfig; -use kimetsu_core::paths::ProjectPaths; + +/// Where the wizard writes the distiller config + secret. Built by the CLI gate +/// from the install scope (workspace dir, or `~/.kimetsu`). +pub struct SetupTarget { + pub project_toml: PathBuf, + pub env_path: PathBuf, + pub gitignore_dir: PathBuf, +} /// Run the wizard against the given reader/writer (real stdin/stdout in /// production; scripted in tests). Returns Ok(true) when the distiller was @@ -16,7 +23,8 @@ use kimetsu_core::paths::ProjectPaths; pub fn run_harvest_setup( reader: &mut R, writer: &mut W, - paths: &ProjectPaths, + target: &SetupTarget, + scope_label: &str, ) -> std::io::Result { write!(writer, "Set up the auto-harvest distiller now? [y/N]: ")?; writer.flush()?; @@ -65,20 +73,19 @@ pub fn run_harvest_setup( model = "claude-haiku-4-5".to_string(); } - apply_distiller_config(paths, &model)?; - // Gitignore `.env` BEFORE writing the secret into it, so the plaintext - // key is never on disk in a tracked/unignored file. - ensure_gitignored(&paths.repo_root, ".env")?; - let env_path = paths.repo_root.join(".env"); - upsert_env_var(&env_path, "ANTHROPIC_API_KEY", &key)?; + apply_distiller_config(&target.project_toml, &model)?; + // Gitignore `.env` BEFORE writing the secret into it. + ensure_gitignored(&target.gitignore_dir, ".env")?; + upsert_env_var(&target.env_path, "ANTHROPIC_API_KEY", &key)?; if !base_url.is_empty() { - upsert_env_var(&env_path, "ANTHROPIC_BASE_URL", &base_url)?; + upsert_env_var(&target.env_path, "ANTHROPIC_BASE_URL", &base_url)?; } writeln!( writer, - "\u{2713} Distiller configured (model {model}). Key stored in .env (gitignored). \ - Note: the key was entered in plain text." + "\u{2713} Distiller configured for {scope_label} (model {model}). \ + Key stored in {} (gitignored). Note: the key was entered in plain text.", + target.env_path.display() )?; Ok(true) } @@ -89,22 +96,23 @@ fn read_line(reader: &mut R) -> std::io::Result { Ok(line) } -/// Flip the distiller on in the project config, anchored strictly at -/// `paths` (the install workspace). Loads an existing `project.toml` if -/// present, else starts from a default — it does NOT call `init_project` -/// (which would climb to an enclosing git repo and open the brain DB), so -/// the config + secret never land in a parent repository. -fn apply_distiller_config(paths: &ProjectPaths, model: &str) -> std::io::Result<()> { - // KimetsuResult's error is Box; it implements - // Display, so to_string() works without naming a concrete error type. - let io_err = |e: Box| std::io::Error::other(e.to_string()); - let mut config = if paths.project_toml.exists() { - ProjectConfig::from_toml(&fs::read_to_string(&paths.project_toml)?).map_err(io_err)? +/// Flip the distiller on in the project config, writing to `project_toml` +/// directly. Loads an existing `project.toml` if present, else starts from a +/// default — it does NOT call `init_project` (which would climb to an +/// enclosing git repo and open the brain DB), so the config + secret never +/// land in a parent repository. +fn apply_distiller_config(project_toml: &Path, model: &str) -> std::io::Result<()> { + let io_err = + |e: Box| std::io::Error::other(e.to_string()); + let mut config = if project_toml.exists() { + ProjectConfig::from_toml(&fs::read_to_string(project_toml)?).map_err(io_err)? } else { - let project_id = paths - .repo_root - .file_name() - .and_then(|name| name.to_str()) + // /.kimetsu/project.toml -> project_id from . + let project_id = project_toml + .parent() + .and_then(|p| p.parent()) + .and_then(|p| p.file_name()) + .and_then(|n| n.to_str()) .unwrap_or("workspace") .to_string(); ProjectConfig::default_for_project(project_id) @@ -114,11 +122,10 @@ fn apply_distiller_config(paths: &ProjectPaths, model: &str) -> std::io::Result< config.learning.distiller.model = model.to_string(); config.learning.distiller.api_key_env = "ANTHROPIC_API_KEY".to_string(); config.learning.distiller.base_url_env = "ANTHROPIC_BASE_URL".to_string(); - let toml = config.to_toml().map_err(io_err)?; - if let Some(parent) = paths.project_toml.parent() { + if let Some(parent) = project_toml.parent() { fs::create_dir_all(parent)?; } - fs::write(&paths.project_toml, toml) + fs::write(project_toml, config.to_toml().map_err(io_err)?) } /// Insert or replace `NAME=value` in a `.env` file (created if missing). @@ -167,46 +174,35 @@ mod tests { use super::*; use std::io::Cursor; - // `at_root` anchors strictly at the temp dir (no git climbing), which is - // exactly how the CLI gate builds the paths in production. - fn paths_for(root: &Path) -> ProjectPaths { - ProjectPaths::at_root(root) + fn target_at(dir: &Path) -> SetupTarget { + SetupTarget { + project_toml: dir.join(".kimetsu").join("project.toml"), + env_path: dir.join(".env"), + gitignore_dir: dir.to_path_buf(), + } } #[test] fn wizard_writes_env_and_config() { let root = std::env::temp_dir().join(format!( "kimetsu_wizard_{}", - std::time::SystemTime::now() - .duration_since(std::time::UNIX_EPOCH) - .unwrap() - .as_nanos() + std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() )); fs::create_dir_all(root.join(".kimetsu")).unwrap(); - let paths = paths_for(&root); - - // y -> claude -> key -> base url (LiteLLM) -> blank model (default). - let mut input = Cursor::new( - "y\nclaude\nsk-litellm-123\nhttp://localhost:4000\n\n" - .as_bytes() - .to_vec(), - ); + let mut input = + Cursor::new("y\nclaude\nsk-litellm-123\nhttp://localhost:4000\n\n".as_bytes().to_vec()); let mut output = Vec::new(); - let configured = run_harvest_setup(&mut input, &mut output, &paths).unwrap(); + let configured = + run_harvest_setup(&mut input, &mut output, &target_at(&root), "this workspace").unwrap(); assert!(configured); - let env = fs::read_to_string(paths.repo_root.join(".env")).unwrap(); + let env = fs::read_to_string(root.join(".env")).unwrap(); assert!(env.contains("ANTHROPIC_API_KEY=sk-litellm-123")); assert!(env.contains("ANTHROPIC_BASE_URL=http://localhost:4000")); - let toml = fs::read_to_string(&paths.project_toml).unwrap(); + let toml = fs::read_to_string(root.join(".kimetsu").join("project.toml")).unwrap(); assert!(toml.contains("enabled = true")); assert!(toml.contains("claude-haiku-4-5")); - assert!( - fs::read_to_string(paths.repo_root.join(".gitignore")) - .unwrap() - .contains(".env") - ); - + assert!(fs::read_to_string(root.join(".gitignore")).unwrap().contains(".env")); fs::remove_dir_all(root).ok(); } @@ -220,11 +216,11 @@ mod tests { .as_nanos() )); fs::create_dir_all(&root).unwrap(); - let paths = paths_for(&root); + let paths = target_at(&root); let mut input = Cursor::new(b"n\n".to_vec()); let mut output = Vec::new(); - assert!(!run_harvest_setup(&mut input, &mut output, &paths).unwrap()); - assert!(!paths.repo_root.join(".env").exists()); + assert!(!run_harvest_setup(&mut input, &mut output, &paths, "this workspace").unwrap()); + assert!(!root.join(".env").exists()); fs::remove_dir_all(root).ok(); } @@ -238,11 +234,11 @@ mod tests { .as_nanos() )); fs::create_dir_all(&root).unwrap(); - let paths = paths_for(&root); + let paths = target_at(&root); let mut input = Cursor::new(b"y\ngemini\n".to_vec()); let mut output = Vec::new(); - assert!(!run_harvest_setup(&mut input, &mut output, &paths).unwrap()); - assert!(!paths.repo_root.join(".env").exists()); + assert!(!run_harvest_setup(&mut input, &mut output, &paths, "this workspace").unwrap()); + assert!(!root.join(".env").exists()); fs::remove_dir_all(root).ok(); } diff --git a/crates/kimetsu-cli/src/main.rs b/crates/kimetsu-cli/src/main.rs index 3056c05..9fb9b6c 100644 --- a/crates/kimetsu-cli/src/main.rs +++ b/crates/kimetsu-cli/src/main.rs @@ -1000,11 +1000,17 @@ fn plugin(command: PluginCommand) -> KimetsuResult<()> { // Anchor strictly at the install workspace (NOT `discover`, // which climbs to an enclosing git repo and could write the // distiller config + secret .env into a parent repository). - let paths = kimetsu_core::paths::ProjectPaths::at_root(&workspace); + let p = kimetsu_core::paths::ProjectPaths::at_root(&workspace); + let setup_target = harvest_setup::SetupTarget { + project_toml: p.project_toml.clone(), + env_path: p.repo_root.join(".env"), + gitignore_dir: p.repo_root.clone(), + }; let stdin = std::io::stdin(); let mut reader = stdin.lock(); let mut stdout = std::io::stdout(); - if let Err(err) = harvest_setup::run_harvest_setup(&mut reader, &mut stdout, &paths) + if let Err(err) = + harvest_setup::run_harvest_setup(&mut reader, &mut stdout, &setup_target, "this workspace") { eprintln!("kimetsu plugin install: distiller setup skipped: {err}"); } From 8ce16af340b68766109ef610230f32501d6bc952 Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 21:00:51 -0300 Subject: [PATCH 09/11] feat: scope-aware wizard target + resolve_distiller-based Stop suppression Co-Authored-By: Claude Sonnet 4.6 --- crates/kimetsu-cli/src/distiller.rs | 35 ---------------- crates/kimetsu-cli/src/main.rs | 62 ++++++++++++++++++++--------- 2 files changed, 44 insertions(+), 53 deletions(-) diff --git a/crates/kimetsu-cli/src/distiller.rs b/crates/kimetsu-cli/src/distiller.rs index 32fcc85..d304fb0 100644 --- a/crates/kimetsu-cli/src/distiller.rs +++ b/crates/kimetsu-cli/src/distiller.rs @@ -256,25 +256,6 @@ pub fn resolve_distiller(workspace: &Path) -> Option { resolve_distiller_with(workspace, global_dir) } -/// True when a global distiller (`~/.kimetsu/project.toml`) is enabled. Used by -/// the Stop hook to suppress its end-of-session cue (the distiller owns it). -pub fn global_distiller_enabled() -> bool { - let dir = if user_brain_enabled() { - user_kimetsu_dir() - } else { - None - }; - global_distiller_enabled_in(dir) -} - -fn global_distiller_enabled_in(global_dir: Option) -> bool { - global_dir - .and_then(|dir| std::fs::read_to_string(dir.join("project.toml")).ok()) - .and_then(|text| ProjectConfig::from_toml(&text).ok()) - .map(|c| c.learning.distiller.enabled) - .unwrap_or(false) -} - /// Testable core: `global_dir` is injected (the `~/.kimetsu` dir, or `None`). fn resolve_distiller_with( workspace: &Path, @@ -536,22 +517,6 @@ mod tests { std::fs::write(dir.join("project.toml"), toml).unwrap(); } - #[test] - fn global_distiller_enabled_reads_global_toml() { - let gdir = std::env::temp_dir().join(format!( - "km_gde_{}", - std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() - )); - write_distiller_toml(&gdir, true, "claude-haiku-4-5"); - assert!(global_distiller_enabled_in(Some(gdir.clone()))); - - write_distiller_toml(&gdir, false, "claude-haiku-4-5"); - assert!(!global_distiller_enabled_in(Some(gdir.clone()))); - - assert!(!global_distiller_enabled_in(None)); - std::fs::remove_dir_all(gdir).ok(); - } - #[test] fn resolve_distiller_global_when_no_workspace() { let ws = std::env::temp_dir().join(format!( diff --git a/crates/kimetsu-cli/src/main.rs b/crates/kimetsu-cli/src/main.rs index 9fb9b6c..9bd1250 100644 --- a/crates/kimetsu-cli/src/main.rs +++ b/crates/kimetsu-cli/src/main.rs @@ -997,22 +997,47 @@ fn plugin(command: PluginCommand) -> KimetsuResult<()> { let interactive = args.setup_harvest || (std::io::stdin().is_terminal() && std::io::stdout().is_terminal()); if matches!(target, BridgeTarget::ClaudeCode) && !args.no_setup && interactive { - // Anchor strictly at the install workspace (NOT `discover`, - // which climbs to an enclosing git repo and could write the - // distiller config + secret .env into a parent repository). - let p = kimetsu_core::paths::ProjectPaths::at_root(&workspace); - let setup_target = harvest_setup::SetupTarget { - project_toml: p.project_toml.clone(), - env_path: p.repo_root.join(".env"), - gitignore_dir: p.repo_root.clone(), + let target_for_scope = match scope { + InstallScope::Global => match kimetsu_core::paths::user_kimetsu_dir() { + Some(dir) => Some(( + harvest_setup::SetupTarget { + project_toml: dir.join("project.toml"), + env_path: dir.join(".env"), + gitignore_dir: dir, + }, + "globally (all projects, ~/.kimetsu)", + )), + None => { + eprintln!( + "kimetsu plugin install: cannot resolve ~/.kimetsu; skipping distiller setup." + ); + None + } + }, + InstallScope::Workspace => { + let p = kimetsu_core::paths::ProjectPaths::at_root(&workspace); + Some(( + harvest_setup::SetupTarget { + project_toml: p.project_toml.clone(), + env_path: p.repo_root.join(".env"), + gitignore_dir: p.repo_root.clone(), + }, + "this workspace", + )) + } }; - let stdin = std::io::stdin(); - let mut reader = stdin.lock(); - let mut stdout = std::io::stdout(); - if let Err(err) = - harvest_setup::run_harvest_setup(&mut reader, &mut stdout, &setup_target, "this workspace") - { - eprintln!("kimetsu plugin install: distiller setup skipped: {err}"); + if let Some((setup_target, label)) = target_for_scope { + let stdin = std::io::stdin(); + let mut reader = stdin.lock(); + let mut stdout = std::io::stdout(); + if let Err(err) = harvest_setup::run_harvest_setup( + &mut reader, + &mut stdout, + &setup_target, + label, + ) { + eprintln!("kimetsu plugin install: distiller setup skipped: {err}"); + } } } } @@ -1927,11 +1952,12 @@ fn brain_stop_hook(args: StopHookArgs) -> KimetsuResult<()> { .and_then(|v| v.as_bool()) .unwrap_or(false); let paths = kimetsu_core::paths::ProjectPaths::discover(&workspace).ok(); - let (auto_harvest, distiller_enabled) = paths + let auto_harvest = paths .as_ref() .and_then(|p| project::load_config(p).ok()) - .map(|c| (c.learning.auto_harvest, c.learning.distiller.enabled)) - .unwrap_or((true, false)); + .map(|c| c.learning.auto_harvest) + .unwrap_or(true); + let distiller_enabled = distiller::resolve_distiller(&workspace).is_some(); if should_emit_stop_harvest_cue(auto_harvest, distiller_enabled) && !stop_active From 57f10b8defea84975e794d8007d3ad173e9f5e8e Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 21:08:44 -0300 Subject: [PATCH 10/11] docs: document the global distiller (~/.kimetsu); cargo fmt Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 6 +++- README.md | 4 ++- crates/kimetsu-cli/src/distiller.rs | 37 +++++++++++++++++++------ crates/kimetsu-cli/src/harvest_setup.rs | 24 +++++++++++----- 4 files changed, 53 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9907418..70508a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,7 +22,11 @@ ADDED `propose_or_merge_memory`. When the distiller is enabled, the Stop hook's end-of-session cue is suppressed (the distiller owns end-of-session; the mid-session PostToolUse resolved-failure cue stays). `AnthropicProvider` - gained an optional base URL for the LiteLLM case. + gained an optional base URL for the LiteLLM case. With `--scope global` the + wizard configures the distiller once in `~/.kimetsu/` (config + `.env`); that + global distiller then runs in every project and records into the user brain + (`~/.kimetsu/brain.db`, available everywhere), with a per-project distiller + taking precedence when one is configured. * **Auto-harvested memories (in-agent).** The hooks now drive memory *generation*, not just retrieval. When a command that failed earlier in the session succeeds (PostToolUse), or a non-trivial session ends with nothing diff --git a/README.md b/README.md index af7d1ec..9495dd0 100644 --- a/README.md +++ b/README.md @@ -209,7 +209,9 @@ install claude-code` offers to set up a **SessionEnd distiller**: a cheap configured model (Anthropic `claude-haiku-4-5`, or a LiteLLM endpoint via `ANTHROPIC_BASE_URL`) that distills each session itself at the end and records the lessons. The wizard stores the key in a gitignored `.env`; skip it with -`--no-setup`. +`--no-setup`. Run it with `--scope global` to configure the distiller once in +`~/.kimetsu/` — it then distills every project's sessions into your user brain +(available everywhere), unless that project has its own distiller. ```bash kimetsu brain search "build failures" diff --git a/crates/kimetsu-cli/src/distiller.rs b/crates/kimetsu-cli/src/distiller.rs index d304fb0..dc5a43c 100644 --- a/crates/kimetsu-cli/src/distiller.rs +++ b/crates/kimetsu-cli/src/distiller.rs @@ -14,7 +14,7 @@ use kimetsu_brain::project; use kimetsu_core::config::ProjectConfig; use kimetsu_core::env_file::resolve_env_value; use kimetsu_core::memory::{MemoryKind, MemoryScope}; -use kimetsu_core::paths::{user_brain_enabled, user_kimetsu_dir, ProjectPaths}; +use kimetsu_core::paths::{ProjectPaths, user_brain_enabled, user_kimetsu_dir}; use serde::Deserialize; /// Max characters of transcript view fed to the distiller (keeps the @@ -334,8 +334,7 @@ pub fn run_session_end_hook(workspace: &Path) { ) else { return; }; - let recorded = - distill_and_record(&resolved.record_start, &view, &mut provider, resolved.scope); + let recorded = distill_and_record(&resolved.record_start, &view, &mut provider, resolved.scope); if recorded > 0 { println!( "[Kimetsu] distilled {recorded} lesson{} at session end.", @@ -461,7 +460,12 @@ mod tests { let mut provider = MockProvider::new([text_response( "[{\"lesson\":\"Set USERPROFILE for global installs\",\"tags\":[\"cargo\",\"windows\"],\"confidence\":0.9}]", )]); - let n = distill_and_record(&root, "user: a\nuser: b", &mut provider, MemoryScope::Project); + let n = distill_and_record( + &root, + "user: a\nuser: b", + &mut provider, + MemoryScope::Project, + ); assert_eq!(n, 1); let memories = kimetsu_brain::project::list_memories(&root).expect("list"); assert!( @@ -521,14 +525,20 @@ mod tests { fn resolve_distiller_global_when_no_workspace() { let ws = std::env::temp_dir().join(format!( "km_rd_ws_{}", - std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() )); // Intentionally NOT a git repo: exercises discover's non-repo fallback // so the workspace tier finds no config and we fall through to global. std::fs::create_dir_all(&ws).unwrap(); let gdir = std::env::temp_dir().join(format!( "km_rd_g_{}", - std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() )); write_distiller_toml(&gdir, true, "claude-haiku-4-5"); std::fs::write(gdir.join(".env"), "ANTHROPIC_API_KEY=sk-global\n").unwrap(); @@ -549,7 +559,10 @@ mod tests { fn resolve_distiller_workspace_wins() { let ws = std::env::temp_dir().join(format!( "km_rd_wsw_{}", - std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() )); std::fs::create_dir_all(ws.join(".kimetsu")).unwrap(); assert!( @@ -573,7 +586,10 @@ mod tests { let gdir = std::env::temp_dir().join(format!( "km_rd_gw_{}", - std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() )); write_distiller_toml(&gdir, true, "g-model"); std::fs::write(gdir.join(".env"), "ANTHROPIC_API_KEY=sk-global\n").unwrap(); @@ -608,7 +624,10 @@ mod tests { .unwrap() .expect("user brain exists"); let mems = kimetsu_brain::user_brain::list_user_memories(&conn).unwrap(); - assert!(mems.iter().any(|m| m.text.contains("Global lesson kept everywhere"))); + assert!( + mems.iter() + .any(|m| m.text.contains("Global lesson kept everywhere")) + ); }); std::fs::remove_dir_all(dir).ok(); } diff --git a/crates/kimetsu-cli/src/harvest_setup.rs b/crates/kimetsu-cli/src/harvest_setup.rs index a145ea6..51feb6e 100644 --- a/crates/kimetsu-cli/src/harvest_setup.rs +++ b/crates/kimetsu-cli/src/harvest_setup.rs @@ -102,8 +102,7 @@ fn read_line(reader: &mut R) -> std::io::Result { /// enclosing git repo and open the brain DB), so the config + secret never /// land in a parent repository. fn apply_distiller_config(project_toml: &Path, model: &str) -> std::io::Result<()> { - let io_err = - |e: Box| std::io::Error::other(e.to_string()); + let io_err = |e: Box| std::io::Error::other(e.to_string()); let mut config = if project_toml.exists() { ProjectConfig::from_toml(&fs::read_to_string(project_toml)?).map_err(io_err)? } else { @@ -186,14 +185,21 @@ mod tests { fn wizard_writes_env_and_config() { let root = std::env::temp_dir().join(format!( "kimetsu_wizard_{}", - std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos() + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() )); fs::create_dir_all(root.join(".kimetsu")).unwrap(); - let mut input = - Cursor::new("y\nclaude\nsk-litellm-123\nhttp://localhost:4000\n\n".as_bytes().to_vec()); + let mut input = Cursor::new( + "y\nclaude\nsk-litellm-123\nhttp://localhost:4000\n\n" + .as_bytes() + .to_vec(), + ); let mut output = Vec::new(); let configured = - run_harvest_setup(&mut input, &mut output, &target_at(&root), "this workspace").unwrap(); + run_harvest_setup(&mut input, &mut output, &target_at(&root), "this workspace") + .unwrap(); assert!(configured); let env = fs::read_to_string(root.join(".env")).unwrap(); @@ -202,7 +208,11 @@ mod tests { let toml = fs::read_to_string(root.join(".kimetsu").join("project.toml")).unwrap(); assert!(toml.contains("enabled = true")); assert!(toml.contains("claude-haiku-4-5")); - assert!(fs::read_to_string(root.join(".gitignore")).unwrap().contains(".env")); + assert!( + fs::read_to_string(root.join(".gitignore")) + .unwrap() + .contains(".env") + ); fs::remove_dir_all(root).ok(); } From 0a994682892d0dbf53852366401650a9598b9cdd Mon Sep 17 00:00:00 2001 From: RodCor Date: Tue, 2 Jun 2026 22:59:45 -0300 Subject: [PATCH 11/11] feat: codex integration --- CHANGELOG.md | 23 +- README.md | 15 +- crates/kimetsu-agent/Cargo.toml | 4 +- crates/kimetsu-agent/src/lib.rs | 1 + crates/kimetsu-agent/src/openai.rs | 469 ++++++++++++++++++++++++ crates/kimetsu-chat/src/bridge.rs | 77 +++- crates/kimetsu-chat/src/lib.rs | 2 +- crates/kimetsu-chat/src/mcp_server.rs | 2 +- crates/kimetsu-cli/src/distiller.rs | 130 ++++++- crates/kimetsu-cli/src/harvest_setup.rs | 192 ++++++++-- crates/kimetsu-cli/src/main.rs | 62 +++- crates/kimetsu-core/src/config.rs | 2 +- docs/HOW-KIMETSU-WORKS.md | 25 +- 13 files changed, 918 insertions(+), 86 deletions(-) create mode 100644 crates/kimetsu-agent/src/openai.rs diff --git a/CHANGELOG.md b/CHANGELOG.md index 70508a6..02842ef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,18 +11,22 @@ caveat that pre-1.0 minor bumps may include breaking changes ADDED * **Credentialed SessionEnd distiller (opt-in).** A second, deterministic memory-harvest path alongside the v0.9.0 in-agent harvester. `kimetsu plugin - install claude-code` now runs an interactive wizard (on a TTY; skip with - `--no-setup`, force with `--setup-harvest`) that configures a cheap distiller - model — Anthropic (recommended `claude-haiku-4-5`) or any Anthropic-compatible - endpoint incl. **LiteLLM** via `ANTHROPIC_BASE_URL`. The key + base URL are - written to a gitignored `.env`; the selection lands in `[learning.distiller]` - in `project.toml`. A new `SessionEnd` hook then runs `kimetsu brain - session-end-hook`, which (when enabled + credentialed) distills the + install claude-code` and `kimetsu plugin install codex` now run an + interactive wizard (on a TTY; skip with `--no-setup`, force with + `--setup-harvest`) that configures a cheap distiller + model: Anthropic (recommended `claude-haiku-4-5`), OpenAI (recommended + `gpt-5.4-mini`), or compatible endpoints via `ANTHROPIC_BASE_URL` / + `OPENAI_BASE_URL`. The key + base URL are written to a gitignored `.env`; + the selection lands in `[learning.distiller]` + in `project.toml`. A new `SessionEnd` hook for Claude Code runs + `kimetsu brain session-end-hook`; Codex uses its supported `Stop` hook with + `--distill-on-stop`. When enabled + credentialed, the distiller reads the transcript with that model and records lessons through the confidence-gated `propose_or_merge_memory`. When the distiller is enabled, the Stop hook's end-of-session cue is suppressed (the distiller owns end-of-session; the mid-session PostToolUse resolved-failure cue stays). `AnthropicProvider` - gained an optional base URL for the LiteLLM case. With `--scope global` the + gained an optional base URL for the LiteLLM case, and the distiller gained + an OpenAI Responses API provider. With `--scope global` the wizard configures the distiller once in `~/.kimetsu/` (config + `.env`); that global distiller then runs in every project and records into the user brain (`~/.kimetsu/brain.db`, available everywhere), with a per-project distiller @@ -32,7 +36,8 @@ ADDED session succeeds (PostToolUse), or a non-trivial session ends with nothing recorded (Stop), Kimetsu emits a `[kimetsu-harvest]` cue telling the agent to dispatch a new background **`kimetsu-memory-harvester`** subagent (installed - at `.claude/agents/`, pinned to a cheap model). The subagent distills 0–3 + at `.claude/agents/` for Claude Code and `.codex/agents/` for Codex, pinned + to a cheap model). The subagent distills 0–3 generalizable lessons and records them through the confidence-gated `kimetsu_brain_record` path — no separate API key or kimetsu-side model credentials, billed in-agent at the cheap model's rate, non-blocking. Cues diff --git a/README.md b/README.md index 9495dd0..8ff8144 100644 --- a/README.md +++ b/README.md @@ -182,7 +182,7 @@ cover Claude Code and Codex: ```bash kimetsu plugin install claude --workspace . # writes .mcp.json + .claude/settings.json -kimetsu plugin install codex --workspace . # writes .codex/config.toml + .codex/hooks.json + skill +kimetsu plugin install codex --workspace . # writes .codex/config.toml + .codex/hooks.json + skill + agent # Install globally for every project (writes to ~/.claude, ~/.claude.json, ~/.codex): kimetsu plugin install claude --scope global @@ -205,11 +205,14 @@ in-agent distiller) that records the lesson for next time — no extra API key. Turn it off with `[learning] auto_harvest = false` in `.kimetsu/project.toml`. For a deterministic harvest that doesn't depend on the agent, `kimetsu plugin -install claude-code` offers to set up a **SessionEnd distiller**: a cheap -configured model (Anthropic `claude-haiku-4-5`, or a LiteLLM endpoint via -`ANTHROPIC_BASE_URL`) that distills each session itself at the end and records -the lessons. The wizard stores the key in a gitignored `.env`; skip it with -`--no-setup`. Run it with `--scope global` to configure the distiller once in +install claude-code` and `kimetsu plugin install codex` offer to set up a +**SessionEnd distiller**: a cheap configured model (Anthropic +`claude-haiku-4-5`, OpenAI `gpt-5.4-mini`, or a compatible endpoint via +`ANTHROPIC_BASE_URL` / `OPENAI_BASE_URL`) that distills each session itself at +the end and records the lessons. Claude Code runs it from `SessionEnd`; Codex +runs it from the supported `Stop` hook with `--distill-on-stop`. The wizard +stores the key in a gitignored `.env`; skip it with `--no-setup`. Run it with +`--scope global` to configure the distiller once in `~/.kimetsu/` — it then distills every project's sessions into your user brain (available everywhere), unless that project has its own distiller. diff --git a/crates/kimetsu-agent/Cargo.toml b/crates/kimetsu-agent/Cargo.toml index 6eb247d..29134fe 100644 --- a/crates/kimetsu-agent/Cargo.toml +++ b/crates/kimetsu-agent/Cargo.toml @@ -9,8 +9,8 @@ homepage.workspace = true documentation.workspace = true readme.workspace = true rust-version.workspace = true -description = "Transport-neutral agent runtime for kimetsu: tools, prompts, verify-loop, providers (claude_code, anthropic)." -keywords = ["kimetsu", "agent", "llm", "anthropic", "claude"] +description = "Transport-neutral agent runtime for kimetsu: tools, prompts, verify-loop, providers (claude_code, anthropic, openai)." +keywords = ["kimetsu", "agent", "llm", "anthropic", "openai"] categories = ["development-tools"] [features] diff --git a/crates/kimetsu-agent/src/lib.rs b/crates/kimetsu-agent/src/lib.rs index 9afbaf5..656b1a0 100644 --- a/crates/kimetsu-agent/src/lib.rs +++ b/crates/kimetsu-agent/src/lib.rs @@ -4,6 +4,7 @@ pub mod bench; pub mod claude_code; pub mod harness; pub mod model; +pub mod openai; pub mod pipeline; pub mod swe_bench; pub mod tools; diff --git a/crates/kimetsu-agent/src/openai.rs b/crates/kimetsu-agent/src/openai.rs new file mode 100644 index 0000000..58a32b0 --- /dev/null +++ b/crates/kimetsu-agent/src/openai.rs @@ -0,0 +1,469 @@ +use std::time::Duration; + +use kimetsu_core::KimetsuResult; +use kimetsu_core::secret::SecretString; +use reqwest::blocking::Client; +use serde_json::{Value, json}; + +use crate::model::{ + MessageContent, MessageRole, ModelProvider, ModelRequest, ModelResponse, StopReason, + TokenUsage, ToolCall, +}; + +const RESPONSES_URL: &str = "https://api.openai.com/v1/responses"; + +#[derive(Debug, Clone)] +pub struct OpenAiProvider { + client: Client, + api_key: SecretString, + model: String, + /// When set, requests POST to an OpenAI-compatible Responses API + /// endpoint. Accepts either a root URL (`/v1/responses`) or a + /// conventional OpenAI-style base URL ending in `/v1` + /// (`/responses`). + base_url: Option, +} + +impl OpenAiProvider { + pub fn model_name(&self) -> &str { + &self.model + } + + /// Build a provider directly from resolved distiller values. Empty + /// `base_url` is normalized to `None`. + pub fn for_distiller( + model: impl Into, + api_key: impl Into, + base_url: Option, + timeout_secs: u64, + ) -> KimetsuResult { + let client = Client::builder() + .timeout(Duration::from_secs(timeout_secs)) + .build()?; + Ok(Self { + client, + api_key: SecretString::new(api_key.into()), + model: model.into(), + base_url: base_url.filter(|value| !value.trim().is_empty()), + }) + } +} + +fn responses_url(base_url: &Option) -> String { + match base_url { + Some(base) => { + let base = base.trim_end_matches('/'); + if base.ends_with("/v1") { + format!("{base}/responses") + } else { + format!("{base}/v1/responses") + } + } + None => RESPONSES_URL.to_string(), + } +} + +impl ModelProvider for OpenAiProvider { + fn complete(&mut self, request: ModelRequest) -> KimetsuResult { + let body = build_request_body(&self.model, &request); + let response = self + .client + .post(responses_url(&self.base_url)) + .bearer_auth(self.api_key.expose_secret()) + .header("content-type", "application/json") + .json(&body) + .send()?; + + let status = response.status(); + let response_text = response.text()?; + if !status.is_success() { + return Err(format!( + "openai request failed ({status}): {}", + response_error_summary(&response_text) + ) + .into()); + } + + parse_response(&response_text) + } +} + +fn build_request_body(model: &str, request: &ModelRequest) -> Value { + let mut system_parts = Vec::new(); + let mut input_parts = Vec::new(); + + for message in &request.messages { + let text = message_text(message); + if text.trim().is_empty() { + continue; + } + if matches!(message.role, MessageRole::System) { + system_parts.push(text); + } else { + input_parts.push(format!("{}: {text}", role_name(&message.role))); + } + } + + let mut body = json!({ + "model": model, + "input": input_parts.join("\n\n"), + "max_output_tokens": request.max_output_tokens, + }); + + if !system_parts.is_empty() { + body["instructions"] = json!(system_parts.join("\n\n")); + } + + body +} + +fn role_name(role: &MessageRole) -> &'static str { + match role { + MessageRole::System => "system", + MessageRole::User => "user", + MessageRole::Assistant => "assistant", + MessageRole::Tool => "tool", + } +} + +fn message_text(message: &crate::model::ModelMessage) -> String { + message + .content + .iter() + .filter_map(content_text) + .collect::>() + .join("\n") +} + +fn content_text(content: &MessageContent) -> Option { + match content { + MessageContent::Text { text } => Some(text.clone()), + MessageContent::ToolCall { name, input, .. } => { + Some(format!("[tool {name}: {}]", compact_json(input))) + } + MessageContent::ToolResult { name, output, .. } => { + Some(format!("[tool result {name}: {}]", compact_json(output))) + } + } +} + +fn compact_json(value: &Value) -> String { + serde_json::to_string(value).unwrap_or_else(|_| value.to_string()) +} + +fn parse_response(response_text: &str) -> KimetsuResult { + let value: Value = serde_json::from_str(response_text)?; + let mut nested_text_parts = Vec::new(); + let mut saw_refusal = false; + let mut tool_calls = Vec::new(); + + for item in value + .get("output") + .and_then(Value::as_array) + .into_iter() + .flatten() + { + match item.get("type").and_then(Value::as_str) { + Some("message") | None => { + for content in item + .get("content") + .and_then(Value::as_array) + .into_iter() + .flatten() + { + match content.get("type").and_then(Value::as_str) { + Some("output_text") => { + if let Some(text) = content.get("text").and_then(Value::as_str) + && !text.trim().is_empty() + { + nested_text_parts.push(text.to_string()); + } + } + Some("refusal") => { + saw_refusal = true; + if let Some(text) = content.get("refusal").and_then(Value::as_str) + && !text.trim().is_empty() + { + nested_text_parts.push(text.to_string()); + } + } + _ => {} + } + } + } + Some("function_call") | Some("custom_tool_call") => { + if let Some(call) = parse_tool_call(item) { + tool_calls.push(call); + } + } + _ => {} + } + } + + let text_parts = if nested_text_parts.is_empty() { + value + .get("output_text") + .and_then(Value::as_str) + .filter(|text| !text.trim().is_empty()) + .map(|text| vec![text.to_string()]) + .unwrap_or_default() + } else { + nested_text_parts + }; + + let stop_reason = if !tool_calls.is_empty() { + StopReason::ToolUse + } else if saw_refusal { + StopReason::Refusal + } else { + map_stop_reason(&value) + }; + + Ok(ModelResponse { + text: if text_parts.is_empty() { + None + } else { + Some(text_parts.join("\n")) + }, + tool_calls, + stop_reason, + usage: parse_usage(&value), + }) +} + +fn parse_tool_call(item: &Value) -> Option { + let name = item.get("name").and_then(Value::as_str)?.to_string(); + let id = item + .get("call_id") + .or_else(|| item.get("id")) + .and_then(Value::as_str) + .unwrap_or("") + .to_string(); + let input = item + .get("arguments") + .or_else(|| item.get("input")) + .map(parse_tool_input) + .unwrap_or(Value::Null); + Some(ToolCall { id, name, input }) +} + +fn parse_tool_input(value: &Value) -> Value { + if let Some(text) = value.as_str() { + serde_json::from_str(text).unwrap_or_else(|_| json!({ "input": text })) + } else { + value.clone() + } +} + +fn map_stop_reason(value: &Value) -> StopReason { + if !value.get("error").unwrap_or(&Value::Null).is_null() { + return StopReason::Error; + } + match value.get("status").and_then(Value::as_str) { + Some("completed") | None => StopReason::EndTurn, + Some("incomplete") => match value + .pointer("/incomplete_details/reason") + .and_then(Value::as_str) + { + Some("max_output_tokens") => StopReason::MaxTokens, + Some("content_filter") => StopReason::Refusal, + _ => StopReason::Error, + }, + Some("failed") => StopReason::Error, + _ => StopReason::Error, + } +} + +fn parse_usage(value: &Value) -> TokenUsage { + TokenUsage { + input_tokens: value_u32(value.pointer("/usage/input_tokens")), + output_tokens: value_u32(value.pointer("/usage/output_tokens")), + cost_usd: 0.0, + cache_creation_input_tokens: 0, + cache_read_input_tokens: value_u32( + value.pointer("/usage/input_tokens_details/cached_tokens"), + ), + } +} + +fn value_u32(value: Option<&Value>) -> u32 { + value + .and_then(Value::as_u64) + .and_then(|n| u32::try_from(n).ok()) + .unwrap_or_default() +} + +fn response_error_summary(response_text: &str) -> String { + let parsed = serde_json::from_str::(response_text).ok(); + let message = parsed + .as_ref() + .and_then(|value| value.pointer("/error/message")) + .and_then(Value::as_str) + .or_else(|| { + parsed + .as_ref() + .and_then(|value| value.get("message")) + .and_then(Value::as_str) + }) + .unwrap_or(response_text); + truncate(message, 700) +} + +fn truncate(value: &str, max_chars: usize) -> String { + let mut out = value.chars().take(max_chars).collect::(); + if value.chars().count() > max_chars { + out.push_str("..."); + } + out +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::model::ModelMessage; + + #[test] + fn debug_format_does_not_leak_api_key() { + let token = "sk-DEFINITELY-LEAKED-IF-BROKEN-1234567890abcdef"; + let provider = OpenAiProvider { + client: Client::new(), + api_key: SecretString::new(token), + model: "gpt-5.4-mini".into(), + base_url: None, + }; + let dbg = format!("{:?}", provider); + assert!( + !dbg.contains("DEFINITELY-LEAKED-IF-BROKEN"), + "Debug-print MUST NOT include the inner token: {dbg}" + ); + assert!(dbg.contains("REDACTED")); + assert!(dbg.contains("gpt-5.4-mini")); + } + + #[test] + fn responses_url_uses_base_when_set() { + assert_eq!(responses_url(&None), RESPONSES_URL); + assert_eq!( + responses_url(&Some("http://localhost:4000".to_string())), + "http://localhost:4000/v1/responses" + ); + assert_eq!( + responses_url(&Some("http://localhost:4000/v1".to_string())), + "http://localhost:4000/v1/responses" + ); + assert_eq!( + responses_url(&Some("http://localhost:4000/v1/".to_string())), + "http://localhost:4000/v1/responses" + ); + } + + #[test] + fn for_distiller_builds_provider_with_base_url() { + let p = OpenAiProvider::for_distiller( + "gpt-5.4-mini", + "sk-test", + Some("http://localhost:4000/v1".to_string()), + 60, + ) + .expect("build"); + assert_eq!(p.model_name(), "gpt-5.4-mini"); + assert_eq!(p.base_url.as_deref(), Some("http://localhost:4000/v1")); + } + + #[test] + fn request_maps_system_and_user_to_responses_body() { + let request = ModelRequest { + messages: vec![ + ModelMessage { + role: MessageRole::System, + content: vec![MessageContent::Text { + text: "Use strict JSON.".to_string(), + }], + }, + ModelMessage::user_text("Distill the transcript."), + ModelMessage::assistant_text("Previous assistant text."), + ], + tools: Vec::new(), + tool_choice: crate::model::ToolChoice::None, + max_output_tokens: 1024, + temperature: 0.2, + metadata: Value::Null, + }; + + let body = build_request_body("gpt-5.4-mini", &request); + assert_eq!(body["model"], "gpt-5.4-mini"); + assert_eq!(body["instructions"], "Use strict JSON."); + assert_eq!(body["max_output_tokens"], 1024); + assert!(body["input"].as_str().unwrap().contains("user: Distill")); + assert!( + body["input"] + .as_str() + .unwrap() + .contains("assistant: Previous") + ); + assert!(body.get("temperature").is_none()); + } + + #[test] + fn response_maps_output_text_and_usage() { + let response = parse_response( + r#"{ + "status": "completed", + "output": [ + { + "type": "message", + "role": "assistant", + "content": [ + {"type": "output_text", "text": "[{\"lesson\":\"Pin the linker\"}]"} + ] + } + ], + "usage": { + "input_tokens": 10, + "input_tokens_details": {"cached_tokens": 3}, + "output_tokens": 5, + "total_tokens": 15 + } + }"#, + ) + .expect("parse response"); + + assert_eq!( + response.text.as_deref(), + Some("[{\"lesson\":\"Pin the linker\"}]") + ); + assert!(matches!(response.stop_reason, StopReason::EndTurn)); + assert_eq!(response.usage.input_tokens, 10); + assert_eq!(response.usage.output_tokens, 5); + assert_eq!(response.usage.cache_read_input_tokens, 3); + } + + #[test] + fn response_maps_top_level_output_text() { + let response = parse_response( + r#"{ + "status": "completed", + "output_text": "[]", + "usage": {"input_tokens": 1, "output_tokens": 1} + }"#, + ) + .expect("parse response"); + + assert_eq!(response.text.as_deref(), Some("[]")); + assert!(matches!(response.stop_reason, StopReason::EndTurn)); + } + + #[test] + fn response_maps_incomplete_max_tokens() { + let response = parse_response( + r#"{ + "status": "incomplete", + "incomplete_details": {"reason": "max_output_tokens"}, + "output": [] + }"#, + ) + .expect("parse response"); + + assert!(matches!(response.stop_reason, StopReason::MaxTokens)); + } +} diff --git a/crates/kimetsu-chat/src/bridge.rs b/crates/kimetsu-chat/src/bridge.rs index 103c0f4..1a048ff 100644 --- a/crates/kimetsu-chat/src/bridge.rs +++ b/crates/kimetsu-chat/src/bridge.rs @@ -207,7 +207,8 @@ Optional mode: - Kimetsu brain is the preferred first step for non-trivial work. - If native MCP tools are unavailable and the task is small, note that Kimetsu brain context was unavailable and continue. - For broad work, fix the plugin/MCP setup first so `kimetsu_brain_context` is available. -- Installed Codex hooks use `.codex/hooks.json` and the `UserPromptSubmit` event to run `kimetsu brain context-hook --workspace .`. Optional mode does not block when Kimetsu returns no relevant context. +- Installed Codex hooks use `.codex/hooks.json`: `UserPromptSubmit` runs `kimetsu brain context-hook --workspace .`, `Stop` runs `kimetsu brain stop-hook --workspace . --distill-on-stop`, and proactive hooks wrap Bash tool calls. Optional mode does not block when Kimetsu returns no relevant context. +- The installer also writes `.codex/agents/kimetsu-memory-harvester.toml`. When a `[kimetsu-harvest]` cue appears and no credentialed distiller is configured, spawn that custom agent in the background to record any durable lesson. Kimetsu brain tools retrieve and manage durable context. Kimetsu bridge tools discover and install reusable capabilities. Continue the actual task with the host harness's normal file, shell, edit, and verification tools. "#; @@ -232,11 +233,41 @@ Required mode: - Treat missing Kimetsu MCP access as a setup blocker for non-trivial tasks. - Continue without Kimetsu only when the user explicitly waives it or the task is trivial. - State whether `kimetsu_benchmark_context` or `kimetsu_brain_context` was called and how many capsules were returned when reporting benchmark or audit results. -- Installed Codex hooks use `.codex/hooks.json` and the `UserPromptSubmit` event to run `kimetsu brain context-hook --workspace .`; benchmark wrappers should inspect MCP transcripts for required Kimetsu usage. +- Installed Codex hooks use `.codex/hooks.json`: `UserPromptSubmit` runs `kimetsu brain context-hook --workspace .`, `Stop` runs `kimetsu brain stop-hook --workspace . --distill-on-stop`, and proactive hooks wrap Bash tool calls; benchmark wrappers should inspect MCP transcripts for required Kimetsu usage. +- The installer also writes `.codex/agents/kimetsu-memory-harvester.toml`. When a `[kimetsu-harvest]` cue appears and no credentialed distiller is configured, spawn that custom agent in the background to record any durable lesson. Kimetsu brain tools retrieve and manage durable context. Kimetsu bridge tools discover and install reusable capabilities. Continue the actual task with the host harness's normal file, shell, edit, and verification tools after loading Kimetsu context. "#; +/// Codex custom agent installed at `.codex/agents/kimetsu-memory-harvester.toml`. +/// It mirrors the Claude Code harvester agent but uses Codex's standalone TOML +/// custom-agent schema. +const CODEX_MEMORY_HARVESTER_AGENT: &str = r#"name = "kimetsu-memory-harvester" +description = "Distills durable, generalizable lessons from the recent session and records them to the Kimetsu brain. Spawn in the background when a [kimetsu-harvest] hook cue appears, or after solving a non-obvious problem." +model = "gpt-5.3-codex-spark" +model_reasoning_effort = "medium" +sandbox_mode = "read-only" +developer_instructions = """ +You are Kimetsu's memory harvester. Given the recent conversation/session context, extract durable lessons worth remembering across future sessions and record them. + +What qualifies: +- A non-obvious fix for a command/tool that failed and was then resolved; capture the root cause and fix, generalized beyond one repo path. +- A convention, gotcha, or environment quirk that cost real effort to discover. +- A reusable approach or anti-pattern confirmed by the outcome. + +What does not qualify: +- Trivial or well-known facts, one-liners, restatements of docs. +- Anything specific to a single throwaway value with no general lesson. + +How to record: +- For each qualifying lesson, at most 3, call kimetsu_brain_record with a concrete actionable lesson, 2-5 domain tags, an optional one-line context, and confidence in [0,1]. +- Use kind = "anti_pattern" for things to avoid, "convention" for project norms, otherwise the default. +- If nothing qualifies, do nothing and finish. + +Constraints: do not modify files, run shell commands, or take any action other than calling Kimetsu brain MCP tools. Quality over quantity. +""" +"#; + pub fn bridge_scan(workspace: &Path, config: &SkillConfig) -> Result { let workspace = normalize_path(workspace); let registry = SkillRegistry::discover(&workspace, config)?; @@ -485,6 +516,12 @@ fn plugin_install_inner( true, )?; files.push(normalize_path(&skill)); + let agents = codex_dir.join("agents"); + fs::create_dir_all(&agents) + .map_err(|err| format!("create {}: {err}", agents.display()))?; + let harvester = agents.join("kimetsu-memory-harvester.toml"); + write_text_file(&harvester, CODEX_MEMORY_HARVESTER_AGENT, true)?; + files.push(normalize_path(&harvester)); write_codex_hooks(&codex_dir, proactive, &mut files)?; } BridgeTarget::Kimetsu => { @@ -548,7 +585,7 @@ fn upsert_kimetsu_hook( } } -/// Merge Kimetsu's `UserPromptSubmit` hook (plus the v0.8 proactive +/// Merge Kimetsu's `UserPromptSubmit`/`Stop` hooks (plus the v0.8 proactive /// `PreToolUse`/`PostToolUse` Bash hooks when `proactive`) into /// `.codex/hooks.json`, preserving any other hooks the user has — even on /// the same events. Idempotent: re-running never duplicates. @@ -604,6 +641,19 @@ fn write_codex_hooks( }] }), ); + upsert_kimetsu_hook( + hooks_obj, + "Stop", + serde_json::json!({ + "matcher": "", + "hooks": [{ + "type": "command", + "command": "kimetsu brain stop-hook --workspace . --distill-on-stop", + "statusMessage": "Checking Kimetsu memory capture", + "timeout": 180 + }] + }), + ); if proactive { upsert_kimetsu_hook( hooks_obj, @@ -1160,6 +1210,11 @@ mod tests { assert!(optional_text.contains("kimetsu_benchmark_context")); assert!(optional_text.contains("kimetsu_benchmark_record_outcome")); assert!(!optional_text.contains("kimetsu_harbor")); + let codex_harvester = root.join(".codex/agents/kimetsu-memory-harvester.toml"); + let harvester_text = fs::read_to_string(&codex_harvester).expect("codex harvester"); + let _: toml::Value = toml::from_str(&harvester_text).expect("harvester toml"); + assert!(harvester_text.contains("name = \"kimetsu-memory-harvester\"")); + assert!(harvester_text.contains("kimetsu_brain_record")); let codex_config = root.join(".codex/config.toml"); let config_text = fs::read_to_string(&codex_config).expect("codex config"); assert!(config_text.contains("[mcp_servers.kimetsu]")); @@ -1173,6 +1228,10 @@ mod tests { hooks_json["hooks"]["UserPromptSubmit"][0]["hooks"][0]["command"].as_str(), Some("kimetsu brain context-hook --workspace .") ); + assert_eq!( + hooks_json["hooks"]["Stop"][0]["hooks"][0]["command"].as_str(), + Some("kimetsu brain stop-hook --workspace . --distill-on-stop") + ); // v0.8: proactive on by default wires the Bash PreToolUse/PostToolUse hooks. assert_eq!( hooks_json["hooks"]["PostToolUse"][0]["matcher"].as_str(), @@ -1226,6 +1285,10 @@ mod tests { let hooks_text = fs::read_to_string(root.join(".codex/hooks.json")).expect("codex hooks"); let hooks_json: serde_json::Value = serde_json::from_str(&hooks_text).expect("hooks json"); assert!(hooks_json["hooks"]["UserPromptSubmit"].is_array()); + assert_eq!( + hooks_json["hooks"]["Stop"][0]["hooks"][0]["command"].as_str(), + Some("kimetsu brain stop-hook --workspace . --distill-on-stop") + ); assert!( hooks_json["hooks"]["PreToolUse"].is_null(), "proactive disabled must not write PreToolUse" @@ -1350,6 +1413,10 @@ mod tests { ups[1]["hooks"][0]["command"], "kimetsu brain context-hook --workspace ." ); + assert_eq!( + value["hooks"]["Stop"][0]["hooks"][0]["command"], + "kimetsu brain stop-hook --workspace . --distill-on-stop" + ); assert_eq!( value["hooks"]["PreToolUse"][0]["hooks"][0]["command"], "kimetsu brain pretool-hook --workspace ." @@ -1474,6 +1541,10 @@ mod tests { .unwrap(); assert!(home.join(".codex/config.toml").is_file()); assert!(home.join(".codex/hooks.json").is_file()); + assert!( + home.join(".codex/agents/kimetsu-memory-harvester.toml") + .is_file() + ); assert!(!ws.join(".codex").exists()); fs::remove_dir_all(ws).ok(); diff --git a/crates/kimetsu-chat/src/lib.rs b/crates/kimetsu-chat/src/lib.rs index 863b1cf..f2993c5 100644 --- a/crates/kimetsu-chat/src/lib.rs +++ b/crates/kimetsu-chat/src/lib.rs @@ -8,7 +8,7 @@ //! 1. Zero dependence on Terminal-Bench / Harbor — chat is its own //! product, not a benchmark subset. //! 2. Reuse the entire 20-tool surface, prompts, brain integration, -//! providers (`claude_code` today; `anthropic` natively next), and +//! providers (`claude_code`, `anthropic`, and distiller-specific OpenAI), and //! MP-18's iterative goal verify with no per-feature porting. //! 3. Tool runtime swaps to host-side `LocalShellExecutor` — commands //! execute against the user's actual filesystem. diff --git a/crates/kimetsu-chat/src/mcp_server.rs b/crates/kimetsu-chat/src/mcp_server.rs index a6080c0..271e741 100644 --- a/crates/kimetsu-chat/src/mcp_server.rs +++ b/crates/kimetsu-chat/src/mcp_server.rs @@ -60,7 +60,7 @@ const BRIDGE_EXPORT_DESCRIPTION: &str = "Export a canonical or discovered skill const BRIDGE_SYNC_DESCRIPTION: &str = "Bulk-import all discovered non-Kimetsu skills into .kimetsu/extensions. Use for setup or migration, not during a narrow task unless the user asked to synchronize capabilities. This writes files and may touch many skill bundles."; -const PLUGIN_INSTALL_DESCRIPTION: &str = "Install Kimetsu MCP/plugin wiring for a target harness in this workspace. For codex, writes .codex/config.toml, .codex/hooks.json, and the kimetsu-bridge skill; for claude-code, writes .mcp.json, command docs, and .claude/settings.json hooks. Set mode=optional to recommend brain-first usage, or mode=required to tell the host harness that non-trivial work must load Kimetsu brain context. Installed guidance tells benchmark agents to prefer kimetsu_benchmark_context and record outcomes through kimetsu_benchmark_record_outcome. Set scope=workspace (default) to install into this workspace, or scope=global to install into the user's home (~/.claude, ~/.claude.json, ~/.codex) for all sessions. Existing user hooks are preserved (merged, not replaced)."; +const PLUGIN_INSTALL_DESCRIPTION: &str = "Install Kimetsu MCP/plugin wiring for a target harness in this workspace. For codex, writes .codex/config.toml, .codex/hooks.json, the kimetsu-bridge skill, and the kimetsu-memory-harvester custom agent; for claude-code, writes .mcp.json, command docs, and .claude/settings.json hooks. Set mode=optional to recommend brain-first usage, or mode=required to tell the host harness that non-trivial work must load Kimetsu brain context. Installed guidance tells benchmark agents to prefer kimetsu_benchmark_context and record outcomes through kimetsu_benchmark_record_outcome. Set scope=workspace (default) to install into this workspace, or scope=global to install into the user's home (~/.claude, ~/.claude.json, ~/.codex) for all sessions. Existing user hooks are preserved (merged, not replaced)."; #[derive(Debug, Clone)] pub struct McpServeConfig { diff --git a/crates/kimetsu-cli/src/distiller.rs b/crates/kimetsu-cli/src/distiller.rs index dc5a43c..505b528 100644 --- a/crates/kimetsu-cli/src/distiller.rs +++ b/crates/kimetsu-cli/src/distiller.rs @@ -10,6 +10,7 @@ use kimetsu_agent::anthropic::AnthropicProvider; use kimetsu_agent::model::{ MessageContent, MessageRole, ModelMessage, ModelProvider, ModelRequest, ToolChoice, }; +use kimetsu_agent::openai::OpenAiProvider; use kimetsu_brain::project; use kimetsu_core::config::ProjectConfig; use kimetsu_core::env_file::resolve_env_value; @@ -236,6 +237,7 @@ pub fn distill_and_record( /// The distiller selected for this session: which model/key/endpoint to use, /// and how to record (project vs the global user brain). pub struct ResolvedDistiller { + pub provider: String, pub model: String, pub key: String, pub base_url: Option, @@ -267,10 +269,11 @@ fn resolve_distiller_with( { let d = &config.learning.distiller; if d.enabled - && d.provider == "anthropic" + && let Some(provider) = normalize_distiller_provider(&d.provider) && let Some(key) = resolve_env_value(&paths.repo_root, &d.api_key_env) { return Some(ResolvedDistiller { + provider: provider.to_string(), model: d.model.clone(), key, base_url: resolve_env_value(&paths.repo_root, &d.base_url_env), @@ -287,10 +290,11 @@ fn resolve_distiller_with( { let d = &config.learning.distiller; if d.enabled - && d.provider == "anthropic" + && let Some(provider) = normalize_distiller_provider(&d.provider) && let Some(key) = resolve_env_value(&dir, &d.api_key_env) { return Some(ResolvedDistiller { + provider: provider.to_string(), model: d.model.clone(), key, base_url: resolve_env_value(&dir, &d.base_url_env), @@ -303,6 +307,14 @@ fn resolve_distiller_with( None } +fn normalize_distiller_provider(provider: &str) -> Option<&'static str> { + match provider.trim().to_ascii_lowercase().as_str() { + "anthropic" | "claude" => Some("anthropic"), + "openai" | "oai" | "gpt" => Some("openai"), + _ => None, + } +} + /// `kimetsu brain session-end-hook` entry. Reads the SessionEnd payload /// from stdin, and if the distiller is enabled + credentialed, distills /// the transcript and records lessons. Silent no-op otherwise. @@ -319,28 +331,52 @@ pub fn run_session_end_hook(workspace: &Path) { else { return; }; - let Some(resolved) = resolve_distiller(workspace) else { - return; - }; + run_distiller_for_transcript(workspace, transcript_path); +} + +/// Run the configured distiller against a known transcript path. Used by +/// SessionEnd hooks where available, and by Codex Stop hooks because current +/// Codex releases expose Stop but not SessionEnd. +pub fn run_distiller_for_transcript(workspace: &Path, transcript_path: &str) -> Option { + let resolved = resolve_distiller(workspace)?; let view = build_transcript_view(transcript_path, MAX_VIEW_CHARS); if view.trim().is_empty() { - return; + return Some(0); } - let Ok(mut provider) = AnthropicProvider::for_distiller( - &resolved.model, - resolved.key, - resolved.base_url, - resolved.timeout_secs, - ) else { - return; + let mut provider: Box = match resolved.provider.as_str() { + "anthropic" => match AnthropicProvider::for_distiller( + &resolved.model, + resolved.key, + resolved.base_url, + resolved.timeout_secs, + ) { + Ok(provider) => Box::new(provider), + Err(_) => return None, + }, + "openai" => match OpenAiProvider::for_distiller( + &resolved.model, + resolved.key, + resolved.base_url, + resolved.timeout_secs, + ) { + Ok(provider) => Box::new(provider), + Err(_) => return None, + }, + _ => return None, }; - let recorded = distill_and_record(&resolved.record_start, &view, &mut provider, resolved.scope); + let recorded = distill_and_record( + &resolved.record_start, + &view, + provider.as_mut(), + resolved.scope, + ); if recorded > 0 { println!( "[Kimetsu] distilled {recorded} lesson{} at session end.", if recorded == 1 { "" } else { "s" } ); } + Some(recorded) } #[cfg(test)] @@ -510,13 +546,31 @@ mod tests { /// `model` sections are required by serde (no `#[serde(default)]` on those /// `ProjectConfig` fields). fn write_distiller_toml(dir: &std::path::Path, enabled: bool, model: &str) { + write_distiller_toml_with_provider( + dir, + enabled, + "anthropic", + model, + "ANTHROPIC_API_KEY", + "ANTHROPIC_BASE_URL", + ); + } + + fn write_distiller_toml_with_provider( + dir: &std::path::Path, + enabled: bool, + provider: &str, + model: &str, + api_key_env: &str, + base_url_env: &str, + ) { std::fs::create_dir_all(dir).unwrap(); let mut config = ProjectConfig::default_for_project("test"); config.learning.distiller.enabled = enabled; - config.learning.distiller.provider = "anthropic".to_string(); + config.learning.distiller.provider = provider.to_string(); config.learning.distiller.model = model.to_string(); - config.learning.distiller.api_key_env = "ANTHROPIC_API_KEY".to_string(); - config.learning.distiller.base_url_env = "ANTHROPIC_BASE_URL".to_string(); + config.learning.distiller.api_key_env = api_key_env.to_string(); + config.learning.distiller.base_url_env = base_url_env.to_string(); let toml = config.to_toml().unwrap(); std::fs::write(dir.join("project.toml"), toml).unwrap(); } @@ -545,6 +599,7 @@ mod tests { let r = resolve_distiller_with(&ws, Some(gdir.clone())).expect("global resolved"); assert_eq!(r.scope, MemoryScope::GlobalUser); + assert_eq!(r.provider, "anthropic"); assert_eq!(r.model, "claude-haiku-4-5"); assert_eq!(r.key, "sk-global"); @@ -596,6 +651,7 @@ mod tests { let r = resolve_distiller_with(&ws, Some(gdir.clone())).expect("workspace resolved"); assert_eq!(r.scope, MemoryScope::Project); + assert_eq!(r.provider, "anthropic"); assert_eq!(r.model, "ws-model"); assert_eq!(r.key, "sk-ws"); @@ -603,6 +659,46 @@ mod tests { std::fs::remove_dir_all(gdir).ok(); } + #[test] + fn resolve_distiller_openai_workspace() { + let ws = std::env::temp_dir().join(format!( + "km_rd_oai_{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + std::fs::create_dir_all(ws.join(".kimetsu")).unwrap(); + assert!( + kimetsu_core::paths::git_init_boundary(&ws), + "git_init_boundary failed - git needed for workspace isolation" + ); + { + let mut config = ProjectConfig::default_for_project("ws"); + config.learning.distiller.enabled = true; + config.learning.distiller.provider = "openai".to_string(); + config.learning.distiller.model = "gpt-5.4-mini".to_string(); + config.learning.distiller.api_key_env = "OPENAI_API_KEY".to_string(); + config.learning.distiller.base_url_env = "OPENAI_BASE_URL".to_string(); + let toml = config.to_toml().unwrap(); + std::fs::write(ws.join(".kimetsu").join("project.toml"), toml).unwrap(); + } + std::fs::write( + ws.join(".env"), + "OPENAI_API_KEY=sk-openai\nOPENAI_BASE_URL=http://localhost:4000/v1\n", + ) + .unwrap(); + + let r = resolve_distiller_with(&ws, None).expect("workspace resolved"); + assert_eq!(r.scope, MemoryScope::Project); + assert_eq!(r.provider, "openai"); + assert_eq!(r.model, "gpt-5.4-mini"); + assert_eq!(r.key, "sk-openai"); + assert_eq!(r.base_url.as_deref(), Some("http://localhost:4000/v1")); + + std::fs::remove_dir_all(ws).ok(); + } + #[test] fn distill_and_record_global_writes_to_user_brain() { let dir = std::env::temp_dir().join(format!( diff --git a/crates/kimetsu-cli/src/harvest_setup.rs b/crates/kimetsu-cli/src/harvest_setup.rs index 51feb6e..72d8c35 100644 --- a/crates/kimetsu-cli/src/harvest_setup.rs +++ b/crates/kimetsu-cli/src/harvest_setup.rs @@ -1,7 +1,8 @@ //! Interactive `kimetsu plugin install` wizard that configures the -//! credentialed SessionEnd distiller: collects an API key (+ optional -//! LiteLLM base URL) + model, writes a gitignored `.env`, and flips -//! `[learning.distiller]` on in the workspace project.toml. +//! credentialed SessionEnd distiller: collects a provider API key +//! (+ optional compatible endpoint base URL) + model, writes a +//! gitignored `.env`, and flips `[learning.distiller]` on in the +//! workspace project.toml. use std::fs; use std::io::{BufRead, Write}; @@ -17,6 +18,16 @@ pub struct SetupTarget { pub gitignore_dir: PathBuf, } +#[derive(Debug, Clone, Copy)] +struct DistillerProviderChoice { + provider: &'static str, + api_key_env: &'static str, + base_url_env: &'static str, + default_model: &'static str, + key_prompt: &'static str, + base_url_prompt: &'static str, +} + /// Run the wizard against the given reader/writer (real stdin/stdout in /// production; scripted in tests). Returns Ok(true) when the distiller was /// configured, Ok(false) when the user declined or aborted. @@ -32,59 +43,71 @@ pub fn run_harvest_setup( return Ok(false); } - write!(writer, "Harness [claude/codex] (codex not yet supported): ")?; + write!(writer, "Harness [claude/codex] [claude]: ")?; writer.flush()?; let harness = read_line(reader)?.trim().to_lowercase(); match harness.as_str() { - "codex" => { - writeln!( - writer, - "Codex distiller is not supported yet — skipping setup." - )?; - return Ok(false); - } + "codex" | "openai-codex" | "cx" => {} // Blank defaults to claude; accept the common aliases. "" | "claude" | "claude-code" | "cc" => {} other => { - writeln!(writer, "Unrecognized harness '{other}' — skipping setup.")?; + writeln!(writer, "Unrecognized harness '{other}' - skipping setup.")?; return Ok(false); } } - write!(writer, "Anthropic API key (or LiteLLM key): ")?; + write!( + writer, + "Distiller provider [anthropic/openai] [anthropic]: " + )?; + writer.flush()?; + let provider_input = read_line(reader)?.trim().to_string(); + let Some(choice) = resolve_distiller_provider(&provider_input) else { + writeln!( + writer, + "Unrecognized distiller provider '{provider_input}' - skipping setup." + )?; + return Ok(false); + }; + + write!(writer, "{}", choice.key_prompt)?; writer.flush()?; let key = read_line(reader)?.trim().to_string(); if key.is_empty() { - writeln!(writer, "No key entered — skipping setup.")?; + writeln!(writer, "No key entered - skipping setup.")?; return Ok(false); } - write!( - writer, - "ANTHROPIC_BASE_URL (optional; blank for Anthropic, set for LiteLLM): " - )?; + write!(writer, "{}", choice.base_url_prompt)?; writer.flush()?; let base_url = read_line(reader)?.trim().to_string(); - write!(writer, "Model [claude-haiku-4-5]: ")?; + write!(writer, "Model [{}]: ", choice.default_model)?; writer.flush()?; let mut model = read_line(reader)?.trim().to_string(); if model.is_empty() { - model = "claude-haiku-4-5".to_string(); + model = choice.default_model.to_string(); } - apply_distiller_config(&target.project_toml, &model)?; + apply_distiller_config( + &target.project_toml, + choice.provider, + &model, + choice.api_key_env, + choice.base_url_env, + )?; // Gitignore `.env` BEFORE writing the secret into it. ensure_gitignored(&target.gitignore_dir, ".env")?; - upsert_env_var(&target.env_path, "ANTHROPIC_API_KEY", &key)?; + upsert_env_var(&target.env_path, choice.api_key_env, &key)?; if !base_url.is_empty() { - upsert_env_var(&target.env_path, "ANTHROPIC_BASE_URL", &base_url)?; + upsert_env_var(&target.env_path, choice.base_url_env, &base_url)?; } writeln!( writer, - "\u{2713} Distiller configured for {scope_label} (model {model}). \ + "\u{2713} Distiller configured for {scope_label} ({} model {model}). \ Key stored in {} (gitignored). Note: the key was entered in plain text.", + choice.provider, target.env_path.display() )?; Ok(true) @@ -96,12 +119,40 @@ fn read_line(reader: &mut R) -> std::io::Result { Ok(line) } +fn resolve_distiller_provider(input: &str) -> Option { + match input.trim().to_ascii_lowercase().as_str() { + "" | "anthropic" | "claude" | "claude-code" | "litellm" => Some(DistillerProviderChoice { + provider: "anthropic", + api_key_env: "ANTHROPIC_API_KEY", + base_url_env: "ANTHROPIC_BASE_URL", + default_model: "claude-haiku-4-5", + key_prompt: "Anthropic API key (or Anthropic-compatible LiteLLM key): ", + base_url_prompt: "ANTHROPIC_BASE_URL (optional; blank for Anthropic, set for LiteLLM): ", + }), + "openai" | "oai" | "gpt" => Some(DistillerProviderChoice { + provider: "openai", + api_key_env: "OPENAI_API_KEY", + base_url_env: "OPENAI_BASE_URL", + default_model: "gpt-5.4-mini", + key_prompt: "OpenAI API key (or OpenAI-compatible endpoint key): ", + base_url_prompt: "OPENAI_BASE_URL (optional; blank for OpenAI, accepts root or /v1): ", + }), + _ => None, + } +} + /// Flip the distiller on in the project config, writing to `project_toml` /// directly. Loads an existing `project.toml` if present, else starts from a /// default — it does NOT call `init_project` (which would climb to an /// enclosing git repo and open the brain DB), so the config + secret never /// land in a parent repository. -fn apply_distiller_config(project_toml: &Path, model: &str) -> std::io::Result<()> { +fn apply_distiller_config( + project_toml: &Path, + provider: &str, + model: &str, + api_key_env: &str, + base_url_env: &str, +) -> std::io::Result<()> { let io_err = |e: Box| std::io::Error::other(e.to_string()); let mut config = if project_toml.exists() { ProjectConfig::from_toml(&fs::read_to_string(project_toml)?).map_err(io_err)? @@ -117,10 +168,10 @@ fn apply_distiller_config(project_toml: &Path, model: &str) -> std::io::Result<( ProjectConfig::default_for_project(project_id) }; config.learning.distiller.enabled = true; - config.learning.distiller.provider = "anthropic".to_string(); + config.learning.distiller.provider = provider.to_string(); config.learning.distiller.model = model.to_string(); - config.learning.distiller.api_key_env = "ANTHROPIC_API_KEY".to_string(); - config.learning.distiller.base_url_env = "ANTHROPIC_BASE_URL".to_string(); + config.learning.distiller.api_key_env = api_key_env.to_string(); + config.learning.distiller.base_url_env = base_url_env.to_string(); if let Some(parent) = project_toml.parent() { fs::create_dir_all(parent)?; } @@ -192,7 +243,7 @@ mod tests { )); fs::create_dir_all(root.join(".kimetsu")).unwrap(); let mut input = Cursor::new( - "y\nclaude\nsk-litellm-123\nhttp://localhost:4000\n\n" + "y\nclaude\n\nsk-litellm-123\nhttp://localhost:4000\n\n" .as_bytes() .to_vec(), ); @@ -207,6 +258,7 @@ mod tests { assert!(env.contains("ANTHROPIC_BASE_URL=http://localhost:4000")); let toml = fs::read_to_string(root.join(".kimetsu").join("project.toml")).unwrap(); assert!(toml.contains("enabled = true")); + assert!(toml.contains("provider = \"anthropic\"")); assert!(toml.contains("claude-haiku-4-5")); assert!( fs::read_to_string(root.join(".gitignore")) @@ -216,6 +268,70 @@ mod tests { fs::remove_dir_all(root).ok(); } + #[test] + fn wizard_accepts_codex_harness() { + let root = std::env::temp_dir().join(format!( + "kimetsu_wizard_codex_{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + fs::create_dir_all(root.join(".kimetsu")).unwrap(); + let mut input = Cursor::new( + "y\ncodex\nopenai\nsk-openai-codex\nhttp://localhost:4000/v1\n\n" + .as_bytes() + .to_vec(), + ); + let mut output = Vec::new(); + let configured = + run_harvest_setup(&mut input, &mut output, &target_at(&root), "this workspace") + .unwrap(); + assert!(configured); + + let out = String::from_utf8(output).unwrap(); + assert!(!out.contains("not supported")); + let env = fs::read_to_string(root.join(".env")).unwrap(); + assert!(env.contains("OPENAI_API_KEY=sk-openai-codex")); + assert!(env.contains("OPENAI_BASE_URL=http://localhost:4000/v1")); + assert!(!env.contains("ANTHROPIC_API_KEY")); + let toml = fs::read_to_string(root.join(".kimetsu").join("project.toml")).unwrap(); + assert!(toml.contains("enabled = true")); + assert!(toml.contains("provider = \"openai\"")); + assert!(toml.contains("gpt-5.4-mini")); + assert!(toml.contains("api_key_env = \"OPENAI_API_KEY\"")); + assert!(toml.contains("base_url_env = \"OPENAI_BASE_URL\"")); + fs::remove_dir_all(root).ok(); + } + + #[test] + fn wizard_accepts_openai_custom_model() { + let root = std::env::temp_dir().join(format!( + "kimetsu_wizard_openai_model_{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + fs::create_dir_all(root.join(".kimetsu")).unwrap(); + let mut input = Cursor::new( + "y\ncodex\nopenai\nsk-openai-custom\n\nmy-distiller-model\n" + .as_bytes() + .to_vec(), + ); + let mut output = Vec::new(); + let configured = + run_harvest_setup(&mut input, &mut output, &target_at(&root), "this workspace") + .unwrap(); + assert!(configured); + + let toml = fs::read_to_string(root.join(".kimetsu").join("project.toml")).unwrap(); + assert!(toml.contains("provider = \"openai\"")); + assert!(toml.contains("model = \"my-distiller-model\"")); + assert!(!toml.contains("model = \"gpt-5.4-mini\"")); + fs::remove_dir_all(root).ok(); + } + #[test] fn wizard_declined_writes_nothing() { let root = std::env::temp_dir().join(format!( @@ -252,6 +368,24 @@ mod tests { fs::remove_dir_all(root).ok(); } + #[test] + fn wizard_unrecognized_provider_aborts() { + let root = std::env::temp_dir().join(format!( + "kimetsu_wizard_bad_provider_{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + fs::create_dir_all(&root).unwrap(); + let paths = target_at(&root); + let mut input = Cursor::new(b"y\nclaude\ngemini\n".to_vec()); + let mut output = Vec::new(); + assert!(!run_harvest_setup(&mut input, &mut output, &paths, "this workspace").unwrap()); + assert!(!root.join(".env").exists()); + fs::remove_dir_all(root).ok(); + } + #[test] fn upsert_env_var_replaces_existing() { let dir = std::env::temp_dir().join(format!( diff --git a/crates/kimetsu-cli/src/main.rs b/crates/kimetsu-cli/src/main.rs index 9bd1250..e2f28a8 100644 --- a/crates/kimetsu-cli/src/main.rs +++ b/crates/kimetsu-cli/src/main.rs @@ -370,9 +370,9 @@ enum BrainCommand { /// silently otherwise. #[command(name = "posttool-hook")] PostToolHook(ProactiveHookArgs), - /// Claude Code SessionEnd hook — runs the credentialed distiller. + /// Host SessionEnd hook — runs the credentialed distiller. #[command(name = "session-end-hook")] - SessionEndHook(StopHookArgs), + SessionEndHook(SessionEndHookArgs), } #[derive(Debug, Subcommand)] @@ -441,6 +441,17 @@ struct StopHookArgs { /// Override the brain workspace path (defaults to current directory). #[arg(long)] workspace: Option, + /// Codex compatibility: run the credentialed distiller from Stop because + /// current Codex hooks expose Stop but not SessionEnd. + #[arg(long)] + distill_on_stop: bool, +} + +#[derive(Debug, Args)] +struct SessionEndHookArgs { + /// Override the brain workspace path (defaults to current directory). + #[arg(long)] + workspace: Option, } #[derive(Debug, Args)] @@ -993,10 +1004,13 @@ fn plugin(command: PluginCommand) -> KimetsuResult<()> { for file in report.files { println!(" {}", file.display()); } - // Offer interactive distiller setup for Claude Code on a TTY. + // Offer interactive distiller setup for host targets on a TTY. let interactive = args.setup_harvest || (std::io::stdin().is_terminal() && std::io::stdout().is_terminal()); - if matches!(target, BridgeTarget::ClaudeCode) && !args.no_setup && interactive { + if matches!(target, BridgeTarget::ClaudeCode | BridgeTarget::Codex) + && !args.no_setup + && interactive + { let target_for_scope = match scope { InstallScope::Global => match kimetsu_core::paths::user_kimetsu_dir() { Some(dir) => Some(( @@ -1910,11 +1924,12 @@ fn brain_stop_hook(args: StopHookArgs) -> KimetsuResult<()> { // line), NOT an inline array — stream it line-by-line so a long // session's transcript (tens of MB) never lands in memory at once. // Fall back to an inline `transcript` array for other harnesses/tests. - let (turn_count, recorded) = match session + let transcript_path = session .get("transcript_path") .and_then(|v| v.as_str()) .filter(|p| !p.trim().is_empty()) - { + .map(str::to_string); + let (turn_count, recorded) = match transcript_path.as_deref() { Some(path) => count_transcript_jsonl(path), None => { let messages: Vec = session @@ -1958,19 +1973,42 @@ fn brain_stop_hook(args: StopHookArgs) -> KimetsuResult<()> { .map(|c| c.learning.auto_harvest) .unwrap_or(true); let distiller_enabled = distiller::resolve_distiller(&workspace).is_some(); + let sid = session.get("session_id").and_then(|v| v.as_str()); + let state_path = paths + .as_ref() + .map(|p| proactive_state::session_path(&p.kimetsu_dir, sid)); + + if args.distill_on_stop + && distiller_enabled + && !stop_active + && let Some(path) = transcript_path.as_deref() + { + let mut state = state_path + .as_ref() + .map(|path| proactive_state::load(path)) + .unwrap_or_default(); + if !state.harvest_cued() { + let _ = distiller::run_distiller_for_transcript(&workspace, path); + if let Some(state_path) = state_path.as_ref() { + state.note_harvest_cue(proactive_state::now_unix()); + proactive_state::save(state_path, &state); + } + return Ok(()); + } + } if should_emit_stop_harvest_cue(auto_harvest, distiller_enabled) && !stop_active && let Some(paths) = paths.as_ref() { - let sid = session.get("session_id").and_then(|v| v.as_str()); - let state_path = proactive_state::session_path(&paths.kimetsu_dir, sid); + let state_path = + state_path.unwrap_or_else(|| proactive_state::session_path(&paths.kimetsu_dir, sid)); let mut state = proactive_state::load(&state_path); if !state.harvest_cued() { println!( "[kimetsu-harvest] No lessons recorded this non-trivial session. If anything \ - durable was learned, dispatch the kimetsu-memory-harvester subagent \ - (run_in_background: true) to capture it — otherwise call kimetsu_brain_record." + durable was learned, run the kimetsu-memory-harvester agent in the background \ + to capture it — otherwise call kimetsu_brain_record." ); state.note_harvest_cue(proactive_state::now_unix()); proactive_state::save(&state_path, &state); @@ -2169,8 +2207,8 @@ fn proactive_hook(event: ProactiveEvent, args: ProactiveHookArgs) -> KimetsuResu let cmd = hook.command.as_deref().unwrap_or("the command"); let cue = format!( "[kimetsu-harvest] You just resolved a previously failing command (`{cmd}`). \ - If this revealed a durable, generalizable lesson, dispatch the \ - kimetsu-memory-harvester subagent in the background (run_in_background: true) \ + If this revealed a durable, generalizable lesson, run the \ + kimetsu-memory-harvester agent in the background \ to record it via kimetsu_brain_record." ); print_tool_use_context(event, &cue)?; diff --git a/crates/kimetsu-core/src/config.rs b/crates/kimetsu-core/src/config.rs index 182f2dd..9775fd2 100644 --- a/crates/kimetsu-core/src/config.rs +++ b/crates/kimetsu-core/src/config.rs @@ -111,7 +111,7 @@ impl Default for LearningSection { /// Credentialed SessionEnd distiller config. Secret values (the API key, /// optional base URL) live in `.env` under the env-var names below; only -/// non-secret selection lives here. +/// non-secret selection lives here. `provider` is `anthropic` or `openai`. #[derive(Debug, Clone, Serialize, Deserialize)] pub struct DistillerSection { #[serde(default)] diff --git a/docs/HOW-KIMETSU-WORKS.md b/docs/HOW-KIMETSU-WORKS.md index abdc165..d688557 100644 --- a/docs/HOW-KIMETSU-WORKS.md +++ b/docs/HOW-KIMETSU-WORKS.md @@ -326,12 +326,17 @@ The core hook pattern is the same across hosts: calls, and prints a one-line post-turn banner — either confirming how many lessons were captured or nudging the model to record one after a non-trivial, un-captured session. +- **`SessionEnd` → `kimetsu brain session-end-hook`** runs the optional + credentialed distiller when the host exposes SessionEnd. Claude Code uses + this event; Codex uses its supported `Stop` event with `--distill-on-stop` + for the same deterministic harvest path. These are plain CLI subcommands, so the same pattern works under any -harness that can run a command on a prompt, stop, or tool event. The -Codex installer wires prompt-time context and proactive tool hooks; the -Claude Code installer wires prompt-time context, stop summaries, and -proactive tool hooks. +harness that can run a command on a prompt, stop, session-end, or tool +event. The Codex installer wires prompt-time context, stop summaries, +Stop-time distilling, proactive tool hooks, and a +`kimetsu-memory-harvester` custom agent; the Claude Code installer wires +the same flow through `.claude/settings.json` and its subagent file. ### Proactive recall (mid-work) @@ -451,13 +456,23 @@ max_total_files = 50_000 max_total_tool_calls = 60 max_total_model_turns = 30 max_total_cost_usd = 250.0 # advisory under subscription providers + +[learning] +auto_harvest = true + +[learning.distiller] +enabled = false +provider = "anthropic" # or "openai" +model = "claude-haiku-4-5" # OpenAI default: "gpt-5.4-mini" +api_key_env = "ANTHROPIC_API_KEY" # or "OPENAI_API_KEY" +base_url_env = "ANTHROPIC_BASE_URL" # or "OPENAI_BASE_URL" ``` Environment variables that override at runtime: | Variable | Effect | |----------|--------| -| `ANTHROPIC_API_KEY` / `CLAUDE_CODE_OAUTH_TOKEN` / `OPENAI_API_TOKEN` | Provider credentials | +| `ANTHROPIC_API_KEY` / `CLAUDE_CODE_OAUTH_TOKEN` / `OPENAI_API_KEY` | Provider credentials | | `KIMETSU_USER_BRAIN=0` | Disable the user brain (project-only memories) | | `KIMETSU_BRAIN_EMBEDDER=noop\|bge\|jina-v2-base-code\|...` | Pick the embedder (or disable) |