Skip to content

release: v0.9.0 — auto-harvested memories#15

Merged
RodCor merged 51 commits into
mainfrom
feat/auto-harvest-memories
Jun 3, 2026
Merged

release: v0.9.0 — auto-harvested memories#15
RodCor merged 51 commits into
mainfrom
feat/auto-harvest-memories

Conversation

@RodCor

@RodCor RodCor commented Jun 2, 2026

Copy link
Copy Markdown
Owner

Summary

v0.9.0 — the Kimetsu hooks now drive memory generation, not just retrieval ("the other way" the brain was missing), plus a credentialed end-of-session distiller, Codex parity, and a round of non-destructive install hardening.

This branch is stacked on the v0.8.4 work (#14) and folds in the global distiller / OpenAI provider (#16), so this PR is the full v0.8.4 + v0.9.0 delta against main.

What's in this branch

🧠 Auto-harvested memories (in-agent, no credentials)

The hooks now generate memories. When a command that failed earlier in the session later succeeds (PostToolUse), or a non-trivial session ends having recorded nothing (Stop), Kimetsu emits a throttled [kimetsu-harvest] cue telling the agent to dispatch a new background kimetsu-memory-harvester subagent (installed at .claude/agents/ for Claude Code and .codex/agents/ for Codex, pinned to a cheap model). The subagent distills 0–3 generalizable lessons and records them through the confidence-gated kimetsu_brain_record path — no separate API key, billed in-agent at the cheap model's rate, non-blocking. Disable with [learning] auto_harvest = false.

🔑 Credentialed SessionEnd distiller (opt-in)

A second, deterministic harvest path alongside the in-agent harvester. kimetsu plugin install claude-code / ... codex now runs an interactive wizard (on a TTY; skip with --no-setup, force with --setup-harvest) configuring a cheap distiller model — Anthropic (claude-haiku-4-5), OpenAI (gpt-5.4-mini), or a compatible endpoint via ANTHROPIC_BASE_URL / OPENAI_BASE_URL. The key + base URL go to a gitignored .env; the selection lands in [learning.distiller] in project.toml. A SessionEnd hook (Claude Code) runs kimetsu brain session-end-hook; when credentialed it reads the transcript with that model and records via propose_or_merge_memory. When the distiller is enabled it owns end-of-session — the Stop harvest cue is suppressed, while the mid-session PostToolUse resolved-failure cue stays. AnthropicProvider gained an optional base URL (LiteLLM); a new OpenAI Responses-API provider was added.

🤖 Codex parity

  • kimetsu plugin install codex writes .codex/config.toml, .codex/hooks.json, the kimetsu-bridge skill, and a .codex/agents/kimetsu-memory-harvester.toml custom agent — the Codex equivalent of the Claude harvester subagent (Codex's standalone TOML agent format). So the in-agent auto-harvest flow works on Codex too.
  • Codex hooks are merged, not replaced. UserPromptSubmitkimetsu brain context-hook, Stopkimetsu brain stop-hook --distill-on-stop, plus proactive Bash-wrapping PreToolUse/PostToolUse hooks. Existing user hooks on the same events are preserved alongside Kimetsu's; re-install is idempotent.
  • Distiller via the Stop hook. Current Codex releases expose Stop but not SessionEnd, so the credentialed distiller runs from Codex's Stop hook via --distill-on-stop (Claude Code uses the dedicated SessionEnd event).
  • Wizard supports Codex. Selecting the codex harness configures the OpenAI provider (default gpt-5.4-mini) or a compatible endpoint via OPENAI_BASE_URL; the key lands in the gitignored .env and the selection in [learning.distiller].
  • New OpenAI Responses-API provider (OpenAiProvider, crates/kimetsu-agent/src/openai.rs) powers the OpenAI/Codex distiller path.

🌍 Global distiller in ~/.kimetsu/

With --scope global the wizard configures the distiller once in ~/.kimetsu/ (config + .env). That global distiller then runs in every project and records into the user brain (~/.kimetsu/brain.db, available everywhere), with a per-project distiller taking precedence when one is configured.

📄 CLAUDE.md merge (append, never replace)

kimetsu plugin install now merges its guidance into an existing CLAUDE.md — workspace .claude/CLAUDE.md or global ~/.claude/CLAUDE.md — inside <!-- kimetsu:begin -->/<!-- kimetsu:end --> markers: write-if-missing / append-if-absent / replace-the-marked-region-in-place / repair-a-corrupt-half-block / BOM-tolerant. The user's content is never overwritten. --force no longer overwrites CLAUDE.md (the whole install is idempotent and non-destructive; the flag is retained only for compatibility).

🔧 Non-destructive install hardening (v0.8.4, bundled)

  • Hook merge. plugin install merges its hooks into existing Claude settings.json / Codex hooks.json instead of replacing them — hooks you already have, even on the same events (UserPromptSubmit, PreToolUse, …), are preserved with Kimetsu's group added alongside. Idempotent (no duplicate groups).
  • --scope workspace|global. Install the Kimetsu surface into the workspace (default) or the user's home (~/.claude/ + ~/.claude.json, ~/.codex/) for every session. Also exposed as the scope arg on the kimetsu_plugin_install MCP tool.
  • BOM-tolerant reads. Strips a leading UTF-8 BOM before parsing an existing settings.json / hooks.json / config.toml / .mcp.json (older Windows editors no longer break the install).
  • Polish. --scope global on the workspace-only kimetsu target warns instead of silently no-op'ing; --workspace is canonicalized leniently so a global install doesn't fail on a missing path.

🐛 Stop hook fix

The Stop hook counted kimetsu_brain_record calls from a non-existent inline transcript array; Claude Code actually passes a transcript_path to a JSONL file. It now streams the JSONL line-by-line (multi-MB transcripts never load fully into memory), keeps the inline array as a fallback, and counts both the bare and MCP-namespaced (mcp__kimetsu__kimetsu_brain_record) tool names — so the "lessons recorded" banner and the end-of-session cue actually fire.

🧹 Cleaner help & tool menus

Stripped internal version-history prefixes (v0.x:, MP-…:) from --help text and the MCP tool/argument descriptions so menus read as plain present-tense descriptions (code comments unchanged).

Test Plan

  • cargo test --workspaceall green (333 tests). New coverage includes:
    • failure→resolution + harvest-throttle proactive state; count_brain_record_calls bare + namespaced shapes; [learning] / [learning.distiller] config back-compat; harvester subagent install.
    • Codex: write_codex_hooks idempotent merge + Stop --distill-on-stop + proactive PostToolUse; .codex/agents/kimetsu-memory-harvester.toml install; wizard accepts the codex harness → OpenAI (wizard_accepts_codex_harness, wizard_accepts_openai_custom_model); OpenAiProvider Responses-URL / base-URL resolution.
    • resolve_distiller precedence (workspace > global) under an env-locked temp user brain; distill_and_record GlobalUser routing; string-aware lesson extraction.
    • the full merge_claude_md matrix (fresh / preserve / idempotent / in-place upgrade / begin-without-end repair / BOM) + install_preserves_existing_user_claude_md.
  • cargo fmt --all --check clean; cargo build --workspace warning-clean after the 0.9.0 bump.
  • Real-binary smoke (Windows): PostToolUse failure→success emits the [kimetsu-harvest] cue; Stop reads a JSONL transcript and reports recorded lessons (verified with the namespaced tool name) vs cues a harvest when none; plugin install into a workspace with a pre-existing CLAUDE.md preserves the user content and is idempotent (exactly one marker block after a re-install).

Merge notes

Bundles #14 (v0.8.4) and #16 (global distiller / OpenAI provider). Merge this and close #14, or merge #14 first — the overlap reconciles automatically.

RodCor and others added 30 commits June 1, 2026 18:20
publish-npm extracted the Windows archive with unzip, but PowerShell Compress-Archive writes ZIP entries with backslash separators that ubuntu's unzip flattens into a literal filename, so the win32 binary was not found and the job failed after publishing the three Unix platform packages. Extract the Windows zip with 7z (interprets the backslash as a path separator) plus a suffix-match find fallback, and make publishing idempotent so a re-run skips already-published versions.

v0.8.2 published @kimetsu-ai/{linux-x64,darwin-x64,darwin-arm64}; the complete set incl. win32 + the main kimetsu-ai package ships in v0.8.3.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`kimetsu plugin install` now merges its hooks into existing Claude
settings.json / Codex hooks.json instead of replacing them: hooks you
already have — even on the same events Kimetsu uses (UserPromptSubmit,
PreToolUse, ...) — are preserved, with Kimetsu's group added alongside.
Re-running is idempotent (no duplicate groups) and the MCP config and
generated docs refresh without needing --force.

Adds --scope workspace|global (default workspace). Global installs the
surface into the user's home for every session: ~/.claude + ~/.claude.json
(mcpServers) for Claude Code, ~/.codex for Codex. Also exposed as the
`scope` argument on the kimetsu_plugin_install MCP tool.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tate

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PostToolUse: when a command that failed earlier this session succeeds,
cue the agent (throttled) to dispatch the kimetsu-memory-harvester
subagent. Stop: read the real transcript_path JSONL (fixes a latent bug
where the inline transcript was never present in Claude Code), count
records correctly, and cue a harvest at end of a non-trivial session
that recorded nothing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Real Claude Code JSONL transcripts carry tool_use names as
mcp__kimetsu__kimetsu_brain_record, not the bare form, so the record
count was always 0 — the 'lessons recorded' banner never fired and the
no-record harvest cue misfired even when the agent did record. Match
both bare and namespaced names; raise the non-trivial threshold to suit
line-counting; test fixtures now use the namespaced form.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hooks now drive memory *generation*, not just retrieval. When a
command that failed earlier in the session succeeds, or a non-trivial
session ends with nothing recorded, Kimetsu cues the agent to dispatch a
background kimetsu-memory-harvester subagent (installed at .claude/agents,
pinned to a cheap model) that distills generalizable lessons and records
them through the confidence-gated kimetsu_brain_record path. No separate
API key or kimetsu-side model credentials; in-agent and non-blocking;
toggle with [learning] auto_harvest. Also fixes the Stop hook to read the
real transcript_path JSONL and match the MCP-namespaced record tool name,
and strips internal version-history prefixes from --help and the MCP tool
menus.

Bundles the v0.8.4 plugin-install work (hook merge + --scope) this branch
is stacked on.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lish

- BOM tolerance (#1): strip a leading UTF-8 BOM before parsing existing
  settings.json/hooks.json/config.toml/.mcp.json, so a config saved by a
  BOM-emitting editor (older Windows Notepad) no longer fails install with
  'expected value at line 1 column 1'.
- Bounded transcript read (#7): the Stop hook now streams the transcript
  JSONL line-by-line (count_transcript_jsonl) instead of loading the whole
  file, so a long session's multi-MB transcript never lands in memory.
- Install polish (#3): document that --force overwrites an existing
  CLAUDE.md (incl. the global one); warn instead of silently ignoring
  --scope global for the workspace-only kimetsu target; canonicalize
  --workspace leniently so a global install doesn't fail on a missing path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RodCor and others added 15 commits June 2, 2026 18:43
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-hook

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… a parent repo

The wizard located its config dir via ProjectPaths::discover(), which walks up
to the nearest enclosing git repository. For a user whose home (or any parent)
is a git repo and who installs into a non-git subdir, that wrote the distiller
config + secret .env into the PARENT repo instead of the install workspace.

Add ProjectPaths::at_root() (non-climbing) and use it for the wizard. Rewrite
apply_distiller_config to write project.toml directly (load-or-default) instead
of init_project (which also climbs and opens the brain DB). Now the wizard
writes strictly under --workspace.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@RodCor
RodCor force-pushed the feat/auto-harvest-memories branch from d89fd54 to 93c34cc Compare June 3, 2026 02:21
RodCor and others added 6 commits June 2, 2026 23:22
… replace)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…gate

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@RodCor
RodCor merged commit 021200f into main Jun 3, 2026
12 checks passed
@RodCor
RodCor deleted the feat/auto-harvest-memories branch June 3, 2026 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant