Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ Two harness details are load-bearing rather than incidental:

**The indexing hook and a doctor check are coupled through a file.** `sync-memories.mts` writes `.claude/.kb-index/memory-loop.log` when a run fails or leaves documents unembedded, and deletes it on success; the "Memory indexing completed" check reports the file's existence. Move or rename it on one side and the check passes forever without testing anything.

**Memory retrievals are recorded in `.claude/memories/.usage.tsv`, and the file is a contract with the shared-memories pack.** A `get`, `multi_get` or `Read` of a memory sets its row to today's UTC date. The file has a `# since:` header and one line per memory: `<file>\t<YYYY-MM-DD>`. It lives inside `memories/` so it travels with the memories however a team shares them. Shared-memories allowlists the path in its naming guardrail and merges it with git's `union` driver. Union keeps both sides' lines, so the format must stay correct under any mix of lines: readers take the latest date per file, the earliest `since`, and drop orphan, malformed and future-dated rows. Retrievals only ever **append** a line, and only when the memory's latest row is not already today, because a read-merge-rewrite lost a third of the rows when parallel `get` calls ran together. Compaction to the normalized form runs on `UserPromptSubmit`, and only once the file holds more than `max(USAGE_COMPACT_FLOOR, 2 × distinct memories)` lines, so a compaction commit is rare. Before renaming, it carries over rows appended while it ran. The hook is the only writer, and the audit must never edit the file. Reads in a session where memory-audit ran, or in a turn where continuous-learning ran, are not counted, since those reads come from reviewing or saving memories, not from using one. The data has no consumer yet: no audit rule acts on it until enough has been collected to choose one.

**The index is reached by `--index`, never by a project-local `.qmd/`.** Two reasons, and the second is the dangerous one. A user may keep their own `.qmd/` at the project root for their own code, which this pack must not touch. And a project-local `.qmd/index.yml` falls under qmd's trust gate, which covers a non-default `models.embed` — for a non-interactive caller the gate does not prompt or fail, it *skips*, silently substituting a much weaker default model. Named indexes are never gated. `QMD_CONFIG_DIR` and `INDEX_PATH` are what move a named index back under the project directory.

**Reranking and query expansion are disabled by pointing their model slots at the embedding model.** The MCP `query` tool hard-defaults `rerank: true` with no server-side way to turn it off, and a *missing* model is downloaded mid-query with no progress output. An embedding model has no ranking head, so qmd fails to build a ranking context, warns, and falls back to RRF — measured at MRR 0.792 against 0.800 for an explicit `rerank: false`, and it buys back zero R@5 versus a real reranker. Those numbers come from a 20-query fixture over this project's own memories, kept outside the repo — nothing here reproduces them, so treat them as recorded measurements rather than something CI re-checks. The fallback is not a *strict* no-op: on a later fixture, a document RRF ranked 9th was dropped entirely when `rerank: true` was passed. Top ranks were unaffected, which is why this is a safe structural disable, but do not describe it as "reranking is off and nothing changes".
Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,12 @@ memory/

**[shared-memories](https://github.com/mcs-cli/shared-memories)** extends The Memory Loop by syncing `.claude/memories/` across teammates through a dedicated Git repository. Install both for team-shared memory: `mcs-cli/memory` captures and retrieves knowledge, while `shared-memories` distributes it.

The loop records the day each memory was last retrieved in `.claude/memories/.usage.tsv`, which travels with the memories. `shared-memories` sets up its merge rule for you. If your team commits `.claude/memories/` in the project repo instead, add this line to `.gitattributes` so concurrent updates merge without conflicts:

```
.claude/memories/.usage.tsv merge=union
```

## You might also like

| Pack | Description |
Expand Down
122 changes: 109 additions & 13 deletions hooks/kb-gate.mts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ const MEMORY_WRITE_TOOLS = ["Write", "Edit", "NotebookEdit"];
const MAX_MEMORY_WRITE_DENIALS_PER_TURN = 3;
const MEMORY_DELETE = /(?:^|[^\w-])(?:rm|unlink|mv)(?=\s)/;
const SLASH_SKILL = new RegExp(`(?:^|[\\s>])/(?:[\\w-]+:)?(${Object.keys(MEMORY_SKILL_SCOPES).join("|")})\\b`, "g");
const RETRIEVAL_TOOLS = ["mcp__memory-loop__get", "mcp__memory-loop__multi_get", "Read"];
// qmd heads every returned document with this line, which is the only place a docid or glob resolves to a file.
const RETRIEVED_RESOURCE = /\[Resource from memory-loop at qmd:\/\/memories\/([^\]\s]+\.md)\]/g;
const DAY = /^\d{4}-\d{2}-\d{2}$/;
const USAGE_COMPACT_FLOOR = 50;
const TODAY = new Date().toISOString().slice(0, 10);

function memorySkill(name: unknown): string | undefined {
const skill = String(name ?? "").replace(/^\//, "").split(":").at(-1) ?? "";
Expand All @@ -26,9 +32,75 @@ function realPath(path: string): string {
return attempt(() => realpathSync(path)) ?? join(attempt(() => realpathSync(dirname(path))) ?? dirname(path), basename(path));
}

function mentionsMemories(path: string): boolean {
const lexical = resolve(path);
return lexical.includes("memories") || realPath(lexical).includes("memories");
}

function readIfExists(file: string): string | undefined {
return existsSync(file) ? attempt(() => read(file)) : "";
}

function rows(file: string): string[][] | undefined {
return readIfExists(file)?.split("\n").filter(Boolean).map(line => line.split("\t"));
}

function isMemory(memories: string, name: string): boolean {
return name.endsWith(".md") && attempt(() => statSync(`${memories}/${name}`).isFile()) === true;
}

function parseUsage(content: string): { since: string; dates: Map<string, string>; lines: number } {
const dates = new Map<string, string>();
let since = "";
const lines = content.split("\n").filter(Boolean);
for (const [first, date, ...rest] of lines.map(line => line.split("\t"))) {
const header = /^# since: (\S+)$/.exec(first!)?.[1];
if (header && DAY.test(header) && header <= TODAY) {
if (!since || header < since) since = header;
continue;
}
// A future date would pin a memory as fresh forever, so a skewed clock's row is dropped, not trusted.
if (!date || rest.length > 0 || !DAY.test(date) || date > TODAY) continue;
if (date > (dates.get(first!) ?? "")) dates.set(first!, date);
}
return { since, dates, lines: lines.length };
}

function recordRetrievals(usage: string, retrieved: string[]): string[] {
const content = readIfExists(usage) ?? "";
const { dates } = parseUsage(content);
const fresh = retrieved.filter(name => dates.get(name) !== TODAY);
if (fresh.length === 0) return fresh;
const header = content ? (content.endsWith("\n") ? "" : "\n") : `# since: ${TODAY}\n`;
// One append per hook, never a rewrite, so concurrent hooks cannot erase each other's rows.
appendFileSync(usage, `${header}${fresh.map(name => `${name}\t${TODAY}\n`).join("")}`);
return fresh;
}

function compactUsage(usage: string, memories: string, state: string): void {
const content = readIfExists(usage);
if (!content || content.split("\n").length <= USAGE_COMPACT_FLOOR) return;
const { since, dates, lines } = parseUsage(content);
if (lines <= Math.max(USAGE_COMPACT_FLOOR, 2 * dates.size)) return;
const kept = [...dates.keys()].filter(name => isMemory(memories, name)).sort();
const compacted = `# since: ${since || TODAY}\n${kept.map(name => `${name}\t${dates.get(name)}\n`).join("")}`;
// Staged outside memories/ so the shared-memories autopush never stages a half-written temp file.
const temporary = `${state}/usage.${process.pid}.tmp`;
try {
// Rows appended while this ran are carried over; a file rewritten under us is left for the next turn.
const latest = read(usage);
if (!latest.startsWith(content)) return;
writeFileSync(temporary, compacted + latest.slice(content.length));
renameSync(temporary, usage);
} finally {
attempt(() => rmSync(temporary, { force: true }));
}
}

function gate(): void {
if (MODE === "off") return;
const payload = JSON.parse(read(0));
const stdin = read(0);
const payload = JSON.parse(stdin);
const event = payload.hook_event_name || "";
const session = String(payload.session_id ?? "unknown").replace(/[^A-Za-z0-9._-]/g, "_");
const agentId = payload.agent_id || "";
Expand All @@ -38,15 +110,17 @@ function gate(): void {
if (!["UserPromptSubmit", "PostToolUse", "PreToolUse", "SubagentStart"].includes(event)) return;
const isSkillCall = event === "PostToolUse" && payload.tool_name === "Skill";
if (isSkillCall && !memorySkill(payload.tool_input?.skill)) return;
const isRetrieval = event === "PostToolUse" && RETRIEVAL_TOOLS.includes(payload.tool_name);
const isMemoryWrite = event === "PreToolUse" && MEMORY_WRITE_TOOLS.includes(payload.tool_name);
const target = payload.tool_input?.file_path || payload.tool_input?.notebook_path || "";
if (isMemoryWrite && !`${resolve(target)}\n${realPath(resolve(target))}`.includes("memories")) return;
if (target && !mentionsMemories(target)) return;
const command = String(payload.tool_input?.command ?? "");
const isMemoryDelete = event === "PreToolUse" && payload.tool_name === "Bash" && command.includes("memories") && MEMORY_DELETE.test(command);
if (event === "PreToolUse" && payload.tool_name === "Bash" && !isMemoryDelete) return;

const root = projectRoot();
const memories = `${root}/.claude/memories`;
const usage = `${memories}/.usage.tsv`;
const state = `${root}/.claude/.kb-gate`;
const log = `${root}/.claude/.kb-gate.log`;
const turnFile = `${state}/${session}.turn`;
Expand All @@ -64,11 +138,6 @@ function gate(): void {
process.stdout.write(`${JSON.stringify({ hookSpecificOutput: { hookEventName: event, ...fields } })}\n`);
}

function rows(file: string): string[][] | undefined {
const content = existsSync(file) ? attempt(() => read(file)) : "";
return content?.split("\n").filter(Boolean).map(line => line.split("\t"));
}

function denials(phase: string): string[][] | undefined {
return rows(denialsFile)?.filter(fields => Number(fields[0]) === turn && fields[1] === phase);
}
Expand All @@ -91,6 +160,18 @@ function gate(): void {
}
}

function activeMemorySkill(includeTurnScoped: boolean): string | undefined {
return rows(skillsFile)?.find(([at, name]) => MEMORY_SKILL_SCOPES[name!] === "session" || (includeTurnScoped && MEMORY_SKILL_SCOPES[name!] === "turn" && Number(at) === turn))?.[1];
}

function memoryName(path: string): string | undefined {
const lexical = resolve(root, path);
if (lexical.startsWith(`${memories}/`)) return lexical.slice(memories.length + 1);
const real = realPath(lexical);
const memoriesReal = attempt(() => realpathSync(memories)) ?? memories;
return real.startsWith(`${memoriesReal}/`) ? real.slice(memoriesReal.length + 1) : undefined;
}

if (event === "UserPromptSubmit") {
mkdirSync(state, { recursive: true });
if (turn === 0) attempt(() => sweep(state));
Expand All @@ -102,6 +183,7 @@ function gate(): void {
renameSync(`${state}/log.tmp`, log);
});
attempt(() => rmSync(`${state}/log.tmp`, { force: true }));
attempt(() => compactUsage(usage, memories, state));
attempt(() => writeFileSync(denialsFile, ""));
attempt(() => writeFileSync(turnFile, `${turn + 1}\n`));
logEvent({ decision: "turn_start", turn: turn + 1 });
Expand All @@ -119,6 +201,23 @@ function gate(): void {
return;
}

if (isRetrieval) {
if (!isDirectory(memories)) return;
const names = target
? [memoryName(target)]
: [...stdin.matchAll(RETRIEVED_RESOURCE)].map(match => match[1]);
const retrieved = [...new Set(names)].filter((name): name is string => !!name && isMemory(memories, name));
if (retrieved.length === 0) return;
const skill = activeMemorySkill(true);
if (skill) {
logEvent({ tool: payload.tool_name, decision: "skip", skip_reason: "memory_skill", skill });
return;
}
const fresh = recordRetrievals(usage, retrieved);
if (fresh.length > 0) logEvent({ tool: payload.tool_name, decision: "retrieved", files: fresh });
return;
}

if (event === "PostToolUse") {
const input = payload.tool_input;
const query = (input?.query || (input?.searches ?? []).map((search: { query?: string }) => search.query ?? "").join(" ")).replace(/\n+$/, "");
Expand Down Expand Up @@ -157,11 +256,8 @@ mcp__memory-loop__query.
const phase = isMemoryWrite ? "memory_write" : "memory_delete";
let subject = command.slice(0, 200);
if (isMemoryWrite) {
const lexical = resolve(root, target);
subject = realPath(lexical);
const memoriesReal = attempt(() => realpathSync(memories)) ?? memories;
const inside = (path: string, directory: string) => path.startsWith(`${directory}/`);
if (!inside(lexical, memories) && !inside(subject, memoriesReal)) return;
subject = realPath(resolve(root, target));
if (memoryName(target) === undefined) return;
}
function logMemory(decision: string, extra = {}): void {
logEvent({ phase, tool: payload.tool_name, target: subject, agent_type: agentType, decision, ...extra });
Expand All @@ -171,7 +267,7 @@ mcp__memory-loop__query.
output({ permissionDecision: "deny", permissionDecisionReason: "Sub-agents cannot change .claude/memories/. Report the proposed memory change back instead; the main thread applies it through the continuous-learning or memory-audit skill, which run the search, pre-write and approval checks." });
return;
}
const skill = rows(skillsFile)?.find(([at, name]) => MEMORY_SKILL_SCOPES[name!] === "session" || (isMemoryWrite && MEMORY_SKILL_SCOPES[name!] === "turn" && Number(at) === turn))?.[1];
const skill = activeMemorySkill(isMemoryWrite);
if (skill) {
logMemory("allow", { skill });
return;
Expand Down
12 changes: 12 additions & 0 deletions techpack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,18 @@ components:
source: hooks/kb-gate.mts
destination: kb-gate.mts

- id: hook-kb-gate-retrieval
displayName: KB gate — record memory retrievals
description: Records the day each memory was last retrieved in .claude/memories/.usage.tsv, so a team can see which memories are still read
hookInterpreter: node --experimental-strip-types --disable-warning=ExperimentalWarning
hookEvent: PostToolUse
hookMatcher: "mcp__memory-loop__get|mcp__memory-loop__multi_get|Read"
# Every Read starts this hook, and it never needs to block, so it stays off the turn's critical path.
hookAsync: true
hook:
source: hooks/kb-gate.mts
destination: kb-gate.mts

- id: hook-kb-gate-memory-write
displayName: KB gate — check memory writes
description: Blocks writes, deletes and moves in .claude/memories/ unless a memory skill is active
Expand Down
5 changes: 3 additions & 2 deletions tests/contracts.test.mts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ for (const type of ["commonjs", "module"]) {
const f = fixture(t);
writeFileSync(`${f.project}/package.json`, JSON.stringify({ type }));
const hooks = manifest.split(/^ - id: /m).filter(component => /\n hookEvent:/.test(component));
assert.equal(hooks.length, 9);
assert.equal(hooks.length, 10);
assert.match(manifest, /hook:\n source: hooks\/shared.mts\n destination: shared.mts/);
const installed = `${f.project}/.claude/hooks/memory`;
mkdirSync(installed, { recursive: true });
Expand All @@ -42,8 +42,9 @@ for (const type of ["commonjs", "module"]) {
assert.match(manifest, /hookMatcher: "Agent\|Task"/);
assert.match(manifest, /hookMatcher: "mcp__memory-loop__query"/);
assert.match(manifest, /hookMatcher: "Skill"/);
assert.match(manifest, /hookMatcher: "mcp__memory-loop__get\|mcp__memory-loop__multi_get\|Read"/);
assert.match(manifest, /hookMatcher: "Write\|Edit\|NotebookEdit\|Bash"/);
assert.equal((manifest.match(/hookAsync: true/g) ?? []).length, 2);
assert.equal((manifest.match(/hookAsync: true/g) ?? []).length, 3);
assert.equal((manifest.match(/hookTimeout: 120/g) ?? []).length, 2);
});
}
Expand Down
Loading
Loading