diff --git a/README.md b/README.md index de42975..bfd1c5b 100644 --- a/README.md +++ b/README.md @@ -63,6 +63,14 @@ The embedding module now uses raw `fetch` instead of the `openai` SDK, making it - Ollama, llama.cpp, vLLM (local models) - Any endpoint that implements `POST /embeddings` +### Operational safeguards + +Three small safeguards make graph-memory cheaper and safer to run in busy OpenClaw deployments: + +- **Readonly subagent/helper sessions**: subagents and short-lived helper sessions can still inherit recall context, but they no longer write noisy long-term memory into the shared graph. This keeps maintenance focused on human-facing sessions instead of ephemeral worker chatter. +- **Permanent-error LLM cooldown**: repeated `400/401/403/404/422` failures from `config.llm` now trigger a temporary cooldown instead of hammering the provider every turn. This turns broken credentials or disabled accounts into a contained failure instead of runaway token spend. +- **Community summary reuse**: community summaries are now keyed by member signatures. If a community has not changed, graph-memory skips regeneration; if the same member set reappears under a different community id, it reuses the cached summary and embedding. This cuts unnecessary summary LLM calls without changing recall quality. + ### Windows one-click installer v2.0 ships a **Windows installer** (`.exe`). Download from [Releases](https://github.com/adoresever/graph-memory/releases): @@ -144,6 +152,7 @@ assemble (zero LLM) afterTurn (async, non-blocking) ├─ LLM extracts triples → gm_nodes + gm_edges ├─ Every 7 turns: PageRank + community detection + community summaries + │ └─ unchanged communities reuse cached summaries/embeddings └─ User sends new message → extract auto-interrupted session_end @@ -152,6 +161,7 @@ session_end Next session → before_prompt_build ├─ Dual-path recall (precise + generalized) + ├─ Subagent/helper sessions stay recall-only └─ Personalized PageRank ranking → inject into context ``` @@ -323,6 +333,7 @@ sqlite3 ~/.openclaw/graph-memory.db "SELECT id, summary FROM gm_communities;" | `recall` works but `gm_messages` is empty | `plugins.slots.contextEngine` not set | Add `"contextEngine": "graph-memory"` to `plugins.slots` | | `FTS5 search mode` instead of `vector search ready` | Embedding not configured or API key invalid | Check `config.embedding` credentials | | `No LLM available` error | LLM config missing after plugin reinstall | Re-add `config.llm` to `plugins.entries.graph-memory` | +| Repeated `LLM API 403/404/422` errors | Broken account, credentials, or provider-side permanent failure | Fix the provider config; graph-memory now enters a temporary cooldown instead of retrying every turn | | No `extracted` log after `afterTurn` | Gateway restart caused turn_index overlap | Update to v2.0 (fixes msgSeq persistence) | | `content.filter is not a function` | OpenClaw expects array content | Update to v2.0 (adds content normalization) | | Nodes are empty after many messages | `compactTurnCount` not reached | Default is 7 messages. Keep chatting or set a lower value | diff --git a/README_CN.md b/README_CN.md index 1d1f335..e69400d 100644 --- a/README_CN.md +++ b/README_CN.md @@ -65,6 +65,14 @@ Embedding 模块改用原生 `fetch` 替代 `openai` SDK,开箱即用兼容** - Ollama、llama.cpp、vLLM(本地模型) - 任何实现了 `POST /embeddings` 的端点 +### 运行期护栏 + +下面三个护栏让 graph-memory 在复杂 OpenClaw 部署里更省钱,也更不容易失控: + +- **subagent / helper session 只读化**:子代理和临时 helper 仍然可以继承 recall 结果,但不会再把噪音写进共享图谱。这样长期记忆更聚焦在面向用户的主会话,而不是短命 worker 的中间过程。 +- **永久性 LLM 错误冷却**:`config.llm` 如果连续返回 `400/401/403/404/422` 这类永久性错误,graph-memory 会进入短暂冷却,而不是每轮继续轰炸 provider。坏处不是“继续烧 token”,而是变成明确、可恢复的故障信号。 +- **社区摘要去重与复用**:社区摘要现在按成员签名缓存。成员没变就跳过重算;同一组成员换了社区 id 也可以复用已有摘要和 embedding。在不影响召回质量的前提下,大幅减少重复摘要调用。 + ### Windows 一键安装包 v2.0 提供 **Windows 安装包**(`.exe`)。从 [Releases](https://github.com/adoresever/graph-memory/releases) 页面下载: @@ -146,6 +154,7 @@ assemble(零 LLM) afterTurn(后台异步,不阻塞用户对话) ├─ LLM 提取三元组 → gm_nodes + gm_edges ├─ 每 7 轮:PageRank + 社区检测 + 社区摘要生成 + │ └─ 未变化的社区复用已有摘要/embedding └─ 用户发新消息时自动中断提取 session_end @@ -154,6 +163,7 @@ session_end 下次新对话 → before_prompt_build ├─ 双路径召回(精确 + 泛化) + ├─ subagent / helper session 只读消费 recall └─ 个性化 PageRank 排序 → 注入上下文 ``` @@ -325,6 +335,7 @@ sqlite3 ~/.openclaw/graph-memory.db "SELECT id, summary FROM gm_communities;" | `recall` 正常但 `gm_messages` 为空 | 没设置 `plugins.slots.contextEngine` | 在 `plugins.slots` 中添加 `"contextEngine": "graph-memory"` | | 显示 `FTS5 search mode` | Embedding 未配置或 API Key 无效 | 检查 `config.embedding` 的密钥和地址 | | `No LLM available` 错误 | 重装插件后 LLM 配置丢失 | 重新添加 `config.llm` 到 `plugins.entries.graph-memory` | +| 持续出现 `LLM API 403/404/422` 错误 | 账号、凭证或 provider 侧出现永久性故障 | 修复 provider 配置;graph-memory 现在会进入临时冷却,而不是每轮继续重试 | | `afterTurn` 后没有 `extracted` 日志 | 重启导致 turn_index 重叠 | 升级到 v2.0(修复了 msgSeq 持久化) | | `content.filter is not a function` | OpenClaw 要求 content 为数组 | 升级到 v2.0(添加了 content 规范化) | | 对话很多轮但节点为空 | 消息数未达到提取阈值 | 默认需要积累消息。继续对话或调低 `compactTurnCount` | diff --git a/index.ts b/index.ts index 62d11d6..070e993 100755 --- a/index.ts +++ b/index.ts @@ -28,6 +28,7 @@ import { sanitizeToolUseResultPairing } from "./src/format/transcript-repair.ts" import { runMaintenance } from "./src/graph/maintenance.ts"; import { invalidateGraphCache, computeGlobalPageRank } from "./src/graph/pagerank.ts"; import { detectCommunities } from "./src/graph/community.ts"; +import { ReadonlySessionRegistry } from "./src/session-policy.ts"; import { DEFAULT_CONFIG, type GmConfig } from "./src/types.ts"; // ─── 从 OpenClaw config 读 provider/model ──────────────────── @@ -158,10 +159,24 @@ const graphMemoryPlugin = { const msgSeq = new Map(); const recalled = new Map(); const turnCounter = new Map(); // 社区维护计数器 + const readonlySessions = new ReadonlySessionRegistry(); // ── 提取串行化(同 session Promise chain,不同 session 并行)──── const extractChain = new Map>(); + function isReadonlySession(sessionKey?: string): boolean { + return readonlySessions.has(sessionKey); + } + + function cleanupSessionState(sessionKey: string | undefined, forgetReadonly = false): void { + if (!sessionKey) return; + extractChain.delete(sessionKey); + msgSeq.delete(sessionKey); + recalled.delete(sessionKey); + turnCounter.delete(sessionKey); + if (forgetReadonly) readonlySessions.clear(sessionKey); + } + /** 存一条消息到 gm_messages(同步,零 LLM) */ function ingestMessage(sessionId: string, message: any): void { let seq = msgSeq.get(sessionId); @@ -245,6 +260,7 @@ const graphMemoryPlugin = { if (prompt.includes("/new or /reset") || prompt.includes("new session was started")) return; const sid = ctx?.sessionId ?? ctx?.sessionKey; + if (isReadonlySession(sid)) return; api.logger.info(`[graph-memory] recall query: "${prompt.slice(0, 80)}"`); @@ -286,6 +302,7 @@ const graphMemoryPlugin = { isHeartbeat?: boolean; }) { if (isHeartbeat) return { ingested: false }; + if (isReadonlySession(sessionId)) return { ingested: false }; ingestMessage(sessionId, message); return { ingested: true }; }, @@ -301,7 +318,7 @@ const graphMemoryPlugin = { tokenBudget?: number; prompt?: string; // Added in OpenClaw 2026.03.28: prompt-aware retrieval }) { - const activeNodes = getBySession(db, sessionId); + const activeNodes = isReadonlySession(sessionId) ? [] : getBySession(db, sessionId); const activeEdges = activeNodes.flatMap((n) => [ ...edgesFrom(db, n.id), ...edgesTo(db, n.id), @@ -378,6 +395,10 @@ const graphMemoryPlugin = { force?: boolean; currentTokenCount?: number; }) { + if (isReadonlySession(sessionId)) { + return { ok: true, compacted: false, reason: "readonly session" }; + } + // compact 仍然保留作为兜底,但主要提取在 afterTurn 完成 const msgs = getUnextracted(db, sessionId, 50); @@ -444,6 +465,7 @@ const graphMemoryPlugin = { tokenBudget?: number; }) { if (isHeartbeat) return; + if (isReadonlySession(sessionId)) return; // Messages are already persisted by ingest() — only slice to // determine the new-message count for extraction triggering. @@ -503,20 +525,26 @@ const graphMemoryPlugin = { parentSessionKey: string; childSessionKey: string; }) { + readonlySessions.markReadonly(childSessionKey); const rec = recalled.get(parentSessionKey); if (rec) recalled.set(childSessionKey, rec); - return { rollback: () => { recalled.delete(childSessionKey); } }; + return { + rollback: () => { + cleanupSessionState(childSessionKey, true); + }, + }; }, async onSubagentEnded({ childSessionKey }: { childSessionKey: string }) { - recalled.delete(childSessionKey); - msgSeq.delete(childSessionKey); + cleanupSessionState(childSessionKey, true); }, async dispose() { extractChain.clear(); msgSeq.clear(); recalled.clear(); + turnCounter.clear(); + readonlySessions.clearAll(); }, }; @@ -533,6 +561,8 @@ const graphMemoryPlugin = { if (!sid) return; try { + if (isReadonlySession(sid)) return; + const nodes = getBySession(db, sid); if (nodes.length) { const summary = ( @@ -581,10 +611,7 @@ const graphMemoryPlugin = { } catch (err) { api.logger.error(`[graph-memory] session_end error: ${err}`); } finally { - extractChain.delete(sid); - msgSeq.delete(sid); - recalled.delete(sid); - turnCounter.delete(sid); + cleanupSessionState(sid, true); } }); @@ -651,6 +678,12 @@ const graphMemoryPlugin = { p: { name: string; type: string; description: string; content: string; relatedSkill?: string }, ) { const sid = ctx?.sessionKey ?? ctx?.sessionId ?? "manual"; + if (isReadonlySession(sid)) { + return { + content: [{ type: "text", text: "subagent session is running in read-only graph-memory mode." }], + details: { readonly: true, sessionKey: sid }, + }; + } const { node } = upsertNode(db, { type: p.type as any, name: p.name, description: p.description, content: p.content, @@ -704,12 +737,19 @@ const graphMemoryPlugin = { ); api.registerTool( - (_ctx: any) => ({ + (ctx: any) => ({ name: "gm_maintain", label: "Graph Memory Maintenance", description: "手动触发图维护:运行去重、PageRank 重算、社区检测。通常 session_end 时自动运行,这个工具用于手动触发。", parameters: Type.Object({}), async execute(_toolCallId: string, _params: any) { + const sid = ctx?.sessionKey ?? ctx?.sessionId; + if (isReadonlySession(sid)) { + return { + content: [{ type: "text", text: "subagent session is running in read-only graph-memory mode." }], + details: { readonly: true, sessionKey: sid }, + }; + } const embedFn = (recaller as any).embed ?? undefined; const result = await runMaintenance(db, cfg, llm, embedFn); const text = [ diff --git a/src/engine/llm-guard.ts b/src/engine/llm-guard.ts new file mode 100644 index 0000000..883b7aa --- /dev/null +++ b/src/engine/llm-guard.ts @@ -0,0 +1,39 @@ +const RETRYABLE_STATUSES = new Set([429, 500, 502, 503, 529]); +const PAUSING_STATUSES = new Set([400, 401, 403, 404, 422]); + +export function extractLlmStatus(error: unknown): number | null { + const text = String(error ?? ""); + const match = text.match(/\bLLM API (\d{3})\b/); + if (!match) return null; + return Number(match[1]); +} + +export class LlmFailureGuard { + private pausedUntil = 0; + + constructor( + private readonly cooldownMs = 10 * 60_000, + private readonly now = () => Date.now(), + ) {} + + canRun(): boolean { + return this.now() >= this.pausedUntil; + } + + remainingMs(): number { + return Math.max(0, this.pausedUntil - this.now()); + } + + reset(): void { + this.pausedUntil = 0; + } + + tripIfNeeded(error: unknown): boolean { + const status = extractLlmStatus(error); + if (status == null || RETRYABLE_STATUSES.has(status) || !PAUSING_STATUSES.has(status)) { + return false; + } + this.pausedUntil = Math.max(this.pausedUntil, this.now() + this.cooldownMs); + return true; + } +} diff --git a/src/engine/llm.ts b/src/engine/llm.ts index de7fd48..786d0a6 100755 --- a/src/engine/llm.ts +++ b/src/engine/llm.ts @@ -14,6 +14,8 @@ * 内置:429/5xx 重试 3 次 + 30s 超时 */ +import { LlmFailureGuard } from "./llm-guard.ts"; + export interface LlmConfig { apiKey?: string; baseURL?: string; @@ -52,51 +54,70 @@ export function createCompleteFn( llmConfig?: LlmConfig, anthropicApiKey?: string, ): CompleteFn { + const guard = new LlmFailureGuard(); + return async (system, user) => { - // ── 路径 A(优先):pluginConfig.llm 直接调 OpenAI 兼容 API ── - if (llmConfig?.apiKey && llmConfig?.baseURL) { - const baseURL = llmConfig.baseURL.replace(/\/+$/, ""); - const llmModel = llmConfig.model ?? model; - const res = await fetchRetry(`${baseURL}/chat/completions`, { + if (!guard.canRun()) { + const seconds = Math.max(1, Math.ceil(guard.remainingMs() / 1000)); + throw new Error( + `[graph-memory] LLM paused for ${seconds}s after a previous permanent API error`, + ); + } + + try { + // ── 路径 A(优先):pluginConfig.llm 直接调 OpenAI 兼容 API ── + if (llmConfig?.apiKey && llmConfig?.baseURL) { + const baseURL = llmConfig.baseURL.replace(/\/+$/, ""); + const llmModel = llmConfig.model ?? model; + const res = await fetchRetry(`${baseURL}/chat/completions`, { + method: "POST", + headers: { + "Content-Type": "application/json", + "Authorization": `Bearer ${llmConfig.apiKey}`, + }, + body: JSON.stringify({ + model: llmModel, + messages: [ + ...(system.trim() ? [{ role: "system", content: system.trim() }] : []), + { role: "user", content: user }, + ], + temperature: 0.1, + }), + }); + if (!res.ok) { + const errText = await res.text().catch(() => ""); + throw new Error(`[graph-memory] LLM API ${res.status}: ${errText.slice(0, 200)}`); + } + const data = await res.json() as any; + const text = data.choices?.[0]?.message?.content ?? ""; + if (!text) throw new Error("[graph-memory] LLM returned empty content"); + guard.reset(); + return text; + } + + // ── 路径 B:Anthropic API ────────────────────────────── + if (!anthropicApiKey) { + throw new Error( + "[graph-memory] No LLM available. 在 openclaw.json 的 graph-memory config 中配置 llm.apiKey + llm.baseURL", + ); + } + const res = await fetchRetry("https://api.anthropic.com/v1/messages", { method: "POST", - headers: { - "Content-Type": "application/json", - "Authorization": `Bearer ${llmConfig.apiKey}`, - }, - body: JSON.stringify({ - model: llmModel, - messages: [ - ...(system.trim() ? [{ role: "system", content: system.trim() }] : []), - { role: "user", content: user }, - ], - temperature: 0.1, - }), + headers: { "Content-Type": "application/json", "x-api-key": anthropicApiKey, "anthropic-version": "2023-06-01" }, + body: JSON.stringify({ model: llmConfig?.model ?? model, max_tokens: 4096, system, messages: [{ role: "user", content: user }] }), }); - if (!res.ok) { - const errText = await res.text().catch(() => ""); - throw new Error(`[graph-memory] LLM API ${res.status}: ${errText.slice(0, 200)}`); - } + if (!res.ok) throw new Error(`[graph-memory] Anthropic API ${res.status}`); const data = await res.json() as any; - const text = data.choices?.[0]?.message?.content ?? ""; - if (text) return text; - throw new Error("[graph-memory] LLM returned empty content"); - } - - // ── 路径 B:Anthropic API ────────────────────────────── - if (!anthropicApiKey) { - throw new Error( - "[graph-memory] No LLM available. 在 openclaw.json 的 graph-memory config 中配置 llm.apiKey + llm.baseURL", - ); + const text = data.content?.[0]?.text ?? ""; + if (!text) throw new Error("[graph-memory] Anthropic API returned empty content"); + guard.reset(); + return text; + } catch (error) { + if (guard.tripIfNeeded(error)) { + const seconds = Math.max(1, Math.ceil(guard.remainingMs() / 1000)); + throw new Error(`${String(error)}; pausing graph-memory LLM calls for ${seconds}s`); + } + throw error; } - const res = await fetchRetry("https://api.anthropic.com/v1/messages", { - method: "POST", - headers: { "Content-Type": "application/json", "x-api-key": anthropicApiKey, "anthropic-version": "2023-06-01" }, - body: JSON.stringify({ model: llmConfig?.model ?? model, max_tokens: 4096, system, messages: [{ role: "user", content: user }] }), - }); - if (!res.ok) throw new Error(`[graph-memory] Anthropic API ${res.status}`); - const data = await res.json() as any; - const text = data.content?.[0]?.text ?? ""; - if (text) return text; - throw new Error("[graph-memory] Anthropic API returned empty content"); }; -} \ No newline at end of file +} diff --git a/src/graph/community.ts b/src/graph/community.ts index 4382fec..e2a1b24 100755 --- a/src/graph/community.ts +++ b/src/graph/community.ts @@ -24,8 +24,15 @@ * - kg_stats 展示社区分布 */ +import { createHash } from "node:crypto"; import { DatabaseSync, type DatabaseSyncInstance } from "@photostructure/sqlite"; -import { updateCommunities } from "../store/store.ts"; +import { + getCommunitySummary, + getCommunitySummaryBySignature, + pruneCommunitySummaries, + updateCommunities, + upsertCommunitySummary, +} from "../store/store.ts"; export interface CommunityResult { labels: Map; @@ -168,7 +175,6 @@ export function getCommunityPeers(db: DatabaseSyncInstance, nodeId: string, limi import type { CompleteFn } from "../engine/llm.ts"; import type { EmbedFn } from "../engine/embed.ts"; -import { upsertCommunitySummary, pruneCommunitySummaries } from "../store/store.ts"; const COMMUNITY_SUMMARY_SYS = `你是知识图谱摘要引擎。根据节点列表,用简短的描述概括这组节点的主题领域。 要求: @@ -176,6 +182,10 @@ const COMMUNITY_SUMMARY_SYS = `你是知识图谱摘要引擎。根据节点列 - 描述涵盖的工具/技术/任务领域 - 不要使用"社区"这个词`; +function buildCommunityMemberSignature(memberIds: string[]): string { + return createHash("sha1").update([...memberIds].sort().join(",")).digest("hex"); +} + /** * 为所有社区生成 LLM 摘要描述 + embedding 向量 * @@ -192,6 +202,25 @@ export async function summarizeCommunities( for (const [communityId, memberIds] of communities) { if (memberIds.length === 0) continue; + const memberSignature = buildCommunityMemberSignature(memberIds); + + const current = getCommunitySummary(db, communityId); + if (current?.memberSignature === memberSignature && current.summary.trim()) { + continue; + } + + const reusable = getCommunitySummaryBySignature(db, memberSignature); + if (reusable?.summary.trim()) { + upsertCommunitySummary( + db, + communityId, + reusable.summary, + memberIds.length, + reusable.embedding, + memberSignature, + ); + continue; + } const placeholders = memberIds.map(() => "?").join(","); const members = db.prepare(` @@ -238,7 +267,7 @@ export async function summarizeCommunities( } } - upsertCommunitySummary(db, communityId, cleaned, memberIds.length, embedding); + upsertCommunitySummary(db, communityId, cleaned, memberIds.length, embedding, memberSignature); generated++; } catch (err) { console.log(` [WARN] community summary failed for ${communityId}: ${err}`); @@ -246,4 +275,4 @@ export async function summarizeCommunities( } return generated; -} \ No newline at end of file +} diff --git a/src/session-policy.ts b/src/session-policy.ts new file mode 100644 index 0000000..92336cd --- /dev/null +++ b/src/session-policy.ts @@ -0,0 +1,56 @@ +/** + * graph-memory — session policy + * + * Subagent sessions should be able to consume inherited recall context + * without writing new long-term memory into the shared graph. + */ + +function normalizeSessionKey(sessionKey?: string | null): string { + return sessionKey?.trim() ?? ""; +} + +export function isHelperSessionKey(sessionKey?: string | null): boolean { + const normalized = normalizeSessionKey(sessionKey).toLowerCase(); + if (!normalized) return false; + return ( + normalized.startsWith("temp:") || + normalized.startsWith("slug-generator-") || + normalized === "slug-gen" + ); +} + +export function isSubagentSessionKey(sessionKey?: string | null): boolean { + const normalized = normalizeSessionKey(sessionKey).toLowerCase(); + if (!normalized) return false; + return normalized.startsWith("subagent:") || normalized.includes(":subagent:"); +} + +export class ReadonlySessionRegistry { + private readonlySessions = new Set(); + + markReadonly(sessionKey?: string | null): void { + const normalized = normalizeSessionKey(sessionKey); + if (!normalized) return; + this.readonlySessions.add(normalized); + } + + clear(sessionKey?: string | null): void { + const normalized = normalizeSessionKey(sessionKey); + if (!normalized) return; + this.readonlySessions.delete(normalized); + } + + has(sessionKey?: string | null): boolean { + const normalized = normalizeSessionKey(sessionKey); + if (!normalized) return false; + return ( + this.readonlySessions.has(normalized) || + isSubagentSessionKey(normalized) || + isHelperSessionKey(normalized) + ); + } + + clearAll(): void { + this.readonlySessions.clear(); + } +} diff --git a/src/store/db.ts b/src/store/db.ts index 60b24cd..81f3aab 100755 --- a/src/store/db.ts +++ b/src/store/db.ts @@ -5,6 +5,7 @@ * Email: Wywelljob@gmail.com */ +import { createHash } from "node:crypto"; import { DatabaseSync, type DatabaseSyncInstance } from "@photostructure/sqlite"; import { mkdirSync } from "fs"; import { homedir } from "os"; @@ -51,7 +52,16 @@ export function closeDb(): void { function migrate(db: DatabaseSyncInstance): void { db.exec(`CREATE TABLE IF NOT EXISTS _migrations (v INTEGER PRIMARY KEY, at INTEGER NOT NULL)`); const cur = (db.prepare("SELECT MAX(v) as v FROM _migrations").get() as any)?.v ?? 0; - const steps = [m1_core, m2_messages, m3_signals, m4_fts5, m5_vectors, m6_communities]; + const steps = [ + m1_core, + m2_messages, + m3_signals, + m4_fts5, + m5_vectors, + m6_communities, + m7_community_signature, + m8_backfill_community_signatures, + ]; for (let i = cur; i < steps.length; i++) { steps[i](db); db.prepare("INSERT INTO _migrations (v,at) VALUES (?,?)").run(i + 1, Date.now()); @@ -180,12 +190,49 @@ function m5_vectors(db: DatabaseSyncInstance): void { function m6_communities(db: DatabaseSyncInstance): void { db.exec(` CREATE TABLE IF NOT EXISTS gm_communities ( - id TEXT PRIMARY KEY, - summary TEXT NOT NULL, - node_count INTEGER NOT NULL DEFAULT 0, - embedding BLOB, - created_at INTEGER NOT NULL, - updated_at INTEGER NOT NULL + id TEXT PRIMARY KEY, + summary TEXT NOT NULL, + node_count INTEGER NOT NULL DEFAULT 0, + embedding BLOB, + member_signature TEXT, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL ); `); } + +function m7_community_signature(db: DatabaseSyncInstance): void { + const cols = db.prepare("PRAGMA table_info(gm_communities)").all() as Array<{ name?: string }>; + const hasMemberSignature = cols.some((col) => col.name === "member_signature"); + if (!hasMemberSignature) { + db.exec("ALTER TABLE gm_communities ADD COLUMN member_signature TEXT"); + } + db.exec("CREATE INDEX IF NOT EXISTS ix_gm_communities_member_signature ON gm_communities(member_signature)"); +} + +function m8_backfill_community_signatures(db: DatabaseSyncInstance): void { + const missing = db.prepare(` + SELECT id FROM gm_communities + WHERE member_signature IS NULL OR member_signature='' + `).all() as Array<{ id: string }>; + + for (const row of missing) { + const members = db.prepare(` + SELECT id FROM gm_nodes + WHERE community_id=? AND status='active' + ORDER BY id + `).all(row.id) as Array<{ id: string }>; + + if (!members.length) continue; + + const memberSignature = createHash("sha1") + .update(members.map((member) => member.id).join(",")) + .digest("hex"); + + db.prepare(` + UPDATE gm_communities + SET member_signature=?, updated_at=updated_at + WHERE id=? + `).run(memberSignature, row.id); + } +} diff --git a/src/store/store.ts b/src/store/store.ts index ec472a9..97f8712 100755 --- a/src/store/store.ts +++ b/src/store/store.ts @@ -531,39 +531,85 @@ export interface CommunitySummary { id: string; summary: string; nodeCount: number; + memberSignature: string | null; createdAt: number; updatedAt: number; } export function upsertCommunitySummary( - db: DatabaseSyncInstance, id: string, summary: string, nodeCount: number, embedding?: number[], + db: DatabaseSyncInstance, + id: string, + summary: string, + nodeCount: number, + embedding?: number[] | Uint8Array, + memberSignature?: string, ): void { const now = Date.now(); - const blob = embedding ? new Uint8Array(new Float32Array(embedding).buffer) : null; + const blob = embedding + ? embedding instanceof Uint8Array + ? embedding + : new Uint8Array(new Float32Array(embedding).buffer) + : null; const ex = db.prepare("SELECT id FROM gm_communities WHERE id=?").get(id) as any; if (ex) { if (blob) { - db.prepare("UPDATE gm_communities SET summary=?, node_count=?, embedding=?, updated_at=? WHERE id=?") - .run(summary, nodeCount, blob, now, id); + db.prepare("UPDATE gm_communities SET summary=?, node_count=?, embedding=?, member_signature=?, updated_at=? WHERE id=?") + .run(summary, nodeCount, blob, memberSignature ?? null, now, id); } else { - db.prepare("UPDATE gm_communities SET summary=?, node_count=?, updated_at=? WHERE id=?") - .run(summary, nodeCount, now, id); + db.prepare("UPDATE gm_communities SET summary=?, node_count=?, member_signature=?, updated_at=? WHERE id=?") + .run(summary, nodeCount, memberSignature ?? null, now, id); } } else { - db.prepare("INSERT INTO gm_communities (id, summary, node_count, embedding, created_at, updated_at) VALUES (?,?,?,?,?,?)") - .run(id, summary, nodeCount, blob, now, now); + db.prepare("INSERT INTO gm_communities (id, summary, node_count, embedding, member_signature, created_at, updated_at) VALUES (?,?,?,?,?,?,?)") + .run(id, summary, nodeCount, blob, memberSignature ?? null, now, now); } } export function getCommunitySummary(db: DatabaseSyncInstance, id: string): CommunitySummary | null { const r = db.prepare("SELECT * FROM gm_communities WHERE id=?").get(id) as any; if (!r) return null; - return { id: r.id, summary: r.summary, nodeCount: r.node_count, createdAt: r.created_at, updatedAt: r.updated_at }; + return { + id: r.id, + summary: r.summary, + nodeCount: r.node_count, + memberSignature: r.member_signature ?? null, + createdAt: r.created_at, + updatedAt: r.updated_at, + }; +} + +export function getCommunitySummaryBySignature( + db: DatabaseSyncInstance, + memberSignature: string, +): (CommunitySummary & { embedding?: Uint8Array }) | null { + const r = db.prepare(` + SELECT * FROM gm_communities + WHERE member_signature=? + ORDER BY updated_at DESC + LIMIT 1 + `).get(memberSignature) as any; + if (!r) return null; + return { + id: r.id, + summary: r.summary, + nodeCount: r.node_count, + memberSignature: r.member_signature ?? null, + embedding: r.embedding ? (r.embedding as Uint8Array) : undefined, + createdAt: r.created_at, + updatedAt: r.updated_at, + }; } export function getAllCommunitySummaries(db: DatabaseSyncInstance): CommunitySummary[] { return (db.prepare("SELECT * FROM gm_communities ORDER BY node_count DESC").all() as any[]) - .map(r => ({ id: r.id, summary: r.summary, nodeCount: r.node_count, createdAt: r.created_at, updatedAt: r.updated_at })); + .map(r => ({ + id: r.id, + summary: r.summary, + nodeCount: r.node_count, + memberSignature: r.member_signature ?? null, + createdAt: r.created_at, + updatedAt: r.updated_at, + })); } export type ScoredCommunity = { id: string; summary: string; score: number; nodeCount: number }; @@ -640,4 +686,4 @@ export function pruneCommunitySummaries(db: DatabaseSyncInstance): number { ) `).run(); return result.changes; -} \ No newline at end of file +} diff --git a/test/graph.test.ts b/test/graph.test.ts index d70dbae..6d16482 100755 --- a/test/graph.test.ts +++ b/test/graph.test.ts @@ -11,7 +11,7 @@ import { describe, it, expect, beforeEach } from "vitest"; import { DatabaseSync, type DatabaseSyncInstance } from "@photostructure/sqlite"; import { createTestDb, insertNode, insertEdge } from "./helpers.ts"; import { personalizedPageRank, computeGlobalPageRank, invalidateGraphCache } from "../src/graph/pagerank.ts"; -import { detectCommunities, getCommunityPeers } from "../src/graph/community.ts"; +import { detectCommunities, getCommunityPeers, summarizeCommunities } from "../src/graph/community.ts"; import { detectDuplicates, dedup } from "../src/graph/dedup.ts"; import { runMaintenance } from "../src/graph/maintenance.ts"; import { saveVector } from "../src/store/store.ts"; @@ -194,6 +194,26 @@ describe("Community Detection", () => { const { count } = detectCommunities(db); expect(count).toBe(0); }); + + it("reuses existing summaries when a community is unchanged", async () => { + const a = insertNode(db, { name: "docker-build" }); + const b = insertNode(db, { name: "docker-push" }); + insertEdge(db, { fromId: a, toId: b }); + + const { communities } = detectCommunities(db); + let llmCalls = 0; + const llm = async () => { + llmCalls += 1; + return "docker deployment skills"; + }; + + const first = await summarizeCommunities(db, communities, llm); + const second = await summarizeCommunities(db, communities, llm); + + expect(first).toBe(1); + expect(second).toBe(0); + expect(llmCalls).toBe(1); + }); }); // ═══════════════════════════════════════════════════════════════ @@ -298,4 +318,4 @@ describe("runMaintenance", () => { expect(result.pagerank.topK).toHaveLength(0); expect(result.community.count).toBe(0); }); -}); \ No newline at end of file +}); diff --git a/test/helpers.ts b/test/helpers.ts index aa3eace..90ebe29 100755 --- a/test/helpers.ts +++ b/test/helpers.ts @@ -113,6 +113,19 @@ export function createTestDb(): DatabaseSyncInstance { ); `); + // m6: 社区摘要 + db.exec(` + CREATE TABLE IF NOT EXISTS gm_communities ( + id TEXT PRIMARY KEY, + summary TEXT NOT NULL, + node_count INTEGER NOT NULL DEFAULT 0, + embedding BLOB, + member_signature TEXT, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL + ); + `); + return db; } @@ -176,4 +189,4 @@ export function insertEdge( "test-session", Date.now(), ); -} \ No newline at end of file +} diff --git a/test/llm-guard.test.ts b/test/llm-guard.test.ts new file mode 100644 index 0000000..ac1e86d --- /dev/null +++ b/test/llm-guard.test.ts @@ -0,0 +1,29 @@ +import { describe, expect, it } from "vitest"; + +import { LlmFailureGuard } from "../src/engine/llm-guard.ts"; + +describe("LlmFailureGuard", () => { + it("pauses after permanent 4xx API errors", () => { + let now = 1_000; + const guard = new LlmFailureGuard(60_000, () => now); + + expect(guard.canRun()).toBe(true); + expect( + guard.tripIfNeeded(new Error('[graph-memory] LLM API 403: {"error":"User not found or inactive"}')), + ).toBe(true); + expect(guard.canRun()).toBe(false); + + now += 59_000; + expect(guard.canRun()).toBe(false); + + now += 2_000; + expect(guard.canRun()).toBe(true); + }); + + it("ignores retryable errors", () => { + const guard = new LlmFailureGuard(60_000, () => 1_000); + + expect(guard.tripIfNeeded(new Error("[graph-memory] LLM API 429: rate limited"))).toBe(false); + expect(guard.canRun()).toBe(true); + }); +}); diff --git a/test/session-policy.test.ts b/test/session-policy.test.ts new file mode 100644 index 0000000..cea9445 --- /dev/null +++ b/test/session-policy.test.ts @@ -0,0 +1,47 @@ +import { describe, expect, it } from "vitest"; + +import { + ReadonlySessionRegistry, + isHelperSessionKey, + isSubagentSessionKey, +} from "../src/session-policy.ts"; + +describe("subagent session detection", () => { + it("detects OpenClaw subagent session keys", () => { + expect(isSubagentSessionKey("agent:minimax-clerk:subagent:0cc464e3-3244-4443-9dbf-cea199b73abb")).toBe(true); + expect(isSubagentSessionKey("subagent:one-shot")).toBe(true); + expect(isSubagentSessionKey("agent:main:feishu:default:direct:ou_df0924becc2951992502da488004bf1d")).toBe(false); + expect(isSubagentSessionKey("temp:slug-generator")).toBe(false); + }); +}); + +describe("helper session detection", () => { + it("detects helper session keys", () => { + expect(isHelperSessionKey("temp:slug-generator")).toBe(true); + expect(isHelperSessionKey("slug-generator-1775243719190")).toBe(true); + expect(isHelperSessionKey("slug-gen")).toBe(true); + expect(isHelperSessionKey("agent:main:feishu:default:direct:ou_df0924becc2951992502da488004bf1d")).toBe(false); + }); +}); + +describe("readonly session registry", () => { + it("tracks explicit readonly child sessions", () => { + const registry = new ReadonlySessionRegistry(); + + expect(registry.has("agent:main:task:1")).toBe(false); + + registry.markReadonly("agent:main:task:1"); + expect(registry.has("agent:main:task:1")).toBe(true); + + registry.clear("agent:main:task:1"); + expect(registry.has("agent:main:task:1")).toBe(false); + }); + + it("treats helper sessions as readonly by default", () => { + const registry = new ReadonlySessionRegistry(); + + expect(registry.has("temp:slug-generator")).toBe(true); + expect(registry.has("slug-generator-1775243719190")).toBe(true); + expect(registry.has("slug-gen")).toBe(true); + }); +});