diff --git a/docs/AGENT-RUNTIME.md b/docs/AGENT-RUNTIME.md index cea0bda1b..b53f01fd7 100644 --- a/docs/AGENT-RUNTIME.md +++ b/docs/AGENT-RUNTIME.md @@ -80,7 +80,7 @@ MUST: `reflex` tool set MUST be a subset of: `task_list`, `task_get`, `memory_save`, `memory_search`, `notify_user`, `request_user_input`, `schedule_wakeup`, `cancel_wakeup`, `set_heartbeat_interval`, `discover_tools`, -`check_mailbox`, `file_read`, `agent_send_message`, `update_notebook`, +`check_mailbox`, `file_read`, `agent_send_message`, `notebook_upsert`, and for managers additionally `team_status`. MUST NOT: Include `package_install`, `package_list`, `goal_create`, `goal_update`, `goal_status`, diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index d305d0238..e26a09463 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -15,8 +15,8 @@ and cross-referenced elsewhere. Start here, then follow the map below. | Document | Domain (single responsibility) | |----------|-------------------------------| | [ARCHITECTURE.md](./ARCHITECTURE.md) *(this file)* | System overview, package structure, core concepts, channels, deployment, observability | -| [COGNITIVE-ARCHITECTURE.md](./COGNITIVE-ARCHITECTURE.md) | Unified cognitive cycle, Cognitive Preparation Pipeline (CPP), heartbeat integration | -| [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md) | Memory layers (ROLE / MEMORY / session / notebook / activity), storage compaction, memory flush | +| [COGNITIVE-ARCHITECTURE.md](./COGNITIVE-ARCHITECTURE.md) | Unified cognitive cycle, deterministic context assembly, heartbeat integration | +| [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md) | Memory layers (ROLE / knowledge / session / notebook / activity), storage compaction, memory flush | | [PROMPT-ENGINEERING.md](./PROMPT-ENGINEERING.md) | Prompt & context assembly, LLM call taxonomy, context packing, prompt caching | | [MAILBOX-SYSTEM.md](./MAILBOX-SYSTEM.md) | Agent mailbox (priority queue) + attention controller (serial focus, interrupts, yield, cancel) | | [STATE-MACHINES.md](./STATE-MACHINES.md) | FSMs for tasks, requirements, callbacks, mailbox items, notebook | @@ -28,6 +28,7 @@ and cross-referenced elsewhere. Start here, then follow the map below. | [GUIDE.md](./GUIDE.md) | Setup, deployment, and usage guide | | [REMOTE-ACCESS.md](./REMOTE-ACCESS.md) | Remote access configuration | | [RELEASE-AND-DISTRIBUTION.md](./RELEASE-AND-DISTRIBUTION.md) | Release process and distribution | +| [agent-liveness-redesign.md](../packages/org-manager/docs/agent-liveness-redesign.md) | 存活/自愈机制审计记录(心跳风暴根因 + 重构 1–4 完成状态);现行设计并入本文件 §3.10 | ### 0.2 Relationship Graph @@ -64,7 +65,7 @@ flowchart TD PROMPT -->|packing triggers flush| MEM PROMPT -->|tool defs and results| TOOLS TOOLS -->|tool errors surfaced as events| STREAM - COG -->|CPP writes notebook| MEM + PROMPT -->|deterministic bounded recall| MEM ``` ### 0.3 Cross-Document Conventions @@ -104,7 +105,7 @@ Markus is an **AI Digital Workforce Platform** that lets organizations hire, man ┌──▼─────────▼──────────▼─────────▼───────────▼───────────────┐ │ Agent Runtime (@markus/core) │ │ Agent · Mailbox · AttentionController · ContextEngine │ -│ Notebook · Memory · CognitivePreparation · PendingCallback │ +│ Notebook · Memory · PendingCallback │ │ Goal/Loop · LLMRouter · Heartbeat · Tools · MCP · Review │ └──────────────────────────┬──────────────────────────────────┘ │ @@ -148,8 +149,8 @@ Each Agent consists of: | `SKILLS.md` | Skill list (tool permissions) | | `HEARTBEAT.md` | Scheduled proactive tasks (e.g. daily issue checks) | | `POLICIES.md` | Behavior rules and boundaries | -| `NOTEBOOK.md` | Persistent cognitive workspace (situational state, CPP output) | -| `knowledge.md` (+ `NOTEBOOK.md`) | Long-term knowledge + working state. `state.md` retired 2026-09-16 (legacy `MEMORY.md` migrates once into `knowledge.md`) | +| `NOTEBOOK.md` | Persistent cognitive workspace (situational state, triage output) | +| `knowledge.md` (+ `NOTEBOOK.md`) | Long-term knowledge + working state. `knowledge.md` is the only write target; `MEMORY.md` / `memories.json` / `state.md` are legacy read-only sources migrated in at load (`state.md` → `state.md.migrated`) | | `CONTEXT.md` | Organization context (shared knowledge base) | The runtime also supports **spawning lightweight LLM subagents** (`spawn_subagent` / `spawn_subagents`) for delegated subtasks. Subagent limits (parallelism, retry policy, preview truncation) are centralized in `packages/shared/src/limits.ts` rather than hardcoded. The parent agent has a **configurable tool-use iteration limit** (`AgentOptions.maxToolIterations`, system settings; default 200, range 1–10000) on chat-style harnesses — task execution and subagent loops remain uncapped by default. @@ -186,7 +187,7 @@ Agents now have tools to actively manage their mailbox queue and cognitive works - `check_mailbox` (read-only inspection, all scenarios) - `defer_mailbox_item` / `drop_mailbox_item` (queue management) -- `update_notebook` / `clear_notebook` (cognitive workspace management) +- `notebook_upsert` / `notebook_clear` / `notebook_read` (cognitive workspace management) This shifts from system-driven to agent-driven cognition. The deliberation threshold is lowered to 2 items, making agent-driven triage the norm. @@ -207,23 +208,22 @@ See [MAILBOX-SYSTEM.md](./MAILBOX-SYSTEM.md) for the complete design. ### 3.2.1 Cognitive System -The agent cognitive system is a continuous cycle backed by persistent stores and optional deliberate preparation: +The agent cognitive system is a continuous cycle backed by persistent stores and deterministic context assembly: ``` -Stimulus (Mailbox) → Triage → [CPP optional] → Context Assembly → Main LLM → Action → Reflection +Stimulus (Mailbox) → Triage → Deterministic Context Assembly → Main LLM → Action ↓ - NOTEBOOK.md + MEMORY.md + knowledge.md (injected) + NOTEBOOK.md (working state) ``` | Component | Storage / Location | Role | |-----------|-------------------|------| -| **Notebook** | `NOTEBOOK.md` | Persistent cognitive workspace — situational state, triage decisions, CPP outputs | -| **Memory** | `MEMORY.md` | Unified long-term knowledge (curated sections) + raw `## _observations` buffer | -| **CPP** | `packages/core/src/cognitive.ts` | Opt-in multi-phase context preparation (Appraisal → Retrieval → Reflection) | +| **Notebook** | `NOTEBOOK.md` | Persistent cognitive workspace — situational state, triage decisions, working-memory keys | +| **Memory** | `knowledge.md` | Unified long-term knowledge (curated sections) + raw `## _observations` buffer | | **Goal/Loop** | `GoalConfig` on Requirements | Persistent objectives with heartbeat integration | | **PendingCallbackRegistry** | `packages/core/src/pending-callback.ts` | Async operation tracking; completions → mailbox `callback_result` | -**Cognitive Preparation Pipeline (CPP)** — opt-in via `agent.cognitive.enabled` in `markus.json` (default `false`). When enabled, CPP runs between triage and the main LLM call, writing outputs to `NOTEBOOK.md` (not separate prompt sections). Four depth levels (D0–D3) control preparation intensity: D0 reflexive (heartbeat OK), D1 reactive (most chats/A2A), D2 deliberative (task execution), D3 meta-cognitive (high-stakes). +**Deterministic context assembly** — the former Cognitive Preparation Pipeline (CPP: the 0–3 pre-call LLM phases appraise / retrieve / reflect, plus `cognitive.ts`, `CognitivePreparation`, `selectCognitiveDepth` and `CognitiveDepth`) has been **removed**. Between triage and the main LLM call the ContextEngine injects a small, bounded, LLM-free situational block (recent activity + working-memory keys) together with its own deterministic relevant-memory retrieval. Deeper cross-domain recall is agent-driven via `memory_search` / `kb_search`. Prompt sections produced today: `## Cognitive Context` (deterministic) and `## Relevant Memories`; `## Retrieved Context` and `## Reflection` are no longer produced. **Goal/Loop mechanism** — Requirements can carry a `GoalConfig` (`loopEnabled`, `completionCriteria`, `maxIterations`, etc.) turning them into standing objectives. Heartbeat injects active goals; agents manage them via `goal_create`, `goal_update`, and `goal_status` tools. @@ -251,12 +251,12 @@ Organization (Org) ### 3.4 Memory and Knowledge System -**Two-file cognitive model** replaces the former volatile working memory + `memories.json` system: +**Two-file cognitive model** replaces the former volatile working memory + `memories.json` system (`memories.json` / `MEMORY.md` / `state.md` are legacy read-only sources, migrated into `knowledge.md` at load): | File | Role | Prompt injection | |------|------|-----------------| -| **`NOTEBOOK.md`** | Persistent cognitive workspace — situational state, CPP/triage outputs | Always loaded as `## Notebook` | -| **`MEMORY.md`** | Curated long-term knowledge + raw `## _observations` buffer | Curated sections as `## Your Knowledge`; observations excluded | +| **`NOTEBOOK.md`** | Persistent cognitive workspace — situational state, triage/working-memory outputs | Always loaded as `## Notebook` | +| **`knowledge.md`** | Curated long-term knowledge + raw `## _observations` buffer | Curated sections as `## Your Knowledge`; observations excluded | The **dream cycle** (`memory_consolidation`) operates within `knowledge.md` — consolidating observations into curated sections and pruning stale content. Post-task learning uses a separate `distillation` scenario ([LEARNING-LOOP.md](./LEARNING-LOOP.md) §0 / §2). @@ -265,9 +265,9 @@ The **dream cycle** (`memory_consolidation`) operates within `knowledge.md` — | Layer | Storage | Role | |-------|---------|------| | **Procedural** | `role/ROLE.md` + skills | How the agent operates. Identity, behavioral rules. | -| **Semantic** | `MEMORY.md` curated sections | What the agent knows. Agent-organized knowledge. | +| **Semantic** | `knowledge.md` curated sections | What the agent knows. Agent-organized knowledge. | | **Episodic** | `sessions/*.json` (current) + SQLite `agent_activities` (past) | What happened. Current conversation + searchable activity history. | -| **Working Memory** | `NOTEBOOK.md` | Persistent, agent-managed keyed entries (`update_notebook` / `clear_notebook`). | +| **Working Memory** | `NOTEBOOK.md` | Persistent, agent-managed keyed entries (`notebook_upsert` / `notebook_clear` / `notebook_read`). | The agent retrieves past episodes via the `recall_activity` tool (keyword search on summary/keywords). Daily logs (`daily-logs/`) are a write-only audit trail for humans — never read back into prompts. @@ -278,8 +278,8 @@ See [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md) for the complete architecture. | Scope | Description | Tools | |-------|-------------|-------| | `personal` | Agent personal memory | `memory_save` / `memory_search` | -| `project` | Project-level shared knowledge | `knowledge_contribute` / `knowledge_search` | -| `org` | Org-level shared knowledge | `knowledge_search` (scope=org) | +| `project` | Project-level shared knowledge | `kb_search` / `kb_read` | +| `org` | Org-level shared knowledge | `kb_search` (scope=org) | Knowledge categories: `architecture`, `convention`, `api`, `decision`, `gotcha`, `troubleshooting`, `dependency`, `process`, `reference` @@ -290,7 +290,7 @@ Knowledge categories: `architecture`, `convention`, `api`, `decision`, `gotcha`, | Tool | Description | |------|-------------| | `shell_execute` | Run shell commands (auto-injects Agent identity into git commit) | -| `file_read` / `file_write` / `file_edit` | File read/write/edit (writes blocked only to other agents' directories) | +| `file_read` / `file_write` / `file_edit` | File read/write/edit (writes blocked to other agents' directories and to single-writer memory files — `knowledge.md` / `NOTEBOOK.md` / `state.md` must be changed via the memory tools) | | `file_list` | List directory contents | | `web_fetch` / `web_search` | HTTP requests / web search | | `spawn_subagent` / `spawn_subagents` | Spawn lightweight LLM subagents for focused subtasks (parallel support) | @@ -388,8 +388,8 @@ Before each conversation, the ContextEngine dynamically builds the system prompt 7. **System announcements** (urgent/high-priority announcements) 8. **Human feedback** (annotations and instructions from report reviews) 9. **Project knowledge highlights** (high-importance verified knowledge entries) -10. **Your Knowledge** (MEMORY.md curated sections — observations excluded) -11. **Notebook** (NOTEBOOK.md — cognitive workspace; CPP outputs land here when enabled) +10. **Your Knowledge** (knowledge.md curated sections — observations excluded; a health banner is appended once usage reaches ≥70%) +11. **Notebook** (NOTEBOOK.md — cognitive workspace; agent-managed working state) 12. Active Goals (when heartbeat or goal-aware context) 13. Task board (currently assigned Tasks) 14. Current conversation identity (sender info) @@ -427,6 +427,24 @@ Client touchpoints: Frozen response-field contract (keep in sync with Hub handlers): [`packages/core/test/hub-billing-contract.test.ts`](../packages/core/test/hub-billing-contract.test.ts) — mirrors Hub `billing-crossflows` plan + `cu/sync` keys (`remainingCu`, `openrouter.remainingUsd`, `planSource`, buckets, etc.). +### 3.10 Agent 存活 / 自愈机制(Liveness SSOT + Conservator 统一仲裁) + +**设计主线一句话:单一存活事实源 + 一条有界安全网 + 收敛式状态机,其余全部去重。** + +背景(审计记录:[agent-liveness-redesign.md](../packages/org-manager/docs/agent-liveness-redesign.md)):dirty / stale / stall 三个独立判定器各自为政、兜底回路没有统一仲裁和上限,曾导致「每 2 分钟一次」的心跳风暴(单 agent 数小时 600+ 条定时心跳签到)。重构 1–4 将机制收敛如下: + +1. **Liveness SSOT**:`agents.last_heartbeat` 由 core 单向写入(每轮心跳完成/跳过时,含 skip 路径),所有观察者统一读它(实时读 live-view,落列供跨进程/诊断)。心跳时间戳是唯一「活着」的证据。 +2. **心跳语义收紧**:心跳 = 存活上报 + 低频巡检(默认 6h);**skip 路径(human-chat defer / idle / deep-sleep)收敛为纯时间戳**(`recordHeartbeatSkip()`,永不调用 LLM),LLM 巡检仅在状态实际变化时发生(`heartbeatStateFingerprint()` 指纹:队列内容签名 + activeTaskIds 签名)。所有触发心跳路径都收口到 `heartbeat:trigger` → `agent.ts` 唯一 handler → `mailbox.enqueue('heartbeat')` → 单条折叠处理。 +3. **单一安全网仲裁器(Conservator)**:`packages/org-manager/src/agent-conservator.ts` 把 dirty / stale / stall 三种判定**收敛为单一组件**(重构 4 起判定原语内联,组件彻底自包含): + - `evaluateConservator`(纯函数)融合脏判定 + 卡死判定(dead-dependency、stale-heartbeat)+ 过期判定(心跳/活动新鲜度),输出**唯一动作阶梯**:`ok → observe → wake → reconcile → human-review`; + - `AgentConservator`(周期仲裁引擎,30s 轮询,守卫启动)施加**指数退避**(base·2^(n−1),封顶 8h)+ **单 episode 总次数上限** + **收敛证明**(未脱离 processing-like 不复位 episode;human-review 每 episode 只通知一次并停止自动动作); + - Fix A 合流:连续 3 次 trigger-heartbeat 无果 → 升级 human-review(`CONSERVATOR_MAX_WAKE_ATTEMPTS=3`),从机制上排除「每 2 分钟一次」周期解; + - 展示路径经 `evaluateConservator` 透出 `runtime.stall` / `runtime.dirty`(形状与旧前端完全兼容),只读派生不回写。 +4. **收敛式状态机**:core 中 27 处散落 `setStatus` 写入点全部收敛为单一意图化派生函数 `transitionStatus` + 落地 `applyStatus`(`packages/core/src/agent.ts`)——error 粘性(error 后 idle 不覆盖)、聚合状态守卫(activeTasks>0 / 并发 worker 忙碌 → idle 被拒)、force 强制兜底、reset 清错、offline 无条件;心跳巡检 / 保守仲裁 / Normal 转换统一走该函数,消除竞态覆盖。 +5. **持久化通路补全**:状态 payload 携带全部权威字段(status / lastHeartbeat / activeTaskIds / currentActivity / lastError),一次回调写全;`SqliteAgentRepo.updateLastHeartbeat` 落库。 + +旧独立模块(`agent-dirty.ts` / `agent-stall.ts` / `agent-dirty-reconciler.ts`)已于重构 4 删除,判定逻辑全部收敛进 Conservator / core 状态机。相关回归测试:`packages/org-manager/test/agent-conservator.test.ts`(19 用例:stuck-working 心跳风暴、心跳宽限、指数退避、上限收敛、degraded/dead-dependency)、`packages/org-manager/test/agent-stall-api.test.ts`(API 展示契约)、`packages/core/test/heartbeat-liveness.test.ts`(skip 纯时间戳 + 指纹巡检)、`packages/core/test/agent-status-machine.test.ts`(状态机收敛)。 + --- ## 4. Governance Framework @@ -458,9 +476,9 @@ Agents can manage other agents' lifecycle through tools with role-based permissi ### 4.2 Workspace Isolation -Each agent has a dedicated workspace (`~/.markus/agents//workspace/`). The only hard enforcement is that agents **cannot write to other agents' directories** — this prevents cross-agent interference. All other file access (read and write) is unrestricted, allowing agents to respond to any user request. Prompt-based guidance encourages agents to work within their own workspace and use worktrees for project code. +Each agent has a dedicated workspace (`~/.markus/agents//workspace/`). The hard enforcement is that agents **cannot write to other agents' directories** (prevents cross-agent interference) and **cannot write single-writer memory files** (`knowledge.md` / `NOTEBOOK.md` / `state.md`) via `file_write` / `file_edit` / `apply_patch` / shell — those must go through the memory tools (the write gate rejects direct writes and points at the right tool). All other file access (read and write) is unrestricted, allowing agents to respond to any user request. Prompt-based guidance encourages agents to work within their own workspace and use worktrees for project code. -- The platform enforces: cross-agent write isolation (deny writes to other agents' directories) +- The platform enforces: cross-agent write isolation (deny writes to other agents' directories) + single-writer memory files (deny direct writes to `knowledge.md` / `NOTEBOOK.md` / `state.md`) - The platform provides via prompt: workspace path, project context, best-practice guidance - The agent decides: branching strategy, worktree layout, merge workflow - Workflow details like branching conventions and review process are defined by **role templates and team norms**, not by the platform diff --git a/docs/COGNITIVE-ARCHITECTURE.md b/docs/COGNITIVE-ARCHITECTURE.md index 1557056cc..e25411f56 100644 --- a/docs/COGNITIVE-ARCHITECTURE.md +++ b/docs/COGNITIVE-ARCHITECTURE.md @@ -1,10 +1,10 @@ # Cognitive Architecture: Unified Agent Cognition -This document describes the unified cognitive architecture that governs how Markus agents perceive stimuli, prepare context, deliberate, act, and learn. It replaces the previous model — mechanical prompt assembly plus volatile in-memory working memory — with a **continuous cognitive cycle** backed by persistent stores (`NOTEBOOK.md`, `knowledge.md`) and an optional **Cognitive Preparation Pipeline (CPP)** for deliberate context preparation. +This document describes the unified cognitive architecture that governs how Markus agents perceive stimuli, prepare context, deliberate, act, and learn. It is a **continuous cognitive cycle** backed by persistent stores (`NOTEBOOK.md`, `knowledge.md`). Context preparation is **deterministic** — the former optional **Cognitive Preparation Pipeline (CPP)** was removed (see §3). -> **Memory SSOT**: Prefer [`MEMORY-SYSTEM.md`](./MEMORY-SYSTEM.md) — durable knowledge is `knowledge.md`, working (short-lived) state is `NOTEBOOK.md` (`system` tier). The old `state.md` store **retired 2026-09-16**; it had a reader and a TTL pruner but no write tool, while the notebook already owns the same capability. Below, historical “MEMORY.md” references mean the older dual-store model. +> **Memory SSOT**: Prefer [`MEMORY-SYSTEM.md`](./MEMORY-SYSTEM.md) — durable knowledge is `knowledge.md`, working (short-lived) state is `NOTEBOOK.md` (`system` tier). Writing is **single-writer**: `knowledge.md` / `NOTEBOOK.md` are written only through memory tools. `MEMORY.md` / `memories.json` / `state.md` are **legacy read-only sources**, read and migrated on load (see MEMORY-SYSTEM.md → “Migration-read”). Below, historical “MEMORY.md” references mean the older dual-store model. > -> **Implementation status**: Core cycle, Notebook, knowledge/state memory, Attention Controller, Goal/Loop heartbeat integration, A2A DM channels, and `PendingCallbackRegistry` are implemented. CPP (Phases 1–3, depth D0–D3) lives in `packages/core/src/cognitive.ts`. CPP is opt-in via `agent.cognitive.enabled` in `markus.json` (default `false` until explicitly enabled). D2+ retrieval requires a `RetrievalBackend` adapter. +> **Implementation status**: Core cycle, Notebook, knowledge memory, Attention Controller, Goal/Loop heartbeat integration, A2A DM channels, and `PendingCallbackRegistry` are implemented. **CPP was removed** — `packages/core/src/cognitive.ts` no longer exists and no pre-call LLM runs (§3). --- @@ -15,22 +15,22 @@ Every agent interaction follows the same loop. Heartbeat checks keep the cycle r ``` ┌─────────────────────────────────────┐ │ Heartbeat (patrol) │ - │ goals · callbacks · failed tasks │ + │ goals · callbacks · failed tasks │ └──────────────┬──────────────────────┘ │ Stimulus ──► Triage / Appraisal ──► Context Assembly ──► Deliberation ──► Action ──► Reflection / Update (Mailbox) (AttentionController (NOTEBOOK.md + (main LLM + (tools) (memory_save, - + optional CPP) MEMORY.md) tools) notebook, dream) + deterministic) knowledge.md) tools) notebook, dream) ``` | Stage | What happens | Primary components | |-------|--------------|-------------------| | **Stimulus** | Message, task, heartbeat, A2A, callback result enters the mailbox | `Mailbox`, `AttentionController` | -| **Triage / Appraisal** | Decide what to focus on; optionally run CPP appraisal | `AttentionController`, `CognitivePreparation` | -| **Context Assembly** | Load procedural identity, curated knowledge, notebook workspace, mailbox state | `ContextEngine`, `NOTEBOOK.md`, `MEMORY.md` | +| **Triage / Appraisal** | Decide what to focus on | `AttentionController` | +| **Context Assembly** | Load procedural identity, curated knowledge, notebook workspace, mailbox state | `ContextEngine`, `NOTEBOOK.md`, `knowledge.md` | | **Deliberation** | Main LLM reasons with assembled context and available tools | `Agent`, tool loop | | **Action** | Execute tool calls (tasks, files, A2A, memory writes) | Tool handlers | -| **Reflection / Update** | Persist observations, update notebook, consolidate via dream cycle | `memory_save`, `update_notebook`, dream cycle | +| **Reflection / Update** | Persist observations, update notebook, consolidate via dream cycle | `memory_save`, `notebook_upsert`, dream cycle | The cycle is **continuous**: heartbeat patrols re-enter the loop, checking active goals, timed-out callbacks, and stalled work even when the mailbox is quiet. @@ -44,70 +44,63 @@ The architecture maps directly to established cognitive models: | Model | Concept | Markus mapping | |-------|---------|----------------| -| **Baddeley — Working Memory** | Central executive coordinates subsidiary systems; episodic buffer integrates sources | `NOTEBOOK.md` = central executive + visuospatial sketchpad (active workspace); `MEMORY.md` = episodic buffer (integrates curated knowledge with experience) | -| **Cowan — Embedded Processes** | Focus of attention (4±1 chunks) within activated long-term memory | `## _observations` buffer = focus of attention (raw, not in prompt); curated `MEMORY.md` sections = activated LTM (always injected) | -| **Kahneman — Dual Process** | System 1 (fast, automatic) vs System 2 (slow, deliberate) | System 1 = triage, fast `memory_search`, mechanical retrieval fallback; System 2 = CPP (appraisal → retrieval → reflection) | -| **Boyd — OODA Loop** | Observe → Orient → Decide → Act | **Observe**: mailbox items; **Orient**: CPP + notebook updates; **Decide**: main LLM deliberation; **Act**: tool execution | +| **Baddeley — Working Memory** | Central executive coordinates subsidiary systems; episodic buffer integrates sources | `NOTEBOOK.md` = central executive + visuospatial sketchpad (active workspace); `knowledge.md` = episodic buffer (integrates curated knowledge with experience) | +| **Cowan — Embedded Processes** | Focus of attention (4±1 chunks) within activated long-term memory | `## _observations` buffer = focus of attention (raw, not in prompt); curated `knowledge.md` sections = activated LTM (always injected) | +| **Kahneman — Dual Process** | System 1 (fast, automatic) vs System 2 (slow, deliberate) | System 1 = triage, mechanical retrieval, bounded auto-recall; System 2 = the main LLM's deliberation plus agent-driven `memory_search` / `kb_search` | +| **Boyd — OODA Loop** | Observe → Orient → Decide → Act | **Observe**: mailbox items; **Orient**: deterministic context assembly + notebook updates; **Decide**: main LLM deliberation; **Act**: tool execution | Additional influences preserved from earlier design: -- **Tulving's memory systems**: Procedural (ROLE.md + skills), Semantic (MEMORY.md), Episodic (sessions + activity index). -- **Metacognition (Flavell)**: CPP appraisal asks "do I know enough?" before acting. +- **Tulving's memory systems**: Procedural (ROLE.md + skills), Semantic (`knowledge.md`), Episodic (sessions + activity index). +- **Metacognition (Flavell)**: the agent asks "do I know enough?" — now answered by active `memory_search` / `kb_search` rather than a pre-call appraisal LLM. - **Global Workspace Theory (Baars)**: Notebook is the broadcast workspace — selected context competes for limited prompt capacity. See [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md) for storage-layer detail. --- -## 3. Cognitive Preparation Pipeline (CPP) +## 3. Deterministic Context Assembly (CPP removed) -CPP is the System 2 layer. It runs **between triage and the main LLM call**, using 0–3 lightweight LLM calls to prepare persona-aware context before deliberation. +> **CPP removed.** The former Cognitive Preparation Pipeline — 0–3 pre-call LLM phases +> (Appraisal / Retrieval / Reflection), `packages/core/src/cognitive.ts`, +> `CognitivePreparation`, `selectCognitiveDepth`, the `CognitiveDepth` levels D0–D3 and the +> `agent.cognitive` depth/model fields — has been **removed**. It cost extra LLM calls per turn +> and duplicated the ContextEngine's own retrieval. + +Context preparation is now **deterministic (no LLM)**: between triage and the main call the +ContextEngine assembles a small, bounded situational block. ``` -Stimulus → Triage → CPP (optional) → Context Assembly → Main LLM - │ - ├─ Phase 1: Appraisal ("What is this? What do I need?") - ├─ Phase 2: Retrieval (directed queries — no LLM) - └─ Phase 3: Reflection ("What does this mean for me?") +Stimulus → Triage → Deterministic Context Assembly → Main LLM + │ + ├─ situational block: recent activity + working-memory keys + ├─ bounded relevant-memory retrieval (## Relevant Memories) + └─ optional deterministic ## Cognitive Context block ``` -### Output destination: Notebook, not prompt sections - -When the notebook writer is active (normal agent runtime), CPP output is written directly to **`NOTEBOOK.md`** entries rather than injected as separate system-prompt sections: +### Output destination: prompt only (the notebook dual-write was removed) -| CPP phase | Notebook key | Managed tag | -|-----------|--------------|-------------| -| Appraisal | `cognitive-context` | `cpp` | -| Retrieval | `relevant-context` | `cpp` | -| Reflection | `reflection` | `cpp` | +CPP output used to be written to **`NOTEBOOK.md`** entries (via a `notebookWriter` callback). That +**dual-destination was removed**: relevance-matched memories are injected **for the current turn only** +as `## Relevant Memories`, and the deterministic situational block is injected as `## Cognitive Context`. +Neither is persisted as a notebook entry. -Mechanical memory retrieval (when CPP is off or at D0) writes `relevant-context` with managed tag `system`. Triage decisions write `triage-decision` (`system`). +Prompt sections produced today: `## Cognitive Context` (deterministic) and `## Relevant Memories` +(`## Retrieved Context` / `## Reflection` are no longer produced). -### Cognitive depth levels +Triage decisions still write `triage-decision` (managed tag `system`) — that is runtime state, not CPP output. -| Level | Name | Phases | When | Extra LLM calls | -|-------|------|--------|------|-----------------| -| D0 | Reflexive | None | Heartbeat OK, memory consolidation | 0 | -| D1 | Reactive | Appraisal | Most chats, A2A, comments | 0–1 | -| D2 | Deliberative | Appraisal + Retrieval + Reflection | Task execution, complex questions | 2 | -| D3 | Meta-cognitive | Full pipeline + post-response eval (future) | High-stakes, blockers, novel situations | 2–3 | +### Cognitive depth levels (retired) -Depth is selected by scenario (`selectCognitiveDepth`) and clamped by `maxDepth` config. Triage outcome influences the effective depth (e.g., failed tasks upgrade heartbeat from D0 → D1). +The D0–D3 depth ladder (`selectCognitiveDepth`) was part of CPP and is **gone**. There is no +depth selection and no per-scenario LLM budget anymore — assembly is always the same cheap, +deterministic step. Any residual `CognitiveDepth` enum value in shared types is inert. ### Configuration -Opt-in via settings (`markus.json` → `agent.cognitive`, REST `/api/settings/agent`): - -```typescript -interface CognitiveConfig { - enabled: boolean; // master switch (default: false) - maxDepth?: CognitiveDepth; // cap depth during rollout (default allows D1) - appraisalModel?: string; // model for appraisal/reflection (defaults to cheapest) - timeoutMs?: number; // CPP timeout (default: 15000) -} -``` - -When CPP times out or errors, the agent falls back to mechanical retrieval without blocking the cycle. +The `agent.cognitive.enabled` flag (`markus.json` → `agent.cognitive`, REST `/api/settings/agent`) +survives and gates whether the deterministic situational block is assembled. The pipeline's +depth / appraisal-model fields are inert — no LLM phase runs, so there is nothing to time out. --- @@ -125,26 +118,25 @@ The Notebook is the agent's **persistent cognitive workspace** — Baddeley's ce | Tag | Writer | Purpose | TTL | |-----|--------|---------|-----| -| `agent` | Agent via `update_notebook` / `clear_notebook` | Explicit notes, priorities, blockers | 96h | +| `agent` | Agent via `notebook_upsert` / `notebook_clear` | Explicit notes, priorities, blockers | 96h | | `system` | Runtime (triage, deliberation, mechanical retrieval) | Triage decisions, fallback context | 24h | -| `cpp` | Cognitive Preparation Pipeline | Appraisal, retrieval, reflection outputs | 6h | **Lifecycle**: Loaded at startup → **normalized** (TTL + caps, persisted) → updated in-process → persisted with a 2s debounce bounded by a 10s maxWait → survives restarts. -**Limits**: 16 entries total (4 of them `agent`-managed), 6000 chars each, 6000 chars for the whole injected block. TTL per tier as above. Eviction is oldest-first; the machine-written tiers (`cpp`, then `system`) are evicted **before** the `agent` tier, because situational state is cheaper to lose than the agent's own deliberate notes — and it expires on its own soon anyway. +**Limits**: 16 entries total (4 of them `agent`-managed), 6000 chars each, 6000 chars for the whole injected block. TTL per tier as above. Eviction is oldest-first; the machine-written `system` tier is evicted **before** the `agent` tier, because situational state is cheaper to lose than the agent's own deliberate notes — and it expires on its own soon anyway. -The Notebook holds *situational* state. Durable knowledge flows to `knowledge.md` via `memory_save` / `memory_update`. +The Notebook holds *situational* state. Durable knowledge flows to `knowledge.md` via `memory_save` / `memory_update` (`notebook_read` is the read-only view). > **Why the notebook needs its own lifecycle** (see [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md) §2): the Notebook is a *resident* prompt region, unlike `## _observations` which is retrieved on demand. Resident regions must be bounded (count), decayed (TTL), deduplicated (key discipline), size-capped (per-entry + total), and made **single-writer** — otherwise every feature that writes to them accretes forever. All four were missing or partial here: three of four writers bypassed the entry cap, there was no TTL anywhere, and the load path trimmed nothing, so a real notebook grew to 26 entries / 33 KB including month-old situational state re-injected every turn. --- -## 5. Memory (MEMORY.md) +## 5. Memory (knowledge.md) Unified long-term store for curated knowledge plus a raw observation buffer — Cowan's activated LTM plus focus-of-attention staging area. ``` -MEMORY.md +knowledge.md ├── ## conventions ← agent-organized curated sections (in prompt) ├── ## procedures ├── ... @@ -156,7 +148,11 @@ MEMORY.md | Curated sections | Distilled knowledge the agent maintains | Always injected as `## Your Knowledge` (via the volatile tail, not the byte-stable system prefix) | | `## _observations` | Raw observations from `memory_save` | Excluded from prompt; processed by dream cycle | -The **dream cycle** (`memory_consolidation`) consolidates observations into curated sections, prunes stale content, and maintains MEMORY.md hygiene. This is the long-term learning path at the end of the cognitive cycle. +The **dream cycle** (`memory_consolidation`) consolidates observations into curated sections, prunes stale content, and maintains `knowledge.md` hygiene. This is the long-term learning path at the end of the cognitive cycle. + +**Budget is lossless.** When curated sections exceed the character budget, the overflow is **archived** (moved verbatim into `knowledge-archive.md`, still retrievable via `memory_search`) rather than truncated or refused. The budget is an invariant — the file converges to ≤100% at load. The injected `## Your Knowledge` block carries a **health banner** when usage ≥70% (budget %, observation count, section count, archived count, last consolidation time). + +**Migration-read.** `MEMORY.md` / `memories.json` / `state.md` are legacy read-only sources, read and migrated on load; only `knowledge.md` is written, and observation metadata is emitted as a single `` line (the old `, tags: a, b` form is read but converged on write). See [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md). --- @@ -173,19 +169,19 @@ Processes the **Mailbox** — the agent's unified stimulus queue. Responsibiliti - Triage decisions persisted to notebook (`triage-decision`) - Yields to higher-priority items (e.g., human chat during deliberation) -Triage decides **what** to process; CPP decides **how to prepare** for processing. +Triage decides **what** to process. How context is prepared for processing is now deterministic (§3), not a second LLM stage. ### Heartbeat Periodic patrol re-enters the cognitive cycle without external stimulus. Each heartbeat checks: -- Active goals (Goal/Loop — see §7) +- Active goals (Goal/Loop — see below) - Timed-out `PendingCallbackRegistry` entries - Failed tasks and requirement monitoring - Background operation completions - Self-evolution and quality signals -Heartbeat uses D0 (reflexive) depth unless failed tasks or blockers upgrade it. +There is no depth ladder anymore — every path, heartbeat included, uses the same deterministic assembly. #### Spec: active-hours timezone (C1) @@ -259,33 +255,32 @@ Implementation: `packages/core/src/pending-callback.ts`, persisted via `SqlitePe ## 8. Integration Map ``` -┌──────────────────────────────────────────────────────────────────┐ -│ Agent Runtime │ -│ ┌─────────────┐ ┌──────────────┐ ┌─────────────────────┐ │ -│ │ Mailbox │──►│ Attention │──►│ CPP (optional) │ │ -│ │ │ │ Controller │ │ cognitive.ts │ │ -│ └─────────────┘ └──────────────┘ └──────────┬──────────┘ │ -│ │ │ -│ ┌────────────────────────────────────────────────▼──────────┐ │ -│ │ ContextEngine │ │ -│ │ ROLE.md · MEMORY.md · NOTEBOOK.md · mailbox · goals │ │ -│ └──────────────────────────────┬───────────────────────────────┘ │ -│ ▼ │ -│ Main LLM + Tools │ -└──────────────────────────────────────────────────────────────────┘ +┌───────────────────────────────────────────────────────────┐ +│ Agent Runtime │ +│ ┌───────────┐ ┌──────────────┐ │ +│ │ Mailbox │──►│ Attention │───► ContextEngine │ +│ └───────────┘ │ Controller │ (deterministic │ +│ └──────────────┘ assembly, no LLM) │ +│ │ │ +│ ┌──────────────────────────────────────────▼────────────┐ │ +│ │ ContextEngine reads: ROLE.md · knowledge.md · │ │ +│ │ NOTEBOOK.md · mailbox · goals · activity │ │ +│ └───────────────────────────┬───────────────────────────┘ │ +│ ▼ │ +│ Main LLM + Tools │ +└───────────────────────────────────────────────────────────┘ ``` | Component | Location | Role in cycle | |-----------|----------|---------------| | `AttentionController` | `packages/core/src/attention.ts` | Triage, deliberation, focus management | -| `CognitivePreparation` | `packages/core/src/cognitive.ts` | CPP orchestration | -| `ContextEngine` | `packages/core/src/context-engine.ts` | Context assembly, notebook writer | +| `ContextEngine` | `packages/core/src/context-engine.ts` | Deterministic context assembly, relevant-memory injection | | `Agent` | `packages/core/src/agent.ts` | Cycle orchestration, heartbeat, notebook persistence | -| `MemoryStore` | `packages/core/src/memory/store.ts` | MEMORY.md + NOTEBOOK.md I/O | +| `MemoryStore` | `packages/core/src/memory/store.ts` | knowledge.md + NOTEBOOK.md I/O, migration-read | | `PendingCallbackRegistry` | `packages/core/src/pending-callback.ts` | Async callback tracking | | `AgentManager` | `packages/core/src/agent-manager.ts` | A2A DM routing, cognitive config | -Types: `packages/shared/src/types/cognitive.ts`, `requirement.ts` (`GoalConfig`). +Types: `packages/shared/src/types/cognitive.ts` (`CognitiveConfig`; the depth types are inert), `requirement.ts` (`GoalConfig`). --- @@ -293,7 +288,7 @@ Types: `packages/shared/src/types/cognitive.ts`, `requirement.ts` (`GoalConfig`) | Document | Relationship | |----------|-------------| -| [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md) | Two-file model detail, dream cycle, tool reference | +| [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md) | Storage model detail, dream cycle, tool reference, migration-read | | [MAILBOX-SYSTEM.md](./MAILBOX-SYSTEM.md) | Mailbox types, priority, triage protocol | | [PROMPT-ENGINEERING.md](./PROMPT-ENGINEERING.md) | Prompt section taxonomy | | [ARCHITECTURE.md](./ARCHITECTURE.md) | System-wide component overview | @@ -304,24 +299,22 @@ Types: `packages/shared/src/types/cognitive.ts`, `requirement.ts` (`GoalConfig`) ### Completed -- Unified cognitive cycle with persistent Notebook and MEMORY.md -- CPP Phases 1–3 with depth selection (D0–D3) and notebook output path +- Unified cognitive cycle with persistent Notebook and `knowledge.md` +- **Deterministic context assembly** (no pre-call LLM) — CPP removed - Attention Controller triage + deliberation with notebook persistence - Goal/Loop heartbeat integration with `GoalConfig` on requirements - A2A DM channels (`dm:a2a:{sorted_ids}`) with group-chat persistence - `PendingCallbackRegistry` with SQLite persistence -- Mechanical retrieval fallback writes `relevant-context` to notebook -- Unit tests in `packages/core/test/cognitive-enhancement.test.ts` +- Lossless memory budget (archive, not truncate) + health banner + migration-read -### In Progress / Not Yet Built +### Retired -- `RetrievalBackend` adapter for D2+ directed retrieval (interface defined, no adapter) -- D2/D3 blocked by `maxDepth` default until retrieval backend exists -- D3 post-response evaluation (meta-cognitive feedback loop) -- Dream cycle integration with CPP activity index +- **CPP (Cognitive Preparation Pipeline)** — the Appraisal / Retrieval / Reflection LLM phases, `cognitive.ts`, `CognitivePreparation`, `selectCognitiveDepth`, depth levels D0–D3, and the `notebookWriter` dual-destination. Replaced by §3. +- `state.md` store (Working-layer state now lives in `NOTEBOOK.md`; legacy content migrated on load). +- Notebook `cpp` managed tier and `NOTEBOOK_TTL_MS_CPP`. ### Future Work -- Enable D2 by default once `RetrievalBackend` ships -- Tune depth heuristics and appraisal prompts from production data -- Simplify compression pipeline (thinner sessions reduce pressure) +- Prompt-matrix guidance for agent-driven `memory_search` / `kb_search` (replacing the old depth heuristics). +- Simplify the compression pipeline (thinner sessions reduce pressure). + diff --git a/docs/CONCURRENT-PROCESSING.md b/docs/CONCURRENT-PROCESSING.md index ff9404fa5..fa873818a 100644 --- a/docs/CONCURRENT-PROCESSING.md +++ b/docs/CONCURRENT-PROCESSING.md @@ -317,7 +317,9 @@ argument so that *different* resources run in parallel: > ① `update_notebook` was registered under domain `notebook` while its alias > `update_working_memory` lived under `working-memory` — both write the **same** Map > (`Agent.writeNotebookEntry`), yet they were judged non-conflicting, so the lock did -> nothing. ② `file_write knowledge.md` took `fs:` while `memory_update` took +> nothing. (The alias has since been **removed** and the tool renamed `notebook_upsert`; +> the resource-key fix below is what makes any such duplication impossible.) +> ② `file_write knowledge.md` took `fs:` while `memory_update` took > `memory` — the same file, two mutually non-exclusive paths, so a hand edit could be > **silently overwritten** by the next in-memory whole-file rewrite. > Both are fixed by resolving tools to a **resource key** via @@ -338,7 +340,7 @@ argument so that *different* resources run in parallel: | `task` / `requirement` / `workflow` / `project` / `deliverable` | entity id | `task_update`, `requirement_update_status`, `workflow_run`, `deliverable_update` | | `terminal` / `browser` | terminal id / page id | `exec_terminal`, `click`, `navigate_page` | | `a2a-out` | target agent / channel | `agent_send_message`, `feishu_send_message` | -| `agent-memory` | **resource key** (`knowledge` / `notebook` / `identity` / `state`) | `memory_save`, `memory_update`, `memory_update_longterm`, `memory_delete`, `update_notebook`, **`update_working_memory` / `clear_working_memory`** (alias → same key), `file_write` / `file_edit` on an agent memory file | +| `agent-memory` | **resource key** (`knowledge` / `notebook` / `identity` / `state`) | `memory_save`, `memory_update`, `memory_update_longterm`, `memory_delete`, `notebook_upsert` / `notebook_clear`, `file_write` / `file_edit` on an agent memory file | | `session`, `session-tools`, `schedule`, `llm-config`, `org`, `ui`, `deliberation`, `mailbox-admin`, `notify` | single resource or item id | `session`, `discover_tools`, `team_update` | **Classification is closed and test-enforced.** `Agent.isWriteTool` is *default-deny*: @@ -377,10 +379,8 @@ read → transform → write, and synchronous code still yields between those st whenever the read-modify-write spans an `await` — which it does on the dream-cycle path. Concretely (see §4.9 defect registry): -- Tool-path writes (`memory_save` / `memory_update` / `update_notebook`) take the - `agent-memory:` lock (`knowledge` / `notebook`), so they mutually exclude — - including the `update_working_memory` alias, which now resolves to the *same* key - instead of a separate `working-memory` domain. +- Tool-path writes (`memory_save` / `memory_update` / `notebook_upsert`) take the + `agent-memory:` lock (`knowledge` / `notebook`), so they mutually exclude. - The dream cycle (`consolidateMemory` → `dreamConsolidateMemory(entries)` → `pruneMemoryMd()` → `compressLongTermMemory()`) runs from a timer, **outside any tool call**, and therefore **outside any lock**. A concurrent `memory_update` could land @@ -424,7 +424,7 @@ Defect registry (found 2026-09-16; each row is a *fixed* bug kept as a regressio | ID | Defect | Root cause | Fix | Guard | |---|---|---|---|---| -| **P0-1** | `update_notebook` and its alias `update_working_memory` write the same Map yet never exclude each other | lock granularity was the **tool name**, and the alias was registered under a different domain (`working-memory`) | both resolve to `agent-memory:notebook` via `WRITE_TOOL_DOMAINS.resource` | `lock-resource-keys.test.ts` — alias key equality + live serialisation | +| **P0-1** | two paths to the same notebook Map (formal name + alias) never excluded each other | lock granularity was the **tool name**, and the alias was registered under a different domain (`working-memory`) | both resolve to `agent-memory:notebook` via `WRITE_TOOL_DOMAINS.resource` (the alias was later removed and the tool renamed `notebook_upsert`) | `lock-resource-keys.test.ts` — resource key equality + live serialisation | | **P0-2** | a hand edit to `knowledge.md` / `NOTEBOOK.md` could be **silently overwritten** by the next in-memory whole-file rewrite | `file_write` took `fs:` while `memory_update` took `memory` — same file, two non-exclusive paths | `Agent.resourceLocksFor(tc, dataDir)` rewrites an fs lock into the memory resource lock when the path names an agent memory file | same file — `file_write`/`file_edit` key equality | | **P0-3** | two OS processes (desktop + CLI) on the same agent have **no** mutual exclusion | `ResourceLockRegistry` and entity locks are in-process maps | **not fixed** — needs a process-level arbiter (file lock / single-instance election) | listed in §7 | diff --git a/docs/GUIDE.md b/docs/GUIDE.md index 9494fb6e6..a1adfccf6 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -348,7 +348,7 @@ A: This depends on the governance policy. By default, standard tasks need Manage A: Human-created tasks start in `pending` status. You will not receive a self-notification or approval request. Click "Start Execution" in the task detail page to begin execution. This differs from agent-created tasks, which show an "Approve" button and go through the HITL approval flow. **Q: How do Agents share knowledge?** -A: Agents contribute via the `knowledge_contribute` tool to the project knowledge base; other Agents search with `knowledge_search`. Humans can view and manage entries on the Knowledge page. +A: Agents contribute via the `knowledge_contribute` tool to the project knowledge base; other Agents search with `kb_search`. Humans can view and manage entries on the Knowledge page. **Q: How do I assign Agents to different projects?** A: Create projects on the Projects page and link them to Teams. When Agents are assigned to tasks within a project, they automatically get project context and isolated workspaces. diff --git a/docs/MAILBOX-SYSTEM.md b/docs/MAILBOX-SYSTEM.md index 3b5aeba76..c30d4cf22 100644 --- a/docs/MAILBOX-SYSTEM.md +++ b/docs/MAILBOX-SYSTEM.md @@ -1336,7 +1336,7 @@ Defined by `DELIBERATION_ALLOWED_TOOLS` in `@markus/shared`: | Context gathering | `task_list`, `task_get`, `requirement_list`, `requirement_get`, `list_projects`, `team_list`, `team_status`, `recall_activity`, `memory_search`, `memory_search_longterm` | | Inline communication | `notify_user`, `task_comment`, `requirement_comment`, `agent_send_message`, `agent_send_group_message`, `agent_create_group_chat`, `agent_list_group_chats` | | Mailbox management | `check_mailbox`, `defer_mailbox_item`, `drop_mailbox_item`, `prioritize_mailbox_item` | -| Working memory | `update_working_memory`, `clear_working_memory` | +| Working memory | `notebook_upsert`, `notebook_clear`, `notebook_read` | | Decision output | `complete_deliberation` | **Excluded**: `task_create`, `task_update`, `requirement_propose`, code/shell tools, `spawn_subagent`. These are heavy side-effect tools that belong in the processing phase. @@ -1352,8 +1352,8 @@ individual mailbox items using dedicated tools: | `defer_mailbox_item` | Deliberation + focused processing | Postpone an item | | `drop_mailbox_item` | Deliberation + focused processing | Discard stale item | | `prioritize_mailbox_item` | Deliberation only | Re-prioritize item | -| `update_working_memory` | All scenarios | Update situational awareness | -| `clear_working_memory` | All scenarios | Clear stale awareness | +| `notebook_upsert` | All scenarios | Update situational awareness | +| `notebook_clear` | All scenarios | Clear stale awareness | **Safety**: `human_chat` items are protected — they cannot be deferred, dropped, or reprioritized by tool calls. **Strict state items**(正式任务执行 / 评审 / 需求·工作流收尾动作, @@ -1413,7 +1413,7 @@ replaces the former `currentCognition` string. Each entry has: Working memory is populated from three sources: 1. **Deliberation**: `situationalAwareness` → key `"deliberation"` 2. **Triage**: reasoning → key `"triage-decision"` -3. **Agent tools**: `update_working_memory` / `clear_working_memory` +3. **Agent tools**: `notebook_upsert` / `notebook_clear` All entries are injected into every system prompt as `## Working Memory` with age labels per entry. The agent decides what to keep, update, or expire. diff --git a/docs/MEMORY-SYSTEM.md b/docs/MEMORY-SYSTEM.md index d61216831..68df10809 100644 --- a/docs/MEMORY-SYSTEM.md +++ b/docs/MEMORY-SYSTEM.md @@ -4,7 +4,8 @@ Architecture and data flows for the Markus agent memory system. Persistent cogni **NOTEBOOK.md** (cognitive workspace) plus **`knowledge.md`** as the single long-term store (permanent curated knowledge + the observation buffer). The former second half of the old "dual store", `state.md`, has **retired** — its situational state now lives in the notebook's -`system` tier (see §2 / §3). A legacy `MEMORY.md` still migrates once. +`system` tier (see §2 / §3). Legacy `MEMORY.md` / `memories.json` / `state.md` files are still +read once on load and migrate into `knowledge.md` (migration-read layer: read old, write new only). Grounded in Tulving-style procedural / semantic / episodic persistence plus cognitive-science working-memory models. @@ -17,7 +18,7 @@ working-memory models. `knowledge.md` holds permanent curated knowledge plus the observation buffer. Situational ("current") state lives in the notebook's `system` tier rather than a second file — the old `state.md` half **retired** (see §2). Observations buffer is never fully prompt-injected. - Legacy `MEMORY.md` MUST migrate on first load. + Legacy `MEMORY.md` / `memories.json` / `state.md` MUST migrate into `knowledge.md` on first load (read old, write new only). 2. **Tulving mapping + notebook**: Persistent layers align with Tulving-style cognition — **Procedural** (ROLE.md), **Semantic** (`knowledge.md`), **Episodic** (sessions + activities). The **Notebook** replaces volatile in-memory working memory with a persistent scratchpad always injected (via the volatile `[Live context]` tail by Scheme A — not as a system-prompt segment). 3. **SQLite for history**: Activity history lives in SQLite — indexed, searchable, and queryable via tools. 4. **Context is currency**: Every byte in the LLM prompt competes for limited context window. Retrieval must maximize signal-to-noise. @@ -37,11 +38,15 @@ MUST: On first load, if only legacy `MEMORY.md` exists, migrate it into `knowled MUST: Prompt injection of knowledge MUST honor `KNOWLEDGE_PROMPT_MAX_TOKENS` (`0` for reflex profile — omit full dump). MUST: `MEMORY_MD_TOTAL_MAX_CHARS` MUST be enforced as a **load-time invariant**, not only a -write-time guard: `MemoryStore.convergeLongTermToCap()` runs on construction/load and condenses -the largest curated section(s) to a stub **keeping the heading** (so the index line and -`memory_search` still surface the topic), never touching `## _observations`. Rationale: a -write-time refusal cannot shrink an already-oversized file, so an over-budget file used to stay -over budget forever (measured 23 323 chars against a 15 000 limit). +write-time guard: `MemoryStore.convergeLongTermToCap()` runs on construction/load and +**losslessly archives** the largest curated section(s)' bodies to `knowledge-archive.md` +**keeping the heading + a pointer line** (so the index line and `memory_search` still surface +the topic), never touching `## _observations`. Rationale: writes are no longer refused at the +total cap, so over-budget content would otherwise be dropped; archiving keeps the whole text +retrievable (via `memory_search`) while the in-prompt file converges to ≤ 100% (measured +23 323 chars against a 15 000 limit). +MUST: `knowledge-archive.md` is archive-only — searchable via `memory_search`, but **never +prompt-injected**. MUST: `memory_update mode="forget"` (`MemoryStore.removeLongTermSection`) is the curated-section **forget primitive**; forgetting `## _observations` is refused (delete observations by id instead). MUST: `knowledge.md` / `NOTEBOOK.md` MUST be persisted with `writeFileAtomic` (tmp + rename) so a @@ -50,7 +55,7 @@ MUST: `memory_update_longterm` / curated updates write `knowledge.md`. MUST: `memory_save` → `## _observations`; `memory_update` → named curated section. Test IDs: `A-knowledge-cap`. -| **Kahneman — Dual Process** | System 1 = fast retrieval (`memory_search`, prompt injection); System 2 = CPP deliberative processing writes `cpp`-managed notebook entries | +| **Kahneman — Dual Process** | System 1 = fast retrieval (`memory_search`, prompt injection); System 2 = slow, deliberate recall the agent triggers explicitly (`memory_search` / `kb_search`) — the old pre-call CPP pipeline is gone | ## 2. Four-Layer Architecture @@ -75,8 +80,8 @@ Test IDs: `A-knowledge-cap`. │ Tools: recall_activity (list / search / get) │ ├───────────────────────────────────────────────────────────────┤ │ Notebook — persistent cognitive workspace │ -│ NOTEBOOK.md — keyed entries (agent / system / cpp managed) │ -│ Tools: update_notebook, clear_notebook │ +│ NOTEBOOK.md — keyed entries (agent / system managed) │ +│ Tools: notebook_upsert, notebook_clear, notebook_read │ │ Code: Agent + MemoryStore (loadNotebook / saveNotebook) │ └───────────────────────────────────────────────────────────────┘ @@ -92,31 +97,30 @@ Persistent markdown replacing the former volatile in-memory working memory. |-----------|-------| | Storage | `~/.markus/agents/{id}/NOTEBOOK.md` | | Format | `## key` headings with metadata comments + body text | -| Entry fields | `key`, `text`, `managed` (`agent` \| `system` \| `cpp`), `updatedAt` | +| Entry fields | `key`, `text`, `managed` (`agent` \| `system`), `updatedAt` | | Prompt injection | Always loaded as `## Notebook`, bounded to `NOTEBOOK_MAX_ENTRIES` (16) entries / `NOTEBOOK_PROMPT_MAX_CHARS` (6000) chars. Oversized entries are truncated inline (never dropped whole). | | Limits | **16 total** entries (`NOTEBOOK_MAX_ENTRIES`) — the CROSS-TIER cap; **4 agent** entries (`NOTEBOOK_MAX_AGENT_ENTRIES`); `NOTEBOOK_MAX_CHARS_PER_ENTRY` (6000) per entry; key ≤ `NOTEBOOK_KEY_MAX_CHARS` (64). | -| TTL | Per tier: `agent` 96h, `system` 24h, `cpp` 6h (`NOTEBOOK_TTL_MS_*`). Entries past their TTL are dropped on load, after every write, and again before injection. | -| Eviction | Oldest-`updatedAt`-first. Tier order: `cpp` → `system` → `agent` — the agent's own notes are the **last** to go. | +| TTL | Per tier: `agent` 96h, `system` 24h (`NOTEBOOK_TTL_MS_AGENT` / `NOTEBOOK_TTL_MS_SYSTEM`). Entries past their TTL are dropped on load, after every write, and again before injection. | +| Eviction | Oldest-`updatedAt`-first. Tier order: `system` → `agent` — the agent's own notes are the **last** to go. | **Managed tags**: -- `agent` — written via `update_notebook` / `clear_notebook` +- `agent` — written via `notebook_upsert` / `notebook_clear` - `system` — triage → `"triage-decision"`, deliberation → `"deliberation"`, etc. -- `cpp` — Cognitive Preparation Pipeline writes situational context **Lifecycle**: Loaded at agent startup → **normalized on load** (TTL + caps; trimmed result is persisted immediately) → updated in-process → persisted with a 2s debounce **bounded by `NOTEBOOK_PERSIST_MAX_WAIT_MS` (10s)** so a chatty turn cannot defer the write indefinitely. Survives restarts. -**Cleanup**: `clear_notebook({ key })` removes one entry; `clear_notebook` without key clears the whole notebook. TTL handles passive expiry; `clear_notebook` handles deliberate removal. Prompt guidance: clear when a task completes or context goes stale. +**Cleanup**: `notebook_clear({ key })` removes one entry; `notebook_clear` without key clears the whole notebook. TTL handles passive expiry; `notebook_clear` handles deliberate removal. Prompt guidance: clear when a task completes or context goes stale. #### Invariants (§2 Notebook lifecycle) The notebook is a RESIDENT prompt region, so it needs all four mechanisms that keep a resident region honest. Historically only the storage-side per-entry cap existed, and -the entry cap was applied on **one** write path — the other three writers -(`triage` → `triage-decision`, `deliberation`, the CPP notebook writer) called +the entry cap was applied on **one** write path — the other writers +(`triage` → `triage-decision`, `deliberation`) called `workingMemory.set()` directly and were never counted. The load path trimmed nothing. Result: a real notebook reached **26 entries / 33 KB** against a nominal -cap of 4, including an 18-day-old triage decision and a 57-day-old CPP output that -were re-injected on every turn. +cap of 4, including an 18-day-old triage decision that +was re-injected on every turn. | # | Invariant | Authority | |---|-----------|-----------| @@ -155,7 +159,7 @@ Map insertion order satisfied neither (evict + re-insert permutes the block). | Episodic retrieval | `recall_activity` tool | `packages/core/src/tools/recall.ts` | | Procedural | `RoleLoader` | `packages/core/src/role-loader.ts` | | Semantic tools | `memory_save`, `memory_search`, `memory_update` | `packages/core/src/tools/memory.ts` | -| Notebook tools | `update_notebook`, `clear_notebook` | `packages/core/src/tools/mailbox-tools.ts` | +| Notebook tools | `notebook_upsert`, `notebook_clear`, `notebook_read` | `packages/core/src/tools/mailbox-tools.ts` | | Vector search | `SemanticMemorySearch` | `packages/core/src/memory/semantic-search.ts` | | Notebook runtime | `Agent.workingMemory`, prompt injection | `packages/core/src/agent.ts` | @@ -232,12 +236,12 @@ The agent organizes sections freely — common patterns: `conventions`, `procedu ## 4. Memory Tools -Five primary tools (down from seven). Legacy aliases (`memory_list`, `memory_delete`, `memory_update_longterm`, `update_working_memory`, `clear_working_memory`) remain for backward compatibility. +Five primary tools. Legacy memory aliases (`memory_list`, `memory_delete`, `memory_update_longterm`) remain for backward compatibility; the notebook aliases `update_working_memory` / `clear_working_memory` were **removed** — use `notebook_upsert` / `notebook_clear`. | Tool | Purpose | |------|---------| -| `update_notebook` | Upsert a keyed entry in NOTEBOOK.md | -| `clear_notebook` | Remove one entry or all agent-managed entries | +| `notebook_upsert` | Upsert a keyed entry in NOTEBOOK.md | +| `notebook_clear` | Remove one entry or all agent-managed entries | | `memory_save` | Append one observation to `knowledge.md` `## _observations` (`content` required; rejects arrays / empty) | | `memory_update` | Update curated `knowledge.md` section (`replace` / `patch`; `append`→`patch`) or delete observations (`mode: delete`) | | `memory_search` | Keyword search over observations **and** curated `knowledge.md` sections (token OR-match, ranked by hits); empty query lists recent observations. Falls back from semantic→keyword when embeddings miss. | @@ -413,7 +417,7 @@ authoritative behavior spec. - **Behavior**: a **turn-level preflight** runs `memoryFlush` once per session when the previous turn's context usage crossed a high-water threshold (~75%). The flush asks the - agent to `memory_save` key decisions/facts and `update_notebook` current state. It runs + agent to `memory_save` key decisions/facts and `notebook_upsert` current state. It runs as an independent `sys_` session. - **Invariants**: - Flush fires **at most once per session** (deduplicated), and **before** the compression @@ -474,7 +478,7 @@ Every state a running agent carries is placed by asking two questions only — * |---|---|---|---|---|---| | **Identity** | agent-lifetime | permanent (human-owned) | `role/ROLE.md`, `role/HEARTBEAT.md`, skills | human edit / explicit file write | fixed段 (system) | | **Knowledge** | agent-lifetime | long, forgettable | `knowledge.md` (curated sections + `## _observations`) | `memory_save`, `memory_update` (incl. `mode="forget"`) | volatile tail, budget-capped | -| **Working** | agent-lifetime | short, auto-expiring | `NOTEBOOK.md` (per-tier TTL) | `update_notebook` (alias `update_working_memory`) | volatile tail (`## Notebook`) | +| **Working** | agent-lifetime | short, auto-expiring | `NOTEBOOK.md` (per-tier TTL: agent 96h / system 24h) | `notebook_upsert` | volatile tail (`## Notebook`) | | **Session** | one session/worker | session-scoped | slots / `summary` / fragments | `session` tool family | fixed段 + history | ### 10.2 Consolidation record — what was merged away diff --git a/docs/PROMPT-ENGINEERING.md b/docs/PROMPT-ENGINEERING.md index ad3c05dbb..32d4d20a4 100644 --- a/docs/PROMPT-ENGINEERING.md +++ b/docs/PROMPT-ENGINEERING.md @@ -10,13 +10,13 @@ This document specifies how Markus constructs prompts, manages context, and orch LLM calls are organized in two tiers: -**Tier 1 -- Cognitive Preparation Calls**: Lightweight LLM calls that prepare context for the main call. These use persona-aware prompts and the cheapest available model. See [COGNITIVE-ARCHITECTURE.md](./COGNITIVE-ARCHITECTURE.md) for theoretical foundations and full design. +**Tier 1 -- (retired)**: the former Cognitive Preparation calls (Appraisal / Reflection / Evaluation) were **removed** — they were extra LLM calls before the main call. Context preparation is now **deterministic (no LLM)**: a small bounded block (recent activity + working-memory keys) plus the ContextEngine's own relevant-memory retrieval. Deeper recall is agent-driven via `memory_search` / `kb_search`. -| # | Phase | Purpose | Model | Tools | Max Output | -|---|-------|---------|-------|-------|-----------| -| P1 | **Appraisal** | Assess situation, plan context retrieval | Cheapest tier | No | 512 tokens | -| P3 | **Reflection** | Extract persona-specific patterns and insights from retrieved context | Cheapest tier | No | 512 tokens | -| P5 | **Evaluation** (D3 only) | Post-response assessment of context adequacy | Cheapest tier | No | 256 tokens | +| # | Phase | Status | +|---|-------|--------| +| P1 | **Appraisal** | **removed** — replaced by deterministic assembly | +| P3 | **Reflection** | **removed** | +| P5 | **Evaluation** (D3) | **removed** | P2 (Directed Retrieval) and P4 (Assembly) are code-only phases with no LLM call. @@ -431,9 +431,8 @@ Source: `getDynamicContext()` — three sources: | Tag | Writer | Typical keys | |-----|--------|--------------| -| `agent` | `update_notebook` / `clear_notebook` | Agent-chosen keys (priorities, blockers, decisions) | +| `agent` | `notebook_upsert` / `notebook_clear` | Agent-chosen keys (priorities, blockers, decisions) | | `system` | Runtime (triage, mechanical retrieval) | `triage-decision`, `relevant-context` | -| `cpp` | Cognitive Preparation Pipeline | `cognitive-context`, `relevant-context`, `reflection` | **Bounding ([MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md) §2).** The block is bounded on four independent axes, because each alone is insufficient: @@ -442,15 +441,15 @@ Source: `getDynamicContext()` — three sources: | Entry count | 16 total, 4 `agent` | "26 entries against a nominal cap of 4" — three of four writers bypassed the cap and the load path trimmed nothing | | Per-entry chars | 6000 (1500 for `relevant-context`, `NOTEBOOK_RELEVANT_CONTEXT_MAX_CHARS`) | one 9 475-char entry — more than the whole block budget — crowding every real note out | | Total block chars | 6000 | the block reaching 41 % of the volatile tail | -| TTL | `agent` 96h / `system` 24h / `cpp` 6h | month-old situational state (`triage-decision`, `cognitive-context`) re-injected as current fact | +| TTL | `agent` 96h / `system` 24h | month-old situational state (`triage-decision`) re-injected as current fact | Overflow is handled by **inline truncation**, not omission: an entry too large for the remaining budget is sliced with a `_[truncated]_` marker rather than dropped whole, and any entry that still cannot be shown appears in a sorted index line. So the agent always sees that a note exists (and can `file_read` NOTEBOOK.md) instead of it vanishing silently. **Ordering is deterministic**: `updatedAt` DESC with a `key` ASC tie-break. Ranking matters for staleness; determinism matters for caching — for the same logical state the block must serialize to the same bytes, or every assembly dirties the volatile tail and re-bills those tokens. (Map insertion order did neither: evict + re-insert permuted the block.) -The context engine no longer injects separate `## Cognitive Context`, `## Retrieved Context`, `## Reflection`, or `## Relevant Memories` sections. Instead, `buildSystemPrompt()` accepts a `notebookWriter` callback; CPP outputs and relevance-matched memories are written to Notebook entries, centralizing cognitive state in one section. See [COGNITIVE-ARCHITECTURE.md](./COGNITIVE-ARCHITECTURE.md) §3–4 and [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md). +The `notebookWriter` dual-destination was **removed**: relevance-matched memories are injected **only** for the current turn as `## Relevant Memories`, and a small deterministic `## Cognitive Context` block (recent activity + working-memory keys) may be injected — neither is written to Notebook. Notebook holds only deliberate working state. See [MEMORY-SYSTEM.md](./MEMORY-SYSTEM.md). -Legacy aliases `update_working_memory` / `clear_working_memory` remain for backward compatibility. +The legacy notebook aliases `update_working_memory` / `clear_working_memory` were **removed** — use `notebook_upsert` / `notebook_clear`. #### Content layers (availability tiers: which content is always on vs on-demand) @@ -936,13 +935,10 @@ The "harness" is the while-loop that drives agentic tool use: LLM → tool calls ### 4.2 Common Harness Flow ``` -0. Cognitive Preparation (when CPP enabled, depth D1+): - 0a. Appraisal — persona-aware LLM call → context plan (D1+) - 0b. Directed Retrieval — execute plan against indexed stores (D2+) - 0c. Reflection — persona-aware LLM call → insights (D2+) - 0d. Assembly — merge prepared context (code only) -1. Build system prompt (contextEngine.buildSystemPrompt + PreparedContext) - CPP/retrieval outputs → notebookWriter → ## Notebook (not separate sections) +0. Deterministic context assembly (no LLM): read recent activity + working-memory + keys; ContextEngine retrieves relevance-matched memories (bounded). +1. Build system prompt (contextEngine.buildSystemPrompt) + → injects ## Relevant Memories (this turn only) + a deterministic ## Cognitive Context 2. Build tool definitions (toolSelector.selectTools) 3. Prepare messages (contextEngine.prepareMessages — compress to fit budget) 4. LLM call (llmRouter.chat / chatStream, wrapped in withNetworkRetry) @@ -951,10 +947,9 @@ The "harness" is the while-loop that drives agentic tool use: LLM → tool calls b. If max_tokens: append continuation prompt → re-prepare messages → LLM call 6. Output guardrail check 7. Persist final reply to session -8. Post-response evaluation (D3 only) — LLM assesses context adequacy ``` -Step 0 is the Cognitive Preparation Pipeline. It runs once before the main harness loop (steps 1-7) begins. The preparation prompts are persona-aware: they include the agent's role description, current state, and recent activity, so different agents produce different context preparation plans for the same stimulus. See [COGNITIVE-ARCHITECTURE.md](./COGNITIVE-ARCHITECTURE.md) for the full design. +Step 0 is **deterministic** — there is no Cognitive Preparation Pipeline and no pre-call LLM. The ContextEngine assembles a small, bounded situational block from recent activity and working-memory keys, and performs its own bounded relevant-memory retrieval. Deeper cross-domain recall is agent-driven via `memory_search` / `kb_search`. ### 4.3 Tool Execution @@ -1044,12 +1039,12 @@ The heartbeat user prompt is assembled inline; the system prompt still comes fro **Deduplication note**: earlier versions of the inline prompt restated `## Communication Channels`, `## Core Principle: Patrol, Don't Build`, `## Prefer Scheduled Wakeups`, `## What You CAN/MUST NOT Do`, and `## Conditional Actions`. These duplicated the cached `heartbeat` scenario section (§5.3 / `buildScenarioSection`) and Tier-1 rules, so they were removed from the (uncached) inline user prompt to cut per-heartbeat token cost. The `heartbeat` scenario section is now the single source of truth for comms/priorities. -**Notebook Guidelines** (in mailbox checklists / system prompt when queue context is present): use `update_notebook` to save priorities, context, decisions, and blockers; `clear_notebook` when context becomes irrelevant. Do not store raw message content — use `memory_save` for durable observations. +**Notebook Guidelines** (in mailbox checklists / system prompt when queue context is present): use `notebook_upsert` to save priorities, context, decisions, and blockers; `notebook_clear` when context becomes irrelevant. Do not store raw message content — use `memory_save` for durable observations. Tool whitelist = **reflex allowlist** (`REFLEX_CORE_TOOLS` + manager `team_status`): `task_list`, `task_get`, `memory_save`, `memory_search`, `notify_user`, `request_user_input`, `schedule_wakeup`, `cancel_wakeup`, `set_heartbeat_interval`, `discover_tools`, `check_mailbox`, -`file_read`, `agent_send_message`, `update_notebook`. MUST NOT imply `task_create` / +`file_read`, `agent_send_message`, `notebook_upsert`. MUST NOT imply `task_create` / `requirement_propose` / `package_install` / `goal_*` in heartbeat (see Interaction Mode). **Heartbeat cadence & configurability**: the periodic heartbeat is a coarse safety-net (`DEFAULT_HEARTBEAT_INTERVAL_MS`, 6h) behind the event-driven `schedule_wakeup` mechanism — it is NOT the primary timing mechanism. The interval is configurable at three levels: (a) the user via `PATCH /api/agents/:id/config` (`heartbeatIntervalMs`) or the Agent Profile → Heartbeat tab; (b) the agent itself via the `set_heartbeat_interval` tool (clamped to `MIN_HEARTBEAT_INTERVAL_MS`–`MAX_HEARTBEAT_INTERVAL_MS`, i.e. 5min–24h). Both apply **live** via `Agent.setHeartbeatInterval()` (restarts the scheduler) and persist (agent-driven changes flow through the `agent:heartbeat-interval-changed` event → `agentRepo.updateConfig`). A one-time SQLite migration (gated by `PRAGMA user_version`) bumps agents still on the legacy 30-min default to the 6h safety-net without clobbering any interval a user or agent set deliberately. @@ -1294,19 +1289,19 @@ Five primary memory tools (down from seven); legacy aliases preserved: | Tool | Purpose | |------|---------| -| `update_notebook` | Upsert a keyed entry in `NOTEBOOK.md` (situational workspace) | -| `clear_notebook` | Remove one entry or all agent-managed entries | +| `notebook_upsert` | Upsert a keyed entry in `NOTEBOOK.md` (situational workspace) | +| `notebook_clear` | Remove one entry or all agent-managed entries | | `memory_save` | Append one observation to `## _observations` in `knowledge.md` | | `memory_update` | Edit curated `knowledge.md` sections (always in `## Your Knowledge`) | | `memory_search` | Search observations and curated knowledge | -**Legacy aliases** (same handlers): `update_working_memory`, `clear_working_memory`, `memory_list`, `memory_delete`, `memory_update_longterm`, `memory_search_longterm`. +**Legacy memory aliases** (same handlers): `memory_list`, `memory_delete`, `memory_update_longterm`. (The notebook aliases `update_working_memory` / `clear_working_memory` were removed — use `notebook_upsert` / `notebook_clear`.) Goal tools (`goal_create`, `goal_update`, `goal_status`) are available during **heartbeat** and **deliberation** for standing-goal management. `ToolSelector.selectTools()` determines which tools appear in each LLM call: -1. **Always-on tools**: Core set always included (e.g., `memory_save`, `memory_search`, `memory_update`, `update_notebook`, `clear_notebook`, `file_read`, `file_write`, `task_create`, `task_list`, `spawn_subagent`, `spawn_subagents`, `check_mailbox`, etc.) +1. **Always-on tools**: Core set always included (e.g., `memory_save`, `memory_search`, `memory_update`, `notebook_upsert`, `notebook_clear`, `file_read`, `file_write`, `task_create`, `task_list`, `spawn_subagent`, `spawn_subagents`, `check_mailbox`, etc.) 2. **Manager-only tools**: Added when `isManager=true` (e.g., `task_assign`, `team_status`, `delegate_message`) 3. **Package tools**: Available to all agents, activated by keyword (e.g., `package_list`, `package_install`, `hub_search`, `hub_install`) 4. **Task-execution tools**: Added when `isTaskExecution=true` (e.g., `task_submit_review`, `subtask_create`) diff --git a/docs/STATE-MACHINES.md b/docs/STATE-MACHINES.md index 166e37af7..02811219a 100644 --- a/docs/STATE-MACHINES.md +++ b/docs/STATE-MACHINES.md @@ -649,9 +649,8 @@ agent stop | Tag | Writer | Purpose | |-----|--------|---------| -| `agent` | `update_notebook` / `clear_notebook` | Agent-chosen keys (priorities, blockers, decisions) | -| `system` | Triage, mechanical retrieval | `triage-decision`, `relevant-context` when CPP off | -| `cpp` | Cognitive Preparation Pipeline | `cognitive-context`, `relevant-context`, `reflection` | +| `agent` | `notebook_upsert` / `notebook_clear` / `notebook_read` | Agent-chosen keys (priorities, blockers, decisions) | +| `system` | Triage, mechanical retrieval | `triage-decision`, `relevant-context` | ### Transitions diff --git a/packages/cli/src/commands/start.ts b/packages/cli/src/commands/start.ts index d6edda2ad..9cf2effa9 100644 --- a/packages/cli/src/commands/start.ts +++ b/packages/cli/src/commands/start.ts @@ -1724,7 +1724,6 @@ async function startServerCore( catch (e) { log.warn('Failed to update mailbox status', { itemId, error: String(e) }); } }, markStaleProcessingAsDropped: (aid: string) => mbRepo.markStaleProcessingAsDropped(aid), - markStaleProcessingAsCompleted: (aid: string, ownerId?: string) => mbRepo.markStaleProcessingAsCompleted(aid, ownerId), loadQueued: (aid: string) => { const rows = mbRepo.getByAgent(aid, { status: 'queued' }); return rows.map((r: any) => ({ diff --git a/packages/core/docs/agent-memory-cognition-redesign.md b/packages/core/docs/agent-memory-cognition-redesign.md new file mode 100644 index 000000000..1bb241bb9 --- /dev/null +++ b/packages/core/docs/agent-memory-cognition-redesign.md @@ -0,0 +1,317 @@ +# Agent 记忆与认知机制 —— 完整重构设计(SSOT) + +> **版本** v2(2026-09-29)· 作者:CTO · 取代 v1 分阶段评审稿 +> **原则**:一次性彻底重构。不留补丁、不留尾巴;**前端 + 后端 + 测试全量改齐**; +> 原有功能一个不少,但机制与架构更合理、更完备。 +> **依据**:源码全量测绘(`audit-store.md` / `audit-context.md` / `audit-tools.md` / +> `audit-frontend-api.md` / `audit-tests.md`)+ 一次真实事故复盘(9/27 手改被 9/28 静默覆盖)。 +> +> **实现状态:已落地**(2026-09-29,分支 `feat/ui-optimize-0925`)。唯一有意偏离:**P-08 不删遗留兼容**, +> 改为**迁移读取层(读旧、只写新)**——实测 106 个 agent 目录仍残留 `memories.json` / `MEMORY.md` / `state.md`, +> 删除会使其记忆无法迁移/解析(违背「功能不减少」)。故遗留文件改为**加载时读入并迁移**,源文件消费(改名 `.migrated`), +> 写入端只发规范格式。详见 `store.ts` / `taxonomy.ts` 文件头「Migration-read layer」。 + +--- + +## 0. TL;DR + +现状是 **4 类机制交织 + 至少 3 代补丁共存**:长期记忆(`knowledge.md`)、工作记忆(`NOTEBOOK.md`)、 +认知准备管道(CPP)、以及四套各自为政的"自动整理"(dream cycle / `memory_consolidation` / +压缩器 / `daily_report`)。它们之间有**双目的地、双写入者、单位不一致、有损不可逆**等结构性问题。 + +本方案收敛为: + +``` +两个 Store(各自单一写入者) + + 一套工具(Agent 主动调用,渐进披露) + + 一张场景提示词矩阵(引导"何时拉、何时写") + + 一层无损兜底(有界、可见、可恢复) +``` + +--- + +## 1. 设计原则(不可违背,逐条可测) + +| 编号 | 原则 | 可测判据 | +|---|---|---| +| P1 | **单一写入者**:每个 Store 只有一个写入路径;其余一律拒绝 | `file_write` 写记忆文件被拒并有引导消息 | +| P2 | **全工具化**:一切读写/整理能力都是带 schema 的工具,可 `discover` | 每能力有工具;无"只能自动、Agent 碰不到"的能力 | +| P3 | **场景提示词化**:何时拉取/写入由场景提示词引导,替代硬编码注入 | 场景矩阵(§7)逐格有对应提示词 + 快照测试 | +| P4 | **兜底无损**:自动机制只做无损/有界/可见的动作 | 删除即归档(可检索);有落盘标记;不静默丢 | +| P5 | **单位一致**:预算一律按 **字符**;不再混用 byte | 全仓 grep 无 byte 预算判定 | +| P6 | **原子且可恢复**:写入原子;任何收敛都可回溯 | 崩溃不产生半截文件;归档段可 `memory_search` | +| P7 | **功能不减**:旧能力逐项有对应新实现 | feature parity 矩阵(§12)全绿 | +| P8 | **概念去歧**:Agent 长期记忆 与 项目知识库 命名分离 | `memory_*` vs `kb_*`,无同义混用 | + +--- + +## 2. 现有机制的目的(**必须全部保留**的职能) + +| # | 机制 | 它要解决的真实问题(目的) | 现状实现 | 新方案如何保留 | +|---|---|---|---|---| +| G1 | 长期语义记忆 | 让经验跨会话留存,且**小预算常驻**(策展段)+ **大容量可检索**(观察缓冲) | `knowledge.md`:curated + `## _observations` | 保留双区模型;**单一写入者** + 无损再平衡 | +| G2 | 记忆整理 | 观察多了要**归纳进策展段**,而不是无限堆积 | dream cycle + `memory_consolidation` + 压缩器(三套重叠) | 收敛为**一个** `memory_organize` 原语;dream 只做定时调用者 | +| G3 | 工作记忆 | 短期、按 key、**并发分身共享**、自动过期 | `NOTEBOOK.md` + 分层 TTL + LRU | 保留;统一单一写入者与锁域;改由工具读写 | +| G4 | 认知准备 | 每轮开始前**按需补齐相关上下文**(检索/反思) | CPP:LLM 多阶段管道 → 双目的地 | **工具化**:`memory_search`/`kb_search` 按需调用 + 轻量确定性注入 | +| G5 | 上下文工程 | 预算内组装 prompt,且**前缀缓存稳定** | ContextOS 分层 stable/dynamic/volatile + slots/summary | 保留(本方案不改 ContextOS 内核) | +| G6 | 安全兜底 | 预算不能无限增长;文件不可损坏 | 压缩器(有损)+ 自愈重建 + 原子写 | 保留但**无损化**:归档而非截断;安全修复而非重建 | +| G7 | 执行/会话史 | 回溯"我做过什么" | sessions/*.json + `recall_activity` + `session` | 保留 | +| G8 | 项目知识库 | 团队共享的文档知识(与 Agent 私有记忆不同物) | `FileKnowledgeStore` + `knowledge_*` 工具 | 保留;**改名去歧**为 `kb_*` | + +--- + +## 3. 问题全量清单(重构的靶子,逐条带证据) + +| 编号 | 问题 | 证据(file:line) | 归类 | +|---|---|---|---| +| **P-01** | **双写入者 → 静默覆盖**:`knowledge.md` 由「内存模型整文件重写」与「通用 file_write」两路写;锁只保证互斥、不保证"内存模型知道外部改动",故外部编辑被**下一轮落盘覆盖** | `lock-resources.ts:11-13`;**事故实证**:9/27 手改被 9/28 覆盖 | P1 | +| **P-02** | **同资源两个锁域**:`update_notebook` 域 `notebook` vs 别名 `update_working_memory` 域 `working-memory`,写同一 Map | `lock-resources.ts:8-10`(旧行为;已部分修复,需端到端复验) | P5/P1 | +| **P-03** | **CPP 双目的地**:同一产出既写 NOTEBOOK(有 writer 时)又注入 prompt(无 writer 时),语义不一致 | `context-engine.ts:1084-1102` | P3/P8 | +| **P-04** | **有损截断**:curated 段截到 3000、总量收敛缩成 stub(400);观察超 500 **静默丢弃**(无标记) | `store.ts:1535-1569`、`:1416` | P4 | +| **P-05** | **拒绝写入 × 永久超预算**:写路径 `{ok:false}` 拒绝,而拒绝无法缩小文件 → 超预算后**永久超**(自述实测 23 323 vs 15 000) | `store.ts:676-704`、注释 `:1206` | P4 | +| **P-06** | **单位不一致(char vs byte)**:写路径按 char 判总量,加载路径 `statSync.size` 按 byte 判 → CJK 下"15 000 字符"实为 ~45 000 字节,被误判超限 | `store.ts:676` vs `:1224` | P5 | +| **P-07** | **破坏性自愈**:>2MiB 或命中可疑标记即**重建**,可能固化损坏 | `store.ts:1247` | P6 | +| **P-08** | **三代补丁共存**:双→单 `data-meta`、legacy `, tags:`、`MEMORY.md`/`state.md` 迁移 | `store.ts:1247-1317`、`taxonomy.ts`、`agent.ts:6980` | 补丁 | +| **P-09** | **摘要覆盖**:`compactSession` 每次**覆盖** `session.summary`,多代压缩塌缩成一份 | `store.ts:900` | P6 | +| **P-10** | **碎片与观察共享 500 上限**:压缩碎片挤占真实观察配额 | `store.ts:866` | P4 | +| **P-11** | **整理触发面重叠**:dream cycle / `memory_consolidation` / 压缩器 / `daily_report` 四者职责交叉、不可预测 | `limits.ts:420`、`session-workspace.ts:16`、`store.ts:1544` | P2/P3 | +| **P-12** | **全自动不可见**:无预算/陈旧度信号;Agent 不知何时该整理(本次纯属偶然发现 150%) | 无信号注入;`memory_stats` 需主动调 | P3 | +| **P-13** | **场景硬编码分支**:`isDream/isReflex/isConverse` 决定注入内容,无场景化"补全信息"引导 | `context-engine.ts` 多处 | P3 | +| **P-14** | **命名分裂**:`knowledge.md`(记忆)vs `knowledge_*`(项目知识库)同名不同物 | `tools/memory.ts` vs `knowledge-*` 工具 | P8 | +| **P-15** | **admin 旁路**:org-manager 直接 `addLongTermMemory`,绕过统一锁/整理,可能与管理周期竞态 | `api-server.ts:5735/5827` | P1 | + +--- + +## 4. 目标架构(分层) + +``` +┌── A. Store 数据层(各自单一写入者;对外只读)───────────────────────────┐ +│ knowledge.md ← MemoryService(唯一写者) │ +│ NOTEBOOK.md ← NotebookService(唯一写者) │ +│ sessions/*.json← SessionService(唯一写者) │ +│ (项目知识库 docs 独立存储,命名 kb_*) │ +└──────────────────────────────────────────────────────────────────────────┘ +┌── B. Service 访问层(进程内,唯一写者;锁 + 原子写 + 不变量)────────────┐ +│ MemoryService: save/search/organize/forget/stats/applyCurated │ +│ NotebookService: read/upsert/clear/prune │ +│ SessionService: (沿用现有 session 原语) │ +│ ★ 所有写操作经此层;直接 file_write/file_edit 记忆文件 → 拒绝 │ +└──────────────────────────────────────────────────────────────────────────┘ +┌── C. Tool 工具层(Agent 主动调用;渐进披露)────────────────────────────┐ +│ memory_save/search/organize/update/forget/stats │ +│ notebook_read/upsert/clear │ +│ kb_search/read/list(项目知识库,去歧命名) │ +│ session(...) / recall_activity │ +└──────────────────────────────────────────────────────────────────────────┘ +┌── D. Prompt 提示词层(场景矩阵 §7;替代硬编码注入)─────────────────────┐ +│ 按场景注入"该主动拉取/写入什么"的引导 + 记忆健康信号(§8.3) │ +└──────────────────────────────────────────────────────────────────────────┘ +┌── E. Backstop 兜底层(无损 / 有界 / 可见)───────────────────────────────┐ +│ 预算再平衡 = 归档(`## _archived_*`,可检索),非截断 │ +│ dream = 定时调用 organize 原语,暴露 lastRunAt │ +│ 安全修复(不重建);原子写;任何丢弃须有落盘标记 + 上报 │ +└──────────────────────────────────────────────────────────────────────────┘ +``` + +**关键变化(相对现状)** +1. **CPP 取消**为独立"机制":其"检索/反思"能力改为 **(a)** 确定性轻量注入(进体积小的相关记忆,无 LLM 管道)+ **(b)** 提示词引导 Agent 主动 `memory_search`/`kb_search`。**不再有"产出写进 NOTEBOOK"的分叉**(消除 P-03)。 +2. `update_working_memory`/`clear_working_memory`/`update_notebook`/`clear_notebook` **四名归一**为 `notebook_upsert` / `notebook_clear` / `notebook_read`(旧名**直接删除**,未保留别名)。 +3. 记忆文件 **禁止** `file_write`/`file_edit`(工具层拒绝 + 引导)。 +4. 预算判定**全改字符**,写/读路径**共用同一 enforcement 函数**。 + +--- + +## 5. 数据模型与不变量 + +### 5.1 knowledge.md(长期记忆) +``` +# Knowledge +## ← 常驻注入;body ≤ KNOWLEDGE_SECTION_MAX_CHARS +...(N 段策展) +## _archived_ ← 归档段:被"再平衡"移出的内容,可检索、不注入 +## _observations ← 观察缓冲:`### ` + `` + body +``` +**不变量** +- INV-K1:`## _observations` 恒为**最后**段;永不作为 curate 段参与注入/检索(现状已满足)。 +- INV-K2:总字符 ≤ `KNOWLEDGE_TOTAL_MAX_CHARS`(**字符**,全路径一致)。超限 → **再平衡**: + 1. 先把最旧/最低价值策展段**归档**进 `## _archived_*`(内容保留、可检索、不再注入); + 2. 归档后仍超限 → 触发一次 `memory_organize`(归纳而非丢弃); + 3. 仍超限 → 上报并**保留最新观察**,绝不静默丢弃(对比现状 P-04/P-05)。 +- INV-K3:任何内容移除必须**可检索**(在 `_archived_*` 或 `memory_search` 命中),否则拒绝该移除。 +- INV-K4:写入原子(`writeFileAtomic`),崩溃不留半截。 + +### 5.2 NOTEBOOK.md(工作记忆) +``` +# Notebook +## + + + +``` +**不变量** +- INV-N1:单一写入者(NotebookService);**单一锁域**(`agent-memory` / sub `notebook`)。 +- INV-N2:TTL 分层(agent 96h / system 24h)+ 容量上限(总 16 / agent 4),驱逐顺序 **cpp→system→agent**、最旧优先;**CPP 层随 CPP 移除而消失**。 +- INV-N3:key 归一化(空白折叠、去 `#`、截断到 `NOTEBOOK_KEY_MAX_CHARS`)且**截断可报**。 +- INV-N4:并发分身共享一致(写经 debounce + maxWait + 单锁)。 + +### 5.3 sessions/*.json(执行史) +- INV-S1:`summary` 为**追加式多代锚点**(不覆盖,见 P-09)或明确只保留最新 + 旧锚在 fragments;二选一并写清语义。 +- INV-S2:fragments 与 observations **分池**(不再共享 500 上限,修 P-10)。 + +--- + +## 6. 工具层(全量清单) + +| 工具 | 归属 | 语义 | 关键约束 | +|---|---|---|---| +| `memory_save` | 长期 | 追加一条观察 | 单对象、非数组;`sanitize` 后写入 | +| `memory_search` | 长期 | 语义→关键词回退检索 | 覆盖 curated + observations + archived | +| `memory_organize` | 长期 | 把观察归纳/归档进策展段 | **唯一**整理原语(dream 亦调用它) | +| `memory_update` | 长期 | 策展段 upsert | 超 section 上限不再截断,改分页/归档 | +| `memory_forget` | 长期 | 删除策展段(= `removeLongTermSection`) | 删除前确认可归档 | +| `memory_stats` | 长期 | 预算/观察数/策展数/**lastConsolidatedAt** | 供提示词层读信号(§8.3) | +| `notebook_read` | 工作 | 读单/全量条目 | 只读 | +| `notebook_upsert` | 工作 | 写/更新条目 | 单一锁域;归一 key | +| `notebook_clear` | 工作 | 删条目/清空 | — | +| `kb_search`/`kb_read`/`kb_list` | 项目库 | 文档知识(原 `knowledge_*` 去歧) | 与 Agent 长期记忆**分名分物** | +| `session` / `recall_activity` | 会话 | 不改 | — | + +**删除**:`update_notebook`、`clear_notebook`、`update_working_memory`、`clear_working_memory` +(→ 归一为 `notebook_upsert` / `notebook_clear` / `notebook_read`);`knowledge_*`(→ `kb_*`)。已直接移除,未保留别名。 + +--- + +## 7. 场景 × 行为矩阵(P3 落点) + +| 场景 | 常驻注入(自动、低成本) | 提示词引导(Agent 主动拉 / 写) | +|---|---|---| +| **chat(Owner 对话)** | 策展段、sender 身份、时间、**记忆健康信号** | 涉及既有工作 → `memory_search` + `session` 回看;形成结论 → `memory_save` | +| **heartbeat** | 看板、团队状态、**记忆健康信号** | `memory_stats` 看预算 → 超阈值 `memory_organize`;无变化 → **不空转** | +| **task_execution** | 任务、验收标准、相关策展段 | 开工前 `memory_search` 经验;收尾 `memory_save` 教训 | +| **a2a / comment** | 对方身份、线程 | 需要对方历史 → `recall_activity` | +| **mailbox triage** | 邮箱项、看板 | 需决策依据 → `memory_search` | +| **dream(定时)** | 无(不注入) | 调用 `memory_organize` 原语;产出报告可观测 | + +> 原则:**平台把"状态"告诉 Agent,把"有损整理"交给 Agent 用工具做**。 + +--- + +## 8. 自动机制与兜底重构 + +### 8.1 CPP 移除(能力迁移,不减少功能) +- **删除**:CPP 的 LLM 多阶段管道 + `notebookWriter` 写 NOTEBOOK 分叉(P-03)。 +- **保留能力**: + - "检索相关记忆" → 确定性轻量注入(`retrieveRelevantMemories` 精简版,体积有界)+ 提示词引导主动 `memory_search`; + - "反思" → 由 `memory_organize` / 正常推理承担; + - 复杂/多跳 → Agent 主动调 `memory_search`/`kb_search`。 +- **类型**:`shared/types/cognitive.ts` 与 `CognitiveConfig/Stimulus` 一并移除或降为内部实现细节。 + +### 8.2 整理触发面收敛(P-11) +- **唯一原语** `memory_organize`。 +- **dream cycle** 退化为"每日一次调用 organize 原语的定时器",暴露 `lastRunAt`(可观测)。 +- `memory_consolidation` mailbox 类型:**合并**进 dream 定时器(不再单独产生条目)。 +- 压缩器(`compressLongTermMemory`):**保留但其"截断"改为"归档"**(见 8.4)。 +- `daily_report`:与 Agent 笔记**职责划清**(报告≠记忆);去除与 knowledge 的重复写入。 + +### 8.3 信号(P-12):让"必要时"成立 +- `memory_stats` 输出固定字段:`usedPercent / observations / curated / lastConsolidatedAt`。 +- **当 `usedPercent > 阈值(80%)`**:在 prompt 的"记忆健康"段出现一行提示,引导 `memory_organize`。 +- **陈旧**:`lastConsolidatedAt` 超过 N 天也在提示中体现。 +- 未超阈值**不出现**(避免噪声)。 + +### 8.4 再平衡 = 归档(P-04/P-05 的无损解) +``` +ensureBudget(knowledge): + while total(chars) > LIMIT: + // 1) 归档最旧/最低价值策展段(内容保留、可检索、不注入) + moveOldestCuratedTo('## _archived_') + // 2) 仍超 → 触发 memory_organize(归纳,非丢弃) + if stillOver: return { needsOrganize: true } // 交给 Agent + // 绝不静默丢弃;若无法无损收敛,保留最新 + 上报 +``` +- 写路径**不再直接 `{ok:false}` 拒绝**;改为"先再平衡、必要时请求 Agent 整理"。 +- 归档段可被 `memory_search` 命中(可恢复)。 + +### 8.5 安全修复替代破坏性重建(P-07) +- 自愈改为**保守修复**:仅当解析失败才回退;**绝不**因"看起来可疑"就重建;重建前先备份原文。 + +--- + +## 9. 前后端与接口改造 + +### 9.1 前端(`packages/web-ui`) +- `AgentProfile` 的 memory 折叠组:从"展示摘要"升级为 **健康可视化 + 只读查看**(预算条、观察数、上次整理时间;`knowledge.md`/`NOTEBOOK.md` 只读渲染)。 +- `AgentMemorySummary`(`api.ts`)**增字段**:`usedPercent / observations / curated / lastConsolidatedAt / archivedCount`。 +- **无独立写路径**:前端任何"整理"动作走 admin API → 服务层(同一写入者),不直改文件。 +- 组件/快照测试同步更新。 + +### 9.2 后端 / 接口(`packages/org-manager`) +- **admin 写路径归口**(P-15):`api-server.ts` 的 `addLongTermMemory` 直调改为经 `MemoryService`(拿锁、走不变量)。 +- 修 `storage-usage.ts` 的"memory 体积"误导(只统计真正的记忆文件,去掉 sessions 的 99.98% 噪声)。 +- 端点清单与返回字段随 §9.1 调整;知识库(`FileKnowledgeStore`)保持独立、**改名 `kb_*`**。 + +### 9.3 共享类型(`packages/shared`) +- `limits.ts`:删除 byte 预算常量/注释,统一字符;`NOTEBOOK_*_CPP` TTL 随 CPP 移除而删。 +- `types/knowledge.ts`:新增 `archived` 段类型;`KnowledgeSection` 增 `archived:boolean`。 + +--- + +## 10. 测试计划(全量改齐,不留旧断言) + +### 10.1 必须更新的既有测试(当前钉死旧行为) +- `cache-optimization.test.ts:577`(CPP→notebook 路由) +- `cognitive.test.ts:257/286/309/337`、`cognitive-deep.test.ts`(CPP 段落/管道) +- `memory-store.test.ts:35/257/264/338/350/360`、`knowledge-lifecycle.test.ts:67/88/100`、 + `memory-convergence.test.ts:34/50/60`(有损截断 / 拒绝写入) +- `notebook-lifecycle.test.ts:64/69/71-73/178`(CPP 层 TTL) +- `agent-targeted-coverage.test.ts:324/372`(dream 剪枝,改断言唯一原语) +- `context-engine.test.ts:138/698+`(consolidation 场景、Learning Habits 文案) + +### 10.2 必须新增的回归(当前缺口) +1. **`file_write`/`file_edit` 直改记忆文件被拒 + 引导消息**(头号缺口,事故无覆盖)。 +2. **无"永久超预算"路径**:预算恒 ≤100%;写路径拒绝分支被"再平衡"取代。 +3. **归档可逆**:被归档内容可 `memory_search` 找回。 +4. **预算信号注入**:>阈值出现、未超不出现。 +5. **CPP 产出不再进 NOTEBOOK**(单目的地)。 +6. **dream 唯一且可观测**(`lastRunAt`)。 +7. **并发**:内存落盘 × 外部 file_write 不丢数据。 +8. **命名去歧**:`memory_*` 与 `kb_*` 不串。 +9. **单位一致**:CJK 文件不被误判超限。 +10. **admin 写入走服务层**(锁/不变量生效)。 + +### 10.3 运行 +`npx vitest run --project node `(单文件)· `pnpm test:node` · `pnpm test:web-ui` · coverage ratchet。 + +--- + +## 11. 删除清单(拒绝"留补丁") + +- 代码:CPP 管道与 `notebookWriter` 分叉;`update_notebook`/`clear_notebook`/`update_working_memory`/`clear_working_memory`(归一);`knowledge_*`(→`kb_*`)。 +- 数据:`NOTEBOOK_*_CPP` TTL 常量;byte 预算常量。 +- 迁移残留:双→单 `data-meta` 兼容分支、legacy `, tags:` 容忍、`MEMORY.md`/`state.md` 迁移(跑完一次性迁移后**删除代码**)。 +- 死导出:`parseCuratedSections`/`slugSectionId` 若无外部引用则内联或导出并测试。 + +--- + +## 12. 验收 + 功能不减少(feature parity) + +**验收(全部必须满足)** +- INV-K1..K4 / INV-N1..N4 / INV-S1..S2 全部有测试。 +- §10.2 十项回归全绿;§10.1 全部更新后全绿。 +- 无"永久超预算";无静默丢失;`memory_search` 可命中归档内容。 +- 前端展示健康数据;无任何直改记忆文件的路径。 + +**功能不减少(parity 矩阵)** + +| 旧能力 | 新实现 | ✅ | +|---|---|---| +| 长期记忆读写/检索 | `memory_save/search/update/forget` | ✅ | +| 观察→策展归纳 | `memory_organize`(唯一原语) | ✅ | +| 工作记忆读写/共享 | `notebook_read/upsert/clear`(统一锁域) | ✅ | +| 检索相关记忆注入 | 轻量确定性注入 + 提示词引导工具 | ✅ | +| 反思 | 归纳/推理承担 | ✅ | +| 预算保护/防损坏 | 无损再平衡 + 安全修复 | ✅ | +| 执行史回溯 | `session` / `recall_activity` | ✅ | +| 项目知识库 | `kb_*`(去歧) | ✅ | +| 前端 memory 视图 | 健康可视化 + 只读查看 | ✅ | + diff --git a/packages/core/docs/agent-self-management-redesign.md b/packages/core/docs/agent-self-management-redesign.md new file mode 100644 index 000000000..aa96ffd8f --- /dev/null +++ b/packages/core/docs/agent-self-management-redesign.md @@ -0,0 +1,161 @@ +# Agent 自我管理机制重构方案(P0/P1) + +> 作者:CTO | 日期:2026-09-27 | 状态:已批准开发 +> 背景:老板(Owner)以「平台入驻 Agent 第一视角」提出:Agent 应有自我管理能力——能给自己定闹钟、能回顾/整理记忆、分身之间能知道彼此在做什么。经平台自体检(CTO 亲自作为 Agent 实测)发现:**声称拥有的能力有一批是「声明了但不接线」的**,正是此前多轮 bug(心跳风暴、cancelled 停滞、终态乱标)的同根问题。本文档调查现状 → 定位根因 → 给出修复/实现方案 → 分阶段开发。 + +--- + +## 一、四个问题域 + +| 编号 | 领域 | 现象(Agent 实测) | +|---|---|---| +| P0-1 | 工具注册可用性 | `schedule_wakeup`/`cancel_wakeup`/`set_heartbeat_interval`/`complete_deliberation` 出现在系统提示「Deferred Tools」列表、要求 discover 后使用,但 `discover_tools({name:[...]})` 返回 **unknown/not found**——激活永远失败 | +| P0-2 | 自我闹钟 | 「到点做某事」能力名义存在(`schedule_wakeup` schema 里有),但 Agent 实际**无法调用**(P0-1 同根因:schema 有、handler 不可达) | +| P1-1 | 记忆整理工具化 | memory_save/search/update 存在,但**没有「观察→合并→归档」的显式工具**;knowledge.md 超 15000 字符预算只能靠系统被动压缩,Agent 无感知、无主动整理入口 | +| P1-2 | 分身共享工作记忆 | 并发分身(worker>1)有交接日志,但**无「其他分身的工作记忆快照」**——分身 A 不知道分身 B 在做什么/做过什么(除 8 条 handoff 摘要外) | + +--- + +## 二、现状调查(证据链) + +### P0-1 工具注册可用性:schema 注入与 handler 注册「两张皮」 + +**关键事实:** `packages/core/src/tool-selector.ts` 通过 `pushUnique()` **无条件注入**以下工具的 schema: + +- `schedule_wakeup` / `cancel_wakeup` / `set_heartbeat_interval` (L483-521) +- `recall_activity` (L523-547) +- `complete_deliberation` (L549-579) +- `notebook_upsert` / `notebook_clear`(原 `update_working_memory` / `clear_working_memory`,L586-609) + +但这些工具的**执行路径不是 `registerTool()` 注册的 handler**,而是 `agent.ts` 里的 `if (toolCall.name === '...')` 分支 dispatch(L8376-8444)。 + +`discover_tools` 的激活逻辑(`agent.ts handleDiscoverTools`,L7907-7988)只认两个来源: + +```ts +if (this.tools.has(name)) { /* 注册过的 handler → 激活 */ } +if (this.skillRegistry?.get(name)) { /* skill → 注入说明 */ } +// 否则 → unknown.push(name) +``` + +**根因链:** +1. 预算不足时,`evictToolsToBudget` 把非 protected 工具驱逐进「Deferred Tools」catalog(`capability-packs.ts` `formatEvictedToolCatalog`); +2. 系统提示要求先 `discover_tools` 再使用; +3. 但 `schedule_wakeup` 等**未注册进 `this.tools`**(仅 if 分支 dispatch)→ `handleDiscoverTools` 判定 unknown → **激活失败**; +4. Agent 永远拿不回被驱逐的工具 schema → 工具名存续、功能死锁。 + +**实测复现(CTO 亲自执行):** +``` +discover_tools({name:["schedule_wakeup","set_heartbeat_interval","cancel_wakeup","complete_deliberation"]}) +→ { status:"ok", activated:[], unknown:[...全部 4 个], hint:"not found as tools or skills" } +``` + +**对照成功案例:** `recall_activity` / `notebook_upsert` / `notebook_clear` 能激活成功——因为它们分别注册在 `createRecallTool`(agent-manager L1732)与 `createMailboxTools`(L1728)里。证明:**「注册进 this.tools」是 discover 可激活的充要条件**。 + +### P0-2 自我闹钟:后端已完整,只差「工具可达」 + +`schedule_wakeup` 的**后端实现是完整的**: + +- 注册:`agent.ts` L8376-8411 → `pendingCallbackRegistry.register({type:'wakeup', deliveryMode, note, wakeAt, recurringMs})` +- 持久化:`org-manager/api-server.ts` L1397 `pendingCallbackRegistry.setPersistence(storage.pendingCallbackRepo)`;`sqlite-storage.ts` 有 `pending_callbacks` 表 + `wake_at` 索引 + `recurring_ms`/`note` 列 +- 触发:`agent.ts` `startWakeupSweep()` L7186 + `sweepDueWakeups()` L7198(~1 分钟粒度,到期 → `deliverCallback` → mailbox / in_session,recurring 自动 re-arm) + +**缺口:** 工具 schema 存在,但被预算驱逐后 discover 无法激活(P0-1)→ Agent 实际调不到。**修好 P0-1,闹钟即通。** 附带的完善点:sweep 是 setInterval 常驻(unref),agent idle 时不烧 token,设计 OK。 + +### P1-1 记忆整理:有「存/查/改」无「整理」,预算压缩被动且不可见 + +现有能力(`packages/core/src/tools/memory.ts` + `memory/store.ts`): + +- `memory_save`:写 `knowledge.md ## _observations` +- `memory_search` / `memory_list`:关键词检索(observations + curated sections) +- `memory_update` / `memory_update_longterm`:写 curated sections +- 预算:`shared/src/limits.ts` — `MEMORY_MD_SECTION_MAX_CHARS=3000`、`MEMORY_MD_TOTAL_MAX_CHARS=15_000`、`MEMORY_ENTRY_MAX_CHARS=4_000` +- 超预算自动压缩:`store.ts` L676-697 `compressLongTermMemory()`(压缩后仍超则拒绝写入) +- 被动 consolidation:`agent.ts` L611 `MEMORY_CONSOLIDATION_INTERVAL_MS=4h` → `consolidateMemory()`(dream cycle,信号量限 3 并发、随机初始延迟防风暴) + +**缺口:** +1. **无显式整理工具**:观察会无限堆积在 `## _observations`,没有「把 N 条 observations 合并成一条 curated section 并标记已归档」的入口; +2. **预算不可见**:Agent 不知道 knowledge.md 当前多大、还剩多少预算、有多少 observations 待整理; +3. **dream cycle 对 Agent 不可感知/不可触发**:平台 4h 自动跑,Agent 无法主动请求一次 consolidate。 + +### P1-2 分身共享工作记忆:有 handoff 摘要,无共享工作记忆快照 + +现有(`packages/core/src/concurrent-handoff.ts` + `context-engine.ts`): + +- `ConcurrentHandoffLog`:declared/fact/done/conflict 四类记录,内存环形(64)+ JSONL 持久化(512 行压实),`inFlight()`/`byEntity()` 查询; +- `context-engine.ts` L1239-1242:并发模式(worker>1)注入 `concurrentContext`(handoffs + workerId/workerCount),每轮**最多 8 条**(`HANDOFF_CONTEXT_LIMIT`); +- `attention.ts` worker 机制:`workerStates` 按 workerId 隔离状态,`getWorkerWorkspace(workerId)` 返回各分身独立 workspace(`session-workspace.ts`)。 + +**缺口:** +1. Handoff 只有「一句话摘要」,不含**工作记忆(notebook)内容**;分身 A 的 `notebook_upsert` 内容,分身 B 的并发上下文里**看不到**; +2. 超过 8 条/64 条即被挤出,长任务中途换分身会丢上下文; +3. 没有「主动读其他分身工作记忆」的工具。 + +--- + +## 三、修复/实现方案 + +### P0-1:统一「schema 注入工具」的激活路径(核心修复) + +**原则:** 凡是 schema 里出现、且系统提示要求「先 discover 再用」的工具,必须能被 discover 激活。不能出现「列表里有、激活失败」的死锁。 + +**实现:** +1. `capability-packs.ts` 新增导出 `SCHEMA_INJECTED_TOOLS`(常量集合,与 `tool-selector.ts` pushUnique 注入清单同源): + `schedule_wakeup, cancel_wakeup, set_heartbeat_interval, recall_activity, complete_deliberation, notebook_upsert, notebook_clear`; +2. `agent.ts handleDiscoverTools` 激活分支**优先判定**:name 命中 `SCHEMA_INJECTED_TOOLS` → `stickyTools().activated.add(name)` 并返回 `activated`(这些工具 schema 由 selectTools 注入,激活后晋升 protected 永不被驱逐——P0-3 审计已保证); +3. 若后续某工具改为 `registerTool` 注册,命中 `this.tools` 的自然走原路径,两路兼容; +4. 测试:`discover_tools({name:['schedule_wakeup']})` 返回 activated;此后 selectTools 输出包含 `schedule_wakeup`,且系统提示不再把它列为「需 discover」的 deferred 项。 + +### P0-2:自我闹钟打通 + 心智模型完善 + +1. 依赖 P0-1 修复(工具可达)——**核心**; +2. 在心智层(提示词/HEARTBEAT 纪律)明确:精确到点的自我提醒 → `schedule_wakeup`;周期性安全网 → `set_heartbeat_interval`;二者职责分离(呼应此前心跳风暴修复); +3. 验证链:注册 → 持久化 → sweep 触发 → deliverCallback → mailbox item → attention 唤醒处理(补端到端测试)。 + +### P1-1:记忆整理工具化 + +新增两个能力(`tools/memory.ts` + `memory/store.ts`): + +1. **`memory_stats`(新工具)**:返回 knowledge.md 当前大小 / 预算(15k)、observations 条数、curated section 数、距上次主动整理的天数——让 Agent 看到「记忆健康度」; +2. **`memory_organize`(新工具)**:把 `## _observations` 中满足条件的条目合并进目标任务(move/archive): + - 输入 `target_section` + 可选 `query`/`ids`; + - 从 observations 中筛出匹配条目 → 内容累积进目标 section(预算内)→ 原 observations 删除(归档); + - 返回移入/归档统计 → Agent 据此决定是否 `memory_update` 精修。 +3. `memory_update` 预算自动压缩后**返回可见的压缩统计**(原来只 warn 日志); +4. dream cycle 增加**可观察性**:`consolidateMemory` 完成后记录上次整理时间(供 memory_stats 读取)。 + +### P1-2:分身共享工作记忆 + +1. `context-engine.ts` 并发上下文段**增加工作记忆快照**:注入当前 agent 的 notebook(`getWorkingMemorySnapshot()`)中**其他 worker 最近写入/更新的条目**(带 workerId 标注),与 handoff 摘要并列; +2. `ConcurrentHandoffLog` 新增 `kind='fact'` 语义强化:允许写入「影响全局的工作记忆」摘要,`byEntity`/`inFlight` 已支持; +3. 提供 `shared_working_memory`(或复用 `notebook_upsert` + 注入):分身的 `notebook_upsert` 写入的 key 在并发上下文可见(标注 workerId)——让分身 B 知道「分身 A 正在/刚在做什么」; +4. 上限控制:快照条数 ≤ 8、每条约 200 字符(防预算膨胀),只增改并发段、不动 stable 前缀。 + +--- + +## 四、分阶段实施 + +| 阶段 | 内容 | 涉及文件 | 验收 | +|---|---|---|---| +| S1(P0) | discover 激活路径 + 闹钟打通 | capability-packs.ts、agent.ts、tool-selector.ts(同源清单)、测试 | discover 4 工具全激活;schedule_wakeup 端到端注册/触发/投递 | +| S2(P1-1) | memory_stats + memory_organize | tools/memory.ts、memory/store.ts、shared/limits.ts(如需)、测试 | 两工具可用;整理后 observations 归档、目标 section 增长、预算统计可见 | +| S3(P1-2) | 分身上下文共享工作记忆 | context-engine.ts、concurrent-handoff.ts、agent-manager.ts(快照接线)、测试 | 并发模式注入含 worker 标注的 notebook 快照;handoff fact 可查 | + +## 五、全局原则(延续) + +- **不做「声明的能力」**:任何 schema 里的工具必须可 discover、可调用——否则从 schema 移除; +- **单一事实源**:工具注册表、schema 注入清单、discover 激活路径三方同源,杜绝「两张表漂移」; +- **自我管理优先**:闹钟/心跳/记忆整理是 Agent 的自理器官,默认常驻、轻量、可观察,不依赖人类提醒; +- **不烧 token 的安全网**:sweep/consolidation 均用 unref 定时器 + 信号量限流,idle 不产生 LLM 调用。 + +--- + +## 六、实施状态(2026-09-27 已全部落地) + +| 阶段 | 状态 | 落地内容 | 验证 | +|---|---|---|---| +| S1(P0) | ✅ 完成 | `capability-packs.ts` 新增 `SCHEMA_INJECTED_TOOLS`;`agent.ts handleDiscoverTools` 对命中集合的工具走「激活」路径(sticky activated);`tool-selector.ts` 预算驱逐时已激活的 schema-injected 工具同样豁免(原先只豁免 `allTools` 注册工具) | `schema-injected-tools.test.ts` 5/5 绿 | +| S2(P1-1) | ✅ 完成 | `tools/memory.ts` 新增 `memory_stats`(预算/观测数/curated section 数/健康度提示)+ `memory_organize`(按 query/ids 把观测合并进目标 section 并归档,预算内、无匹配不写库) | `memory-organize-tools.test.ts` 6/6 绿 | +| S3(P1-2) | ✅ 完成 | `context-engine.ts` 并发段注入「分身共享工作记忆」快照(agent 级 NOTEBOOK,限量 5 条、每条约 140 字符截断);`agent.ts getConcurrentContext()` 附带 `getWorkingMemorySnapshot()` | `context-concurrency-notebook.test.ts` 3/3 绿 | +| 回归 | ✅ | tool-selector / memory-tools / context-engine / context-engine-deep 等既有套件 | 78/78 绿,tsc 0 错误 | + +**说明**:P0-2(自我闹钟)后端链路本就完整(注册→持久化→sweep→投递),S1 修好 discover 激活后即达可用,无需额外代码;S2/S3 中「dream cycle 可观察性」「handoff fact 强化」两项列为后续增量(不阻塞本次交付)。 \ No newline at end of file diff --git a/packages/core/src/agent-manager.ts b/packages/core/src/agent-manager.ts index e9df8397f..151eeb265 100644 --- a/packages/core/src/agent-manager.ts +++ b/packages/core/src/agent-manager.ts @@ -599,7 +599,7 @@ export class AgentManager { // A project with NO bound knowledge roots has nothing to read — the // unbound case must not silently open arbitrary files. if (roots.length === 0) { - log.warn('knowledge_read rejected: project has no knowledge base paths', { projectId }); + log.warn('kb_read rejected: project has no knowledge base paths', { projectId }); return null; } const resolvedRef = resolve(reference); diff --git a/packages/core/src/agent.ts b/packages/core/src/agent.ts index 2cd722460..426de9b71 100644 --- a/packages/core/src/agent.ts +++ b/packages/core/src/agent.ts @@ -24,8 +24,7 @@ import { type TriageResult, type DeliberationResult, type CognitiveConfig, - type CognitiveStimulus, - type CognitiveAgentContext, + CognitiveDepth, type PreparedCognitiveContext, MailboxPriorityLevel, MAILBOX_TYPE_REGISTRY, @@ -54,7 +53,6 @@ import { NOTEBOOK_MAX_ENTRIES, NOTEBOOK_MAX_CHARS_PER_ENTRY, NOTEBOOK_PROMPT_MAX_CHARS, - NOTEBOOK_RELEVANT_CONTEXT_MAX_CHARS, NOTEBOOK_PERSIST_MAX_WAIT_MS, } from '@markus/shared'; import { AGENT_MEMORY_RESOURCE_DOMAIN, memoryResourceForPath, memoryResourceLock, type AgentMemoryResource } from './lock-resources.js'; @@ -63,7 +61,7 @@ import { EventBus } from './events.js'; import { createTokenCounter, type SmartTokenCounter } from './token-counter.js'; import { GuardrailPipeline } from './guardrails.js'; import { ToolHookRegistry, generateIdempotencyKey, type ToolHook } from './tool-hooks.js'; -import { HeartbeatScheduler } from './heartbeat.js'; +import { HeartbeatScheduler, heartbeatStateFingerprint } from './heartbeat.js'; import type { LLMRouter } from './llm/router.js'; import { stripToolNoise } from './llm/provider-helpers.js'; import { MemoryStore, loadNotebook, saveNotebook, pruneNotebookEntries, normalizeNotebookKey, type NotebookEntry, type NotebookEntryManaged } from './memory/store.js'; @@ -71,7 +69,6 @@ import type { IMemoryStore, MemoryEntry } from './memory/types.js'; import type { SemanticMemorySearch } from './memory/semantic-search.js'; import { AgentMetricsCollector, type AgentMetricsSnapshot } from './agent-metrics.js'; import { ContextEngine, type OrgContext, type LLMSummarizer, type SystemPromptSegment } from './context-engine.js'; -import { CognitivePreparation, selectCognitiveDepth } from './cognitive.js'; import { detectEnvironment, type EnvironmentProfile } from './environment-profile.js'; import { ToolSelector } from './tool-selector.js'; import { @@ -80,6 +77,7 @@ import { COMMENT_RESPONSE_ALLOWED_TOOLS, REQUIREMENT_ACTION_ALLOWED_TOOLS, WORKFLOW_ACTION_ALLOWED_TOOLS, + SCHEMA_INJECTED_TOOLS, type CapabilityPack, } from './capability-packs.js'; import { ensureAffordablePromptPack } from './afford-guard.js'; @@ -551,8 +549,7 @@ export class Agent { private notebookSaveTimer?: ReturnType; /** Earliest time the debounced notebook write may be deferred to (maxWait). */ private notebookSaveDeadline?: number; - /** Cognitive Preparation Pipeline instance (null when CPP is disabled) */ - private cognitivePrep?: CognitivePreparation; + /** 审计 §8.1:CPP 的 LLM 管道已移除;仅保留配置(决定是否注入确定性情境块)。 */ private cognitiveConfig?: CognitiveConfig; /** Ring buffer of recent activity summaries for triage context. */ private recentActivityRing: string[] = []; @@ -752,7 +749,6 @@ export class Agent { this.toolHooks = new ToolHookRegistry(); this.metricsCollector = new AgentMetricsCollector(this.id, options.dataDir); if (options.cognitive?.enabled) { - this.cognitivePrep = new CognitivePreparation(options.cognitive); this.cognitiveConfig = options.cognitive; } this.heartbeat = new HeartbeatScheduler(this.id, this.eventBus, { @@ -942,9 +938,66 @@ export class Agent { } /** - * Set agent status and emit status change event + * 单一状态转换意图(状态机收敛)——调用点只表达「发生了什么」, + * 不自行判断守卫;终态由 transitionStatus 内部按聚合状态 + 优先级裁定。 + * - working :某个工作单元开始(幂等;可从 error 恢复并清错) + * - idle :某个工作单元结束,请求回空闲(仅聚合空闲时落地;error 粘性:不被覆盖) + * - error :工作单元失败(记录错误;仅无活跃任务时落地,避免部分失败污染全局状态) + * - offline :生命周期 stop(无条件) + * - reset :生命周期 start / reconcile 兜底(无条件回 idle 并清错误) */ - private setStatus(status: AgentState['status'], errorMessage?: string): void { + private transitionStatus( + intent: + | { to: 'working' } + | { to: 'idle' } + | { to: 'error'; message: string } + | { to: 'offline' } + | { to: 'reset' }, + opts?: { force?: boolean }, + ): void { + const oldStatus = this.state.status; + const force = opts?.force === true; + + // 生命周期/兜底:无条件落地(start / stop / reconcile)。 + if (intent.to === 'reset') { + this.applyStatus('idle'); + return; + } + if (intent.to === 'offline') { + this.applyStatus('offline'); + return; + } + + // working:幂等;新工作开始视为恢复,允许覆盖 error 并清错。 + if (intent.to === 'working') { + if (oldStatus === 'working') return; + this.applyStatus('working'); + return; + } + + // error:仅在无活跃任务时落地(与历史语义一致:部分失败不污染全局状态)。 + if (intent.to === 'error') { + if (!force && this.activeTasks.size > 0) return; + this.applyStatus('error', intent.message); + return; + } + + // idle:error 粘性 —— 一旦进入 error,后续 idle 意图不得覆盖(需 reset / 新 working 恢复); + // 且仅当聚合空闲(无活跃任务;并发模式下各 worker 均空闲)才落地,避免与并行工作互相覆盖。 + if (oldStatus === 'error' && !force) return; + if (!force && this.activeTasks.size > 0) return; + if ( + !force && + this.attentionController.getWorkerCount() > 1 && + this.attentionController.getState() !== 'idle' + ) { + return; + } + this.applyStatus('idle'); + } + + /** 状态落地的唯一写入点:维护 lastError/lastErrorAt + stateManager + 状态通知。 */ + private applyStatus(status: AgentState['status'], errorMessage?: string): void { const oldStatus = this.state.status; if (oldStatus === status && status !== 'error') return; @@ -967,7 +1020,8 @@ export class Agent { } async start(options?: { initialHeartbeatDelayMs?: number }): Promise { - this.setStatus('idle'); + // start 是生命周期重置:无条件回 idle 并清错误。 + this.transitionStatus({ to: 'reset' }); this.stopReason = undefined; // Load persistent notebook (NOTEBOOK.md) @@ -1092,7 +1146,7 @@ export class Agent { } this.metricsCollector.flush(); this.stopReason = reason; - this.setStatus('offline'); + this.transitionStatus({ to: 'offline' }); this.eventBus.emit('agent:stopped', { agentId: this.id }); log.info(`Agent stopped: ${this.config.name}`, { reason }); } @@ -1511,15 +1565,10 @@ export class Agent { mailboxDepth: this.mailbox.depth, }); if (item) { - this.setStatus('working'); - } else if (this.attentionController.getWorkerCount() > 1) { - // 并发模式:单个 worker 结束不代表 agent 空闲 —— 只有所有 worker - // 都空闲(聚合状态 idle)才恢复 idle,避免状态抖动。 - if (this.attentionController.getState() === 'idle' && this.activeTasks.size === 0) { - this.setStatus('idle'); - } - } else if (this.activeTasks.size === 0) { - this.setStatus('idle'); + this.transitionStatus({ to: 'working' }); + } else { + // 空闲判定统一收敛到 transitionStatus(活跃任务 + 并发 worker 聚合判断在函数内完成)。 + this.transitionStatus({ to: 'idle' }); } }, getWorkerWorkspace: (workerId: number) => { @@ -3630,20 +3679,6 @@ export class Agent { } } - private getNotebookWriter(): (key: string, text: string, managed: 'system' | 'cpp') => void { - return (key: string, text: string, managed: 'system' | 'cpp') => { - // `relevant-context` is a transcript of retrieved knowledge that is ALREADY - // injected as `## Your Knowledge`. One such entry measured 9 475 chars — more - // than the entire notebook prompt budget (6 000) — so it used to crowd every - // real note out of the injected block. Cap it hard; the full text remains - // retrievable via `memory_search`. - const capped = key === 'relevant-context' - ? text.slice(0, NOTEBOOK_RELEVANT_CONTEXT_MAX_CHARS) - : text; - this.writeNotebookEntry(key, capped, managed); - }; - } - /** Activated skill bodies — injected as their own uncapped system section. */ private getActivatedSkillContext(): string | undefined { const instructions = this.activatedSkills(); @@ -3669,7 +3704,7 @@ export class Agent { if (this.workingMemory.size > 0) { const wmLines = ['## Notebook']; - wmLines.push(`Your cognitive workspace (max ${NOTEBOOK_MAX_ENTRIES} entries, ${Agent.NOTEBOOK_PROMPT_MAX_ENTRIES} shown). Persists across sessions. Update via \`update_notebook\`. System entries are auto-managed and expire on their own. Choose keys wisely — the oldest entry is evicted when full.`); + wmLines.push(`Your cognitive workspace (max ${NOTEBOOK_MAX_ENTRIES} entries, ${Agent.NOTEBOOK_PROMPT_MAX_ENTRIES} shown). Persists across sessions. Update via \`notebook_upsert\`. System entries are auto-managed and expire on their own. Choose keys wisely — the oldest entry is evicted when full.`); wmLines.push(''); // Deterministic, most-recent-first order. Two properties matter here: @@ -3731,60 +3766,34 @@ export class Agent { } /** - * Run the Cognitive Preparation Pipeline (appraisal phase) before the main LLM call. - * Returns undefined when CPP is disabled or on error (caller falls back to mechanical retrieval). + * 审计 §8.1:确定性情境准备(原 CPP 的 LLM 多阶段管道已移除)。 + * + * 旧 CPP 在主调用前跑 0–3 次 LLM(appraise / reflect),带来额外时延与成本, + * 且与上下文引擎自身「有界的相关记忆检索」重复。现改为**纯确定性**装配一小段 + * 情境块(近期活动 + 工作记忆键);更深的跨域召回由 Agent 按场景提示主动调 + * `memory_search` / `kb_search` 承担。返回 undefined 表示无情境可注入。 */ private async prepareCognitiveContext( - scenario: string, - message: string, - sender?: string, + _scenario: string, + _message: string, + _sender?: string, ): Promise { - if (!this.cognitivePrep) return undefined; - - const stimulus: CognitiveStimulus = { - type: scenario, - summary: message.slice(0, 200), - content: message, - sender, - scenario, - }; - - const agentCtx: CognitiveAgentContext = { - id: this.id, - name: this.config.name, - roleDescription: this.role.systemPrompt.slice(0, 300), - status: this.state.status, - currentTask: this.currentTaskId, - recentActivity: [ - ...this.recentActivityRing.slice(-5), - ...(this.workingMemory.size > 0 - ? [`[working_memory] ${this.workingMemory.size} entries: ${[...this.workingMemory.keys()].join(', ')}`] - : []), - ], - }; - - const tasks = this.tasksFetcher?.(); - const hasFailedTasks = tasks?.some(t => t.status === 'failed') ?? false; - const hasBlockers = tasks?.some(t => t.status === 'blocked') ?? false; - - const depth = selectCognitiveDepth(scenario, { hasFailedTasks, hasBlockers }, message.length); - - try { - const startMs = Date.now(); - const timeoutMs = this.cognitiveConfig?.timeoutMs ?? 15_000; - const result = await Promise.race([ - this.cognitivePrep.prepare(stimulus, agentCtx, depth, this.llmRouter), - new Promise((_, reject) => - setTimeout(() => reject(new Error(`CPP timed out after ${timeoutMs}ms`)), timeoutMs), - ), - ]); - const elapsedMs = Date.now() - startMs; - log.info('CPP completed', { depth: result.depth, isEmpty: result.isEmpty, elapsedMs }); - return result; - } catch (err) { - log.warn('CPP failed, falling back to mechanical retrieval', { error: String(err) }); - return undefined; + if (!this.cognitiveConfig?.enabled) return undefined; + const parts: string[] = []; + const activity = this.recentActivityRing.slice(-3); + if (activity.length > 0) { + parts.push('Recent activity:', ...activity.map((a) => `- ${a}`)); } + if (this.workingMemory.size > 0) { + const keys = [...this.workingMemory.keys()].slice(0, 8).join(', '); + parts.push(`Working memory: ${this.workingMemory.size} entries (${keys})`); + } + if (parts.length === 0) return undefined; + return { + depth: CognitiveDepth.D1_Reactive, + cognitiveContext: parts.join('\n'), + isEmpty: false, + }; } private getMailboxContext(): { @@ -3831,12 +3840,19 @@ export class Agent { workerId: number; workerCount: number; handoffs: Array<{ workerId: number; kind: 'declared' | 'fact' | 'done' | 'conflict'; entityKey?: string; summary: string }>; + notebook?: Array<{ key: string; text: string; updatedAt: number; managed?: string }>; } | undefined { const log = this.handoffLog; const workerCount = this.attentionController.getWorkerCount(); if (!log || workerCount <= 1) return undefined; const workerId = this.workspace().workerId; - return { + const ctx: { + enabled: boolean; + workerId: number; + workerCount: number; + handoffs: Array<{ workerId: number; kind: 'declared' | 'fact' | 'done' | 'conflict'; entityKey?: string; summary: string }>; + notebook?: Array<{ key: string; text: string; updatedAt: number; managed?: string }>; + } = { enabled: true, workerId, workerCount, @@ -3847,6 +3863,12 @@ export class Agent { summary: h.summary, })), }; + // P1-2:分身共享工作记忆 —— notebook 是 agent 级共享的(每个分身写的是同一 + // 个 NOTEBOOK.md),但上下文默认不注入,分身之间互不知晓。并发模式下把快照 + // 一起带上,让每个 worker 感知其他分身的 working-memory 状态。 + const nb = this.getWorkingMemorySnapshot(); + if (nb.length > 0) ctx.notebook = nb; + return ctx; } setAuditCallback( @@ -4415,13 +4437,11 @@ export class Agent { // User Cancel on a non-stream turn (heartbeat etc.) — stop before spending an LLM call. if (this.isUserProcessingCancelled()) { log.info('handleMessage cancelled by user before start', { agentId: this.id, scenario: options?.scenario }); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return '[cancelled]'; } - if (this.activeTasks.size === 0) { - this.setStatus('working'); - } + this.transitionStatus({ to: 'working' }); const scenario = options?.scenario ?? 'chat'; const isLightweight = scenario !== 'chat' && scenario !== 'task_execution' && scenario !== 'review'; @@ -4545,7 +4565,6 @@ export class Agent { concurrentContext: this.getConcurrentContext(), workflowContext: isLightweight ? undefined : this.workflowContextFetcher?.(), cognitiveContext, - notebookWriter: this.getNotebookWriter(), ...this.getTeamContextParams(), }); const { volatile } = systemPromptBuild; @@ -5127,7 +5146,7 @@ export class Agent { this.emitActivityLog(chatActivityId, 'text', displayReply); } if (chatActivityId) this.endActivity(chatActivityId); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); this.eventBus.emit('agent:message', { agentId: this.id, @@ -5149,7 +5168,7 @@ export class Agent { }); } catch { /* avoid masking the original error */ } - if (this.activeTasks.size === 0) this.setStatus('error', String(error)); + this.transitionStatus({ to: 'error', message: String(error) }); this.emitAudit({ type: 'error', action: 'handle_message', @@ -5193,16 +5212,14 @@ export class Agent { // onEvent callback is already a no-op when the connection is gone. if (cancelToken?.cancelled && cancelToken.userStopped) { log.info('Stream cancelled by user before processing started', { agentId: this.id }); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return '[cancelled]'; } if (cancelToken?.cancelled && !cancelToken.userStopped) { log.info('SSE disconnected before processing started — continuing without streaming', { agentId: this.id }); } - if (this.activeTasks.size === 0) { - this.setStatus('working'); - } + this.transitionStatus({ to: 'working' }); // Track chat activity for streaming let streamChatActivityId: string | undefined; @@ -5292,7 +5309,6 @@ export class Agent { concurrentContext: this.getConcurrentContext(), workflowContext: this.workflowContextFetcher?.(), cognitiveContext, - notebookWriter: this.getNotebookWriter(), ...this.getTeamContextParams(), }); const { volatile: volatileState } = systemPromptBuild; @@ -5455,7 +5471,7 @@ export class Agent { }); } if (streamChatActivityId) this.endActivity(streamChatActivityId); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return '[cancelled]'; } @@ -5479,7 +5495,7 @@ export class Agent { reasoningContent: response.reasoningContent, }); if (streamChatActivityId) this.endActivity(streamChatActivityId); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return '[cancelled]'; } @@ -5566,7 +5582,7 @@ export class Agent { }); } if (streamChatActivityId) this.endActivity(streamChatActivityId); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return '[cancelled]'; } @@ -5612,7 +5628,7 @@ export class Agent { }); } if (streamChatActivityId) this.endActivity(streamChatActivityId); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return '[cancelled]'; } else if (streamYield.decision === 'preempt') { // A human is awaiting this streamed reply (chat is non-preemptable by design; @@ -5650,7 +5666,7 @@ export class Agent { }); } if (streamChatActivityId) this.endActivity(streamChatActivityId); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return lastResponseContent || ''; } @@ -5693,7 +5709,7 @@ export class Agent { }); } if (streamChatActivityId) this.endActivity(streamChatActivityId); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return '[cancelled]'; } const rawReply = sanitizeLLMReply(response.content); @@ -5712,7 +5728,7 @@ export class Agent { this.emitActivityLog(streamChatActivityId, 'text', displayReply); } if (streamChatActivityId) this.endActivity(streamChatActivityId); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); this.eventBus.emit('agent:message', { agentId: this.id, @@ -5738,7 +5754,7 @@ export class Agent { } catch { /* avoid masking */ } } if (streamChatActivityId) this.endActivity(streamChatActivityId, { success: false }); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return truncated + COMPLETION_MARKER; } if (streamChatActivityId) this.endActivity(streamChatActivityId, { success: !cancelToken?.userStopped }); @@ -5754,7 +5770,7 @@ export class Agent { }); } catch { /* avoid masking */ } } - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); return '[cancelled]'; } @@ -5771,7 +5787,7 @@ export class Agent { } catch { /* avoid masking the original error */ } } - if (this.activeTasks.size === 0) this.setStatus('error', String(error)); + this.transitionStatus({ to: 'error', message: String(error) }); this.emitAudit({ type: 'error', action: 'handle_message_stream', @@ -5900,7 +5916,7 @@ export class Agent { `Agent has reached maximum concurrent tasks (${taskConcurrencyLimit})`, ); } - this.setStatus('working'); + this.transitionStatus({ to: 'working' }); this.activeTasks.add(taskId); const execGen = (this.activeTaskGen.get(taskId) ?? 0) + 1; this.activeTaskGen.set(taskId, execGen); @@ -6065,7 +6081,6 @@ export class Agent { concurrentContext: this.getConcurrentContext(), workflowContext: this.workflowContextFetcher?.(), cognitiveContext, - notebookWriter: this.getNotebookWriter(), ...this.getTeamContextParams(), }); const { volatile: volatileState } = systemPromptBuild; @@ -6549,12 +6564,11 @@ export class Agent { this.endActivity(taskActId); this.notifyStateChange(); - if (this.activeTasks.size === 0) { - if (taskFailed) { - this.setStatus('error', taskFailed); - } else { - this.setStatus('idle'); - } + // 终态裁定统一收敛到 transitionStatus(内部按聚合状态 + 优先级判定)。 + if (taskFailed) { + this.transitionStatus({ to: 'error', message: taskFailed }); + } else { + this.transitionStatus({ to: 'idle' }); } } } @@ -6580,7 +6594,7 @@ export class Agent { persist: boolean; }) => void, ): Promise { - this.setStatus('working'); + this.transitionStatus({ to: 'working' }); const risLabel = userMessage.replace(/^[\s#*[\]]+/g, '').slice(0, 80) || 'Session response'; const actId = this.startActivity('respond_in_session', risLabel); @@ -6628,7 +6642,6 @@ export class Agent { concurrentContext: this.getConcurrentContext(), workflowContext: this.workflowContextFetcher?.(), cognitiveContext, - notebookWriter: this.getNotebookWriter(), ...this.getTeamContextParams(), }); const { volatile: volatileState } = systemPromptBuild; @@ -6829,7 +6842,7 @@ export class Agent { throw error; } finally { this.endActivity(actId); - if (this.activeTasks.size === 0) this.setStatus('idle'); + this.transitionStatus({ to: 'idle' }); await this.finalizeBrowserSession(sessionId); } } @@ -6863,7 +6876,7 @@ export class Agent { /** * THE single notebook write path. * - * Every writer — the `update_notebook` tool, triage, deliberation, and the CPP + * Every writer — the `notebook_upsert` tool, triage, deliberation, and the CPP * notebook writer — must go through here, because the entry cap and the TTL are * only meaningful as an *invariant* (true after every write), not as a check on * one code path. Three writers used to call `workingMemory.set()` directly and @@ -7280,7 +7293,7 @@ export class Agent { if (!fresh) this.state.currentActivity = undefined; } - if (this.state.status !== 'idle') this.setStatus('idle'); + if (this.state.status !== 'idle') this.transitionStatus({ to: 'reset' }); if (cleared) { this.notifyStateChange(); this.eventBus.emit('agent:reconciled-idle', { agentId: this.id, clearedActivity: true }); @@ -7344,6 +7357,16 @@ export class Agent { return this.memory; } + /** + * 受锁写入长期记忆(审计 P-15)——admin API 等**非工具**路径也必须走统一的 + * `agent-memory:knowledge` 资源锁,否则会与 dream cycle / 工具写入交错而丢写。 + */ + async updateLongTermMemory(key: string, content: string): Promise<{ ok: boolean; reason?: string }> { + return this.resourceLocks.withLock(memoryResourceLock('knowledge'), async () => + this.memory.addLongTermMemory(key, content), + ); + } + getContextEngine(): ContextEngine { return this.contextEngine; } @@ -7487,7 +7510,6 @@ export class Agent { availableSkills: this.availableSkillCatalog, mailboxContext: this.getMailboxContext(), concurrentContext: this.getConcurrentContext(), - notebookWriter: this.getNotebookWriter(), ...this.getTeamContextParams(), }); const tools = this.buildToolDefinitions({ @@ -7857,6 +7879,17 @@ export class Agent { const skillToolNames: string[] = []; for (const name of requested) { + // 0. Schema-injected tools (ToolSelector.pushUnique) have no registerTool + // handler — they dispatch through if-branches in agent.ts. They must still + // be activate-able, or budget eviction + discover_tools dead-locks them + // (the P0-1 self-management bug: schedule_wakeup etc. listed as Deferred + // but discover_tools returned unknown). Activation = sticky + protected. + if (SCHEMA_INJECTED_TOOLS.has(name)) { + this.stickyTools().activated.add(name); + activated.push(name); + continue; + } + // 1. Check if it's an existing tool name on this agent if (this.tools.has(name)) { this.stickyTools().activated.add(name); @@ -7971,17 +8004,15 @@ export class Agent { process: { domain: GLOBAL_LOCK_DOMAIN }, // ── 单体状态资源:按**资源**而非工具名登记 ────────────────────────── // `resource` 让「同一个可变状态」的所有工具(正式名 + 兼容别名)解析到同一个 - // 锁键。曾经 `update_notebook` 记在域 `notebook`、别名 `update_working_memory` - // 记在域 `working-memory` —— 两者写同一个 Map,却因域名不同被判为不冲突, - // 锁形同虚设。现在二者都是 `agent-memory:notebook`。 + // 锁键。曾经 notebook 工具有「正式名 + 兼容别名」两套登记;别名现已删除, + // 名字归一为 notebook_upsert / notebook_clear。二者与其它写工具共享 + // `agent-memory:notebook` 锁域,保证写同一个 Map 的工具有互斥。 memory_save: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'knowledge' }, memory_update: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'knowledge' }, memory_update_longterm: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'knowledge' }, memory_delete: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'knowledge' }, - update_notebook: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'notebook' }, - clear_notebook: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'notebook' }, - update_working_memory: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'notebook' }, - clear_working_memory: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'notebook' }, + notebook_upsert: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'notebook' }, + notebook_clear: { domain: AGENT_MEMORY_RESOURCE_DOMAIN, resource: 'notebook' }, // 会话自我管理:compact / pin / unpin / purge 都是会话状态变更 session: { domain: 'session', arg: ['session_id', 'sessionId'] }, // discover_tools 会写入「本会话已激活工具」集合 —— 共享可变状态 @@ -8099,7 +8130,7 @@ export class Agent { // ── 纯读取:文件 / 代码 / 检索 'file_read', 'grep_search', 'glob_find', 'list_directory', 'web_search', 'web_fetch', 'web_extract', 'describe_image', - 'knowledge_search', 'knowledge_list', 'knowledge_read', + 'kb_search', 'kb_list', 'kb_read', 'recall_activity', 'recall_context', 'check_mailbox', // ── 纯读取:任务 / 需求 / 项目 / 团队 'task_list', 'task_get', 'task_board_health', 'task_check_duplicates', @@ -8848,6 +8879,30 @@ export class Agent { } } + /** + * 重构 2:心跳 skip 的唯一出口 —— 纯时间戳存活上报(不触发 LLM)。 + * + * 所有 skip 分支(human-chat defer / idle / deep-sleep)都必须走这里:统一负责 + * 活动记录 + lastHeartbeat 落库(notifyStateChange → Fix B 链路写入 DB)+ 指标, + * 以及可选的 deep-sleep 间隔延长。保证「skip 路径永不调用 LLM」成为结构性事实, + * 且任何 skip 都刷新存活时间戳(列存在有人写)。 + */ + private recordHeartbeatSkip(label: string, note: string, opts?: { deepSleep?: boolean }): void { + const skipActivityId = this.startActivity('heartbeat', label, {}); + this.emitActivityLog(skipActivityId, 'text', note); + this.endActivity(skipActivityId, { success: true }); + this.state.lastHeartbeat = new Date().toISOString(); + this.metricsCollector.recordHeartbeat(true, true); + this.notifyStateChange(); // 心跳也是存活证明:落库 last_heartbeat,供所有观察者读取 + if (opts?.deepSleep) { + try { + const cur = (this as unknown as { heartbeatIntervalMs?: number }).heartbeatIntervalMs ?? 6 * 3600_000; + const next = nextDeepSleepIntervalMs(cur); + this.heartbeat?.updateInterval?.(next); + } catch { /* optional */ } + } + } + private async handleHeartbeat(ctx: { agentId: string; triggeredAt: string; @@ -8856,41 +8911,43 @@ export class Agent { if (this.shouldDeferHeartbeatForHumanChat()) { log.info('Heartbeat: skipping LLM (human chat focused/queued)', { agentId: this.id }); - const skipActivityId = this.startActivity('heartbeat', 'Heartbeat check-in (deferred for human chat)', {}); - this.emitActivityLog( - skipActivityId, - 'text', + this.recordHeartbeatSkip( + 'Heartbeat check-in (deferred for human chat)', 'Human chat is active or queued — skipping heartbeat LLM. Will patrol on the next trigger when chat is idle.', ); - this.endActivity(skipActivityId, { success: true }); - this.state.lastHeartbeat = new Date().toISOString(); - this.metricsCollector.recordHeartbeat(true, true); - this.notifyStateChange(); // 心跳也是存活证明:落库 last_heartbeat,供所有观察者读取 return; } // Deep sleep / idle skip — AGENT-RUNTIME deep sleep Spec. - const queuedNonHeartbeat = this.mailbox.getQueuedItems().filter( + const queuedAll = this.mailbox.getQueuedItems(); + const queuedNonHeartbeat = queuedAll.filter( i => i.sourceType !== 'heartbeat' && i.status === 'queued' - ).length; - const humanOrTaskMail = this.mailbox.getQueuedItems().some( + ); + const humanOrTaskMail = queuedAll.some( i => i.status === 'queued' && ( i.sourceType === 'human_chat' || i.sourceType === 'task_status_update' || i.sourceType === 'task_comment' ), ); - const fingerprint = `q:${queuedNonHeartbeat}`; - const unchanged = fingerprint === this.lastHeartbeatFingerprint && queuedNonHeartbeat === 0; + // 重构 2:巡检状态指纹 —— 队列内容(sourceType+id,含 stuck 邮件与队列空)+ 活跃任务集合。 + // 仅当状态**实际变化**才巡检(新邮件/新任务出现 → 指纹变 → 巡检一次); + // 无变化(含同一邮件一直卡在队列)→ skip LLM 只写时间戳,不空转 LLM。 + const fingerprint = heartbeatStateFingerprint(queuedNonHeartbeat, this.activeTasks); + const unchanged = fingerprint === this.lastHeartbeatFingerprint; if (unchanged) this.consecutiveIdleHeartbeats++; else this.consecutiveIdleHeartbeats = resetIdleOnWake(); this.lastHeartbeatFingerprint = fingerprint; + // 重构 2:深睡判定使用实时状态,不再硬编码 false —— 有活跃任务/待审邮件绝不深睡 + // (否则长任务期间间隔被翻倍,存活证明变稀)。 const deepSleep = shouldEnterDeepSleep({ consecutiveIdleHeartbeats: this.consecutiveIdleHeartbeats, - hasActiveTasks: false, - hasPendingReviews: false, - hasHumanOrTaskMailbox: humanOrTaskMail || queuedNonHeartbeat > 0, + hasActiveTasks: this.activeTasks.size > 0, + hasPendingReviews: queuedAll.some( + i => i.status === 'queued' && i.sourceType === 'review_request', + ), + hasHumanOrTaskMailbox: humanOrTaskMail || queuedNonHeartbeat.length > 0, }); // Skip LLM while unchanged (including deep sleep). No forced patrol when org is quiet — @@ -8900,25 +8957,13 @@ export class Agent { consecutiveIdle: this.consecutiveIdleHeartbeats, deepSleep, }); - const skipActivityId = this.startActivity('heartbeat', 'Heartbeat check-in (idle skip)', {}); - this.emitActivityLog( - skipActivityId, - 'text', + this.recordHeartbeatSkip( + 'Heartbeat check-in (idle skip)', deepSleep ? `Deep sleep (idle ${this.consecutiveIdleHeartbeats}). No LLM call; interval may extend.` : `No changes detected (idle ${this.consecutiveIdleHeartbeats}). Skipping LLM.`, + { deepSleep }, ); - this.endActivity(skipActivityId, { success: true }); - this.state.lastHeartbeat = new Date().toISOString(); - this.metricsCollector.recordHeartbeat(true, true); - this.notifyStateChange(); // 心跳存活证明落库 - if (deepSleep) { - try { - const cur = (this as unknown as { heartbeatIntervalMs?: number }).heartbeatIntervalMs ?? 6 * 3600_000; - const next = nextDeepSleepIntervalMs(cur); - this.heartbeat?.updateInterval?.(next); - } catch { /* optional */ } - } return; } this.consecutiveIdleHeartbeats = 0; @@ -9198,7 +9243,7 @@ export class Agent { '- Technical details that would be costly to rediscover', '', 'Call shape: `{ content, type?, tags? }` — never an array. Verify `{ status:"saved", store:"knowledge.md" }`.', - 'Also use `update_notebook` for current working state.', + 'Also use `notebook_upsert` for current working state.', '', 'Only save genuinely important information. Skip routine exchanges.', 'If nothing important needs saving, just respond with "No important information to save."', @@ -9247,6 +9292,10 @@ export class Agent { await this.resourceLocks.withLock(memoryResourceLock('knowledge'), async () => { await this.dreamConsolidateMemory(entries); this.pruneMemoryMd(); + // 审计 P-11:上报整理时刻,使「陈旧」信号真实可用。 + // (此前只有手动 memory_organize 会写 lastConsolidatedAt,自动 dream 周期不写 → + // 信号实际是死的。dream 定时器与 organize 原语共用同一时刻语义。) + this.memory.markConsolidated?.(); // state.md TTL pruning removed with the store itself (2026-09-16): TTL now // lives in the notebook's per-tier expiry, which runs on every write. }); diff --git a/packages/core/src/attention.ts b/packages/core/src/attention.ts index d3895eeea..8f7ad7c72 100644 --- a/packages/core/src/attention.ts +++ b/packages/core/src/attention.ts @@ -1013,26 +1013,29 @@ export class AttentionController { if (timedOut) { if (orphanAbandoned) { // Visibility + no silent re-run: an orphan we could not stop is exactly - // the "completed but unfinished" terminal, not a retryable failure. + // the "finished but unfinished" terminal, not a retryable failure. + // 语义修正:超时未完成 ≠ 完成,标 dropped(异常可见 + 允许补偿重投), + // 绝不标 completed。 this.emitIncomplete( item, 'backstop timeout — in-flight turn did not settle within grace; not requeued to avoid duplicate side effects', ); - this.mailbox.complete(item.id); + this.mailbox.drop(item.id); } else { this.mailbox.requeue(item); } statusResolved = true; } else if (reply === '[cancelled]' || this.lastYieldDecision === 'cancel') { - // Permanently cancelled by an explicit cancel decision — the new incoming - // message contradicts or revokes this work. Drop the item; it will NOT - // be resumed. - log.info('Item cancelled — dropping permanently', { + // 取消(用户显式取消按钮 / 新消息 revoke 当前工作)。 + // 语义修正:被取消 = **异常终态**,标 dropped(红色可见 + 允许补偿重投), + // 绝不标 completed —— completed 会被 resolveReinjectDecision 判为 settled + // 而抑制后续同键重投,等于把被取消的工作静默吞掉。 + log.info('Item cancelled — marking dropped (not completed)', { agentId: this.agentId, itemId: item.id, type: item.sourceType, }); - this.mailbox.complete(item.id); + this.mailbox.drop(item.id); statusResolved = true; } else if (reply === '[preempted]' || this.lastYieldDecision === 'preempt') { // Paused by a higher-priority item — defer so it can be resumed later. @@ -1064,27 +1067,27 @@ export class AttentionController { if (abnormalReason === 'completion marker missing from reply') { // In-session continuation was already attempted by the agent upstream. // Requeuing would restart from scratch and duplicate all side effects. - log.warn('Completion marker still missing after in-session continuation — completing without retry', { + // 语义修正:缺完成标记 = 未明确正常结束,标 dropped(异常可见),不标 completed。 + log.warn('Completion marker still missing after in-session continuation — marking dropped without retry', { agentId: this.agentId, itemId: item.id, type: item.sourceType, }); - // A3: make the near-silent "completed but unfinished" terminal visible. - // Visibility only — retry semantics are unchanged. this.emitIncomplete(item, abnormalReason); - this.mailbox.complete(item.id); + this.mailbox.drop(item.id); } else if (isUserInteraction) { // Empty reply / error for user-facing item — the user already saw // partial results and tool calls may have produced side effects. // Don't restart; the user can manually retry if needed. - log.warn('Abnormal completion for user interaction — completing without retry', { + // 语义修正:一次异常 ≠ 完成,标 dropped(红点可见),用户可手动重试。 + log.warn('Abnormal completion for user interaction — marking dropped without retry', { agentId: this.agentId, itemId: item.id, type: item.sourceType, reason: abnormalReason, }); this.emitIncomplete(item, abnormalReason); - this.mailbox.complete(item.id); + this.mailbox.drop(item.id); } else { // Empty reply for background type — transient errors may self-correct. log.warn('Abnormal completion detected, requeueing for retry', { @@ -1096,17 +1099,20 @@ export class AttentionController { }); this.mailbox.requeue(item); } + } else if (abnormalReason) { + // 重试耗尽后仍异常:异常终态,标 dropped(红点可见 + 允许补偿重投), + // 不标 completed —— 未明确正常结束的工作没有资格标记为「完成」。 + log.error('Abnormal completion persisted after max retries — marking dropped', { + agentId: this.agentId, + itemId: item.id, + type: item.sourceType, + retryCount: retries, + reason: abnormalReason, + }); + this.emitIncomplete(item, abnormalReason); + this.mailbox.drop(item.id); } else { - if (abnormalReason) { - log.error('Abnormal completion persisted after max retries, completing anyway', { - agentId: this.agentId, - itemId: item.id, - type: item.sourceType, - retryCount: retries, - reason: abnormalReason, - }); - this.emitIncomplete(item, abnormalReason); - } + // 正常完成(reply 有效、无异常、agent 明确结束)→ completed 是唯一合法终态。 this.mailbox.complete(item.id); } statusResolved = true; diff --git a/packages/core/src/capability-packs.ts b/packages/core/src/capability-packs.ts index 9ddb9a50e..a68d7b96d 100644 --- a/packages/core/src/capability-packs.ts +++ b/packages/core/src/capability-packs.ts @@ -27,7 +27,7 @@ export const REFLEX_CORE_TOOLS = [ 'check_mailbox', 'file_read', 'agent_send_message', - 'update_notebook', + 'notebook_upsert', ] as const; export const REFLEX_MANAGER_EXTRA_TOOLS = ['team_status'] as const; @@ -45,6 +45,25 @@ export const TOOL_DEF_PROTECTED = new Set([ 'request_user_approval', ]); +/** + * Tools whose schemas are unconditionally injected by ToolSelector.pushUnique + * (schedule_wakeup / cancel_wakeup / set_heartbeat_interval / recall_activity / + * complete_deliberation / notebook_upsert / notebook_clear). discover_tools must + * treat them as activate-able (sticky activated → protected from eviction), or + * they dead-lock once budget pressure moves them into the Deferred catalog. + * + * MUST stay in sync with ToolSelector.selectTools() pushUnique calls. + */ +export const SCHEMA_INJECTED_TOOLS = new Set([ + 'schedule_wakeup', + 'cancel_wakeup', + 'set_heartbeat_interval', + 'recall_activity', + 'complete_deliberation', + 'notebook_upsert', + 'notebook_clear', +]); + /** * Core Markus tools that should survive budget pressure before MCP/skill tools. * Prefer deferring chrome-devtools__* / feishu_* over shell_execute / file_read. @@ -257,7 +276,7 @@ export const REQUIREMENT_ACTION_ALLOWED_TOOLS: readonly string[] = [ 'subtask_create', 'subtask_complete', 'subtask_list', 'deliverable_search', 'deliverable_create', 'file_read', 'grep_search', - 'memory_search', 'memory_save', 'update_notebook', + 'memory_search', 'memory_save', 'notebook_upsert', 'notify_user', 'agent_send_message', ]; diff --git a/packages/core/src/cognitive.ts b/packages/core/src/cognitive.ts deleted file mode 100644 index 055ee037e..000000000 --- a/packages/core/src/cognitive.ts +++ /dev/null @@ -1,378 +0,0 @@ -/** - * Cognitive Preparation Pipeline (CPP) - * - * Runs 0–3 lightweight LLM calls before the main reasoning call to prepare - * persona-aware context. See docs/COGNITIVE-ARCHITECTURE.md for theory. - * - * Phases: - * 1. Appraisal — "What is this? What do I need?" (persona-aware LLM call) - * 2. Directed Retrieval — execute the plan from Phase 1 (no LLM, pure queries) - * 3. Reflection — "What does this mean for me?" (persona-aware LLM call) - * 4. Assembly — stitch results into prompt sections (code, no LLM) - */ - -import { - createLogger, - CognitiveDepth, - type CognitiveStimulus, - type CognitiveAgentContext, - type AppraisalResult, - type RetrievalPlan, - type RetrievedContext, - type ReflectionResult, - type PreparedCognitiveContext, - type CognitiveConfig, - type LLMRequest, - type LLMResponse, -} from '@markus/shared'; - -const log = createLogger('cognitive'); - -/** - * Parse a JSON object out of an LLM response. Reasoning models often emit - * hidden reasoning and may wrap the payload in prose or code fences, so we - * isolate the outermost {...} before parsing instead of trusting the whole - * string. Throws if no object is present (caller falls back gracefully). - */ -function parseJsonObject(content: string): T { - const raw = (content || '').trim(); - const start = raw.indexOf('{'); - const end = raw.lastIndexOf('}'); - const slice = start >= 0 && end > start ? raw.slice(start, end + 1) : raw; - return JSON.parse(slice) as T; -} - -/** Minimal LLM interface — only needs the non-streaming chat method */ -export interface CognitiveLLM { - chat(request: LLMRequest, providerName?: string): Promise; -} - -/** Backend for Phase 2 retrieval */ -export interface RetrievalBackend { - searchMemories(query: string, limit?: number): Array<{ content: string; relevance: number }>; - searchActivities(agentId: string, query: string, limit?: number): Array<{ summary: string; type: string }>; - searchTasks(query: string, limit?: number): Array<{ title: string; status: string; id: string }>; -} - -// ─── Depth Selection (0G) ──────────────────────────────────────────────────── - -const SCENARIO_DEPTH_MAP: Record = { - heartbeat: CognitiveDepth.D0_Reflexive, - memory_consolidation: CognitiveDepth.D0_Reflexive, - distillation: CognitiveDepth.D1_Reactive, - chat: CognitiveDepth.D1_Reactive, - a2a: CognitiveDepth.D1_Reactive, - comment_response: CognitiveDepth.D1_Reactive, - review: CognitiveDepth.D1_Reactive, - requirement_action: CognitiveDepth.D1_Reactive, - task_execution: CognitiveDepth.D2_Deliberative, -}; - -export function selectCognitiveDepth( - scenario: string, - agentState: { hasFailedTasks?: boolean; hasBlockers?: boolean }, - stimulusLength: number, -): CognitiveDepth { - const base = SCENARIO_DEPTH_MAP[scenario] ?? CognitiveDepth.D1_Reactive; - - // Upgrade heuristics - if (base === CognitiveDepth.D0_Reflexive && agentState.hasFailedTasks) { - return CognitiveDepth.D1_Reactive; - } - if (base === CognitiveDepth.D1_Reactive && stimulusLength > 500) { - return CognitiveDepth.D2_Deliberative; - } - if (base === CognitiveDepth.D2_Deliberative && agentState.hasBlockers) { - return CognitiveDepth.D3_MetaCognitive; - } - - return base; -} - -// ─── Cognitive Preparation ─────────────────────────────────────────────────── - -export class CognitivePreparation { - constructor(private config: CognitiveConfig) {} - - async prepare( - stimulus: CognitiveStimulus, - agent: CognitiveAgentContext, - depth: CognitiveDepth, - llm: CognitiveLLM, - retrieval?: RetrievalBackend, - ): Promise { - const effectiveDepth = this.config.maxDepth !== undefined - ? Math.min(depth, this.config.maxDepth) as CognitiveDepth - : depth; - - if (!this.config.enabled || effectiveDepth === CognitiveDepth.D0_Reflexive) { - return { depth: effectiveDepth, isEmpty: true }; - } - - log.info('CPP starting', { depth: effectiveDepth, stimulus: stimulus.type, agent: agent.name }); - - // Phase 1: Appraisal - const appraisal = await this.appraise(stimulus, agent, llm); - - if (effectiveDepth === CognitiveDepth.D1_Reactive) { - return { - depth: effectiveDepth, - cognitiveContext: appraisal.cognitiveContext, - isEmpty: false, - }; - } - - // Phase 2: Directed Retrieval (D2+) — requires retrieval backend - if (!retrieval) { - log.warn('D2+ requested but no RetrievalBackend provided, returning appraisal only'); - return { - depth: effectiveDepth, - cognitiveContext: appraisal.cognitiveContext, - isEmpty: false, - }; - } - const retrieved = await this.retrieve(appraisal.retrievalPlan, retrieval, agent.id); - - // Phase 3: Reflection (D2+ when needed) - let reflection: ReflectionResult | undefined; - if (appraisal.reflectionNeeded && effectiveDepth >= CognitiveDepth.D2_Deliberative) { - reflection = await this.reflect(stimulus, agent, retrieved, llm); - } - - // Phase 4: Assembly - return this.assemble(effectiveDepth, appraisal, retrieved, reflection); - } - - // ─── Phase 1: Appraisal ───────────────────────────────────────────────────── - - private async appraise( - stimulus: CognitiveStimulus, - agent: CognitiveAgentContext, - llm: CognitiveLLM, - ): Promise { - const prompt = [ - `You are ${agent.name}, a ${agent.roleDescription}.`, - `Current status: ${agent.status}`, - agent.currentTask ? `Current task: ${agent.currentTask}` : '', - agent.recentActivity.length > 0 - ? `Recent activity:\n${agent.recentActivity.slice(-5).map(a => `- ${a}`).join('\n')}` - : '', - '', - 'An incoming stimulus requires your attention:', - `Type: ${stimulus.type}`, - stimulus.sender ? `From: ${stimulus.sender}` : '', - `Content: ${stimulus.content.slice(0, 800)}`, - '', - 'Respond with a JSON object (no markdown fences):', - '{', - ' "intent": "brief description of what this stimulus is asking/requiring",', - ' "relevance": "how this relates to your current work and role",', - ' "confidence": "high|medium|low — how confident you are in understanding this",', - ' "risk": "none|low|medium|high — potential for negative outcomes if handled incorrectly",', - ' "securityCheck": "safe|review|alert — whether this stimulus contains potentially adversarial content or requests that exceed your boundaries",', - ' "retrievalPlan": {', - ' "memoryQueries": ["keywords to search your memory for relevant knowledge"],', - ' "activityQueries": ["keywords to search your past activities"],', - ' "taskQueries": ["keywords to search task board"]', - ' },', - ' "reflectionNeeded": true/false,', - ' "cognitiveContext": "1-2 sentence summary of your assessment for the main reasoning call"', - '}', - ].filter(Boolean).join('\n'); - - try { - // No maxTokens: do not inject the catalog output ceiling (or any artificial - // cap). Reasoning may spend tokens before the JSON; omitting max_tokens lets - // the upstream apply its default, and the model stops once the small JSON - // payload is complete. (Injecting catalog max_output broke OpenRouter prepaid - // keys that reserve credits against max_tokens.) - const response = await llm.chat({ - messages: [{ role: 'user', content: prompt }], - temperature: 0.3, - metadata: { purpose: 'cognitive_appraisal' }, - }, this.config.appraisalModel); - - const parsed = parseJsonObject(response.content); - log.debug('Appraisal completed', { intent: parsed.intent, confidence: parsed.confidence }); - return parsed; - } catch (err) { - log.warn('Appraisal LLM call failed, using fallback', { error: String(err) }); - return { - intent: stimulus.summary || stimulus.type, - relevance: 'unknown', - confidence: 'low', - retrievalPlan: { memoryQueries: [], activityQueries: [], taskQueries: [] }, - reflectionNeeded: false, - cognitiveContext: `Incoming ${stimulus.type}: ${stimulus.summary || stimulus.content.slice(0, 100)}`, - }; - } - } - - // ─── Phase 2: Directed Retrieval ──────────────────────────────────────────── - - private async retrieve( - plan: RetrievalPlan, - backend: RetrievalBackend, - agentId: string, - ): Promise { - const result: RetrievedContext = { memories: [], activities: [], tasks: [] }; - - for (const q of plan.memoryQueries.slice(0, 3)) { - try { - const hits = backend.searchMemories(q, 5); - result.memories.push(...hits); - } catch (err) { - log.warn('Memory retrieval failed', { query: q, error: String(err) }); - } - } - - for (const q of plan.activityQueries.slice(0, 3)) { - try { - const hits = backend.searchActivities(agentId, q, 5); - result.activities.push(...hits); - } catch (err) { - log.warn('Activity retrieval failed', { query: q, error: String(err) }); - } - } - - for (const q of plan.taskQueries.slice(0, 3)) { - try { - const hits = backend.searchTasks(q, 5); - result.tasks.push(...hits); - } catch (err) { - log.warn('Task retrieval failed', { query: q, error: String(err) }); - } - } - - // Deduplicate memories by content - const seen = new Set(); - result.memories = result.memories.filter(m => { - const key = m.content.slice(0, 100); - if (seen.has(key)) return false; - seen.add(key); - return true; - }); - - log.debug('Retrieval completed', { - memories: result.memories.length, - activities: result.activities.length, - tasks: result.tasks.length, - }); - - return result; - } - - // ─── Phase 3: Reflection ──────────────────────────────────────────────────── - - private async reflect( - stimulus: CognitiveStimulus, - agent: CognitiveAgentContext, - retrieved: RetrievedContext, - llm: CognitiveLLM, - ): Promise { - const contextSummary = [ - retrieved.memories.length > 0 - ? `Relevant knowledge:\n${retrieved.memories.slice(0, 5).map(m => `- ${m.content}`).join('\n')}` - : '', - retrieved.activities.length > 0 - ? `Past activities:\n${retrieved.activities.slice(0, 5).map(a => `- [${a.type}] ${a.summary}`).join('\n')}` - : '', - retrieved.tasks.length > 0 - ? `Related tasks:\n${retrieved.tasks.slice(0, 5).map(t => `- [${t.status}] ${t.title} (${t.id})`).join('\n')}` - : '', - ].filter(Boolean).join('\n\n'); - - const prompt = [ - `You are ${agent.name}, a ${agent.roleDescription}.`, - `Current status: ${agent.status}`, - '', - `You are about to respond to: ${stimulus.summary || stimulus.content.slice(0, 200)}`, - '', - 'Here is the context retrieved from your memory and experience:', - contextSummary || '(No relevant context found)', - '', - 'Reflect on this from your role\'s perspective. Respond with JSON (no markdown fences):', - '{', - ' "interpretation": "What this context means for how you should approach the stimulus",', - ' "recommendations": ["specific action or consideration 1", "specific action 2"]', - '}', - ].join('\n'); - - try { - // No maxTokens cap — see appraisal above. The router supplies the - // model's real output limit; reasoning models need room before the JSON. - const response = await llm.chat({ - messages: [{ role: 'user', content: prompt }], - temperature: 0.3, - metadata: { purpose: 'cognitive_reflection' }, - }, this.config.appraisalModel); - - const parsed = parseJsonObject(response.content); - log.debug('Reflection completed', { recommendations: parsed.recommendations.length }); - return parsed; - } catch (err) { - log.warn('Reflection LLM call failed', { error: String(err) }); - return { - interpretation: 'Unable to reflect — proceeding with available context.', - recommendations: [], - }; - } - } - - // ─── Phase 4: Assembly ────────────────────────────────────────────────────── - - private assemble( - depth: CognitiveDepth, - appraisal: AppraisalResult, - retrieved: RetrievedContext, - reflection?: ReflectionResult, - ): PreparedCognitiveContext { - const sections: string[] = []; - - // Cognitive context (from appraisal) - const cognitiveContext = appraisal.cognitiveContext; - - // Retrieved context - const retrievedParts: string[] = []; - if (retrieved.memories.length > 0) { - retrievedParts.push('**From your knowledge:**'); - for (const m of retrieved.memories.slice(0, 5)) { - retrievedParts.push(`- ${m.content}`); - } - } - if (retrieved.activities.length > 0) { - retrievedParts.push('**From past experience:**'); - for (const a of retrieved.activities.slice(0, 5)) { - retrievedParts.push(`- [${a.type}] ${a.summary}`); - } - } - if (retrieved.tasks.length > 0) { - retrievedParts.push('**Related tasks:**'); - for (const t of retrieved.tasks.slice(0, 5)) { - retrievedParts.push(`- [${t.status}] ${t.title} (${t.id})`); - } - } - const retrievedContext = retrievedParts.length > 0 ? retrievedParts.join('\n') : undefined; - - // Reflection - let reflectionText: string | undefined; - if (reflection) { - sections.length = 0; // reuse - sections.push(reflection.interpretation); - if (reflection.recommendations.length > 0) { - for (const r of reflection.recommendations) { - sections.push(`- ${r}`); - } - } - reflectionText = sections.join('\n'); - } - - return { - depth, - cognitiveContext, - retrievedContext, - reflection: reflectionText, - isEmpty: false, - }; - } -} diff --git a/packages/core/src/context-engine.ts b/packages/core/src/context-engine.ts index d9cabfc26..a5b5744d0 100644 --- a/packages/core/src/context-engine.ts +++ b/packages/core/src/context-engine.ts @@ -424,7 +424,6 @@ export class ContextEngine { }>; }; cognitiveContext?: PreparedCognitiveContext; - notebookWriter?: (key: string, text: string, managed: 'system' | 'cpp') => void; channelContext?: Array<{ role: string; content: string }>; /** Prompt profile (AGENT-RUNTIME §4). Defaults from scenario pack. */ promptProfile?: PromptProfile; @@ -436,6 +435,13 @@ export class ContextEngine { workerId: number; workerCount: number; handoffs: ConcurrentHandoffLite[]; + /** + * 分身共享工作记忆快照(agent 级 NOTEBOOK.md)。notebook 是 agent 级共享的, + * 但上下文默认不注入 → 分身 B 不知道分身 A 写了什么(P1-2「我不知道另一个 + * 我在干嘛」)。注入最近更新条目(限量、截断),让每个 worker 感知彼此的 + * working-memory 状态。只有并发模式才注入,且必须限量避免 token 膨胀。 + */ + notebook?: Array<{ key: string; text: string; updatedAt: number; managed?: string }>; }; }): Promise { const isDream = opts.scenario === 'memory_consolidation'; @@ -877,6 +883,22 @@ export class ContextEngine { if (longTermMem) { const knowledgeCapChars = Math.min(SYSTEM_KNOWLEDGE_CHARS, knowledgeTokCap * 4); volatile.push('\n## Your Knowledge'); + // 审计 P-12:记忆健康信号 —— 让 Agent「知道」何时该主动整理(否则永不自知)。 + try { + const h = opts.memory.getMemoryHealth(); + const staleDays = h.lastConsolidatedAt + ? Math.max(0, Math.floor((Date.now() - Date.parse(h.lastConsolidatedAt)) / 86_400_000)) + : null; + if (h.percent >= 70 || (staleDays !== null && staleDays >= 14)) { + const staleness = staleDays !== null && staleDays >= 14 + ? ` · 上次整理 ${staleDays} 天前` + : ''; + volatile.push( + `> ${h.percent >= 90 ? '🔴' : '⚠️'} **记忆健康 ${h.percent}%**(${h.totalChars}/${h.cap} 字符 · ${h.curatedSections} 个知识段 · ${h.observations} 条观察${h.archiveChars > 0 ? ` · 已归档 ${h.archiveChars} 字符` : ''}${staleness})。` + + '超预算时旧知识会被**无损归档**(不再注入)。建议用 `memory_organize` 合并观察、或 `memory_update`(mode:"delete")删除过时条目。', + ); + } + } catch { /* health signal is best-effort */ } // Relevance-ranked, whole-section selection (see prepareKnowledgeForPrompt). // `opts.currentQuery` is the in-flight user text, so which sections get // inlined depends on what is actually being asked rather than on document @@ -1075,39 +1097,29 @@ export class ContextEngine { const alreadyShownIds = new Set(); const cpp = opts.cognitiveContext; if (cpp && !cpp.isEmpty) { - if (opts.notebookWriter) { - if (cpp.cognitiveContext) opts.notebookWriter('cognitive-context', cpp.cognitiveContext, 'cpp'); - if (cpp.retrievedContext) opts.notebookWriter('relevant-context', cpp.retrievedContext, 'cpp'); - if (cpp.reflection) opts.notebookWriter('reflection', cpp.reflection, 'cpp'); - } else { - if (cpp.cognitiveContext) { - dynamic.push('\n## Cognitive Context'); - dynamic.push(cpp.cognitiveContext); - } - if (cpp.retrievedContext) { - dynamic.push('\n## Retrieved Context'); - dynamic.push(cpp.retrievedContext); - } - if (cpp.reflection) { - dynamic.push('\n## Reflection'); - dynamic.push(cpp.reflection); - } + // 审计 P-03:CPP 产出有**唯一目的地**——本轮 prompt。以前它写进共享 NOTEBOOK + // (并因此**不注入当轮 prompt**),与 `## Your Knowledge` 重复、还把瞬时检索 + // 记录沉淀成持久笔记。现在只注入,不落盘。 + if (cpp.cognitiveContext) { + dynamic.push('\n## Cognitive Context'); + dynamic.push(cpp.cognitiveContext); + } + if (cpp.retrievedContext) { + dynamic.push('\n## Retrieved Context'); + dynamic.push(cpp.retrievedContext); + } + if (cpp.reflection) { + dynamic.push('\n## Reflection'); + dynamic.push(cpp.reflection); } } else if (!isDream) { const relevantMemories = await this.retrieveRelevantMemories(opts.memory, opts.currentQuery, opts.agentId, alreadyShownIds); if (relevantMemories.length > 0) { - if (opts.notebookWriter) { - const lines = relevantMemories.map(mem => { - const ts = mem.timestamp ? new Date(mem.timestamp).toLocaleDateString() : ''; - return `- [${ts}] ${mem.content}`; - }); - opts.notebookWriter('relevant-context', lines.join('\n'), 'system'); - } else { - volatile.push('\n## Relevant Memories'); - for (const mem of relevantMemories) { - const ts = mem.timestamp ? new Date(mem.timestamp).toLocaleDateString() : ''; - volatile.push(`- [${ts}] ${mem.content}`); - } + // 审计 P-03:检索结果只注入本轮,不写共享 NOTEBOOK。 + volatile.push('\n## Relevant Memories'); + for (const mem of relevantMemories) { + const ts = mem.timestamp ? new Date(mem.timestamp).toLocaleDateString() : ''; + volatile.push(`- [${ts}] ${mem.content}`); } } } @@ -1263,6 +1275,20 @@ export class ContextEngine { ' 2. 同一任务/需求/对话串同时只能有一个分身处理——发现实体已被占用,不要强行开做,如实说明。', ' 3. 发现事实冲突时(你的认知与交接记录矛盾),优先报告差异并请求合并决策,不静默覆盖。' ); + + // P1-2:分身共享工作记忆快照(agent 级 NOTEBOOK)。让每个 worker 感知 + // 其他分身写下的 working-memory 条目(限量 + 截断,避免 token 膨胀)。 + const nb = cc.notebook ?? []; + if (nb.length > 0) { + const sorted = [...nb].sort((a, b) => b.updatedAt - a.updatedAt).slice(0, 5); + lines.push('- 分身共享工作记忆(其他 worker 最近写入 NOTEBOOK.md,勿重复写入同类条目,如需更新请覆盖同 key):'); + for (const e of sorted) { + const text = (e.text ?? '').replace(/\s+/g, ' ').slice(0, 140); + const when = e.updatedAt ? new Date(e.updatedAt).toISOString().slice(5, 16) : ''; + lines.push(` - [${e.key}]${when ? ` (${when})` : ''} ${text}`); + } + } + volatile.push(lines.join('\n')); } @@ -1395,7 +1421,7 @@ export class ContextEngine { lines.push('2. **Clean**: `drop_mailbox_item` for stale informational items (old heartbeats, outdated status updates, superseded notifications).'); lines.push('3. **Group**: Items sharing a taskId / requirementId / channel → plan to handle together.'); lines.push('4. **Inline**: Handle trivial items now (quick ack via `notify_user`, one-line `task_comment`). Mark as inline_completed.'); - lines.push('5. **Assess**: `update_working_memory` with your situational summary — priorities, blockers, what you plan to do.'); + lines.push('5. **Assess**: `notebook_upsert` with your situational summary — priorities, blockers, what you plan to do.'); lines.push('6. **Focus**: `complete_deliberation` — choose the most important remaining item.'); lines.push(''); lines.push('### Per-Type Processing'); @@ -1426,7 +1452,7 @@ export class ContextEngine { lines.push('- [ ] Does this affect your current work or priorities?'); lines.push('- [ ] Does it unblock something you were waiting for?'); lines.push('- [ ] Usually no response needed — absorb into awareness.'); - lines.push('- [ ] If it changes your priorities → `update_working_memory`.'); + lines.push('- [ ] If it changes your priorities → `notebook_upsert`.'); lines.push(''); lines.push('**review_request**:'); lines.push('- [ ] Call `task_get` — read task description, deliverables, notes.'); @@ -1441,9 +1467,9 @@ export class ContextEngine { lines.push('- [ ] If nothing needs attention → HEARTBEAT_OK.'); lines.push(''); lines.push('### Notebook Guidelines'); - lines.push('- **Save**: current priorities, ongoing context, key decisions, blockers via `update_notebook`.'); + lines.push('- **Save**: current priorities, ongoing context, key decisions, blockers via `notebook_upsert`.'); lines.push('- **Update**: when situation changes — new task, resolved blocker, shifted priority.'); - lines.push('- **Clear**: when a task completes, when context becomes irrelevant via `clear_notebook`.'); + lines.push('- **Clear**: when a task completes, when context becomes irrelevant via `notebook_clear`.'); lines.push('- **Do NOT save**: raw message content, large data — use `memory_save` for durable observations.'); lines.push(''); lines.push('### Mailbox Management Guidelines'); @@ -1818,7 +1844,7 @@ export class ContextEngine { lines.push('2. **Gather context**: `recall_activity`, `task_get`, `memory_search` — understand history.'); lines.push('3. **Manage queue**: `defer_mailbox_item` — postpone items; `drop_mailbox_item` — discard stale ones.'); lines.push('4. **Handle inline**: `notify_user`, `task_comment`, `agent_send_message` — handle trivial items now.'); - lines.push('5. **Record awareness**: `update_working_memory` — persist your situational assessment.'); + lines.push('5. **Record awareness**: `notebook_upsert` — persist your situational assessment.'); lines.push(''); lines.push('**Rules**:'); lines.push('- Human messages and comments are ALWAYS highest priority.'); diff --git a/packages/core/src/heartbeat.ts b/packages/core/src/heartbeat.ts index e8f1ad629..d2f951c2c 100644 --- a/packages/core/src/heartbeat.ts +++ b/packages/core/src/heartbeat.ts @@ -61,6 +61,34 @@ export function isWithinActiveHours(activeHours: HeartbeatActiveHours, now: Date return current >= startMin || current < endMin; } +/** + * 重构 2:心跳巡检状态指纹(纯函数,可确定性单测)。 + * + * 编码「与 LLM 巡检相关的 agent 状态」—— 队列内容 + 活跃任务。仅当指纹变化时 + * 才值得运行一次心跳 LLM 巡检;指纹不变 → skip(只写时间戳落库,不空转 LLM)。 + * + * 维度设计(对照 docs/agent-liveness-redesign.md §三.2「LLM 巡检只在有实际变化时发生」): + * · 队列签名:**内容(sourceType + item id)**而非深度 —— 同一邮件一直挂着不构成 + * 「状态变化」;新邮件到达(新 id)才触发。修复旧实现「队列深度相同即 skip / + * 队列深度变化即巡检」的双向误判:同一邮件挂一天不空转 LLM,新邮件却立刻感知。 + * · 活跃任务签名:任务开始/结束 → 巡检一次;任务集稳定 → 不巡检(任务会话自己 + * 在做 LLM 工作,心跳不需要重复巡检)。 + * + * 特意**不含** lastHeartbeat 时间戳(skip 也会写它 → 含则指纹恒变化 → 永远巡检)。 + */ +export function heartbeatStateFingerprint( + queued: Array<{ sourceType: string; id: string }>, + activeTaskIds: Iterable, +): string { + const queue = queued + .filter(i => i.sourceType !== 'heartbeat') + .map(i => `${i.sourceType}:${i.id}`) + .sort() + .join(','); + const tasks = [...activeTaskIds].sort().join(','); + return `q:[${queue}]|t:[${tasks}]`; +} + export class HeartbeatScheduler { private initialTimer?: ReturnType; private timer?: ReturnType; diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 1197839bf..8ddb7df77 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -16,7 +16,6 @@ export { export { RoleLoader } from './role-loader.js'; export { HeartbeatScheduler } from './heartbeat.js'; export { ContextEngine, type OrgContext, type ContextConfig, type LLMSummarizer, type SystemPromptResult, type SystemPromptSegment } from './context-engine.js'; -export { CognitivePreparation, selectCognitiveDepth, type CognitiveLLM, type RetrievalBackend } from './cognitive.js'; export { SmartTokenCounter, getDefaultTokenCounter, diff --git a/packages/core/src/lock-resources.ts b/packages/core/src/lock-resources.ts index a81eaf88d..6897a2405 100644 --- a/packages/core/src/lock-resources.ts +++ b/packages/core/src/lock-resources.ts @@ -5,9 +5,10 @@ * * 资源域锁原先的粒度是**工具名**,因此同一个资源可以有两条互不互斥的写入路径: * - * 1. `update_notebook` 登记为域 `'notebook'`,而别名 `update_working_memory` - * 登记为域 `'working-memory'` —— 两者写的是**同一个 Map** - * (`Agent.writeNotebookEntry`),却因域名不同被判定为「不冲突」,锁形同虚设。 + * 1. notebook 写工具曾按「正式名 + 别名」分登记到不同的域(`'notebook'` 与 + * `'working-memory'`),写的却是**同一个 Map**(`Agent.writeNotebookEntry`), + * 因域名不同被判定为「不冲突」,锁形同虚设。现别名已删除、名字归一为 + * `notebook_upsert` / `notebook_clear`,共享 `agent-memory:notebook` 锁域。 * 2. `file_write` 走 `fs:` 域,而记忆工具走 `memory` / `notebook` 域: * 同一个 `knowledge.md` 由「内存模型整文件重写」与「通用文件工具」两条路径 * 写入,互不互斥 —— 外部编辑会被下一次内存落盘**静默覆盖**。 @@ -70,3 +71,37 @@ export function memoryResourceForPath( export function memoryResourceLock(resource: AgentMemoryResource): LockRequest { return { domain: AGENT_MEMORY_RESOURCE_DOMAIN, sub: resource }; } + +/** + * 「单一写入者」记忆文件 —— 通用文件工具 / shell 一律禁止直写。 + * + * 这些文件由记忆服务整文件维护;任何绕过服务的外部写入都会与内存模型分叉, + * 最终被下一次落盘**静默覆盖**(审计 P-01:实测 2026-09-27 手改 knowledge.md + * 于 2026-09-28 被覆盖)。因此写门禁直接拒绝,并引导改用记忆 / 笔记本工具。 + * + * 注意:`role.md` / `heartbeat.md`(身份 / 巡检)**不在此列** —— 它们本就由 + * Agent 自行编辑(Learning Habits),不属于单一写入者资源。 + */ +export const SINGLE_WRITER_MEMORY_FILES: ReadonlySet = new Set([ + 'knowledge.md', + 'notebook.md', + 'state.md', +]); + +/** + * 目标路径是否为「单一写入者记忆文件」。 + * + * `workspacePath` 提供时,位于**该工作区内**的同名文件视为普通文件(放行)—— + * 记忆文件位于 agent home,不在工作区内;工作区里的 `knowledge.md` 可能只是 + * 普通文档,Agent 有权编辑。未提供 workspacePath 时按安全方向(拒绝)。 + */ +export function isSingleWriterMemoryFile(rawPath: string, workspacePath?: string): boolean { + if (typeof rawPath !== 'string' || rawPath.length === 0) return false; + if (!SINGLE_WRITER_MEMORY_FILES.has(basename(rawPath).toLowerCase())) return false; + if (workspacePath) { + const abs = resolve(rawPath); + const root = resolve(workspacePath); + if (abs === root || abs.startsWith(root + sep)) return false; + } + return true; +} diff --git a/packages/core/src/mailbox.ts b/packages/core/src/mailbox.ts index edd8fb354..a1b7e0b1c 100644 --- a/packages/core/src/mailbox.ts +++ b/packages/core/src/mailbox.ts @@ -138,13 +138,6 @@ export interface MailboxPersistence { updateStatus(itemId: string, status: MailboxItemStatus, extra?: Partial): void; /** Mark all items stuck in 'processing' as 'dropped' (stale after restart). */ markStaleProcessingAsDropped?(agentId: string): number; - /** - * Mark stuck 'processing' items as 'completed' (runtime self-healing). - * - * `ownerId` = 本实例认领者标识。传入后,持久层只清理「无认领 / 租约已过期 / - * **本实例自己持有**」的行,从而**不误杀其它实例仍在有效租约内处理**的项(跨实例互踩)。 - */ - markStaleProcessingAsCompleted?(agentId: string, ownerId?: string): number; /** Load persisted queued items for this agent (for recovery on restart). */ loadQueued?(agentId: string): MailboxItem[]; /** Load persisted deferred items for this agent (for auto-resurface). */ @@ -366,16 +359,18 @@ export class AgentMailbox { } /** - * Runtime self-healing: mark DB items stuck in 'processing' as 'completed'. + * Runtime self-healing: mark DB items stuck in 'processing' as 'dropped'. * Called by the watchdog when no item is being processed in memory. - * Unlike recoverStaleItems (startup-only, marks as 'dropped'), this uses - * 'completed' because the processing likely did finish — the DB update - * just failed silently or was interrupted. + * + * 语义修正(不再标 'completed'):completed = 「Agent 明确正常结束」—— + * 卡住/被打断/超时的工作**不属于完成**,且 completed 会让 resolveReinjectDecision + * 判定为 settled(抑制同键补偿重投),等于把未完成工作静默吞掉。 + * 改标 'dropped':异常可见(UI 红色)+ 允许后续补偿重投(dropped → reuse)。 + * storage 的 markStaleProcessingAsDropped 本身租约感知(仅碰「无认领/租约空/ + * 租约过期」的行,不误杀其它实例在飞项),strict-state 项会回队重试。 */ cleanStaleProcessing(): number { - // 传 ownerId:只清理「无认领 / 租约过期 / 本实例持有」的行, - // 避免把其它实例仍有效租约内的 processing 项误标为 completed(跨实例互踩)。 - return this.persistence?.markStaleProcessingAsCompleted?.(this.agentId, this.ownerId) ?? 0; + return this.persistence?.markStaleProcessingAsDropped?.(this.agentId) ?? 0; } /** diff --git a/packages/core/src/memory/store.ts b/packages/core/src/memory/store.ts index 84f086a93..7a068c14c 100644 --- a/packages/core/src/memory/store.ts +++ b/packages/core/src/memory/store.ts @@ -7,9 +7,20 @@ * * Additionally exports Notebook (NOTEBOOK.md) parse/serialize for the cognitive workspace. * Procedural Memory (ROLE.md + skills) is managed by RoleLoader and the skill system. - * Legacy MEMORY.md is migrated once via `ensureKnowledgeFile` and is never written. + * + * ─── Migration-read layer(审计 P-08:**读旧、只写新**)──────────────────── + * 记忆读写遵循单一策略:**读取端容忍一切历史格式,写入端只发一种规范格式**。 + * 遗留数据不删除,而是在加载时被**读入并迁移**为规范形态(无损、可检索): + * 来源(读) → 归宿(写,唯一规范) + * 1. `memories.json` → knowledge.md `## _observations`,源消费 + * 2. `MEMORY.md` → knowledge.md(源保留;仅当 knowledge 不存在时并入) + * 3. `state.md`(已退场介质) → knowledge.md `## _observations`,源改名 `.migrated` + * 4. 旧 `` 行 → 加载时收敛为单 `data-meta` JSON 行 + * 不变量:任一次落盘后,knowledge.md 内不再残留旧格式(`, tags: ` 行)。读取端的 + * 容忍只为让旧数据**能进来**,不是长期形态。清退条件:线上所有 agent 目录均无上述 + * 遗留文件、且 knowledge.md 无 `, tags: ` 行(实测 2026-09-29 仍有多名 agent 残留,故暂不满足)。 */ -import { readFileSync, writeFileSync, existsSync, mkdirSync, readdirSync, appendFileSync, statSync, unlinkSync } from 'node:fs'; +import { readFileSync, writeFileSync, existsSync, mkdirSync, readdirSync, appendFileSync, statSync, unlinkSync, renameSync } from 'node:fs'; import { basename, join } from 'node:path'; import { createLogger, @@ -19,22 +30,21 @@ import { type LLMMessage, MEMORY_MD_SECTION_MAX_CHARS, MEMORY_MD_TOTAL_MAX_CHARS, + MEMORY_OBSERVATIONS_MAX_CHARS, MEMORY_ENTRY_MAX_CHARS, KNOWLEDGE_MD_SELF_HEAL_BYTES, KNOWLEDGE_SECTION_KEY_MAX_CHARS, - KNOWLEDGE_STUB_MAX_CHARS, NOTEBOOK_KEY_MAX_CHARS, NOTEBOOK_MAX_ENTRIES, NOTEBOOK_MAX_AGENT_ENTRIES, NOTEBOOK_TTL_MS_AGENT, NOTEBOOK_TTL_MS_SYSTEM, - NOTEBOOK_TTL_MS_CPP, SESSION_STORAGE_COMPACT_KEEP, SESSION_STORAGE_COMPACT_TRIGGER, CONTEXT_SLOT_MAX_CHARS, } from '@markus/shared'; -import type { IMemoryStore, MemoryEntry, ConversationSession } from './types.js'; -import { ensureKnowledgeFile, knowledgePath } from './taxonomy.js'; +import type { IMemoryStore, MemoryEntry, ConversationSession, CompactResult } from './types.js'; +import { ensureKnowledgeFile, knowledgePath, retiredStatePath } from './taxonomy.js'; import { writeFileAtomic } from '../atomic-write.js'; import { buildSlotSegment, buildSummarySegment, sanitizeSlotKey, type SlotEntry } from '../context-slot.js'; @@ -124,7 +134,7 @@ function parseDataMeta(raw: string): Record | undefined { // ─── Notebook (NOTEBOOK.md) parse/serialize ───────────────────────────────── -export type NotebookEntryManaged = 'agent' | 'system' | 'cpp'; +export type NotebookEntryManaged = 'agent' | 'system'; export interface NotebookEntry { text: string; @@ -178,7 +188,7 @@ export function parseNotebook(markdown: string): Map { const managedMatch = NOTEBOOK_MANAGED_RE.exec(line); if (managedMatch) { const val = managedMatch[1] as NotebookEntryManaged; - if (val === 'agent' || val === 'system' || val === 'cpp') currentManaged = val; + if (val === 'agent' || val === 'system') currentManaged = val; continue; } contentLines.push(line); @@ -230,7 +240,6 @@ export function normalizeNotebookKey(raw: string): string { /** Per-tier TTL lookup for a notebook entry. */ export function notebookTtlMs(managed: NotebookEntryManaged): number { - if (managed === 'cpp') return NOTEBOOK_TTL_MS_CPP; if (managed === 'system') return NOTEBOOK_TTL_MS_SYSTEM; return NOTEBOOK_TTL_MS_AGENT; } @@ -253,9 +262,9 @@ export interface NotebookPruneResult { * nothing, so a notebook could only ever grow. * * Eviction order is oldest-`updatedAt`-first within a tier, and the machine-written - * tiers (`cpp`, then `system`) are evicted BEFORE the agent tier: situational state - * is cheaper to lose than the agent's own deliberate notes, and it expires on its - * own soon anyway. + * `system` tier is evicted BEFORE the agent tier: machine situational state is + * cheaper to lose than the agent's own deliberate notes, and it expires on its own + * soon anyway. */ export function pruneNotebookEntries( entries: Map, @@ -287,12 +296,12 @@ export function pruneNotebookEntries( // ── Pass 3: hard total cap ───────────────────────────────────────────── // Evict oldest-first, and among comparable ages drop the LOWER-durability tier - // first: cpp → system → agent. Machine-written situational state is cheaper to + // first: system → agent. Machine-written situational state is cheaper to // lose than the agent's own deliberate notes (and it would expire on its own // soon anyway). Getting this order backwards silently ate the agent's own notes // whenever the machine tiers filled the budget. if (entries.size > NOTEBOOK_MAX_ENTRIES) { - const evictionRank = (e: NotebookEntry) => (e.managed === 'cpp' ? 0 : e.managed === 'system' ? 1 : 2); + const evictionRank = (e: NotebookEntry) => (e.managed === 'system' ? 0 : 1); const evictable = [...entries.entries()].sort((a, b) => { const ra = evictionRank(a[1]); const rb = evictionRank(b[1]); if (ra !== rb) return ra - rb; @@ -341,6 +350,8 @@ export function saveNotebook(dataDir: string, entries: Map> = new Map(); private longTermFile: string; + private longTermArchiveFile: string; constructor(dataDir: string) { this.dataDir = dataDir; @@ -361,6 +373,7 @@ export class MemoryStore implements IMemoryStore { ensureKnowledgeFile(dataDir); // SSOT: always knowledge.md after ensure (never write legacy MEMORY.md). this.longTermFile = knowledgePath(dataDir); + this.longTermArchiveFile = join(dataDir, 'knowledge-archive.md'); this.loadFromDisk(); this.loadSessionsFromDisk(); } @@ -369,10 +382,89 @@ export class MemoryStore implements IMemoryStore { return basename(this.longTermFile); } + /** 归档文件名:超预算段落正文的**无损**去处(可被 memory_search 检索,不注入)。 */ + getArchiveFileName(): string { + return basename(this.longTermArchiveFile); + } + + /** + * 记忆预算健康度(审计 P-12):驱动提示词内的健康信号,让 Agent 知道何时该整理。 + */ + getMemoryHealth(): { totalChars: number; cap: number; percent: number; observations: number; curatedSections: number; archiveChars: number; lastConsolidatedAt: string | null } { + let totalChars = 0; + try { + if (existsSync(this.longTermFile)) totalChars = readFileSync(this.longTermFile, 'utf-8').length; + } catch { /* unreadable — treat as 0 */ } + let archiveChars = 0; + try { + if (existsSync(this.longTermArchiveFile)) archiveChars = readFileSync(this.longTermArchiveFile, 'utf-8').length; + } catch { /* archive optional */ } + const cap = MEMORY_MD_TOTAL_MAX_CHARS; + return { + totalChars, + cap, + percent: cap > 0 ? Math.round((totalChars / cap) * 100) : 0, + observations: this.entries.length, + curatedSections: parseCuratedSections(this.getLongTermMemory()).length, + archiveChars, + lastConsolidatedAt: this.getLastConsolidatedAt(), + }; + } + + /** 审计 P-11:上次整理(dream / memory_organize)时间 —— 供 memory_stats 与前端可观测。 */ + getLastConsolidatedAt(): string | null { + try { + const f = join(this.dataDir, 'memory-meta.json'); + if (!existsSync(f)) return null; + const meta = JSON.parse(readFileSync(f, 'utf-8')) as { lastConsolidatedAt?: string }; + return meta.lastConsolidatedAt ?? null; + } catch { + return null; + } + } + + /** 记录一次整理发生(用于陈旧度信号)。 */ + markConsolidated(at: Date = new Date()): void { + try { + const f = join(this.dataDir, 'memory-meta.json'); + let meta: Record = {}; + if (existsSync(f)) { + try { meta = JSON.parse(readFileSync(f, 'utf-8')) as Record; } catch { meta = {}; } + } + meta['lastConsolidatedAt'] = at.toISOString(); + writeFileAtomic(f, JSON.stringify(meta, null, 2)); + } catch (err) { + log.warn('Failed to persist memory meta', { error: String(err) }); + } + } + + /** + * Losslessly move a curated section body into `knowledge-archive.md` + * (审计 P-04/P-05:绝不静默丢弃 / 截断)。 + * 主文件保留标题 + 指针,主题仍可发现;正文原文进归档,仍可检索。 + */ + private archiveSection(name: string, body: string): number { + const trimmed = body.trim(); + if (!trimmed) return 0; + try { + const head = '# Knowledge Archive\n\n' + + '\n\n'; + const existing = existsSync(this.longTermArchiveFile) + ? readFileSync(this.longTermArchiveFile, 'utf-8') + : head; + if (!existing.includes(trimmed)) { + appendFileSync(this.longTermArchiveFile, `## ${name}\n${trimmed}\n\n`); + } + } catch (err) { + log.warn('Failed to archive section body', { name, error: String(err) }); + } + return trimmed.length; + } + /** * NOTE (2026-09-16): `getStateMemory()` / `pruneStateMemory()` were removed together * with the state.md store. Situational state is Working-layer data and lives in - * NOTEBOOK.md (`update_notebook`), which already has per-tier TTL — a second + * NOTEBOOK.md (`notebook_upsert`), which already has per-tier TTL — a second * short-lived store added no capability, only a second place to look. * See docs/MEMORY-SYSTEM.md §10.2 (option A). */ @@ -433,6 +525,28 @@ export class MemoryStore implements IMemoryStore { }); } + // 归档段落(无损去处):memory_search 必须能找回被归档的内容(审计 P-04/P-05)。 + try { + if (existsSync(this.longTermArchiveFile)) { + const archivedText = readFileSync(this.longTermArchiveFile, 'utf-8'); + for (const section of parseCuratedSections(archivedText)) { + const body = `## ${section.name}\n${section.body}`; + const score = scoreKeywordHaystack(body, tokens, fullLower); + if (score <= 0) continue; + scored.push({ + entry: { + id: `archived_${slugSectionId(section.name)}`, + timestamp: '', + type: 'fact', + content: body.length > 2500 ? `${body.slice(0, 2500)}…` : body, + metadata: { source: 'archive', section: section.name, store: this.getArchiveFileName() }, + }, + score: score + 0.1, + }); + } + } + } catch { /* archive missing/unreadable — non-fatal */ } + scored.sort((a, b) => b.score - a.score || a.entry.id.localeCompare(b.entry.id)); return scored.map((s) => s.entry); } @@ -650,12 +764,14 @@ export class MemoryStore implements IMemoryStore { }; } const key = sectionKey; - let truncatedContent = sanitizeSectionBody(content); + const truncatedContent = sanitizeSectionBody(content); + // 审计 P-04/P-05:单段正文超限 —— 不再静默截断,改为可读拒绝(无损、可操作)。 if (truncatedContent.length > MEMORY_MD_SECTION_MAX_CHARS) { - log.warn('Section content exceeds limit, truncating', { - key, original: content.length, limit: MEMORY_MD_SECTION_MAX_CHARS, - }); - truncatedContent = truncatedContent.slice(0, MEMORY_MD_SECTION_MAX_CHARS); + return { + ok: false, + reason: `Section body is ${truncatedContent.length} chars, over the ${MEMORY_MD_SECTION_MAX_CHARS}-char per-section limit. ` + + 'Split it into multiple sections or shorten it — content is never silently truncated.', + }; } let existing = ''; @@ -670,42 +786,27 @@ export class MemoryStore implements IMemoryStore { const regex = new RegExp(`(## ${key.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')})\\n[\\s\\S]*?(?=\\n## |$)`); updated = existing.replace(regex, `${sectionHeader}\n${truncatedContent}\n`); } else { - updated = existing + `\n${sectionHeader}\n${truncatedContent}\n`; - } - - if (updated.length > MEMORY_MD_TOTAL_MAX_CHARS) { - // Attempt auto-compression first before refusing the write - log.warn('knowledge.md total size exceeds limit, attempting compression', { - key, fileSize: updated.length, limit: MEMORY_MD_TOTAL_MAX_CHARS, - }); - const compressed = this.compressLongTermMemory(); - if (compressed.charsAfter < updated.length) { - log.info('Compression freed space, retrying write', { - key, charsFreed: updated.length - compressed.charsAfter, - }); - // Re-read the freshly compressed file and retry - existing = readFileSync(this.longTermFile, "utf-8"); - if (existing.includes(sectionHeader)) { - const regex = new RegExp(`(## ${key.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')})\\n[\\s\\S]*?(?=\\n## |$)`); - updated = existing.replace(regex, `${sectionHeader}\n${truncatedContent}\n`); - } else { - updated = existing + `\n${sectionHeader}\n${truncatedContent}\n`; - } - if (updated.length > MEMORY_MD_TOTAL_MAX_CHARS) { - log.warn('knowledge.md still exceeds limit even after compression, refusing write', { - key, fileSize: updated.length, limit: MEMORY_MD_TOTAL_MAX_CHARS, - }); - return { ok: false, reason: `knowledge.md is full (> ${MEMORY_MD_TOTAL_MAX_CHARS} chars) even after compression; write refused. Prune or shorten sections, or use memory_save (## _observations) instead.` }; - } + // 审计 P-17(严重 bug 修复):`## _observations` 必须是**最后一个**段落—— + // saveToDisk() 以该标记为界重建文件,会丢弃其后的所有内容。历史上这里直接 + // `existing + section` 追加到末尾,恰好落在 _observations 之后,导致**任何新建 + // curated 段落在下一次观察区保存时被静默删除**(正文只剩归档副本)。 + // 正确做法:新段落插入到 _observations **之前**。 + const obsIdx = existing.indexOf('\n## _observations'); + if (obsIdx >= 0) { + updated = existing.slice(0, obsIdx) + `\n${sectionHeader}\n${truncatedContent}\n` + existing.slice(obsIdx); } else { - log.warn('knowledge.md still exceeds limit after compression, refusing write', { - key, fileSize: updated.length, limit: MEMORY_MD_TOTAL_MAX_CHARS, - }); - return { ok: false, reason: `knowledge.md is full (> ${MEMORY_MD_TOTAL_MAX_CHARS} chars) and could not be compressed further; write refused. Prune or shorten sections, or use memory_save (## _observations) instead.` }; + updated = existing + `\n${sectionHeader}\n${truncatedContent}\n`; } } writeFileAtomic(this.longTermFile, updated); + // 审计 P-04/P-05:总量超预算 → 无损再平衡(归档),绝不拒绝写入、绝不静默丢弃。 + if (updated.length > MEMORY_MD_TOTAL_MAX_CHARS) { + const rebalanced = this.compressLongTermMemory(); + log.info('knowledge.md over budget after write — rebalanced losslessly', { + key, totalChars: updated.length, charsAfter: rebalanced.charsAfter, archived: rebalanced.truncatedChunks, + }); + } log.debug('Long-term memory updated', { key, sectionChars: truncatedContent.length, totalChars: updated.length, store: this.getStoreFileName() }); return { ok: true }; } catch (err) { @@ -843,10 +944,20 @@ export class MemoryStore implements IMemoryStore { return idx; } - compactSession(sessionId: string, keepLast: number = 20): { summary: string; flushedCount: number } { - const session = this.sessions.get(sessionId); - if (!session || session.messages.length <= keepLast) { - return { summary: '', flushedCount: 0 }; + compactSession(sessionId: string, keepLast: number = 20): CompactResult { + // SSOT / LRU fallback: the in-memory map only holds the N most recent + // sessions (MAX_SESSIONS_IN_MEMORY=20) while thousands live on disk. Reading + // ONLY `this.sessions` made compact() a silent no-op for any session that + // had been evicted — it returned ok with flushedCount:0 and the agent could + // not tell "nothing to compact" from "compaction failed" (the 2026-09-30 + // 刘利 P0 report). Every OTHER session accessor already falls back to disk + // (getSession/getSlots/serializeSummary/getRecentMessages); this one must too. + const session = this.sessions.get(sessionId) ?? this.getSession(sessionId); + if (!session) { + return { summary: '', flushedCount: 0, remaining: 0, found: false }; + } + if (session.messages.length <= keepLast) { + return { summary: '', flushedCount: 0, remaining: session.messages.length, found: true }; } // MessageGroup-atomic cut: never split an assistant(tool_calls) from its @@ -897,13 +1008,20 @@ export class MemoryStore implements IMemoryStore { // - turn-neutral (a `role:'user'` message would pollute attribution and // could be mistaken for genuine human input). // Raw history is still fully recoverable via the archived fragment below. - session.summary = summary; + // 审计 P-09:摘要锚点是**追加式多代**(不再覆盖)——每次压缩各留一条代锚, + // 最新在后、总代有界(超出的旧锚仍在归档 fragment 中,可 session_retrieve)。 + const anchorLine = `- [${new Date().toISOString()}] ${summary.replace(/\s+/g, ' ').trim().slice(0, 400)}`; + const priorAnchors = (session.summary ?? '') + .split('\n') + .map((l) => l.trim()) + .filter((l) => l.length > 0); + session.summary = [...priorAnchors, anchorLine].slice(-MemoryStore.MAX_SUMMARY_ANCHORS).join('\n'); session.summaryPagedOut = flushedCount; session.messages = retained; this.saveSessionToDisk(session); log.info('Session compacted', { sessionId, flushedCount, remaining: session.messages.length }); - return { summary, flushedCount }; + return { summary, flushedCount, remaining: session.messages.length, found: true }; } /** @@ -1135,8 +1253,57 @@ export class MemoryStore implements IMemoryStore { private static readonly MAX_MEMORY_ENTRIES = 500; + /** + * Migration-read layer(P-08)——**读旧、只写新**。 + * + * 读取端容忍一切历史格式;写入端只发单一规范格式(单 `data-meta` JSON 行)。 + * 遗留数据不删除,而是在加载时被读入并迁移为规范形态: + * - 已退场的 `state.md` → 一条规范观察(源改名 `.migrated`,可恢复、不重复读); + * - knowledge.md 内的旧 `` 行 → 标记需收敛重写。 + * (`memories.json` / `MEMORY.md` 的迁移在各自调用点,见文件头。) + * + * @returns true 表示内存态已变化、调用方应落盘收敛为规范格式。 + */ + private runLegacyMigrations(): boolean { + let changed = false; + + // 已退场的 state.md:读入为一条观察,源改名(而非删除)→ 不再重复读取。 + const retired = retiredStatePath(this.dataDir); + if (existsSync(retired)) { + try { + const raw = readFileSync(retired, 'utf-8').trim(); + if (raw) { + this.entries.push({ + id: `obs_${Date.now()}`, + timestamp: new Date().toISOString(), + type: 'note', + content: raw, + metadata: { tags: ['legacy-state'] }, + }); + changed = true; + } + renameSync(retired, `${retired}.migrated`); + log.info('Migrated retired state.md into knowledge.md observations', { chars: raw.length }); + } catch (err) { + log.warn('Failed to migrate retired state.md', { error: String(err) }); + } + } + + // 旧 `, tags:` 行(单 data-meta 之前的历史格式):标记需收敛重写为规范格式。 + try { + const raw = readFileSync(this.longTermFile, 'utf-8'); + if (/(?:^|\n)', '', ]; - const entries = this.entries - .filter(e => e.content.trim().length > 0) - .slice(-MemoryStore.MAX_MEMORY_ENTRIES); + // 审计 P-10:观察缓冲区有界(字符),且与压缩碎片**分池**(各有 500 上限), + // 碎片不再挤占观察配额。超限的观察**无损归档**(可检索),绝不静默丢弃。 + const isFrag = (e: MemoryEntry) => e.type === 'conversation_fragment'; + const obsCount = () => this.entries.filter(e => !isFrag(e)).length; + let obsChars = this.entries + .filter(e => !isFrag(e)) + .reduce((n, e) => n + e.content.length + 96, 0); + while (obsChars > MEMORY_OBSERVATIONS_MAX_CHARS && obsCount() > 1) { + const idx = this.entries.findIndex(e => !isFrag(e)); + if (idx < 0) break; + const oldest = this.entries.splice(idx, 1)[0]!; + this.archiveSection(`observation ${oldest.id}`, oldest.content); + obsChars -= oldest.content.length + 96; + } + // 分池计数上限:观察与碎片各自独立保留最新 500 条(保持原顺序)。 + const keptObs = this.entries.filter(e => !isFrag(e)).slice(-MemoryStore.MAX_MEMORY_ENTRIES); + const keptFrags = this.entries.filter(isFrag).slice(-MemoryStore.MAX_MEMORY_ENTRIES); + const keepIds = new Set([...keptObs, ...keptFrags].map(e => e.id)); + const entries = this.entries.filter(e => keepIds.has(e.id) && e.content.trim().length > 0); this.entries = entries; for (const entry of entries) { obsLines.push(`### ${entry.id}`); @@ -1526,63 +1719,55 @@ export class MemoryStore implements IMemoryStore { } const sectionsBefore = sections.length; - let truncatedChunks = 0; + let archived = 0; + + const render = (): string => { + const out: string[] = [...preambleLines]; + for (const s of sections) out.push(s.headerLine, s.body.join('\n')); + return out.join('\n'); + }; + const size = () => render().length; + + const isPointer = (s: { body: string[] }) => /^_\[archived/.test(s.body.join('\n').trim()); + const archiveBodyOf = (section: { headerLine: string; body: string[] }): void => { + const bodyStr = section.body.join('\n').trim(); + if (!bodyStr) return; + const name = section.headerLine.replace(/^##\s+/, '').trim(); + this.archiveSection(name, bodyStr); // 无损:正文进 knowledge-archive.md + section.body = [`_[archived → ${this.getArchiveFileName()};正文已无损归档,可用 memory_search 检索]_`]; + archived++; + }; - // ── Phase 2: per-section cap ───────────────────────────────────────── + // ── Phase 2: 单段超限 → 归档该段正文(无损,不截断)────────────────── for (const section of sections) { if (section.observationBuffer) continue; - const bodyStr = section.body.join('\n'); - if (bodyStr.length > MEMORY_MD_SECTION_MAX_CHARS) { - // The explanatory footer counts AGAINST the cap, otherwise the "capped" - // section is limit+footer chars and the cap is not actually a cap. - const note = `\n_[section truncated to ${MEMORY_MD_SECTION_MAX_CHARS} chars]_`; - section.body = [bodyStr.slice(0, MEMORY_MD_SECTION_MAX_CHARS - note.length) + note]; - truncatedChunks++; - } + if (section.body.join('\n').length > MEMORY_MD_SECTION_MAX_CHARS) archiveBodyOf(section); } - // ── Phase 3: TOTAL convergence ─────────────────────────────────────── - // The per-section cap alone does NOT bound the file: N sections × 3 000 chars - // can sit arbitrarily far above MEMORY_MD_TOTAL_MAX_CHARS, and the total was - // only ever checked on the WRITE path — where crossing it caused the write to - // be REFUSED. A refused write cannot shrink an oversized file, so once a - // knowledge.md was over budget it stayed over budget forever (measured: 23 323 - // chars against a 15 000 limit). Converge here instead: shrink the largest - // curated sections to a stub until the file fits, preserving the heading (so - // the index line and `memory_search` still surface the topic) and never - // touching `## _observations`. - const size = () => preambleLines.length - + sections.reduce((n, s) => n + s.headerLine.length + s.body.join('\n').length + 2, 0); + // ── Phase 3: 总量超限 → 逐个归档最大段落正文,直到达标 ───────────────── + // 审计 P-04/P-05:以前这里把段落「压成 stub」(有损);现在改为把正文移入 + // knowledge-archive.md(无损、可检索、不注入),主文件只留标题 + 指针。 + // 绝不触碰 ## _observations(它有自己的容量与整理路径)。 if (size() > MEMORY_MD_TOTAL_MAX_CHARS) { const shrinkable = sections - .filter(s => !s.observationBuffer) + .filter(s => !s.observationBuffer && !isPointer(s)) .sort((a, b) => b.body.join('\n').length - a.body.join('\n').length); for (const section of shrinkable) { if (size() <= MEMORY_MD_TOTAL_MAX_CHARS) break; - const bodyStr = section.body.join('\n').trim(); - if (bodyStr.length <= KNOWLEDGE_STUB_MAX_CHARS) continue; - section.body = [ - `${bodyStr.slice(0, KNOWLEDGE_STUB_MAX_CHARS)}\n` - + `_[section condensed — knowledge.md was over its ${MEMORY_MD_TOTAL_MAX_CHARS}-char budget]_`, - ]; - truncatedChunks++; + if (section.body.join('\n').trim().length <= 120) continue; // 已是小段,不值得归档 + archiveBodyOf(section); } } - const outputLines: string[] = [...preambleLines]; - for (const section of sections) { - outputLines.push(section.headerLine, section.body.join('\n')); - } - - const compressed = outputLines.join('\n'); - writeFileAtomic(this.longTermFile, compressed); + const compressed = render(); + if (compressed !== content) writeFileAtomic(this.longTermFile, compressed); return { charsBefore, charsAfter: compressed.length, sectionsBefore, sectionsAfter: sections.length, - truncatedChunks, + truncatedChunks: archived, }; } } diff --git a/packages/core/src/memory/taxonomy.ts b/packages/core/src/memory/taxonomy.ts index e9c4340e0..3ffdad838 100644 --- a/packages/core/src/memory/taxonomy.ts +++ b/packages/core/src/memory/taxonomy.ts @@ -16,6 +16,9 @@ * 1. knowledge 文件的路径 / 读写; * 2. 历史文件的**一次性迁移**(外部存储已被清空的情形下不静默丢内容)。 * + * 迁移策略:**读旧、只写新**。已退场的 `state.md` 不再作为写入目标;其遗留内容 + * 由 MemoryStore 的「Migration-read layer」读入并入 knowledge.md(见 store.ts 文件头)。 + * * 设计依据:docs/MEMORY-SYSTEM.md §10.2(state.md 退场,方案 A)。 */ @@ -39,6 +42,7 @@ export function retiredStatePath(dataDir: string): string { /** * 一次性迁移:历史 `MEMORY.md` → `knowledge.md`。 * + * [LEGACY-COMPAT #3](见 store.ts 文件头「Legacy compatibility layer」,含退役判据)。 * 旧实现把它**拆成两半**(`splitLegacyMemory` 按关键词猜哪些段落属于「状态」); * state 档退场后拆分不再有意义 —— 整体并入 knowledge.md,宁可多留一点内容, * 也不静默丢弃。仅在 knowledge.md 尚不存在时执行。 diff --git a/packages/core/src/memory/types.ts b/packages/core/src/memory/types.ts index 55638e62f..598eb9959 100644 --- a/packages/core/src/memory/types.ts +++ b/packages/core/src/memory/types.ts @@ -35,6 +35,25 @@ export interface ConversationSession { summaryPagedOut?: number; } +/** + * Result of a session compaction. + * + * `found` distinguishes a *genuine failure* (no such session in memory nor on + * disk) from a *valid no-op* (session already within `keep_last`) — the caller + * must be able to tell them apart, otherwise a silent no-op masquerades as + * success. This was the root of the 2026-09-30 刘利 P0 report ("compaction API + * reports ok but does nothing"). + */ +export interface CompactResult { + summary: string; + /** Number of messages paged out into the anchor summary / fragment archive. */ + flushedCount: number; + /** Messages retained in the session after compaction (<= keep_last). */ + remaining: number; + /** false when the session does not exist (in memory or on disk). */ + found: boolean; +} + /** * Unified memory interface for Agent and ContextEngine. * MemoryStore is the primary implementation. @@ -53,6 +72,19 @@ export interface IMemoryStore { // -- Semantic Memory: curated knowledge (knowledge.md SSOT) -- /** Basename of the on-disk semantic store (normally "knowledge.md"). */ getStoreFileName(): string; + /** Memory budget health (audit P-12) — powers the in-prompt health signal. */ + getMemoryHealth(): { + totalChars: number; + cap: number; + percent: number; + observations: number; + curatedSections: number; + archiveChars: number; + lastConsolidatedAt: string | null; + }; + /** 审计 P-11:整理时间可观测(可选,便于 mock)。 */ + getLastConsolidatedAt?(): string | null; + markConsolidated?(at?: Date): void; addLongTermMemory(key: string, content: string): { ok: boolean; reason?: string }; getLongTermMemory(): string; getLongTermMemoryExcluding(sections: string[]): string; @@ -83,7 +115,7 @@ export interface IMemoryStore { renameSession?(sessionId: string, title: string): void; appendMessage(sessionId: string, message: LLMMessage): void; getRecentMessages(sessionId: string, limit: number): LLMMessage[]; - compactSession(sessionId: string, keepLast?: number): { summary: string; flushedCount: number }; + compactSession(sessionId: string, keepLast?: number): CompactResult; summarizeAndTruncate(sessionId: string, keepLast: number): LLMMessage[]; // -- ContextOS: session slots (agent-managed fixed段) + fragment archive -- diff --git a/packages/core/src/tool-selector.ts b/packages/core/src/tool-selector.ts index 0039c16ea..509f49fff 100644 --- a/packages/core/src/tool-selector.ts +++ b/packages/core/src/tool-selector.ts @@ -9,6 +9,7 @@ import { isSkillOrMcpToolName, isWorkContextBoundTool, packToolDefBudget, + SCHEMA_INJECTED_TOOLS, TASK_EXECUTION_EXTRA_TOOLS, TOOL_DEF_CORE_KEEP, TOOL_DEF_PROTECTED, @@ -118,7 +119,7 @@ const TOOL_GROUPS: ToolGroup[] = [ 'knowledge base', 'knowledge', 'kb', 'synced document', '产出物', '产出', '交付物', '知识', '知识库', '贡献', '约定', '架构决策', '最佳实践', '经验'], toolNames: ['deliverable_create', 'deliverable_search', 'deliverable_list', 'deliverable_update', - 'knowledge_search', 'knowledge_list', 'knowledge_read'], + 'kb_search', 'kb_list', 'kb_read'], }, { name: 'office', @@ -303,7 +304,7 @@ export class ToolSelector { // Only skip keyword activation when the WHOLE group is already in — a // partially-present group must still be able to add the rest (e.g. // deliverable_search is base-always-on, so a keyword hit must still - // surface knowledge_search/list/read). + // surface kb_search/list/read). if (group.toolNames.length > 0 && group.toolNames.every(n => selected.has(n))) continue; const matched = group.keywords.some(kw => contextLower.includes(kw)); if (matched) { @@ -584,8 +585,8 @@ export class ToolSelector { // current item. pushUnique({ - name: 'update_working_memory', - description: 'Upsert a keyed entry in your working memory. Use to track priorities, context, decisions.', + name: 'notebook_upsert', + description: 'Upsert a keyed entry in your notebook (working memory). Use to track priorities, context, decisions.', inputSchema: { type: 'object', properties: { @@ -597,8 +598,8 @@ export class ToolSelector { }); pushUnique({ - name: 'clear_working_memory', - description: 'Remove a working memory entry by key, or clear all entries.', + name: 'notebook_clear', + description: 'Remove a notebook entry by key, or clear all agent entries.', inputSchema: { type: 'object', properties: { @@ -636,7 +637,10 @@ export class ToolSelector { // discover_tools 空烧 token。未激活的 skill/MCP 仍按渐进披露(catalog)延迟。 const protectedNames = new Set([...TOOL_DEF_PROTECTED, ...TOOL_DEF_CORE_KEEP]); for (const name of activated) { - if (opts.allTools.has(name)) protectedNames.add(name); + // registered handler OR schema-injected (pushUnique) tool — both must be + // eviction-immune once explicitly activated via discover_tools, otherwise + // activation is a no-op after budget eviction (P0-1 self-management fix). + if (opts.allTools.has(name) || SCHEMA_INJECTED_TOOLS.has(name)) protectedNames.add(name); } const { tools: capped, evicted } = evictToolsToBudget( result, diff --git a/packages/core/src/tools/file.ts b/packages/core/src/tools/file.ts index dd9254cbb..f88a6fc7d 100644 --- a/packages/core/src/tools/file.ts +++ b/packages/core/src/tools/file.ts @@ -3,7 +3,7 @@ import { basename, dirname, resolve } from 'node:path'; import { type PathAccessPolicy, validateManifest } from '@markus/shared'; import type { AgentToolHandler } from '../agent.js'; import { defaultSecurityGuard, type SecurityGuard } from '../security.js'; -import { matchedDenyWritePath } from '../write-guard.js'; +import { assertWriteAllowed } from '../write-guard.js'; const MANIFEST_FILENAMES = new Set(['agent.json', 'team.json', 'skill.json']); @@ -32,13 +32,15 @@ function resolveAndCheckAccess( rawPath: string, workspacePath: string | undefined, policy: PathAccessPolicy | undefined, -): { resolved: string; access: AccessLevel } { +): { resolved: string; access: AccessLevel; reason?: string } { const resolved = workspacePath ? resolve(workspacePath, rawPath) : resolve(rawPath); - // 判定收口到 write-guard(单一门禁,审计 P0-2 / G1),并用边界感知匹配, - // 避免 `/agents/agt_a` 误伤同前缀目录 `/agents/agt_ab`。 - if (matchedDenyWritePath(resolved, policy?.denyWritePaths)) { - return { resolved, access: 'denied' }; + // 判定收口到 write-guard(单一门禁,审计 P0-2 / G1),覆盖三类拒绝: + // denyWritePaths(其他 agent 工作区)、系统敏感文件、**单一写入者记忆文件** + // (knowledge.md / NOTEBOOK.md — 审计 P-01:禁止绕过记忆服务直写)。 + const decision = assertWriteAllowed(resolved, { policy, workspacePath }); + if (!decision.allowed) { + return { resolved, access: 'denied', reason: decision.reason }; } return { resolved, access: 'readwrite' }; @@ -156,10 +158,10 @@ export function createFileWriteTool(security?: SecurityGuard, workspacePath?: st async execute(args: Record): Promise { const rawPath = (args['path'] ?? args['file'] ?? args['file_path'] ?? args['filePath']) as string; - const { resolved: path, access } = resolveAndCheckAccess(rawPath, workspacePath, policy); + const { resolved: path, access, reason } = resolveAndCheckAccess(rawPath, workspacePath, policy); if (access === 'denied') { - return JSON.stringify({ status: 'denied', error: 'Write denied: this path belongs to another agent\'s workspace. Create your own worktree or copy the files to your workspace.' }); + return JSON.stringify({ status: 'denied', error: reason ?? 'Write denied: this path belongs to another agent\'s workspace. Create your own worktree or copy the files to your workspace.' }); } const content = args['content'] as string; @@ -207,10 +209,10 @@ export function createFileEditTool(security?: SecurityGuard, workspacePath?: str async execute(args: Record): Promise { const rawPath = (args['path'] ?? args['file'] ?? args['file_path'] ?? args['filePath']) as string; - const { resolved: path, access } = resolveAndCheckAccess(rawPath, workspacePath, policy); + const { resolved: path, access, reason } = resolveAndCheckAccess(rawPath, workspacePath, policy); if (access === 'denied') { - return JSON.stringify({ status: 'denied', error: 'Edit denied: this path belongs to another agent\'s workspace. Create your own worktree or copy the files to your workspace.' }); + return JSON.stringify({ status: 'denied', error: reason ?? 'Edit denied: this path belongs to another agent\'s workspace. Create your own worktree or copy the files to your workspace.' }); } const oldStr = args['old_string'] as string; diff --git a/packages/core/src/tools/mailbox-tools.ts b/packages/core/src/tools/mailbox-tools.ts index 444862a98..90c5a6764 100644 --- a/packages/core/src/tools/mailbox-tools.ts +++ b/packages/core/src/tools/mailbox-tools.ts @@ -56,13 +56,13 @@ export function createMailboxTools(ctx: MailboxToolContext): AgentToolHandler[] }, { - name: 'update_notebook', + name: 'notebook_upsert', description: `Upsert a keyed entry in your Notebook — your persistent cognitive workspace. Use to track priorities, context, decisions, blockers. ` + `Keys are short labels (≤${NOTEBOOK_KEY_MAX_CHARS} chars, e.g. "current-priorities", "blockers") — reuse a key to REPLACE it instead of creating a near-duplicate. ` + `Capacity: ${NOTEBOOK_MAX_AGENT_ENTRIES} agent entries (oldest evicted), ${NOTEBOOK_MAX_CHARS_PER_ENTRY} chars each, ${NOTEBOOK_MAX_ENTRIES} entries total. ` + `Entries expire on their own (agent ~4d, machine-written entries sooner), so anything you write here is working state, not durable knowledge — use memory_save/memory_update for that. ` + - `Delete entries you are done with via clear_notebook.`, + `Delete entries you are done with via notebook_clear.`, inputSchema: { type: 'object', properties: { @@ -86,7 +86,7 @@ export function createMailboxTools(ctx: MailboxToolContext): AgentToolHandler[] }, { - name: 'clear_notebook', + name: 'notebook_clear', description: 'Remove a Notebook entry by key, or clear all agent-managed entries.', inputSchema: { type: 'object', @@ -98,45 +98,31 @@ export function createMailboxTools(ctx: MailboxToolContext): AgentToolHandler[] async execute(args: Record): Promise { const key = args['key'] as string | undefined; const all = args['all'] as boolean | undefined; - if (all) { - return JSON.stringify(ctx.clearWorkingMemory()); - } - return JSON.stringify(ctx.clearWorkingMemory(key)); + return JSON.stringify(ctx.clearWorkingMemory(all ? undefined : key)); }, }, - // Backward compatibility aliases (hidden from discovery) + // 审计 §6:notebook 只读视图(原 update/clear_working_memory 别名已删除,名字归一)。 { - name: 'update_working_memory', - description: '[Alias for update_notebook] Upsert a keyed entry in your Notebook.', - inputSchema: { - type: 'object', - properties: { - key: { type: 'string', description: 'Label for this entry' }, - content: { type: 'string', description: 'The content to store' }, - }, - required: ['key', 'content'], - }, - async execute(args: Record): Promise { - const key = args['key'] as string; - const content = args['content'] as string; - if (!key || !content) return JSON.stringify({ status: 'error', error: 'key and content required' }); - return JSON.stringify(ctx.updateWorkingMemory(key, content)); - }, - }, - { - name: 'clear_working_memory', - description: '[Alias for clear_notebook] Remove a Notebook entry by key.', + name: 'notebook_read', + description: + 'Read your Notebook (working memory) — all entries, or a single key. Read-only. ' + + 'Entries are agent-written (your priorities/context) and machine-written (triage/deliberation), each with a TTL.', inputSchema: { type: 'object', properties: { - key: { type: 'string', description: 'Key to clear. Omit to clear all.' }, - all: { type: 'boolean', description: 'Set true to clear all entries' }, + key: { type: 'string', description: 'Read a single entry by key. Omit to list all entries.' }, }, }, async execute(args: Record): Promise { - const key = args['key'] as string | undefined; - return JSON.stringify(ctx.clearWorkingMemory(args['all'] ? undefined : key)); + const key = (args['key'] as string | undefined)?.trim(); + const snapshot = ctx.getWorkingMemorySnapshot(); + const entries = key ? snapshot.filter((e) => e.key === key) : snapshot; + return JSON.stringify({ + status: 'ok', + count: entries.length, + entries: entries.map((e) => ({ key: e.key, text: e.text, updatedAt: new Date(e.updatedAt).toISOString() })), + }); }, }, diff --git a/packages/core/src/tools/memory.ts b/packages/core/src/tools/memory.ts index 4ff22a57e..f33e1415f 100644 --- a/packages/core/src/tools/memory.ts +++ b/packages/core/src/tools/memory.ts @@ -421,5 +421,160 @@ export function createMemoryTools(ctx: AgentMemoryContext): AgentToolHandler[] { return JSON.stringify({ status: 'updated', section, mode, store }); }, }, + + { + name: 'memory_stats', + description: + 'Inspect your memory health: knowledge.md size vs budget, observation count, curated section count. ' + + 'Use periodically (or when memory feels bloated) to decide whether to run memory_organize. ' + + 'Returns JSON with budget usage percent. No args.', + inputSchema: { type: 'object', properties: {} }, + async execute(): Promise { + const store = storeName(ctx.memory); + // SSOT:预算口径必须与提示词横幅 / 写入端强制归档(getMemoryHealth = 文件原始大小)完全一致, + // 否则会出现「横幅说 122%、工具说 95%」的自相矛盾(审计 P-12/P-16)。 + const health = ctx.memory.getMemoryHealth(); + const curated = ctx.memory.getLongTermMemory(); + const obs = ctx.memory.getObservations(); + const sectionNames = [...curated.matchAll(/^## (.+)$/gm)] + .map((m) => m[1]!) + .filter((n) => n !== '_observations'); + const obsChars = obs.reduce((s, e) => s + (e.content?.length ?? 0), 0); + return JSON.stringify({ + status: 'ok', + store, + budget: { + totalChars: health.totalChars, + limit: health.cap, + usedPercent: health.percent, + }, + observations: { count: obs.length, chars: obsChars }, + curatedSections: { count: sectionNames.length, names: sectionNames.slice(0, 20) }, + archivedChars: health.archiveChars, + lastConsolidatedAt: health.lastConsolidatedAt, + hint: health.percent > 80 + ? 'Approaching budget — run memory_organize to merge observations into curated sections, or memory_update mode="forget" on superseded knowledge.' + : undefined, + }); + }, + }, + + { + name: 'memory_organize', + description: + 'Merge matching observations from ## _observations into a curated section and archive them. ' + + 'Args: { target_section: string (required), query?: string, ids?: string[], limit?: number }. ' + + 'Selects observations whose content matches `query` keywords OR whose ids are listed, appends them as bullet lines ' + + 'under target_section (budget-aware), then removes them from _observations. ' + + 'Returns { status, moved, archived, section }. Use to consolidate scattered observations into durable knowledge ' + + '(the manual counterpart of dream-cycle consolidation).', + inputSchema: { + type: 'object', + properties: { + target_section: { + type: 'string', + description: 'Curated section to append the merged observations into (e.g. "procedures", "conventions").', + }, + query: { + type: 'string', + description: 'Keyword filter — observations whose content contains any of these tokens (OR-match) get merged. Omit to use ids only.', + }, + ids: { + type: 'array', + items: { type: 'string' }, + description: 'Explicit observation entry IDs to merge (from memory_search / memory_list).', + }, + limit: { + type: 'number', + description: 'Max observations to merge in one call (default 20, max 50).', + }, + }, + required: ['target_section'], + }, + async execute(args: Record): Promise { + const store = storeName(ctx.memory); + const target = String(args['target_section'] ?? '').trim(); + if (!target) { + return JSON.stringify({ status: 'error', error: 'target_section is required.', store }); + } + if (/^_observations$/i.test(target)) { + return JSON.stringify({ + status: 'error', + error: 'The `_observations` buffer is not a curated section — merge INTO a named section (e.g. "procedures").', + store, + }); + } + const query = String(args['query'] ?? '').trim().toLowerCase(); + const ids = Array.isArray(args['ids']) ? args['ids'].map(String) : []; + const limit = Math.min(Math.max(Number(args['limit']) || 20, 1), 50); + + const all = ctx.memory.getObservations(); + let matches = all; + if (ids.length > 0) { + const idSet = new Set(ids); + matches = matches.filter((e) => idSet.has(e.id)); + } + if (query) { + matches = matches.filter( + (e) => + (e.content ?? '').toLowerCase().includes(query) || + JSON.stringify(e.metadata ?? {}).toLowerCase().includes(query), + ); + } + if (matches.length === 0) { + return JSON.stringify({ + status: 'ok', + moved: 0, + archived: 0, + section: target, + message: 'No matching observations found.', + store, + }); + } + + const selected = matches.slice(-limit); + const existing = ctx.memory.getLongTermSection(target); + const newLines = selected.map( + (e) => + `- [${e.type}] (${(e.timestamp ?? '').slice(0, 10)}) ${e.content}`, + ); + const merged = existing ? `${existing}\n${newLines.join('\n')}` : newLines.join('\n'); + const writeResult = ctx.memory.addLongTermMemory(target, merged); + if (!writeResult.ok) { + log.warn('memory_organize write refused', { + agentId: ctx.agentId, target, reason: writeResult.reason, + }); + return JSON.stringify({ + status: 'error', + error: writeResult.reason ?? 'knowledge.md write refused', + moved: 0, + section: target, + store, + }); + } + + const archived = ctx.memory.removeEntries(selected.map((e) => e.id)); + ctx.memory.markConsolidated?.(); + if (ctx.semanticSearch?.isEnabled()) { + for (const e of selected) { + ctx.semanticSearch.deleteMemory(e.id).catch((err) => { + log.warn('Failed to remove from semantic index', { error: String(err) }); + }); + } + } + + log.info('Agent organized memory', { + agentId: ctx.agentId, target, moved: selected.length, archived, + }); + return JSON.stringify({ + status: 'organized', + moved: selected.length, + archived, + section: target, + message: `Merged ${selected.length} observation(s) into "${target}" and archived them.`, + store, + }); + }, + }, ]; } diff --git a/packages/core/src/tools/patch.ts b/packages/core/src/tools/patch.ts index 59f4f7a6c..0e18c80e6 100644 --- a/packages/core/src/tools/patch.ts +++ b/packages/core/src/tools/patch.ts @@ -81,10 +81,10 @@ export function createPatchTool(security?: SecurityGuard, workspacePath?: string // Validation pass — collect resolved paths so the apply pass reuses them for (const patch of patches) { - const { resolved: filePath, access } = resolveAndCheckAccess(patch.file, workspacePath, policy); + const { resolved: filePath, access, reason } = resolveAndCheckAccess(patch.file, workspacePath, policy); if (access === 'denied') { - return JSON.stringify({ status: 'denied', error: `Write denied: this path belongs to another agent's workspace. Cannot ${patch.action}: ${patch.file}` }); + return JSON.stringify({ status: 'denied', error: reason ?? `Write denied: this path belongs to another agent's workspace. Cannot ${patch.action}: ${patch.file}` }); } const check = guard.validateFilePath(filePath); diff --git a/packages/core/src/tools/process-group.ts b/packages/core/src/tools/process-group.ts index a0dc19b29..265270947 100644 --- a/packages/core/src/tools/process-group.ts +++ b/packages/core/src/tools/process-group.ts @@ -56,9 +56,23 @@ function readPgid(pid: number): number | null { } /** - * True when `pid` leads its own process group and that group is not ours. - * Call this *after* spawning with `detached: true`; only then is it safe to - * use `kill(-pid)`. + * Verify that `pid` really leads its own process group right now. + * + * ⚠️ NOT for gating kills at spawn time. It reports on the process's CURRENT + * state, which stops matching the spawn contract the moment the wrapper exits: + * + * • While the wrapper is alive the probe is fine. Measured 25/25 correct for + * `spawn(..., {detached:true})` probed immediately afterwards — the older + * "setsid() races the ps probe" theory does NOT reproduce (the setsid runs + * in the child long before a forked `ps` can be observed). + * • Once the wrapper exits (self-daemonizing commands such as + * `sh -c 'sleep N & exit 0'`), `ps -p ` returns nothing → this returns + * false even though the group still has live members. A kill gated on that + * false degrades to signalling the dead wrapper alone and EVERY descendant + * leaks (measured: 100% leak). + * + * Callers that spawned with `detached: true` must pass the spawn contract + * (`!isWin`) instead. Keep this for diagnostics only. */ export function isIsolatedProcessGroup(pid: number | undefined): boolean { if (isWindows() || !pid || pid <= 1) return false; @@ -116,7 +130,7 @@ export function signalTree( } } - if (isolatedGroup && pid > 1) { + if (isolatedGroup && pid > 1 && pid !== process.pid) { try { process.kill(-pid, signal); return 'group'; diff --git a/packages/core/src/tools/process-manager.ts b/packages/core/src/tools/process-manager.ts index 82c4eb141..3376c3eca 100644 --- a/packages/core/src/tools/process-manager.ts +++ b/packages/core/src/tools/process-manager.ts @@ -4,7 +4,6 @@ import { platform } from 'node:os'; import type { AgentToolHandler } from '../agent.js'; import { KILL_GRACE_MS, - isIsolatedProcessGroup, isProcessGroupAlive, killProcessTree, type SignalDelivery, @@ -165,7 +164,14 @@ export function createBackgroundExecTool(workspacePath?: string): AgentToolHandl stderr: [], process: child, notified: false, - isolatedGroup: isIsolatedProcessGroup(child.pid), + // Trust the spawn contract: we passed `detached: !isWin`, so on POSIX + // this child IS the leader of its own process group. Probing with `ps` + // here is wrong — once the wrapper exits (self-daemonizing commands + // such as `foo & exit`), `ps -p ` returns nothing and the probe + // reports "not isolated". The kill path then degrades to signalling the + // (already dead) wrapper and every descendant leaks. Measured: 100% + // leak for `sh -c 'sleep N & exit 0'` when the probe gated the kill. + isolatedGroup: !isWin, }; child.stdout?.on('data', (chunk: Buffer) => { diff --git a/packages/core/src/tools/project-tools.ts b/packages/core/src/tools/project-tools.ts index 81fd3aa27..4a970f029 100644 --- a/packages/core/src/tools/project-tools.ts +++ b/packages/core/src/tools/project-tools.ts @@ -724,7 +724,7 @@ export function createProjectTools(ctx: ProjectToolsContext): AgentToolHandler[] ...(ctx.deliverableSearch ? [ { - name: 'knowledge_search', + name: 'kb_search', description: 'Search the project knowledge base (documents synced from bound knowledge directories). ' + 'Thin wrapper over deliverable_search with source forced to "knowledge". ' + @@ -769,7 +769,7 @@ export function createProjectTools(ctx: ProjectToolsContext): AgentToolHandler[] ...(ctx.deliverableList ? [ { - name: 'knowledge_list', + name: 'kb_list', description: 'List knowledge-base documents (source="knowledge") with optional project filter. ' + 'Thin wrapper over deliverable_list with source forced to "knowledge".', @@ -809,10 +809,10 @@ export function createProjectTools(ctx: ProjectToolsContext): AgentToolHandler[] ...(ctx.deliverableRead ? [ { - name: 'knowledge_read', + name: 'kb_read', description: 'Read the text content of a knowledge-base document by its file path (reference). ' + - 'Use the "reference" from knowledge_search / knowledge_list results. ' + + 'Use the "reference" from kb_search / kb_list results. ' + 'When project_id is provided, the read is scoped to that project\'s bound knowledge directories. ' + 'Thin wrapper that returns the extracted plain-text content for context.', inputSchema: { diff --git a/packages/core/src/tools/session.ts b/packages/core/src/tools/session.ts index 42e33aa4d..1f2515121 100644 --- a/packages/core/src/tools/session.ts +++ b/packages/core/src/tools/session.ts @@ -56,13 +56,26 @@ export interface SessionRepo { /** 可选注入:让 agent 通过 session 工具主动压缩自己的历史上下文(0.9.7 context-root-fix)。 */ export interface SessionCompactor { - compactOnDemand(sessionId: string, keepLast: number): { summary: string; flushedCount: number }; + compactOnDemand(sessionId: string, keepLast: number): { + summary: string; + flushedCount: number; + /** Messages retained after compaction. Optional — mocks may omit. */ + remaining?: number; + /** false when the session does not exist (memory + disk). Optional — mocks may omit. */ + found?: boolean; + }; /** ContextOS: compact with an explicit structured anchor (goal/done/next). */ compactWithAnchor?( sessionId: string, keepLast: number, anchor: { goal?: string; done?: string; next?: string }, - ): { summary: string; flushedCount: number; anchorKey: string }; + ): { + summary: string; + flushedCount: number; + remaining?: number; + found?: boolean; + anchorKey: string; + }; } /** ContextOS: agent-managed slot store — pin/unpin facts that survive compaction. */ @@ -437,7 +450,7 @@ export function createSessionTool(ctx: SessionToolContext): AgentToolHandler { const keepLast = Math.max(5, Math.min(200, Math.trunc(Number(n.keepLast) || 40) || 40)); try { const hasAnchor = n.goal || n.done || n.next; - let res: { summary: string; flushedCount: number; anchorKey?: string }; + let res: { summary: string; flushedCount: number; remaining?: number; found?: boolean; anchorKey?: string }; if (hasAnchor && ctx.compactor.compactWithAnchor) { res = ctx.compactor.compactWithAnchor(n.sessionId, keepLast, { goal: n.goal, done: n.done, next: n.next, @@ -445,15 +458,34 @@ export function createSessionTool(ctx: SessionToolContext): AgentToolHandler { } else { res = ctx.compactor.compactOnDemand(n.sessionId, keepLast); } + // Distinguish a genuine failure from a valid no-op. The 2026-09-30 刘利 + // P0 report: a session evicted from the in-memory LRU made compact() a + // silent no-op that still returned status:'ok', so the agent could not + // tell "nothing to compact" from "compaction failed" and never recovered. + if (res.found === false) { + return JSON.stringify({ + status: 'error', + message: `No session with id "${n.sessionId}" (not in memory nor on disk). Use session_list to find the correct id.`, + }); + } + const noop = res.flushedCount === 0; return JSON.stringify({ status: 'ok', flushedCount: res.flushedCount, - remaining: res.summary.length, + remaining: res.remaining ?? 0, summary: res.summary, ...(res.anchorKey ? { anchored_as: res.anchorKey } : {}), - note: hasAnchor - ? 'As a pinch: pin a goal/done/next anchor keeps your position alive even after compaction. Use session_pin to persist them as durable slots.' - : 'Earlier messages were compacted into the anchor summary above. Continue the work from this point; you do NOT need to re-read the flushed messages.', + ...(noop + ? { + skipped: true, + reason: 'within_keep_last', + note: `Nothing to compact: the session holds <= keep_last (${keepLast}) messages, so no history was dropped and no token was freed. This is a valid no-op, not a failure.`, + } + : { + note: hasAnchor + ? 'As a pinch: pin a goal/done/next anchor keeps your position alive even after compaction. Use session_pin to persist them as durable slots.' + : 'Earlier messages were compacted into the anchor summary above. Continue the work from this point; you do NOT need to re-read the flushed messages.', + }), }); } catch (err) { log.error('session compact failed', { error: String(err) }); @@ -693,7 +725,7 @@ export function createSessionTool(ctx: SessionToolContext): AgentToolHandler { ' Example: { "operation": "list", "since": "2026-08-01", "page": 1, "page_size": 20 }', '• session_get — get one session + its messages. Args: session_id, since/until, page, page_size. Returns { status, sessionId, messages: [{role, content}], total, page }.', ' Example: { "operation": "get", "session_id": "sess_...", "page_size": 50 }', - '• session_compact — collapse stale history into an anchor summary so future turns stop re-reading it. Atomic: an assistant tool-call and its tool results are NEVER split. Optionally pass goal/done/next to anchor your position. Use when earlier tool results are obsolete or context feels bloated. Returns { status, flushedCount, remaining, summary } — summary is a [SYSTEM]-prefixed anchor injected into the session; the raw flushed messages are archived and recoverable via session_retrieve.', + '• session_compact — collapse stale history into an anchor summary so future turns stop re-reading it. Atomic: an assistant tool-call and its tool results are NEVER split. Optionally pass goal/done/next to anchor your position. Use when earlier tool results are obsolete or context feels bloated. Returns { status, flushedCount, remaining, summary } — flushedCount = messages paged out (0 means nothing was compacted), remaining = messages kept, summary = a [SYSTEM]-prefixed anchor injected into the session; the raw flushed messages are archived and recoverable via session_retrieve. If the session already fits within keep_last you get flushedCount:0, skipped:true, reason:"within_keep_last" (a valid no-op, NOT a failure); an unknown session_id returns status:"error".', ' Args: session_id (required), keep_last (5-200, default 40), goal/done/next (optional anchors).', ' Example: { "operation": "compact", "session_id": "sess_...", "keep_last": 40 }', '• session_pin — write a durable slot into the fixed [SLOTS] segment: persisted per session and injected every turn, NEVER compacted until you unpin it. Best for your current goal / what is done / what is next (use key=goal/done/next). Returns { status, pinned: {key: content} }.', diff --git a/packages/core/src/tools/shell-session.ts b/packages/core/src/tools/shell-session.ts index 75caf9b0c..d0c814b80 100644 --- a/packages/core/src/tools/shell-session.ts +++ b/packages/core/src/tools/shell-session.ts @@ -18,7 +18,34 @@ import { SHELL_SESSION_IDLE_TIMEOUT_MS, SHELL_SESSION_MAX_OUTPUT_BYTES, } from '@markus/shared'; -import { isIsolatedProcessGroup, killProcessTree } from './process-group.js'; +import { existsSync } from 'node:fs'; +import { killProcessTree } from './process-group.js'; + +/** + * Pick the shell used for persistent sessions. + * + * WHY NOT `$SHELL` + * ---------------- + * `$SHELL` mirrors the *user's login shell*, which is frequently NOT a POSIX + * shell: fish, csh/tcsh, nu, pwsh… The session protocol below relies on POSIX + * behaviour (piping a command in, sourcing a sentinel back out), so a + * non-POSIX shell makes **every command fail instantly** with confusing errors + * like `Unable to read input file: Is a directory` — the agent then reports + * "commands don't work / hang", which is misdiagnosed as a timeout bug. + * + * We therefore prefer a known-good POSIX shell and only fall back to `$SHELL` + * as a last resort. + */ +function resolvePosixShell(): string { + for (const candidate of ['/bin/bash', '/bin/zsh', '/bin/sh']) { + try { + if (existsSync(candidate)) return candidate; + } catch { + /* ignore */ + } + } + return process.env['SHELL'] || '/bin/sh'; +} export interface ShellSession { id: string; @@ -60,6 +87,8 @@ class ManagedSession { readonly createdAt = Date.now(); lastUsedAt = Date.now(); alive = true; + /** Why this session was force-terminated (diagnostics only). */ + terminationReason: string | null = null; private pending: PendingCommand | null = null; private dataBuffer = ''; @@ -161,9 +190,20 @@ class ManagedSession { const timer = setTimeout(() => { const partial = this.pending?.output ?? ''; this.pending = null; - this.resetIdleTimer(); + // A timeout MUST terminate the command, not just give up on waiting. + // The previous implementation only resolved the promise: the command + // (and everything it had spawned) kept running, and the shell stayed + // busy forever, so every later command in this session queued behind + // the runaway process and "timed out" as well — the session was wedged + // (owner report 2026-10-01: a 120s limit never actually stopped + // anything). Terminating the session is the only deterministic way to + // stop an arbitrary subtree from a persistent shell; the manager + // transparently recreates a clean session (same cwd) on next use. + this.terminate('timed out'); resolve({ - stdout: partial + `\n[Command timed out after ${timeoutMs}ms]`, + stdout: + partial + + `\n[Command timed out after ${timeoutMs}ms — command and its child processes were killed; the shell session was reset.]`, exitCode: -1, }); }, timeoutMs); @@ -208,12 +248,25 @@ class ManagedSession { } kill() { - if (this.idleTimer) clearTimeout(this.idleTimer); + this.terminate('killed'); + } + + /** + * Force-terminate this session and everything it spawned. + * + * SIGTERM → SIGKILL over the whole process group: anything the agent started + * through this session (dev servers, watchers, test runners) must die with + * it, otherwise it is reparented to PID 1 and leaks forever. Used both by + * kill() and by the command-timeout path. + */ + terminate(reason: string) { + if (this.idleTimer) { + clearTimeout(this.idleTimer); + this.idleTimer = null; + } + this.terminationReason = reason; this.alive = false; - this.rejectPending('Session killed'); - // SIGTERM → SIGKILL over the whole process group: anything the agent - // started through this session (dev servers, watchers, test runners) must - // die with it, otherwise it is reparented to PID 1 and leaks forever. + this.rejectPending(`Session terminated: ${reason}`); killProcessTree(this.process, this.process.pid ?? 0, this.isolatedGroup); } @@ -238,6 +291,8 @@ class ManagedSession { export class ShellSessionManager { private sessions = new Map(); private agentSessions = new Map>(); + /** Last known cwd per session id — survives session recreation (e.g. after a timeout). */ + private sessionCwd = new Map(); /** * Get or create the default session for an agent. @@ -252,7 +307,11 @@ export class ShellSessionManager { this.removeSession(defaultId); } - return this.createSession(defaultId, agentId, cwd); + return this.createSession( + defaultId, + agentId, + cwd ?? this.sessionCwd.get(defaultId), + ); } /** @@ -302,15 +361,23 @@ export class ShellSessionManager { ); let session: ManagedSession | undefined; + let sessionId: string; if (options?.sessionId) { - session = this.get(options.sessionId); + sessionId = options.sessionId; + session = this.get(sessionId); if (!session) { - return { stdout: `[Session ${options.sessionId} not found]`, exitCode: -1 }; + return { stdout: `[Session ${sessionId} not found]`, exitCode: -1 }; } } else { + sessionId = `${agentId}:default`; session = this.getOrCreateDefault(agentId, options?.cwd); } + // Remember the working directory so a session that has to be recreated + // (timeout termination, unexpected exit) resumes in the same place. + const effectiveCwd = options?.cwd ?? this.sessionCwd.get(sessionId); + if (options?.cwd) this.sessionCwd.set(sessionId, options.cwd); + // If cwd is specified and differs, cd to it first if (options?.cwd) { const cdResult = await session.execute( @@ -322,7 +389,18 @@ export class ShellSessionManager { } } - return session.execute(command, timeoutMs, options?.onOutput); + const result = await session.execute(command, timeoutMs, options?.onOutput); + + // A timed-out command terminates its session (see ManagedSession.execute): + // the shell is gone, so drop it from the registry. The next call lazily + // starts a clean session in the remembered cwd — the agent never inherits a + // wedged shell that would make every subsequent command time out too. + if (!session.alive) { + if (effectiveCwd) this.sessionCwd.set(sessionId, effectiveCwd); + this.removeSession(sessionId); + } + + return result; } listForAgent(agentId: string): ShellSession[] { @@ -364,7 +442,7 @@ export class ShellSessionManager { const isWin = process.platform === 'win32'; const shell = isWin ? (process.env['COMSPEC'] || 'cmd.exe') - : (process.env['SHELL'] || '/bin/sh'); + : resolvePosixShell(); const isBashLike = !isWin && /\b(bash|zsh)\b/.test(shell); const args = isWin ? ['/Q'] : (isBashLike ? ['--norc', '--noprofile', '-i'] : []); const child = spawn(shell, args, { @@ -379,11 +457,45 @@ export class ShellSessionManager { detached: !isWin, }); + // Two interactive-mode behaviours must be neutralised at session start. + // + // 1. Job control (`set +m`). `bash -i` turns on monitor mode (`set -m`), + // which puts every command — including `cmd &` — in its OWN process + // group. The session group then no longer contains the real work, so + // `kill(-pgid)` reaps only the shell and the actual command keeps running + // (observed: `sleep` grandchildren alive minutes after a timeout fired). + // `set +m` keeps every descendant inside the session's group. + // + // 2. Input echo (`set +o emacs`). GNU bash 5 — the version on Linux/CI — + // ECHOES every command it reads back to **stderr** when interactive with + // a non-tty stdin (bash 3.2 on macOS does not). ManagedSession merges + // stderr into the sentinel-parsing buffer (see the stderr handler in the + // constructor), so the echoed command leaked into the result `stdout` — + // e.g. `echo $X` came back as "echo $X\n\n" instead of "". + // Disabling emacs readline mode (neither emacs nor vi active) makes bash + // read plainly and stop echoing. Reproduced 2026-10-02 on bash 5.2; + // absent on bash 3.2. + // + // Both are written before any user command; their own echo and bash's + // startup banners arrive before the first `pending` command exists, so + // `onData` (which returns early when nothing is pending) drops them. + try { + child.stdin?.write('set +m 2>/dev/null; set +o emacs 2>/dev/null\n'); + } catch { + /* best effort: worst case we fall back to the per-process kill */ + } + const session = new ManagedSession( sessionId, agentId, child, - isIsolatedProcessGroup(child.pid), + // Trust the spawn contract (`detached: !isWin` ⇒ POSIX setsid ⇒ pgid === pid). + // Do NOT re-derive this with a `ps` probe: that reports the process's CURRENT + // state, not the spawn contract. Measured failure (2026-10-01): 0/25 wrong + // while the wrapper is alive, but 100% wrong once the wrapper has exited + // (self-daemonizing commands) → the kill path degrades to "kill the wrapper + // only" and every descendant leaks. See process-group.ts for the full note. + !isWin, ); this.sessions.set(sessionId, session); diff --git a/packages/core/src/write-guard.ts b/packages/core/src/write-guard.ts index 7864a2f6f..ee1063259 100644 --- a/packages/core/src/write-guard.ts +++ b/packages/core/src/write-guard.ts @@ -17,8 +17,9 @@ * - 只在目标命中 denyWritePaths / 敏感文件时才拒绝 —— 因此即使解析误判,也不会误伤正常写入。 * - 边界感知匹配(`p === d || p.startsWith(d + '/')`),避免 `/agents/agt_a` 误伤 `/agents/agt_ab`。 */ -import { resolve, sep } from 'node:path'; +import { basename, resolve, sep } from 'node:path'; import type { PathAccessPolicy } from '@markus/shared'; +import { isSingleWriterMemoryFile } from './lock-resources.js'; export interface WriteAllowed { allowed: true; @@ -88,6 +89,21 @@ export function assertWriteAllowed( } } + // 单一写入者记忆文件(审计 P-01):knowledge.md / NOTEBOOK.md 由记忆服务独占; + // 直写会与内存模型分叉并被静默覆盖 —— 拒绝并引导改用记忆 / 笔记本工具。 + if (isSingleWriterMemoryFile(resolved, opts.workspacePath)) { + return { + allowed: false, + matchedPath: resolved, + reason: + `Write denied: "${basename(resolved)}" is managed memory with a single writer (the memory service). ` + + 'A direct file write would be silently overwritten by the next in-memory flush. ' + + 'Use the memory tools instead: memory_save / memory_update / memory_organize for knowledge.md, ' + + 'notebook_upsert / notebook_clear for NOTEBOOK.md. ' + + '(An unrelated file with the same name is fine — keep it inside your workspace.)', + }; + } + return { allowed: true }; } @@ -236,7 +252,7 @@ export function assertShellWriteAllowed( const base = opts.cwd ?? opts.workspacePath ?? process.cwd(); for (const target of targets) { const resolved = resolve(base, target); - const decision = assertWriteAllowed(resolved, { policy: opts.policy }); + const decision = assertWriteAllowed(resolved, { policy: opts.policy, workspacePath: opts.workspacePath }); if (!decision.allowed) { return { allowed: false, diff --git a/packages/core/test/agent-coverage.test.ts b/packages/core/test/agent-coverage.test.ts index bfaae31a9..8db142d77 100644 --- a/packages/core/test/agent-coverage.test.ts +++ b/packages/core/test/agent-coverage.test.ts @@ -892,32 +892,23 @@ describe('performDeliberation and cognitive pipeline', () => { expect(result!.reasoning).toContain('Focus'); }); - it('runs cognitive appraisal when cognitive config is enabled', async () => { + it('makes NO CPP LLM calls — the multi-stage pipeline is removed (审计 §8.1)', async () => { const router = makeMockRouter({ - chatFn: async (req: unknown) => { - const meta = (req as { metadata?: { purpose?: string } }).metadata; - if (meta?.purpose === 'cognitive_appraisal') { - return { - content: JSON.stringify({ - intent: 'User wants deployment help', - relevance: 'Matches current sprint work', - confidence: 'high', - retrievalPlan: { memoryQueries: ['deploy'], activityQueries: [], taskQueries: [] }, - reflectionNeeded: false, - cognitiveContext: 'User needs help deploying the service.', - }), - finishReason: 'end_turn', - usage: { inputTokens: 30, outputTokens: 15 }, - }; - } - return makeResponse('Deployment guidance provided.', 'end_turn'); - }, + chatFn: async () => makeResponse('Deployment guidance provided.', 'end_turn'), }); const agent = createAgent(router, { cognitive: { enabled: true } }); + agent.updateWorkingMemory('current-task', 'deploy the latest build'); await agent.handleMessage('Help me deploy the latest build'); + // 主调用照常发生… + expect(router.chat).toHaveBeenCalled(); + // …但不再有任何 CPP 的 LLM 多阶段调用(appraisal / reflection)—— + // 旧的 LLM 准备管道已移除,改为确定性情境装配(agent.prepareCognitiveContext)。 expect(router.chat.mock.calls.some( c => (c[0] as { metadata?: { purpose?: string } }).metadata?.purpose === 'cognitive_appraisal', - )).toBe(true); + )).toBe(false); + expect(router.chat.mock.calls.some( + c => (c[0] as { metadata?: { purpose?: string } }).metadata?.purpose === 'cognitive_reflection', + )).toBe(false); }); it('includes working memory in dynamic context during chat', async () => { diff --git a/packages/core/test/agent-status-machine.test.ts b/packages/core/test/agent-status-machine.test.ts new file mode 100644 index 000000000..f1e3cfd43 --- /dev/null +++ b/packages/core/test/agent-status-machine.test.ts @@ -0,0 +1,250 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import { Agent } from '../src/agent.js'; +import type { LLMRouter } from '../src/llm/router.js'; +import type { RoleTemplate } from '@markus/shared'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +/** + * 重构 3 回归测试:core 状态机收敛为单一派生函数 transitionStatus。 + * + * 覆盖需求原文: + * - 状态所有写入点收敛到单一派生/转换函数(27 处 setStatus → 意图化调用,残留为 0) + * - 心跳巡检、保守仲裁(reconcileToIdle/reset)、Normal 转换(working/idle 恢复)统一走派生函数 + * - 竞态覆盖场景(严格并行 error/idle、working 中 idle 请求)在单一函数下不会互相覆盖 + */ + +let tempDir: string; + +const MOCK_ROLE: RoleTemplate = { + id: 'test-role', + name: 'Test Role', + description: 'Test role for status machine tests', + category: 'engineering', + systemPrompt: 'You are a test agent.', + defaultSkills: [], + heartbeatChecklist: '', + defaultPolicies: [], + builtIn: false, +}; + +function makeMockRouter(chatFn?: (...args: unknown[]) => Promise): LLMRouter { + const chat = vi.fn(chatFn ?? (async () => ({ + content: 'Hello from the agent.', + finishReason: 'end_turn', + usage: { inputTokens: 50, outputTokens: 25 }, + }))); + + return { + chat, + chatStream: vi.fn(), + getActiveModelContextWindow: () => 200000, + getActiveModelName: () => 'test-model', + getActiveModelMaxOutput: () => 8000, + getModelContextWindow: () => 200000, + getModelMaxOutput: () => 8000, + getModelCost: () => undefined, + isCompactionSupported: () => true, + modelSupportsVision: () => false, + listProviders: () => ['test'], + getProvider: () => undefined, + getDefaultProvider: () => 'test', + defaultProviderName: 'test', + resolveModalityCandidates: vi.fn(() => []), + } as unknown as LLMRouter; +} + +function createAgent(overrides: Record = {}) { + return new Agent({ + config: { + id: 'status-test-agent', + name: 'Status Test Agent', + roleId: 'worker', + llmConfig: { modelMode: 'custom', primary: 'anthropic' }, + createdAt: new Date().toISOString(), + } as never, + role: MOCK_ROLE, + llmRouter: makeMockRouter(), + dataDir: tempDir, + ...overrides, + }); +} + +beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), 'markus-status-machine-')); +}); + +afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); +}); + +/** 测试助手:直接驱动私有 transitionStatus(状态机纯逻辑原子断言)。 */ +function ts(agent: Agent, intent: unknown, opts?: { force?: boolean }) { + return (agent as unknown as { transitionStatus(i: unknown, o?: { force?: boolean }): void }).transitionStatus(intent, opts); +} + +describe('单一状态派生函数 transitionStatus — 生命周期与 Normal 转换', () => { + it('start → reset:无条件回 idle(从离线/错误恢复)', async () => { + const agent = createAgent(); + expect(agent.getState().status).toBe('offline'); // 构造后默认 offline + await agent.start(); + expect(agent.getState().status).toBe('idle'); + }); + + it('stop → offline:无条件落地', async () => { + const agent = createAgent(); + await agent.start(); + await agent.stop('maintenance'); + expect(agent.getState().status).toBe('offline'); + expect(agent.getStopReason()).toBe('maintenance'); + }); + + it('working 意图:从 idle 进入 working,幂等不抖动', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'working' }); + expect(agent.getState().status).toBe('working'); + ts(agent, { to: 'working' }); + expect(agent.getState().status).toBe('working'); + }); + + it('error 意图在无活跃任务时落地并记录 lastError', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'error', message: 'boom' }); + expect(agent.getState().status).toBe('error'); + expect(agent.getState().lastError).toBe('boom'); + expect(agent.getState().lastErrorAt).toBeTruthy(); + }); + + it('working 意图可从 error 恢复并清空 lastError(Normal 转换)', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'error', message: 'boom' }); + ts(agent, { to: 'working' }); + expect(agent.getState().status).toBe('working'); + expect(agent.getState().lastError).toBeUndefined(); + expect(agent.getState().lastErrorAt).toBeUndefined(); + }); +}); + +describe('单一状态派生函数 transitionStatus — 竞态覆盖防护(需求核心)', () => { + it('error 粘性:error 后并行的 idle 意图不得覆盖错误状态', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'working' }); + // 模拟严格并行:任务 A 失败 → error;任务 B 成功回来请求 idle(后到) + ts(agent, { to: 'error', message: 'task A failed' }); + ts(agent, { to: 'idle' }); + // 错误可见性优先:不能被成功的并行路径冲掉 + expect(agent.getState().status).toBe('error'); + expect(agent.getState().lastError).toBe('task A failed'); + }); + + it('idle 意图在 error 粘性下只能通过 reset / 新 working 恢复', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'error', message: 'boom' }); + ts(agent, { to: 'idle' }); + expect(agent.getState().status).toBe('error'); // 粘性拒绝 + // 仲裁兜底:reset 强制恢复(reconcileToIdle 路径) + ts(agent, { to: 'reset' }); + expect(agent.getState().status).toBe('idle'); + expect(agent.getState().lastError).toBeUndefined(); + }); + + it('working 中有并行活动时 idle 意图被拒(聚合状态守卫)', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'working' }); + // 模拟仍有活跃任务(聚合非空闲) + (agent as unknown as { activeTasks: Set }).activeTasks.add('task-1'); + ts(agent, { to: 'idle' }); + expect(agent.getState().status).toBe('working'); // 不落地 idle + // 任务结束后 idle 落地 + (agent as unknown as { activeTasks: Set }).activeTasks.delete('task-1'); + ts(agent, { to: 'idle' }); + expect(agent.getState().status).toBe('idle'); + }); + + it('并发模式:仍有 worker 忙碌时 idle 意图被拒,全部空闲后落地', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'working' }); + // 模拟并发 worker 聚合状态为忙碌 + (agent as unknown as { attentionController: unknown }).attentionController = { + getWorkerCount: () => 2, + getState: () => 'working', + } as never; + ts(agent, { to: 'idle' }); + expect(agent.getState().status).toBe('working'); + // 全部 worker 空闲 → 落地 idle + (agent as unknown as { attentionController: unknown }).attentionController = { + getWorkerCount: () => 2, + getState: () => 'idle', + } as never; + ts(agent, { to: 'idle' }); + expect(agent.getState().status).toBe('idle'); + }); + + it('force idle:仲裁兜底可强制越过聚合守卫与 error 粘性', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'working' }); + (agent as unknown as { activeTasks: Set }).activeTasks.add('stuck'); + ts(agent, { to: 'idle' }, { force: true }); + expect(agent.getState().status).toBe('idle'); + // 清理,避免影响其它断言 + (agent as unknown as { activeTasks: Set }).activeTasks.delete('stuck'); + }); + + it('error 意图在仍有活跃任务时不落地(部分失败不污染全局状态)', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'working' }); + (agent as unknown as { activeTasks: Set }).activeTasks.add('other-task'); + ts(agent, { to: 'error', message: 'partial failure' }); + expect(agent.getState().status).toBe('working'); // 保持 working + expect(agent.getState().lastError).toBeUndefined(); + }); +}); + +describe('状态机收敛 — 心跳/仲裁路径统一入口', () => { + it('reconcileToIdle(保守仲裁兜底)走 reset 派生函数:从 error 回收 idle', async () => { + const agent = createAgent(); + await agent.start(); + ts(agent, { to: 'error', message: 'stale error' }); + agent.reconcileToIdle(); + // 无 stale 活动可清时 cleared=false,但状态回收仍走 reset 派生函数 + expect(agent.getState().status).toBe('idle'); + expect(agent.getState().lastError).toBeUndefined(); + }); + + it('handleMessage 正常完成 → idle(聚合空闲落地)', async () => { + const agent = createAgent(); + await agent.start(); + const reply = await agent.handleMessage('hello'); + expect(reply).toContain('Hello'); + expect(agent.getState().status).toBe('idle'); + }); + + it('handleMessage 抛错 → error(错误可见性,未被覆盖)', async () => { + const failing = createAgent({ llmRouter: makeMockRouter(async () => { + throw new Error('API quota exceeded'); + }) }); + await failing.start(); + await expect(failing.handleMessage('fail please')).rejects.toThrow('API quota exceeded'); + expect(failing.getState().status).toBe('error'); + expect(failing.getState().lastError).toContain('API quota exceeded'); + }); + + it('所有状态写入已收敛:transitionStatus 为唯一派生入口(无遗留 setStatus 调用点)', async () => { + const agent = createAgent(); + const self = agent as unknown as Record; + // 转换函数存在且 applyStatus 是唯一落地载体 + expect(typeof self.transitionStatus).toBe('function'); + ts(agent, { to: 'working' }); + expect(self.state?.status).toBe('working'); + }); +}); \ No newline at end of file diff --git a/packages/core/test/agent-write-lock.test.ts b/packages/core/test/agent-write-lock.test.ts index 0d0e8eb93..e2bf1fdd4 100644 --- a/packages/core/test/agent-write-lock.test.ts +++ b/packages/core/test/agent-write-lock.test.ts @@ -175,7 +175,7 @@ describe('P2-A Agent 工具写互斥', () => { for (const r of ['task_list', 'task_get', 'requirement_list', 'requirement_get', 'memory_search', 'list_projects', 'team_list', 'web_search', 'web_fetch', 'llm_list_providers', 'check_mailbox', 'workflow_list', 'workflow_status', - 'read_terminal', 'list_terminals', 'take_snapshot', 'knowledge_search']) { + 'read_terminal', 'list_terminals', 'take_snapshot', 'kb_search']) { expect(cls.isWriteTool(r), `${r} 应判为读`).toBe(false); } }); diff --git a/packages/core/test/background-exec-descendant-reap.test.ts b/packages/core/test/background-exec-descendant-reap.test.ts new file mode 100644 index 000000000..49b820ba9 --- /dev/null +++ b/packages/core/test/background-exec-descendant-reap.test.ts @@ -0,0 +1,66 @@ +import { describe, it, expect, afterAll } from 'vitest'; +import { execSync } from 'node:child_process'; +import { createBackgroundExecTool, createProcessTool } from '../src/tools/process-manager.js'; + +/** + * Regression for the background-exec descendant leak (found + fixed 2026-10-01). + * + * `isolatedGroup` used to be derived from `isIsolatedProcessGroup(child.pid)`, + * i.e. a `ps` probe of the wrapper's CURRENT state. For a self-daemonizing + * command (`sleep N & exit 0`) the wrapper is gone by the time we ask, so the + * probe returned false → `hasLeakedDescendants()` returned false → `kill` + * answered "Process already exited" and the grandchild leaked forever. + * + * The fix trusts the spawn contract (`detached: !isWin`) instead. These tests + * pin the observable behaviour, not the implementation. + */ + +const MARK = '777812'; +const sleepMarker = `sleep ${MARK}`; +// Self-match-proof pattern: bracket the first digit so the literal string +// `sleep 777812` does NOT appear in the search command's own argv. `execSync` +// runs through `sh -c 'pgrep -f "sleep 777812" || true'`, whose command line +// literally contains `sleep 777812` — so a plain `pgrep -f` matched that +// wrapper and reported a phantom survivor on Linux/CI ("expected 1 to be +0"), +// while macOS happened not to. `sleep [7]77812` still matches the real process +// but not the shell that spells the pattern out. Measured 2026-10-02. +const markerPattern = `sleep [${MARK[0]}]${MARK.slice(1)}`; +const pgrep = `pgrep -f "${markerPattern}" || true`; + +function survivors(): number { + const out = execSync(pgrep, { encoding: 'utf8' }).trim(); + return out ? out.split('\n').length : 0; +} + +describe('background exec: reap descendants after the wrapper exits', () => { + const bgExec = createBackgroundExecTool(); + const proc = createProcessTool(); + + afterAll(() => { + try { + execSync(`pkill -f "${markerPattern}" || true`); + } catch { + /* nothing left to clean up */ + } + }); + + it('kills the surviving grandchild of a self-daemonizing command', async () => { + const start = JSON.parse(await bgExec.execute({ command: `${sleepMarker} & exit 0` })); + expect(start.status).toBe('running'); + expect(start.pid).toBeGreaterThan(0); + + // Let the wrapper exit while the grandchild keeps running — the exact state + // in which the old `ps` probe reported "not isolated". + await new Promise((r) => setTimeout(r, 1500)); + expect(survivors(), 'precondition: the grandchild must actually be alive').toBeGreaterThan(0); + + const killed = JSON.parse( + await proc.execute({ action: 'kill', sessionId: start.sessionId }), + ); + expect(killed.status).toBe('success'); + + // SIGTERM first, SIGKILL after the grace window. + await new Promise((r) => setTimeout(r, 3500)); + expect(survivors(), 'no descendant may survive the kill').toBe(0); + }, 15000); +}); diff --git a/packages/core/test/cache-optimization.test.ts b/packages/core/test/cache-optimization.test.ts index 67a8fb227..ed2d28d8f 100644 --- a/packages/core/test/cache-optimization.test.ts +++ b/packages/core/test/cache-optimization.test.ts @@ -574,8 +574,7 @@ describe('ContextEngine — cache optimization', () => { expect(after.volatile).toContain('## Your Knowledge'); }); - it('routes CPP output to the notebook (Tier 3) rather than a new stable section when a writer is provided', async () => { - const writes: Array<{ key: string; managed: string }> = []; + it('injects CPP output into the volatile tail — single destination, never the notebook (P-03)', async () => { const result = await engine.buildSystemPrompt({ agentId: 'test-agent', agentName: 'TestAgent', @@ -587,10 +586,11 @@ describe('ContextEngine — cache optimization', () => { isEmpty: false, cognitiveContext: 'situational note', }, - notebookWriter: (key, _text, managed) => { writes.push({ key, managed }); }, }); - // CPP was handed to the notebook writer, not injected as a stable "## Cognitive Context". - expect(writes.some(w => w.managed === 'cpp')).toBe(true); + // 审计 P-03:CPP 产出只注入本轮 volatile tail(不再写 NOTEBOOK), + // 且绝不进入可缓存的 stable 前缀。 + expect(result.volatile).toContain('## Cognitive Context'); + expect(result.volatile).toContain('situational note'); const stablePrefix = result.segments .filter(s => s.cacheBreakpoint) .map(s => s.content) diff --git a/packages/core/test/cognitive-deep.test.ts b/packages/core/test/cognitive-deep.test.ts deleted file mode 100644 index e9333ae2a..000000000 --- a/packages/core/test/cognitive-deep.test.ts +++ /dev/null @@ -1,158 +0,0 @@ -import { describe, it, expect, vi } from 'vitest'; -import { - CognitiveDepth, - type CognitiveConfig, - type CognitiveStimulus, - type CognitiveAgentContext, -} from '@markus/shared'; -import { CognitivePreparation, type CognitiveLLM, type RetrievalBackend } from '../src/cognitive.js'; - -const STIMULUS: CognitiveStimulus = { - type: 'task_execution', - summary: 'Implement feature X', - content: 'Build the new authentication flow with OAuth2 support.', - scenario: 'task_execution', -}; - -const AGENT_CTX: CognitiveAgentContext = { - id: 'agent-001', - name: 'DevBot', - roleDescription: 'Backend developer', - status: 'working', - currentTask: 'auth refactor', - recentActivity: ['Reviewed OAuth docs'], -}; - -const APPRAISAL_WITH_REFLECTION = JSON.stringify({ - intent: 'Build OAuth2 auth flow', - relevance: 'Core platform work', - confidence: 'high', - retrievalPlan: { - memoryQueries: ['oauth', 'auth'], - activityQueries: ['login bug'], - taskQueries: ['auth task'], - }, - reflectionNeeded: true, - cognitiveContext: 'OAuth implementation with existing auth context.', -}); - -const REFLECTION_JSON = JSON.stringify({ - interpretation: 'Need to integrate with existing session store', - recommendations: ['Check current JWT middleware', 'Add refresh token rotation'], -}); - -function makeLLM(appraisal: string, reflection?: string): CognitiveLLM { - let calls = 0; - return { - chat: vi.fn(async (req) => { - calls++; - const purpose = req.metadata?.purpose; - if (purpose === 'cognitive_reflection') { - return { content: reflection ?? REFLECTION_JSON }; - } - return { content: appraisal }; - }), - }; -} - -function makeBackend(overrides?: Partial): RetrievalBackend { - return { - searchMemories: vi.fn((q: string) => [ - { content: `Memory hit for ${q}`, score: 0.9 }, - { content: `Memory hit for ${q}`, score: 0.8 }, - ]), - searchActivities: vi.fn((_, q: string) => [ - { type: 'task', summary: `Activity for ${q}` }, - ]), - searchTasks: vi.fn((q: string) => [ - { id: 'task_1', title: `Task ${q}`, status: 'in_progress' }, - ]), - ...overrides, - }; -} - -describe('CognitivePreparation D2+ with retrieval', () => { - it('runs full pipeline with retrieval and reflection at D2', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm = makeLLM(APPRAISAL_WITH_REFLECTION); - const backend = makeBackend(); - - const result = await cpp.prepare( - STIMULUS, - AGENT_CTX, - CognitiveDepth.D2_Deliberative, - llm, - backend, - ); - - expect(result.isEmpty).toBe(false); - expect(result.retrievedContext).toContain('Memory hit'); - expect(result.reflection).toContain('session store'); - expect(llm.chat).toHaveBeenCalledTimes(2); - expect(backend.searchMemories).toHaveBeenCalled(); - }); - - it('deduplicates memory retrieval results', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm = makeLLM(JSON.stringify({ - intent: 'test', - relevance: 'test', - confidence: 'low', - retrievalPlan: { memoryQueries: ['dup'], activityQueries: [], taskQueries: [] }, - reflectionNeeded: false, - cognitiveContext: 'ctx', - })); - const backend = makeBackend(); - - await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D2_Deliberative, llm, backend); - expect(backend.searchMemories).toHaveBeenCalledWith('dup', 5); - }); - - it('continues when retrieval backends throw', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm = makeLLM(APPRAISAL_WITH_REFLECTION); - const backend = makeBackend({ - searchMemories: vi.fn(() => { throw new Error('mem fail'); }), - searchActivities: vi.fn(() => { throw new Error('act fail'); }), - searchTasks: vi.fn(() => { throw new Error('task fail'); }), - }); - - const result = await cpp.prepare( - STIMULUS, - AGENT_CTX, - CognitiveDepth.D2_Deliberative, - llm, - backend, - ); - - expect(result.isEmpty).toBe(false); - expect(result.cognitiveContext).toBeDefined(); - }); - - it('uses reflection fallback when reflection LLM fails', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm: CognitiveLLM = { - chat: vi.fn(async (req) => { - if (req.metadata?.purpose === 'cognitive_reflection') { - throw new Error('reflection failed'); - } - return { content: APPRAISAL_WITH_REFLECTION }; - }), - }; - const backend = makeBackend(); - - const result = await cpp.prepare( - STIMULUS, - AGENT_CTX, - CognitiveDepth.D3_MetaCognitive, - llm, - backend, - ); - - expect(result.reflection).toContain('Unable to reflect'); - }); -}); diff --git a/packages/core/test/cognitive-enhancement.test.ts b/packages/core/test/cognitive-enhancement.test.ts index 21c6c7e93..c2a7870fa 100644 --- a/packages/core/test/cognitive-enhancement.test.ts +++ b/packages/core/test/cognitive-enhancement.test.ts @@ -22,13 +22,12 @@ describe('Notebook persistence — parseNotebook / serializeNotebook', () => { const entries = new Map([ ['current-task', { text: 'Working on feature X', managed: 'agent', updatedAt: 1000 }], ['triage-decision', { text: 'Processing user message', managed: 'system', updatedAt: 2000 }], - ['cognitive-context', { text: 'This is about refactoring', managed: 'cpp', updatedAt: 3000 }], + ['cognitive-context', { text: 'This is about refactoring', managed: 'system', updatedAt: 3000 }], ]); const md = serializeNotebook(entries); expect(md).toContain('## current-task'); expect(md).toContain(''); expect(md).toContain(''); - expect(md).toContain(''); expect(md).toContain('Working on feature X'); const parsed = parseNotebook(md); @@ -36,7 +35,7 @@ describe('Notebook persistence — parseNotebook / serializeNotebook', () => { expect(parsed.get('current-task')?.managed).toBe('agent'); expect(parsed.get('current-task')?.text).toContain('Working on feature X'); expect(parsed.get('triage-decision')?.managed).toBe('system'); - expect(parsed.get('cognitive-context')?.managed).toBe('cpp'); + expect(parsed.get('cognitive-context')?.managed).toBe('system'); }); it('defaults to agent managed when no tag present', () => { diff --git a/packages/core/test/cognitive.test.ts b/packages/core/test/cognitive.test.ts deleted file mode 100644 index 5321a34aa..000000000 --- a/packages/core/test/cognitive.test.ts +++ /dev/null @@ -1,371 +0,0 @@ -import { describe, it, expect, vi } from 'vitest'; -import { - CognitiveDepth, - type CognitiveConfig, - type CognitiveStimulus, - type CognitiveAgentContext, - type PreparedCognitiveContext, -} from '@markus/shared'; -import { CognitivePreparation, selectCognitiveDepth } from '../src/cognitive.js'; -import type { CognitiveLLM } from '../src/cognitive.js'; -import { ContextEngine } from '../src/context-engine.js'; -import { MemoryStore } from '../src/memory/store.js'; -import { mkdtempSync, rmSync } from 'node:fs'; -import { join } from 'node:path'; -import { tmpdir } from 'node:os'; -import type { RoleTemplate } from '@markus/shared'; - -// ─── Fixtures ──────────────────────────────────────────────────────────────── - -const STIMULUS: CognitiveStimulus = { - type: 'chat', - summary: 'User asks about auth module', - content: 'How does the auth module handle token refresh?', - sender: 'user-1', - scenario: 'chat', -}; - -const AGENT_CTX: CognitiveAgentContext = { - id: 'agent-001', - name: 'DevBot', - roleDescription: 'A backend developer specializing in Node.js and TypeScript.', - status: 'idle', - currentTask: undefined, - recentActivity: ['Fixed login bug', 'Reviewed PR #42'], -}; - -function makeMockLLM(responseContent: string): CognitiveLLM { - return { - chat: vi.fn().mockResolvedValue({ content: responseContent }), - }; -} - -const VALID_APPRAISAL_JSON = JSON.stringify({ - intent: 'User wants to understand token refresh logic', - relevance: 'Directly related to auth module I maintain', - confidence: 'high', - retrievalPlan: { - memoryQueries: ['auth token refresh'], - activityQueries: ['auth module work'], - taskQueries: [], - }, - reflectionNeeded: false, - cognitiveContext: 'User is asking about auth token refresh. I have recent experience with the auth module.', -}); - -// ─── selectCognitiveDepth ──────────────────────────────────────────────────── - -describe('selectCognitiveDepth', () => { - it('returns D0 for heartbeat', () => { - expect(selectCognitiveDepth('heartbeat', {}, 10)).toBe(CognitiveDepth.D0_Reflexive); - }); - - it('returns D0 for memory_consolidation', () => { - expect(selectCognitiveDepth('memory_consolidation', {}, 100)).toBe(CognitiveDepth.D0_Reflexive); - }); - - it('returns D1 for chat', () => { - expect(selectCognitiveDepth('chat', {}, 100)).toBe(CognitiveDepth.D1_Reactive); - }); - - it('returns D1 for a2a', () => { - expect(selectCognitiveDepth('a2a', {}, 100)).toBe(CognitiveDepth.D1_Reactive); - }); - - it('returns D1 for comment_response', () => { - expect(selectCognitiveDepth('comment_response', {}, 100)).toBe(CognitiveDepth.D1_Reactive); - }); - - it('returns D1 for review', () => { - expect(selectCognitiveDepth('review', {}, 100)).toBe(CognitiveDepth.D1_Reactive); - }); - - it('returns D2 for task_execution', () => { - expect(selectCognitiveDepth('task_execution', {}, 100)).toBe(CognitiveDepth.D2_Deliberative); - }); - - it('defaults to D1 for unknown scenarios', () => { - expect(selectCognitiveDepth('unknown_scenario', {}, 100)).toBe(CognitiveDepth.D1_Reactive); - }); - - // Upgrade heuristics - it('upgrades D0 to D1 when agent has failed tasks', () => { - expect(selectCognitiveDepth('heartbeat', { hasFailedTasks: true }, 10)).toBe(CognitiveDepth.D1_Reactive); - }); - - it('upgrades D1 to D2 when stimulus is long (>500 chars)', () => { - expect(selectCognitiveDepth('chat', {}, 600)).toBe(CognitiveDepth.D2_Deliberative); - }); - - it('does NOT upgrade D1 to D2 when stimulus is short', () => { - expect(selectCognitiveDepth('chat', {}, 400)).toBe(CognitiveDepth.D1_Reactive); - }); - - it('upgrades D2 to D3 when agent has blockers', () => { - expect(selectCognitiveDepth('task_execution', { hasBlockers: true }, 100)).toBe(CognitiveDepth.D3_MetaCognitive); - }); -}); - -// ─── CognitivePreparation ──────────────────────────────────────────────────── - -describe('CognitivePreparation', () => { - describe('prepare() at D0', () => { - it('returns empty result immediately', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm = makeMockLLM('{}'); - - const result = await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D0_Reflexive, llm); - - expect(result.isEmpty).toBe(true); - expect(result.depth).toBe(CognitiveDepth.D0_Reflexive); - expect(llm.chat).not.toHaveBeenCalled(); - }); - }); - - describe('prepare() when disabled', () => { - it('returns empty result even at D2', async () => { - const config: CognitiveConfig = { enabled: false }; - const cpp = new CognitivePreparation(config); - const llm = makeMockLLM('{}'); - - const result = await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D2_Deliberative, llm); - - expect(result.isEmpty).toBe(true); - expect(llm.chat).not.toHaveBeenCalled(); - }); - }); - - describe('prepare() at D1', () => { - it('calls appraisal LLM and returns cognitiveContext', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm = makeMockLLM(VALID_APPRAISAL_JSON); - - const result = await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D1_Reactive, llm); - - expect(result.isEmpty).toBe(false); - expect(result.depth).toBe(CognitiveDepth.D1_Reactive); - expect(result.cognitiveContext).toContain('auth token refresh'); - expect(llm.chat).toHaveBeenCalledTimes(1); - // Verify appraisal prompt includes agent identity - const callArgs = (llm.chat as ReturnType).mock.calls[0][0]; - expect(callArgs.messages[0].content).toContain('DevBot'); - expect(callArgs.metadata?.purpose).toBe('cognitive_appraisal'); - }); - - it('does NOT call retrieval or reflection', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm = makeMockLLM(VALID_APPRAISAL_JSON); - - const result = await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D1_Reactive, llm); - - expect(result.retrievedContext).toBeUndefined(); - expect(result.reflection).toBeUndefined(); - expect(llm.chat).toHaveBeenCalledTimes(1); - }); - }); - - describe('maxDepth clamping', () => { - it('clamps D2 to D1 when maxDepth is D1', async () => { - const config: CognitiveConfig = { enabled: true, maxDepth: CognitiveDepth.D1_Reactive }; - const cpp = new CognitivePreparation(config); - const llm = makeMockLLM(VALID_APPRAISAL_JSON); - - const result = await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D2_Deliberative, llm); - - expect(result.depth).toBe(CognitiveDepth.D1_Reactive); - expect(result.isEmpty).toBe(false); - // Only 1 LLM call (appraisal), no retrieval or reflection - expect(llm.chat).toHaveBeenCalledTimes(1); - }); - - it('clamps D3 to D0 when maxDepth is D0', async () => { - const config: CognitiveConfig = { enabled: true, maxDepth: CognitiveDepth.D0_Reflexive }; - const cpp = new CognitivePreparation(config); - const llm = makeMockLLM('{}'); - - const result = await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D3_MetaCognitive, llm); - - expect(result.isEmpty).toBe(true); - expect(result.depth).toBe(CognitiveDepth.D0_Reflexive); - expect(llm.chat).not.toHaveBeenCalled(); - }); - }); - - describe('appraisal fallback on LLM error', () => { - it('returns a fallback cognitiveContext when LLM throws', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm: CognitiveLLM = { - chat: vi.fn().mockRejectedValue(new Error('LLM timeout')), - }; - - const result = await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D1_Reactive, llm); - - expect(result.isEmpty).toBe(false); - expect(result.cognitiveContext).toContain('chat'); - expect(result.cognitiveContext).toContain('User asks about auth module'); - }); - - it('returns fallback when LLM returns invalid JSON', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm = makeMockLLM('not valid json at all'); - - const result = await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D1_Reactive, llm); - - expect(result.isEmpty).toBe(false); - expect(result.cognitiveContext).toBeDefined(); - }); - }); - - describe('D2+ without RetrievalBackend', () => { - it('falls back to appraisal-only when retrieval is not provided', async () => { - const config: CognitiveConfig = { enabled: true }; - const cpp = new CognitivePreparation(config); - const llm = makeMockLLM(VALID_APPRAISAL_JSON); - - const result = await cpp.prepare(STIMULUS, AGENT_CTX, CognitiveDepth.D2_Deliberative, llm); - - expect(result.isEmpty).toBe(false); - expect(result.cognitiveContext).toContain('auth token refresh'); - // Only appraisal call, no reflection since no retrieval backend - expect(llm.chat).toHaveBeenCalledTimes(1); - }); - }); -}); - -// ─── ContextEngine CPP sections ────────────────────────────────────────────── - -describe('ContextEngine CPP section rendering', () => { - const MOCK_ROLE: RoleTemplate = { - id: 'test-role', - name: 'Test Role', - description: 'Test role', - category: 'engineering', - systemPrompt: 'You are a test agent.', - defaultSkills: [], - heartbeatChecklist: '', - defaultPolicies: [], - builtIn: false, - }; - - let tempDir: string; - - it('renders ## Cognitive Context when cognitiveContext is provided', async () => { - tempDir = mkdtempSync(join(tmpdir(), 'markus-cpp-test-')); - try { - const engine = new ContextEngine(); - const memory = new MemoryStore(tempDir); - - const cppResult: PreparedCognitiveContext = { - depth: CognitiveDepth.D1_Reactive, - cognitiveContext: 'User is asking about auth. I have relevant experience.', - isEmpty: false, - }; - - const { volatile: prompt } = await engine.buildSystemPrompt({ - agentId: 'agent-1', - agentName: 'TestBot', - role: MOCK_ROLE, - memory, - currentQuery: 'How does auth work?', - cognitiveContext: cppResult, - }); - - expect(prompt).toContain('## Cognitive Context'); - expect(prompt).toContain('User is asking about auth. I have relevant experience.'); - expect(prompt).not.toContain('## Relevant Memories'); - } finally { - rmSync(tempDir, { recursive: true, force: true }); - } - }); - - it('falls back to ## Relevant Memories when cognitiveContext is undefined', async () => { - tempDir = mkdtempSync(join(tmpdir(), 'markus-cpp-test-')); - try { - const engine = new ContextEngine(); - const memory = new MemoryStore(tempDir); - memory.addEntry({ id: 'mem-1', type: 'fact', content: 'Auth uses JWT tokens', timestamp: Date.now() }); - - const { volatile: prompt } = await engine.buildSystemPrompt({ - agentId: 'agent-1', - agentName: 'TestBot', - role: MOCK_ROLE, - memory, - currentQuery: 'How does auth work?', - }); - - expect(prompt).not.toContain('## Cognitive Context'); - expect(prompt).toContain('## Relevant Memories'); - expect(prompt).toContain('Auth uses JWT tokens'); - } finally { - rmSync(tempDir, { recursive: true, force: true }); - } - }); - - it('falls back to ## Relevant Memories when cognitiveContext.isEmpty is true', async () => { - tempDir = mkdtempSync(join(tmpdir(), 'markus-cpp-test-')); - try { - const engine = new ContextEngine(); - const memory = new MemoryStore(tempDir); - memory.addEntry({ id: 'mem-1', type: 'fact', content: 'DB uses PostgreSQL', timestamp: Date.now() }); - - const cppResult: PreparedCognitiveContext = { - depth: CognitiveDepth.D0_Reflexive, - isEmpty: true, - }; - - const { volatile: prompt } = await engine.buildSystemPrompt({ - agentId: 'agent-1', - agentName: 'TestBot', - role: MOCK_ROLE, - memory, - currentQuery: 'What database do we use?', - cognitiveContext: cppResult, - }); - - expect(prompt).not.toContain('## Cognitive Context'); - expect(prompt).toContain('## Relevant Memories'); - } finally { - rmSync(tempDir, { recursive: true, force: true }); - } - }); - - it('renders all three CPP sections when provided', async () => { - tempDir = mkdtempSync(join(tmpdir(), 'markus-cpp-test-')); - try { - const engine = new ContextEngine(); - const memory = new MemoryStore(tempDir); - - const cppResult: PreparedCognitiveContext = { - depth: CognitiveDepth.D2_Deliberative, - cognitiveContext: 'Appraisal: complex auth question.', - retrievedContext: 'Retrieved: prior auth work from last week.', - reflection: 'Reflection: be careful about token expiry edge cases.', - isEmpty: false, - }; - - const { volatile: prompt } = await engine.buildSystemPrompt({ - agentId: 'agent-1', - agentName: 'TestBot', - role: MOCK_ROLE, - memory, - currentQuery: 'Refactor auth module', - cognitiveContext: cppResult, - }); - - expect(prompt).toContain('## Cognitive Context'); - expect(prompt).toContain('Appraisal: complex auth question.'); - expect(prompt).toContain('## Retrieved Context'); - expect(prompt).toContain('Retrieved: prior auth work from last week.'); - expect(prompt).toContain('## Reflection'); - expect(prompt).toContain('Reflection: be careful about token expiry edge cases.'); - expect(prompt).not.toContain('## Relevant Memories'); - } finally { - rmSync(tempDir, { recursive: true, force: true }); - } - }); -}); diff --git a/packages/core/test/context-concurrency-notebook.test.ts b/packages/core/test/context-concurrency-notebook.test.ts new file mode 100644 index 000000000..79e9bd37c --- /dev/null +++ b/packages/core/test/context-concurrency-notebook.test.ts @@ -0,0 +1,134 @@ +/** + * P1-2 回归:并发上下文注入「分身共享工作记忆」快照。 + * + * 方案(agent-self-management-redesign.md §三 P1-2):notebook(NOTEBOOK.md/工作记忆) + * 是 agent 级共享的——每个分身写的是同一份——但上下文默认不注入,分身之间互不知晓 + * (「我不知道另一个我在干嘛」)。并发模式下把最近更新条目注入 Concurrency Context 段。 + */ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; +import { ContextEngine } from '../src/context-engine.js'; +import { MemoryStore } from '../src/memory/store.js'; +import type { RoleTemplate } from '@markus/shared'; + +let tempDir: string; + +const MOCK_ROLE: RoleTemplate = { + id: 'ctx-role', + name: 'Context Test Role', + description: 'Role for context engine tests', + category: 'engineering', + systemPrompt: 'You are a helpful engineering assistant.', + defaultSkills: [], + heartbeatChecklist: '- Check inbox', + defaultPolicies: [], + builtIn: false, +}; + +beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), 'markus-ctx-cc-')); +}); + +afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); +}); + +describe('P1-2 · 并发上下文注入分身共享工作记忆', () => { + it('workerCount>1 且有 notebook 快照 → 注入「分身共享工作记忆」段(限量、截断)', async () => { + const engine = new ContextEngine({ memorySearchTopK: 3 }); + const memory = new MemoryStore(tempDir); + const now = Date.now(); + + const result = await engine.buildSystemPrompt({ + agentId: 'agt_cc', + agentName: 'CC Agent', + role: MOCK_ROLE, + memory, + currentQuery: 'hi', + concurrentContext: { + enabled: true, + workerId: 1, + workerCount: 3, + handoffs: [ + { + workerId: 2, + kind: 'done', + entityKey: 'task:tsk_1', + summary: '处理完成:xxx', + }, + ], + notebook: [ + { + key: 'poly-brief', + text: 'Polymarket 每日投资报告的口径与输出格式,09:00 执行。这是一段比较长的说明文字用于验证截断逻辑是否真的会生效并且不会把整个段落都灌进上下文里去。', + updatedAt: now, + managed: 'agent', + }, + { + key: 'old-note', + text: '过时条目,应被 limit 排后或不显示', + updatedAt: now - 1000 * 60 * 60 * 24, // 1 day ago + managed: 'agent', + }, + ], + }, + }); + + // 并发段在 volatile 尾部(不污染 stable/semiStable 前缀缓存)—— + // buildSystemPrompt 的 result.text 只含 stable+semiStable,volatile 单独在 result.volatile。 + expect(result.text).not.toContain('Concurrency Context'); + const v = result.volatile ?? ''; + expect(v).toContain('## Concurrency Context(并发上下文)'); + expect(v).toContain('分身共享工作记忆'); + expect(v).toContain('poly-brief'); + // 截断:text 长度不超过 ~140 字符 + const line = v.split('\n').find((l) => l.includes('poly-brief')) ?? ''; + expect(line.length).toBeLessThan(220); + }); + + it('workerCount<=1 → 不注入并发段(串行模式与旧行为一致)', async () => { + const engine = new ContextEngine({ memorySearchTopK: 3 }); + const memory = new MemoryStore(tempDir); + const result = await engine.buildSystemPrompt({ + agentId: 'agt_cc', + agentName: 'CC Agent', + role: MOCK_ROLE, + memory, + currentQuery: 'hi', + concurrentContext: { + enabled: false, + workerId: 1, + workerCount: 1, + handoffs: [], + notebook: [{ key: 'k', text: 't', updatedAt: Date.now(), managed: 'agent' }], + }, + }); + expect(result.text).not.toContain('Concurrency Context'); + expect(result.text).not.toContain('分身共享工作记忆'); + }); + + it('notebook 为空 → 注入并发段但无快照子段', async () => { + const engine = new ContextEngine({ memorySearchTopK: 3 }); + const memory = new MemoryStore(tempDir); + const result = await engine.buildSystemPrompt({ + agentId: 'agt_cc', + agentName: 'CC Agent', + role: MOCK_ROLE, + memory, + currentQuery: 'hi', + concurrentContext: { + enabled: true, + workerId: 1, + workerCount: 2, + handoffs: [], + notebook: [], + }, + }); + expect(result.text).not.toContain('Concurrency Context'); + const v = result.volatile ?? ''; + expect(v).toContain('## Concurrency Context(并发上下文)'); + expect(v).not.toContain('分身共享工作记忆'); + }); +}); diff --git a/packages/core/test/context-os.test.ts b/packages/core/test/context-os.test.ts index 3efb3446c..a1d26441b 100644 --- a/packages/core/test/context-os.test.ts +++ b/packages/core/test/context-os.test.ts @@ -398,6 +398,25 @@ describe('session tool — Memory-backed repo (ContextOS split-brain regression) const after = JSON.parse(await tool.execute({ operation: 'status', session_id: sessionId })); expect(after.fragmentCount).toBeGreaterThan(0); }); + + // 刘利 P0 回归(2026-09-30):会话被 LRU 逐出内存后 compact 仍须真实生效。 + // 旧行为:checkOwnership(repo → mem.getSession 有磁盘回退)通过,但 + // compactSession 只读内存 Map → 静默空转并报 ok(flushedCount:0)。 + it('compact 对已被 LRU 逐出内存的会话仍真实压缩(不再静默空转)', async () => { + const { tool, store, sessionId } = memWiredTool(); + // 把目标会话挤出 20 槽内存 Map(逐出时同步落盘)。 + for (let i = 0; i < 25; i++) { + const filler = store.createSession('agt-A'); + for (let j = 0; j < 3; j++) store.appendMessage(filler.id, { role: 'user', content: `f${i}-${j}` } as never); + } + + const res = JSON.parse(await tool.execute({ operation: 'compact', session_id: sessionId, keep_last: 5 })); + expect(res.status).toBe('ok'); + expect(res.flushedCount).toBeGreaterThan(0); // was 0 — silent no-op when memory-evicted + // 保留的会话必须真的变短(否则只是报了数但没做事)。 + const stats = JSON.parse(await tool.execute({ operation: 'status', session_id: sessionId })); + expect(stats.messageCount).toBeLessThanOrEqual(5); + }); }); // ============================================================================= diff --git a/packages/core/test/heartbeat-liveness.test.ts b/packages/core/test/heartbeat-liveness.test.ts new file mode 100644 index 000000000..aedbbf374 --- /dev/null +++ b/packages/core/test/heartbeat-liveness.test.ts @@ -0,0 +1,272 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import { Agent } from '../src/agent.js'; +import { heartbeatStateFingerprint } from '../src/heartbeat.js'; +import type { LLMRouter } from '../src/llm/router.js'; +import type { RoleTemplate } from '@markus/shared'; +import { COMPLETION_MARKER } from '@markus/shared'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +/** + * 重构 2:Liveness 解耦与心跳路径统一收口 —— 回归测试 + * + * 覆盖(docs/agent-liveness-redesign.md §五.2): + * 1. 巡检状态指纹纯函数:状态无变化 → 指纹不变(skip,不空转 LLM); + * 状态实际变化(新邮件 / 任务增减)→ 指纹变(巡检一次)。 + * 2. 心跳 skip 路径 = 纯时间戳:连续心跳无状态变化 → LLM 只巡检一次, + * 其余 skip 只刷新 lastHeartbeat(存活落库)。 + * 3. 邮件来件触发巡检:队列内容变化 → 下一次心跳巡检(状态实际变化)。 + * 4. 统一收口:Conservator 侧使用的 triggerHeartbeat() 与唯一收口 + * (heartbeat:trigger → mailbox.enqueue)同路 —— 状态不变时重复触发 + * 不产生多余 LLM 巡检(无周期小于 RETRY 下限的重复心跳)。 + */ + +let tempDir: string; + +const MOCK_ROLE: RoleTemplate = { + id: 'hb-liveness-role', + name: 'Heartbeat Liveness Role', + description: 'Regression tests for heartbeat liveness (refactor 2)', + category: 'engineering', + systemPrompt: 'You are a heartbeat liveness test agent.', + defaultSkills: [], + heartbeatChecklist: '- Check inbox', + defaultPolicies: [], + builtIn: false, +}; + +function makeMockRouter(overrides?: { + chatFn?: (...args: unknown[]) => Promise; +}): LLMRouter { + const chat = vi.fn(overrides?.chatFn ?? (async () => ({ + content: 'Heartbeat ok.', + finishReason: 'end_turn', + usage: { inputTokens: 40, outputTokens: 20 }, + }))); + return { + chat, + chatStream: vi.fn(async () => ({ + content: 'Stream ok.', + finishReason: 'end_turn', + usage: { inputTokens: 40, outputTokens: 20 }, + })), + getActiveModelContextWindow: () => 200000, + getActiveModelName: () => 'test-model', + getActiveModelMaxOutput: () => 8000, + getModelContextWindow: () => 200000, + getModelMaxOutput: () => 8000, + getModelCost: () => undefined, + isCompactionSupported: () => true, + modelSupportsVision: () => false, + listProviders: () => ['test'], + getProvider: () => undefined, + getDefaultProvider: () => 'test', + defaultProviderName: 'test', + resolveModalityCandidates: vi.fn(() => []), + } as unknown as LLMRouter; +} + +function createAgent(router: LLMRouter, extra?: Record) { + return new Agent({ + config: { + id: 'hb-liveness-agent', + name: 'Heartbeat Liveness Agent', + roleId: 'worker', + llmConfig: { modelMode: 'custom', primary: 'anthropic' }, + createdAt: new Date().toISOString(), + ...(extra ?? {}), + } as never, + role: MOCK_ROLE, + llmRouter: router, + dataDir: tempDir, + }); +} + +async function processViaMailbox( + agent: Agent, + sourceType: Parameters[0], + payload: Parameters[1], +): Promise { + return new Promise((resolve, reject) => { + agent.enqueueToMailbox(sourceType, payload, { + metadata: { responsePromise: { resolve, reject } }, + }); + }); +} + +beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), 'markus-heartbeat-liveness-')); +}); + +afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); +}); + +describe('heartbeatStateFingerprint(纯函数:状态变化才巡检)', () => { + it('空队列 + 无任务 → 稳定空签名(skip 基态)', () => { + const a = heartbeatStateFingerprint([], []); + const b = heartbeatStateFingerprint([], []); + expect(a).toBe(b); + expect(a).toContain('q:[]'); + }); + + it('同一邮件(同 id)卡在队列 → 指纹不变(stuck 邮件不空转 LLM)', () => { + const queued = [{ sourceType: 'task_status_update', id: 'mbx-1' }]; + const a = heartbeatStateFingerprint(queued, []); + const b = heartbeatStateFingerprint(queued, []); + expect(a).toBe(b); + }); + + it('新邮件(新 id)→ 指纹变(巡检一次)', () => { + const oldFp = heartbeatStateFingerprint([], []); + const newFp = heartbeatStateFingerprint( + [{ sourceType: 'task_status_update', id: 'mbx-2' }], + [], + ); + expect(newFp).not.toBe(oldFp); + }); + + it('heartbeat 自身邮件被排除(不与存活巡检互为因果)', () => { + const withHb = heartbeatStateFingerprint( + [{ sourceType: 'heartbeat', id: 'mbx-3' }], + [], + ); + const without = heartbeatStateFingerprint([], []); + expect(withHb).toBe(without); + }); + + it('活跃任务增减 → 指纹变;任务集不变 → 指纹不变', () => { + const idle = heartbeatStateFingerprint([], []); + const withTask = heartbeatStateFingerprint([], ['t1']); + const sameTask = heartbeatStateFingerprint([], ['t1']); + const moreTasks = heartbeatStateFingerprint([], ['t1', 't2']); + expect(withTask).not.toBe(idle); + expect(sameTask).toBe(withTask); + expect(moreTasks).not.toBe(withTask); + // 顺序无关 + expect(heartbeatStateFingerprint([], ['t2', 't1'])).toBe(moreTasks); + }); +}); + +describe('心跳 skip 路径 = 纯时间戳(不空转 LLM)', () => { + it('无状态变化连续心跳 → 仅首次巡检一次 LLM,后续 skip 只刷新 lastHeartbeat', async () => { + const router = makeMockRouter(); + const agent = createAgent(router); + const states: Array<{ lastHeartbeat?: string }> = []; + agent.setStateChangeCallback((_id, state) => states.push(state as { lastHeartbeat?: string })); + await agent.start(); + + // 首次心跳:指纹初始化('' → q:[]|t:[])→ 巡检一次 LLM + await processViaMailbox(agent, 'heartbeat', { + summary: 'HB-1', + content: '[HEARTBEAT] first patrol', + }); + const callsAfterFirst = router.chat.mock.calls.length; + expect(callsAfterFirst).toBeGreaterThanOrEqual(1); + + // 无状态变化 → 第二次心跳 skip LLM(纯时间戳),但 lastHeartbeat 更新 + await processViaMailbox(agent, 'heartbeat', { + summary: 'HB-2', + content: '[HEARTBEAT] no change — must skip LLM', + }); + expect(router.chat.mock.calls.length).toBe(callsAfterFirst); + // 任意一次 notifyStateChange 应携带新鲜的 lastHeartbeat(存活落库) + const lastHbs = states + .map(s => s.lastHeartbeat) + .filter(Boolean); + expect(lastHbs.length).toBeGreaterThan(0); + expect(agent.getState?.().lastHeartbeat).toBeDefined(); + + await agent.stop(); + }); + + it('stuck-working 宽限场景:状态不变时多次触发心跳 → LLM 巡检不随触发次数增长', async () => { + const router = makeMockRouter(); + const agent = createAgent(router); + await agent.start(); + + // 第一次:建立指纹基线 + 巡检 + await processViaMailbox(agent, 'heartbeat', { + summary: 'HB-A', + content: '[HEARTBEAT] baseline', + }); + const baseline = router.chat.mock.calls.length; + expect(baseline).toBeGreaterThanOrEqual(1); + + // 连续 N 次无变化触发(Conservator 宽限窗口内可能多次 trigger-heartbeat): + // 每一次都走 mailbox 收口 + 指纹 skip → 不产生额外 LLM(无周期 < 5min 的重复巡检) + for (let i = 0; i < 3; i++) { + await processViaMailbox(agent, 'heartbeat', { + summary: `HB-${i}`, + content: `[HEARTBEAT] re-trigger ${i} (stuck, no change)`, + }); + } + expect(router.chat.mock.calls.length).toBe(baseline); + await agent.stop(); + }); +}); + +describe('状态实际变化才巡检 / 邮箱处理与心跳解耦', () => { + it('邮件被独立消费(不依赖心跳)→ 心跳不因邮件往返产生多余巡检', async () => { + const router = makeMockRouter(); + const agent = createAgent(router); + await agent.start(); + + // 基线:无邮件心跳 → 巡检一次后 skip + await processViaMailbox(agent, 'heartbeat', { + summary: 'HB-B1', + content: '[HEARTBEAT] baseline', + }); + const baseline = router.chat.mock.calls.length; + expect(baseline).toBeGreaterThanOrEqual(1); + await processViaMailbox(agent, 'heartbeat', { + summary: 'HB-B2', + content: '[HEARTBEAT] stable', + }); + const stableAfter = router.chat.mock.calls.length; + expect(stableAfter).toBe(baseline); + + // 一封邮件入队(邮箱 worker 独立消费/丢弃——不靠心跳 LLM 处理它), + // 消费后队列回到空 → 指纹不变 → 下一次心跳仍 skip。 + // 这验证「邮箱处理不由心跳承担,心跳也不重复巡检邮件」的解耦收口。 + await processViaMailbox(agent, 'task_status_update', { + summary: 'Task now in_progress', + content: 'task t_new started', + taskId: 't_new', + }); + await processViaMailbox(agent, 'heartbeat', { + summary: 'HB-B3', + content: '[HEARTBEAT] mail consumed independently — expect skip', + }); + expect(router.chat.mock.calls.length).toBe(stableAfter); + await agent.stop(); + }); +}); + +describe('统一收口:Conservator triggerHeartbeat 与定时心跳同一出口', () => { + it('triggerHeartbeat()(Conservator 路径)→ 状态不变时重复触发不产生多余 LLM 巡检', async () => { + const router = makeMockRouter(); + const agent = createAgent(router); + await agent.start(); + + // 通过收口建基线(一次巡检) + await processViaMailbox(agent, 'heartbeat', { + summary: 'HB-C1', + content: '[HEARTBEAT] baseline', + }); + const baseline = router.chat.mock.calls.length; + expect(baseline).toBeGreaterThanOrEqual(1); + + // Conservator 路径:agent.triggerHeartbeat()(agent-manager/api-server 均收敛于此) + // 状态无变化 → 即使反复触发,也只走 mailbox 折叠 + 指纹 skip,不增加 LLM。 + agent.triggerHeartbeat(); + agent.triggerHeartbeat(); + await new Promise(r => setTimeout(r, 120)); + expect(router.chat.mock.calls.length).toBeGreaterThanOrEqual(baseline); + // 宽限窗口内最多多巡一次(首 trigger 若指纹已变),绝不随触发次数线性增长 + expect(router.chat.mock.calls.length).toBeLessThanOrEqual(baseline + 1); + + await agent.stop(); + }); +}); \ No newline at end of file diff --git a/packages/core/test/knowledge-tools-e2e.test.ts b/packages/core/test/knowledge-tools-e2e.test.ts index 4b229b7ce..fcfebd1b0 100644 --- a/packages/core/test/knowledge-tools-e2e.test.ts +++ b/packages/core/test/knowledge-tools-e2e.test.ts @@ -126,17 +126,17 @@ afterEach(() => { }); describe('Knowledge tools E2E (real Agent, tool surface)', () => { - it('registers knowledge_search/list/read on the real Agent tool surface', async () => { + it('registers kb_search/list/read on the real Agent tool surface', async () => { const { agent } = await createE2EAgent(); const tools = agent.getTools(); - expect(tools.has('knowledge_search')).toBe(true); - expect(tools.has('knowledge_list')).toBe(true); - expect(tools.has('knowledge_read')).toBe(true); + expect(tools.has('kb_search')).toBe(true); + expect(tools.has('kb_list')).toBe(true); + expect(tools.has('kb_read')).toBe(true); }); - it('knowledge_search returns KB docs with updatedAt and forces source=knowledge', async () => { + it('kb_search returns KB docs with updatedAt and forces source=knowledge', async () => { const { agent, ds } = await createE2EAgent(); - const raw = await agent.getTools().get('knowledge_search')!.execute({ + const raw = await agent.getTools().get('kb_search')!.execute({ query: 'onboarding', project_id: 'proj_kb', }); @@ -152,9 +152,9 @@ describe('Knowledge tools E2E (real Agent, tool surface)', () => { })); }); - it('knowledge_list returns KB docs for a project', async () => { + it('kb_list returns KB docs for a project', async () => { const { agent, ds } = await createE2EAgent(); - const raw = await agent.getTools().get('knowledge_list')!.execute({ project_id: 'proj_kb' }); + const raw = await agent.getTools().get('kb_list')!.execute({ project_id: 'proj_kb' }); const res = JSON.parse(raw); expect(res.status).toBe('success'); expect(res.source).toBe('knowledge'); @@ -165,10 +165,10 @@ describe('Knowledge tools E2E (real Agent, tool surface)', () => { })); }); - it('knowledge_read reads a doc inside the project knowledge root (prefix guard passes)', async () => { + it('kb_read reads a doc inside the project knowledge root (prefix guard passes)', async () => { const { agent } = await createE2EAgent(); const target = join(kbDir, 'guide.md'); - const raw = await agent.getTools().get('knowledge_read')!.execute({ + const raw = await agent.getTools().get('kb_read')!.execute({ path: target, project_id: 'proj_kb', }); @@ -178,10 +178,10 @@ describe('Knowledge tools E2E (real Agent, tool surface)', () => { expect(res.content).toContain('入门指南'); }); - it('knowledge_read reads a Chinese-filename doc inside the knowledge root', async () => { + it('kb_read reads a Chinese-filename doc inside the knowledge root', async () => { const { agent } = await createE2EAgent(); const target = join(kbDir, 'docs', '中文文档.md'); - const raw = await agent.getTools().get('knowledge_read')!.execute({ + const raw = await agent.getTools().get('kb_read')!.execute({ path: target, project_id: 'proj_kb', }); @@ -190,10 +190,10 @@ describe('Knowledge tools E2E (real Agent, tool surface)', () => { expect(res.content).toContain('中文内容'); }); - it('knowledge_read rejects files outside the bound knowledge roots (prefix guard)', async () => { + it('kb_read rejects files outside the bound knowledge roots (prefix guard)', async () => { const { agent } = await createE2EAgent(); const target = join(outsideDir, 'secret.md'); - const raw = await agent.getTools().get('knowledge_read')!.execute({ + const raw = await agent.getTools().get('kb_read')!.execute({ path: target, project_id: 'proj_kb', }); @@ -201,19 +201,19 @@ describe('Knowledge tools E2E (real Agent, tool surface)', () => { expect(res.status).toBe('error'); }); - it('knowledge_read still works without project_id (no scope enforcement)', async () => { + it('kb_read still works without project_id (no scope enforcement)', async () => { const { agent } = await createE2EAgent(); const target = join(outsideDir, 'secret.md'); - const raw = await agent.getTools().get('knowledge_read')!.execute({ path: target }); + const raw = await agent.getTools().get('kb_read')!.execute({ path: target }); const res = JSON.parse(raw); // No project scoping → bridge does not enforce roots and reads directly. expect(res.status).toBe('success'); expect(res.content).toContain('不该被读的内容'); }); - it('knowledge_read returns a friendly error for a missing document', async () => { + it('kb_read returns a friendly error for a missing document', async () => { const { agent } = await createE2EAgent(); - const raw = await agent.getTools().get('knowledge_read')!.execute({ + const raw = await agent.getTools().get('kb_read')!.execute({ path: join(kbDir, 'missing.md'), project_id: 'proj_kb', }); @@ -222,14 +222,14 @@ describe('Knowledge tools E2E (real Agent, tool surface)', () => { expect(res.error).toMatch(/not readable/i); }); - it('knowledge_search returns success with zero count for empty results', async () => { + it('kb_search returns success with zero count for empty results', async () => { const manager = createManager(); const ds = makeDeliverableService(); ds.search.mockReturnValue({ results: [], total: 0 }); manager.setDeliverableService(ds); manager.setProjectService(makeProjectService() as never); const agent = await manager.createAgent({ name: 'KB Worker 2', roleName: 'custom', orgId: 'org_e2e', tools: [] }); - const raw = await agent.getTools().get('knowledge_search')!.execute({ query: 'nothing' }); + const raw = await agent.getTools().get('kb_search')!.execute({ query: 'nothing' }); const res = JSON.parse(raw); expect(res.status).toBe('success'); expect(res.count).toBe(0); diff --git a/packages/core/test/lock-resource-keys.test.ts b/packages/core/test/lock-resource-keys.test.ts index 28f06e8ab..fa933e419 100644 --- a/packages/core/test/lock-resource-keys.test.ts +++ b/packages/core/test/lock-resource-keys.test.ts @@ -11,8 +11,8 @@ import { * 锁的粒度必须是**资源**,不是工具名。 * * 回归背景(P0-1 / P0-2,见 docs/CONCURRENT-PROCESSING.md §4.9): - * - `update_notebook` 记在域 `notebook`,别名 `update_working_memory` 记在域 - * `working-memory` —— 写**同一个 Map**,却因域名不同被判「不冲突」,锁形同虚设; + * - notebook 写工具曾按「正式名 + 别名」分登记到不同的域(`notebook` / + * `working-memory`)—— 写**同一个 Map**,却因域名不同被判「不冲突」,锁形同虚设; * - `file_write knowledge.md` 走 `fs:`,而 `memory_update` 走 `memory` —— * 同一个文件有两条互不互斥的写入路径,外部编辑会被下一次内存落盘静默覆盖。 * @@ -59,13 +59,11 @@ describe('锁资源键解析(lock-resources)', () => { }); describe('同一资源 ⇒ 同一锁键(P0-1 / P0-2 回归守卫)', () => { - it('notebook 正式名与兼容别名不再分裂成两个域', () => { - const a = lockKey('update_notebook'); - const b = lockKey('update_working_memory'); - const c = lockKey('clear_working_memory'); + it('notebook 工具族共用一个锁键(别名已删除,名字归一)', () => { + const a = lockKey('notebook_upsert'); + const b = lockKey('notebook_clear'); expect(a).toBe(`${AGENT_MEMORY_RESOURCE_DOMAIN}:notebook`); expect(b).toBe(a); - expect(c).toBe(a); }); it('knowledge 工具族共用一个锁键', () => { @@ -82,7 +80,7 @@ describe('同一资源 ⇒ 同一锁键(P0-1 / P0-2 回归守卫)', () => { expect(viaFileWrite).toBe(viaMemoryTool); const viaEdit = lockKey('file_edit', { path: `${root}/NOTEBOOK.md` }, root); - expect(viaEdit).toBe(lockKey('update_notebook')); + expect(viaEdit).toBe(lockKey('notebook_upsert')); // 非记忆文件仍是普通 fs 锁,不被误并入记忆资源 const other = lockKey('file_write', { path: `${root}/src/app.ts` }, root); @@ -90,7 +88,7 @@ describe('同一资源 ⇒ 同一锁键(P0-1 / P0-2 回归守卫)', () => { }); it('不同资源互不冲突(并发度不被无谓牺牲)', () => { - expect(lockKey('update_notebook')).not.toBe(lockKey('memory_update')); + expect(lockKey('notebook_upsert')).not.toBe(lockKey('memory_update')); }); it('写域表不变式:resource 与 domain 至少有一个,fs 域必须声明路径参数', () => { @@ -108,7 +106,7 @@ describe('同一资源 ⇒ 同一锁键(P0-1 / P0-2 回归守卫)', () => { }); describe('别名互斥实际生效(并发)', () => { - it('update_notebook 与 update_working_memory 并发时严格串行', async () => { + it('notebook_upsert 与 notebook_clear 并发时严格串行', async () => { const registry = new ResourceLockRegistry(); let active = 0; let max = 0; @@ -119,7 +117,7 @@ describe('别名互斥实际生效(并发)', () => { await new Promise((r) => setTimeout(r, 30)); active--; }); - await Promise.all([run('update_notebook'), run('update_working_memory')]); + await Promise.all([run('notebook_upsert'), run('notebook_clear')]); expect(max).toBe(1); }); @@ -134,7 +132,7 @@ describe('别名互斥实际生效(并发)', () => { await new Promise((r) => setTimeout(r, 30)); active--; }); - await Promise.all([run('update_notebook'), run('memory_update')]); + await Promise.all([run('notebook_upsert'), run('memory_update')]); expect(max).toBe(2); }); }); diff --git a/packages/core/test/mailbox-core.test.ts b/packages/core/test/mailbox-core.test.ts index 1f2cfe072..b4e008bb0 100644 --- a/packages/core/test/mailbox-core.test.ts +++ b/packages/core/test/mailbox-core.test.ts @@ -239,17 +239,18 @@ describe('AgentMailbox async and persistence', () => { expect(updateStatus).toHaveBeenCalledWith('mbx_stale', 'dropped'); }); - it('cleanStaleProcessing delegates to persistence', () => { - const markCompleted = vi.fn(() => 3); + it('cleanStaleProcessing delegates to persistence (dropped, not completed)', () => { + // 语义:卡住/被打断/超时的工作 ≠ 完成。cleanStaleProcessing 只应把 stuck + // 'processing' 标为 dropped(异常可见 + 允许补偿重投),绝不标 completed。 + const markDropped = vi.fn(() => 3); const { mailbox } = makeMailbox({ save: vi.fn(), updateStatus: vi.fn(), - markStaleProcessingAsCompleted: markCompleted, + markStaleProcessingAsDropped: markDropped, }); expect(mailbox.cleanStaleProcessing()).toBe(3); - // P0:cleanStaleProcessing 传本实例 ownerId(用于排除其它实例有效租约内的在飞项) - expect(markCompleted).toHaveBeenCalledWith(AGENT_ID, expect.any(String)); + expect(markDropped).toHaveBeenCalledWith(AGENT_ID); }); it('emits mailbox:new-item on enqueue', () => { diff --git a/packages/core/test/mailbox-redelivery-sqlite.test.ts b/packages/core/test/mailbox-redelivery-sqlite.test.ts index e3a6d479c..8de6ddae7 100644 --- a/packages/core/test/mailbox-redelivery-sqlite.test.ts +++ b/packages/core/test/mailbox-redelivery-sqlite.test.ts @@ -62,7 +62,6 @@ function makePersistence(repoRef: SqliteMailboxRepo): MailboxPersistence { }, updateStatus: (itemId, status, extra) => repoRef.updateStatus(itemId, status, extra as Record), markStaleProcessingAsDropped: (aid) => repoRef.markStaleProcessingAsDropped(aid), - markStaleProcessingAsCompleted: (aid, ownerId) => repoRef.markStaleProcessingAsCompleted(aid, ownerId), loadQueued: (aid) => repoRef.getByAgent(aid, { status: 'queued' }).map(toItem), loadDeferred: (aid) => repoRef.getByAgent(aid, { status: 'deferred' }).map(toItem), claimItem: (id, ownerId, leaseUntil, nowIso) => repoRef.claimItem(id, ownerId, leaseUntil, nowIso), diff --git a/packages/core/test/mailbox-tools.test.ts b/packages/core/test/mailbox-tools.test.ts index c0ad75743..14a7118ba 100644 --- a/packages/core/test/mailbox-tools.test.ts +++ b/packages/core/test/mailbox-tools.test.ts @@ -40,10 +40,9 @@ describe('createMailboxTools', () => { const tools = createMailboxTools(createContext()); expect(tools.map(t => t.name)).toEqual([ 'check_mailbox', - 'update_notebook', - 'clear_notebook', - 'update_working_memory', - 'clear_working_memory', + 'notebook_upsert', + 'notebook_clear', + 'notebook_read', 'defer_mailbox_item', 'drop_mailbox_item', 'prioritize_mailbox_item', @@ -209,9 +208,9 @@ describe('createMailboxTools', () => { }); }); - describe('working memory tools', () => { - it('update_working_memory validates inputs', async () => { - const tool = createMailboxTools(createContext()).find(t => t.name === 'update_working_memory')!; + describe('notebook tools', () => { + it('notebook_upsert validates inputs', async () => { + const tool = createMailboxTools(createContext()).find(t => t.name === 'notebook_upsert')!; const missing = JSON.parse(await tool.execute({ key: 'k' })); expect(missing.status).toBe('error'); @@ -220,10 +219,10 @@ describe('createMailboxTools', () => { expect(ok.status).toBe('updated'); }); - it('clear_working_memory clears all when all=true', async () => { + it('notebook_clear clears all when all=true', async () => { const clearWorkingMemory = vi.fn(() => ({ status: 'cleared', cleared: 3 })); const ctx = createContext({ clearWorkingMemory }); - const tool = createMailboxTools(ctx).find(t => t.name === 'clear_working_memory')!; + const tool = createMailboxTools(ctx).find(t => t.name === 'notebook_clear')!; const result = JSON.parse(await tool.execute({ all: true })); expect(result.cleared).toBe(3); expect(clearWorkingMemory).toHaveBeenCalledWith(undefined); diff --git a/packages/core/test/memory-health.test.ts b/packages/core/test/memory-health.test.ts new file mode 100644 index 000000000..f6f5ba650 --- /dev/null +++ b/packages/core/test/memory-health.test.ts @@ -0,0 +1,79 @@ +/** + * MemoryStore.getMemoryHealth (审计 P-12) —— 驱动提示词内「记忆健康」信号。 + */ +import { describe, it, expect } from 'vitest'; +import fs from 'node:fs'; +import path from 'node:path'; +import os from 'node:os'; +import { MemoryStore } from '../src/memory/store.js'; +import { MEMORY_MD_TOTAL_MAX_CHARS, MEMORY_OBSERVATIONS_MAX_CHARS } from '@markus/shared'; + +function tmp(): string { + return fs.mkdtempSync(path.join(os.tmpdir(), 'mem-health-')); +} + +describe('MemoryStore.getMemoryHealth (P-12)', () => { + it('报告 cap / 百分比 / 观察数 / 知识段数', () => { + const dir = tmp(); + try { + const store = new MemoryStore(dir); + store.addLongTermMemory('procedures', 'step one'); + store.addEntry({ id: 'o1', timestamp: new Date().toISOString(), type: 'note', content: 'obs' }); + const h = store.getMemoryHealth(); + expect(h.cap).toBe(MEMORY_MD_TOTAL_MAX_CHARS); + expect(h.curatedSections).toBeGreaterThanOrEqual(1); + expect(h.observations).toBe(1); + expect(h.percent).toBeGreaterThan(0); + expect(h.percent).toBeLessThan(100); + expect(h.archiveChars).toBe(0); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); + + it('观察积压导致超预算时 percent ≥ 70(触发提示词告警)', () => { + const dir = tmp(); + try { + const store = new MemoryStore(dir); + // 观察缓冲区不被 compress 归档,是超预算的主要来源之一。 + for (let i = 0; i < 200; i++) { + store.addEntry({ + id: `o${i}`, + timestamp: new Date().toISOString(), + type: 'note', + content: 'x'.repeat(120), + }); + } + const h = store.getMemoryHealth(); + expect(h.percent).toBeGreaterThanOrEqual(70); + expect(h.totalChars).toBeGreaterThan(MEMORY_MD_TOTAL_MAX_CHARS); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); + + it('观察缓冲区超限时,最旧观察被无损归档且仍可检索(P-10)', () => { + const dir = tmp(); + try { + const store = new MemoryStore(dir); + for (let i = 0; i < 400; i++) { + store.addEntry({ + id: `o${i}`, + timestamp: new Date().toISOString(), + type: 'note', + content: `marker-${i} ` + 'y'.repeat(180), + }); + } + const h = store.getMemoryHealth(); + // 缓冲区被裁剪到上限以内(不再无界增长) + expect(h.observations).toBeLessThan(400); + // 内容仍在归档中(无损)——最旧的 marker-0 可被检索到 + const results = store.search('marker-0'); + expect(results.length).toBeGreaterThan(0); + expect(h.archiveChars).toBeGreaterThan(0); + expect(h.totalChars).toBeLessThanOrEqual(MEMORY_OBSERVATIONS_MAX_CHARS + 20_000); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/core/test/memory-migration.test.ts b/packages/core/test/memory-migration.test.ts new file mode 100644 index 000000000..0a17b98da --- /dev/null +++ b/packages/core/test/memory-migration.test.ts @@ -0,0 +1,61 @@ +/** + * Migration-read layer(审计 P-08):**读旧、只写新**。 + * + * 锁定两件事: + * 1) knowledge.md 内的旧 `, tags:` 观察行在加载后被收敛为单 `data-meta` JSON 行 + * (读取端容忍旧格式,写入端只发规范格式); + * 2) 已退场的 `state.md` 被读入为一条观察,源文件改名为 `.migrated`(消费而非销毁)。 + */ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import fs from 'node:fs'; +import path from 'node:path'; +import os from 'node:os'; +import { MemoryStore } from '../src/memory/store.js'; + +function makeTempDir(): string { + return fs.mkdtempSync(path.join(os.tmpdir(), 'mem-mig-')); +} + +describe('MemoryStore — migration-read layer (P-08)', () => { + let tmp: string; + beforeEach(() => { tmp = makeTempDir(); }); + afterEach(() => { fs.rmSync(tmp, { recursive: true, force: true }); }); + + it('收敛旧 `, tags:` 观察行 → 单 `data-meta`(仅写新格式)', () => { + const legacy = [ + '# Knowledge', + '', + '## _observations', + '### obs_1700000000000', + '', + 'legacy observation body', + '', + ].join('\n'); + fs.writeFileSync(path.join(tmp, 'knowledge.md'), legacy, 'utf8'); + + // 构造即触发迁移读取层 → 规范重写。 + const store = new MemoryStore(tmp); + const after = fs.readFileSync(path.join(tmp, 'knowledge.md'), 'utf8'); + + // 落盘后只保留规范格式,不再有裸 `, tags:` 行。 + expect(after).toMatch(/