Skip to content

feat(memory): L3 field cognition, retry failed L3 batches (MEMMY-852), feed scanned sessions into L3 - #739

Open
Anson-Se wants to merge 9 commits into
MemTensor:v1.2.6from
Anson-Se:gh-590-scan-l3
Open

Anson-Se wants to merge 9 commits into
MemTensor:v1.2.6from
Anson-Se:gh-590-scan-l3

Conversation

@Anson-Se

Copy link
Copy Markdown
Contributor

做了什么

这个 PR 包含三件互相依赖的事,按提交顺序:

1. L3 场域认知改造(前 7 个提交)

  • 把 L3 从“每个项目四段整段重写的文字”改成按“本机 / 项目”分开存的短条目,只有两类:
    • 环境事实(env_fact):“条件 -> 现象”,必须有工具输出作证据;
    • 项目规矩(project_constraint):用户希望以后在这个项目里一直遵守的做法。
  • 判断标准是“换一个人在这台电脑或这个项目上工作,这条仍然成立”。
  • 以下内容不记:
    • 个人偏好、单次任务的细节;
    • 工作产出的结论、读代码得到的知识;
    • 通用常识、临时状态;
    • Agent 自身的沙箱和工具限制。
  • 条目支持新增、替换、删除、确认:新规矩会替换旧规矩,问题修好后对应条目会失效。
  • 环境画像由代码扫描生成,模型只读不写。
  • 项目识别:先看用户在桌面端选过的项目,再看 git 仓库,最后看普通文件夹。临时目录、系统和工具目录、依赖包目录不算项目。
  • 有开关 algorithm.l3FieldCognition.enabled(环境变量 MEMMY_L3_FIELD_COGNITION=1),默认关闭。关闭时走原来的 L3,行为不变。
  • 不改 SCHEMA_VERSION(仍为 9),只新增 4 张表,旧数据不转写,旧版 Memmy 仍能打开这个库。

2. MEMMY-852 失败批次补跑(第 8 个提交)

  • 临时失败(空响应、网络错误、超时、限流)分别等 1、5、15、30、60 分钟再重试,第 5 次重试还失败就进死信,后面的批次照常处理。
  • 配置类错误(没配模型、key 错、额度不足)先挂起,不消耗重试次数,每 30 分钟再试一次;没配进化模型时不领取 L3 任务。
  • 新增补跑入口 POST /api/v1/l3/failed-batches/retry,在原任务上复位后重跑。
  • 全部接在现有的 evolution_jobs 队列上(用 runAfter、attempts/max_attempts、failJob 这些现有字段和方法),没有新建队列、定时器或重试表。

3. #590 扫描会话进 L3(第 9 个提交)

  • 扫描新开的会话走新协议,并关闭 Work Memory。
  • 每个来源一轮扫描处理完后,把这一轮碰到的会话交给 L3 处理,不等模型跑完,所以不会拖慢扫描。
  • 托管导入路径(backend ingestion-service)也走同样的处理。
  • 先扫描、后来 Hook 又接管同一会话时,由 Hook 接手。

Closes #590

改动范围

提交 内容
add l3 field cognition tables and flag 新表、开关、仓库层、快照记忆
resolve l3 projects from paths 从路径识别项目
accept user-selected project roots for l3 桌面端把用户选过的项目推给 Memory
build l3 evidence fragments with unit candidates 证据片段及其归属
profile machine and project units for l3 本机画像、项目画像
inject and display l3 field context 会话开头注入 L3、面板显示
update l3 field items from frozen batches 更新流水线、提示词、输入裁剪与上限
retry and resume failed l3 jobs through the job queue MEMMY-852
feed scanned sessions into l3 #590

共 46 个文件,约 3400 行,其中包含测试。

怎么验证的

  • 测试:
    • npm run memory:test 1754 个通过。有 9 个失败,都在已知的“满负载超时”名单里(扫描响应、checkpoint 恢复等),单独运行全部通过。
    • App/backend 992 个通过,失败的 1 个单独运行通过。
    • App/memmy-agent 改动涉及的部分 2810 个通过。
    • typecheck 通过。
  • 离线回放对比:同样 27 个会话(16 个真实会话 + 11 个测试会话),新旧两版都从空 L3 开始,同一个模型,各跑 3 次。存下的条目混在一起交给独立评审盲评,评审不知道条目来自哪个版本。
v1.2.6 本 PR
每次存下的条目 72~94 10~11
其中该存的占比 37% 91%
噪声(一次性、常识、临时状态、Agent 自身限制) 34~58 条 0
模型输入 token 约 445 万 约 45 万
模型调用次数 67~68 44~46
  • 输入 token 少的原因:旧版把整段原始对话原样发给模型,其中工具调用和工具输出占 99%;而且项目契约、领域知识两个字段各发一遍。新版每组只调用一次模型,工具输出只留头尾和报错行,输入上限 65536 字,超出时先丢工具输出。
  • 真机测试:在本机用 Memmy CLI 和 Codex 跑过多个场景,包括规矩替换、问题修好后失效、个人偏好不记、大文件夹里改子项目、扫描会话进 L3,结果都符合预期。
  • Codex 审核 4 轮:前 3 轮提的问题都已修复。第 4 轮提出 1 个阻塞问题(模型上下文较小时会进死信),已修复:输出预算降到 16384,超出上下文时缩小输入再试一次。

已知限制

🤖 Generated with Claude Code

Anson-Se and others added 9 commits October 24, 2026 04:23
Add the storage base for itemized machine/project L3, behind
algorithm.l3FieldCognition (off by default, local SQLite only):

- four new tables (l3_units, l3_items, l3_unit_batches,
  l3_user_projects) created alongside the existing ones without
  bumping SCHEMA_VERSION; old L3 tables and rows are untouched
- L3FieldRepository for units, items, snapshot memories, unit
  deletion, inactive-item purge and the user project list
- register the tables for bundle export/import and clear-all; reject
  imports whose units already exist and drop unfinished unit batches
  from redacted exports
- countL3Units for the per-item length limit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decide which project a path belongs to for the new L3, by reading files
only (no git process):

- the nearer of a user-selected folder and a git repository wins; a
  shared root counts once with the git identity
- git identity is the normalized origin (ssh, scp-like and https forms
  map to host/path), else the common git dir, so a clone's worktrees
  and subdirectories share one project
- a session folder outside any repository is a plain-folder project; a
  tool path only joins the session project it lands in
- temp dirs, Codex dated dirs, Claude scratch workspaces,
  ~/.codex/memories*, Memmy task folders and the home/Desktop/
  Documents/Downloads folders themselves are never projects

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The desktop app now tells Memory which folders the user picked as
projects, so L3 can treat them as project roots:

- POST /api/v1/l3/user-projects replaces the user's whole list and
  the Memmy task-folder roots; relative paths are rejected; the list is
  stored regardless of the L3 field cognition flag
- memmy-agent's long-running gateway reads the project registry every
  minute and pushes only when the list changed or the last push
  failed; a corrupt registry is never pushed
- the contract is added identically to the Memory and App copies, with
  a test that keeps them in sync

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cut a raw turn into evidence fragments for the new L3 so each fact can
be attributed by the paths its own evidence touched:

- one fragment for the user's words, one per tool call with its paired
  result (reusing the existing strict call/result pairing), and one per
  result that matches no call
- paths come from file/dir arguments, cd and git -C targets and absolute
  command arguments; Codex code-mode snippets are read for their
  cmd/workdir literals; relative paths resolve against the call's
  workdir, else the session folder; URLs are ignored
- each fragment gets the project units its paths resolve to, falling
  back to the session folder; long outputs keep head, tail and the
  error lines in between

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- machine profile: probe OS (macOS product version, not the Darwin
  kernel), arch, shell and node/python3/git/npm/brew versions with
  2-second timeouts, and render it in a fixed format by code only
- project profile for the new L3: reuse the existing scan and the
  shared summarizeProjectProfile model call, but feed it only what the
  project declares (runtime versions probed on this machine belong to
  the machine profile); write it into the project unit and refresh its
  snapshot; skip the model when the scan is unchanged and drop the
  result when the unit was deleted meanwhile
- at most one pending profile job per unit; each request carries its
  own scan id because the immutable dedupe index never frees a key
- the old project-profile pipeline is unchanged apart from calling the
  extracted summarizeProjectProfile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Temporary model failures of L3 jobs wait 1, 5, 15, 30 and 60 minutes before
retrying (runAfter); the sixth failure goes to dead letter and later batches
continue. Configuration problems (bad key, quota 40309) wait 30 minutes
without using an attempt, and L3 jobs are not leased at all while no
evolution model is configured. Adds POST /api/v1/l3/failed-batches/retry,
which resets the user's dead-letter l3_unit_update jobs and batches in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conversations first opened by an agent-source scan now use the L3 v2 session
protocol with Work Memory disabled, so their turns register as L3 input like
Hook sessions. When a scan pass finishes a source, the sessions it touched are
frozen through the existing freezeBatches entry (trigger scan_pass; the old
pipeline records it as session_close) and the worker is woken. Hosted imports
(backend importManaged) do the same when an import ends, through
POST /api/v1/l3/scan-sessions/freeze, which only freezes the caller's own
sessions. Freezing only writes rows, so neither scans nor imports wait on L3.
Existing sessions and detached legacy history keep their protocol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@Anson-Se

Copy link
Copy Markdown
Contributor Author

@origami-chen

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants