feat(memory): 存量 worktree 分片迁移工具 — legacy 分片合并/删除 (#2379) - #2519
feat(memory): 存量 worktree 分片迁移工具 — legacy 分片合并/删除 (#2379)#2519KingSystemHaiGo wants to merge 12 commits into
Conversation
…ndy#2379) buildMemoryScopeKey 对本地会话原样透传 workdir 绝对路径, git linked worktree (如 .cindy-worktrees/<name>) 与主仓被拆成两个独立 Store, worktree 会话 memory_search 恒 0 命中 (makecindy#2379 问题一)。 新增 async resolveMemoryScopeKey (memory/scope-resolver.ts) 作为所有 getStore 调用方的统一入口: 本地会话先做 worktree 归一化 —— linked worktree 的 cwd 映射为「主仓根 + cwd 相对 worktree 根的子路径」 (保留按 cwd 隔离语义, 不做整仓合并, 语义收敛见 makecindy#2379 评论); 非 git 目录 / git 不可用 / 探测失败一律回落现有行为; SSH 复合键分支不变。 - 进程内缓存 (正/负结果 60s TTL, in-flight 去重): withStore 是每次 memory 工具调用的热路径, 不能每次 spawn git - .git 标记上溯预检: 非仓库目录直接短路, 不 spawn git (也避开 Windows 子进程 CWD 锁临时目录导致 teardown EPERM) - 探测用 git -C <dir> rev-parse, 相对输出按 -C 目录解析 (兼容旧 git) - agent 启动路径仅在 memory 开启时解析, 关闭零探测开销 - manager.getStore() 归一化兜底, resetWorkdir/runReview/UI 旁路同语义 - buildMemoryScopeKey 同步契约不变 (scope-key.test.ts 不动) 验证: scope-resolver.test.ts 17 用例 (真实临时 git 仓库 + fake probe); maker-core 2301 通过; lizi-mcps 490 通过; 根 pnpm test:unit 全绿。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
`git clone --separate-git-dir` 的 checkout 其 --git-common-dir basename 恰好 也是 .git, 旧实现会把主仓根错误推导到 git 存储目录, 普通会话被静默映射到 无关 Store (Codex review on makecindy#2399)。 - 探测改为单次 rev-parse 同取 toplevel / git-dir / git-common-dir (每次 解析只 spawn 一次 git) - 仅当 gitdir ≠ common-dir (真 linked worktree) 才应用映射; 普通 clone / submodule / separate-git-dir 一律原样返回 - 新增 fake probe 用例 + 真实 `git clone --separate-git-dir` 端到端用例 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
…akecindy#2379) Codex review on makecindy#2399 第二轮两条跟进: - 主仓本身是 --separate-git-dir 布局时, common-dir 的 dirname 不是工作树, mainRoot 改为 `git worktree list --porcelain` 第一条记录 (git 保证主工作 树排第一, 布局带 core.worktree 指针时取到真实主 checkout)。已知限制: clone --separate-git-dir 不写 core.worktree 时 git 自身也无法反推真实 checkout (worktree list 把 gitdir 父目录报为主工作树), resolver 跟随 git 的 canonical 答案并在代码注释中记录。 - 真实 Git 组合矩阵 (多 worktree / separate-git-dir / 主仓内 cwd) 迁出默认 unit tier: 新增 scope-resolver.git-integration.test.ts 由 `pnpm test:git-integration` 执行, unit 层只保留一条真实 Git smoke + fake probe 全量覆盖 (engineering-conventions §3.1)。maker-core 在 test-workspaces.config.mjs 增加 git-integration tier 声明与 unit exclude, 契约测试同步更新。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
…第二阶段) makecindy#2399 合入前, worktree 会话的 memory 落到独立分片目录, 归一化生效后 不再被访问。本 PR 提供一次性迁移: 扫描 maker-memory 根下所有分片, 空分片直接删除, 有内容分片合并进 canonical(主仓)分片。 - migrate.ts: planLegacyShardMigration(纯只读扫描) + runLegacyShardMigration (幂等执行)。同名同内容跳过 / 同名不同内容 = 冲突不覆盖(保留源目录); SSH 分片与无 meta.json 目录不碰; 可选 backupRoot 迁移前备份 - migrate.test.ts: 11 个 unit 用例(fake resolver, 不 spawn git) - migrate.git-integration.test.ts: 2 个真实 git worktree 端到端用例 - scripts/migrate-maker-memory.mjs: CLI(dry-run 默认 / --apply 执行), 遵循 node --import tsx 先例 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
|
| Filename | Overview |
|---|---|
| packages/maker-core/src/memory/migrate.ts | 新增 legacy 分片迁移的计划、合并和安全删除流程；rename-then-remove 会使仍在运行的 legacy store 写入原路径时失败。 |
| packages/maker-core/src/memory/migrate.test.ts | 覆盖分片识别、冲突、备份、幂等及多种 rename 前内容竞态，但未覆盖目录被 rename 后活动 store 的实际写入。 |
| packages/maker-core/src/memory/scope-resolver.ts | 新增本地 Git worktree 到 canonical 主仓作用域的解析和缓存逻辑。 |
| packages/maker-core/src/memory/manager.ts | getStore 在缓存和创建 store 前统一解析本地 canonical scope key。 |
| scripts/migrate-maker-memory.mjs | 新增迁移计划预览、显式 apply、备份目录及 JSON 输出 CLI。 |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[扫描 legacy 分片] --> B{是否为空}
B -->|是| C[复查内容]
C --> D[rename 到 trash]
D --> E[最终复查]
E --> F[删除或恢复]
B -->|否| G{canonical 是否存在}
G -->|否| H[rename 为 canonical]
G -->|是| I[逐文件合并]
I --> J[冲突与内容复查]
J --> K[rename 到 trash]
K --> L[最终复查并删除或恢复]
Prompt To Fix All With AI
### Issue 1
packages/maker-core/src/memory/migrate.ts:307
**活动分片写入被中断**
如果存量会话在源分片被 rename 到 trash 后、迁移结束前写入记忆，常驻 store 仍会对原目录执行 `fs.writeFile`；由于该目录已不存在且写路径不会重新创建目录或重试，写入会以 ENOENT 失败。慢路径第 400 行的同类 rename 也存在相同窗口。
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (9): Last reviewed commit: "fix(memory): Windows 正斜杠 scope key 拼写统一 ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8617327a57
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…#2519) - Greptile P1 (migrate.ts): 未识别 .md 文件 (不符合 <type>_<slug>.md 规则 但仍含内容) 不再随源目录删除 — 检测到即保留源目录供人工处理 - Codex P1 (migrate.ts): 已归档/删除的 Cindy worktree 加静态路径推导 (deriveCanonicalFromCindyWorktreePath, .cindy-worktrees/<name> 形态), resolver live 探测失败时不再把旧记录判为孤儿 - Greptile Summary: 空分片删除前重新校验 (扫描后新增分片文件 → 跳过删除 并报告, 防过期快照删掉新写入数据) 新增 3 个 unit 用例覆盖 (14 total)。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
|
@dash-s-cindy[bot] / Greptile Summary 第三条（扫描与执行竞态）已修复（commit 6f4edaa）： 空分片删除前新增重新校验： |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6f4edaaa5a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…#2519) - Greptile P1 (migrate.ts): 只有未识别 .md 的 legacy 分片不再按「空」删 — 归入 mergeCandidates 走数据保全路径 (canonical 不存在 rename 快路径 / 已存在则未识别文件保留源目录), 未指定备份也不会丢用户内容 - Greptile P1 (migrate.ts): 空分片删除改 rename-then-remove — 复查放到 rename 之后只看待删临时目录, 压缩 countShardFiles→fs.rm 之间的竞态窗口; 删除前同时检查新增分片文件与未识别 .md - Codex P1 (migrate.ts): 静态推导同时认 .cindy-worktrees 与 .xdt-worktrees (品牌迁移前旧形态), 老版托管 worktree 归档后不再判孤儿 新增 2 个 unit 用例 (16 total)。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
rename-then-remove 的 rename 后复查只查 countShardFiles, 未识别 .md (notes.md 等) 在首次复查后、rename 前写入时仍会被递归删除。最终复查 补 findUnrecognizedMdFiles, 任一命中即恢复原目录名并报告。 新增 unit 用例: 扫描后写入未识别 .md → 跳过删除并报告 (17 total)。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c6f87e30ff
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…y#2519) - Codex P1 (migrate.ts): 慢路径删源前复查新增合法分片 — gainedValid = 当前合法数 − mergeFilesInto 快照数, >0 即保留源目录并报告 (快照后新写入 的 <type>_<slug>.md 既没被合并也不算未识别, 直接删会丢) - Codex P1 (migrate.ts): deriveCanonicalFromCindyWorktreePath 改段解析, 同时接受 / 与 \ 分隔符 — Desktop 存储把 Windows workingDir 归一化为 正斜杠 (C:/repo/.cindy-worktrees/...), 只认 path.sep 会漏判成孤儿 新增 2 个 unit 用例 (19 total)。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b381b803fb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… on makecindy#2519) - Codex P1: SSH 分片判定从目录名前缀改为 meta.absPath 的 ssh: scope key 前缀 — sanitizeWorkdir 允许本地路径 (如 /ssh/proj) 恰好产出 ssh- 开头的 目录名, 前缀误判会把本地 legacy 分片跳过成孤儿 - Greptile P1: 未识别内容检查从 .md 扩展为一切非系统非分片文件 — notes.txt / data.yaml 等非 Markdown 遗留内容不再随源目录删除 - Greptile P1: 删源前对已合并分片做源/目标逐字节内容对比 (findChangedAfterMerge) — 存量会话在复制后更新已有记忆时数量复查检测不到, 内容对比兜底 新增 3 个 unit 用例 (22 total)。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9472bf5fce
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…akecindy#2519) - Codex P1: 同数替换 (删 A 建 B) 时数量不变, gainedValid 检测不到 — 新增 diffShardFilenames 文件名集合对比 (快照 vs 当前), added/missing 任一命中即保留源目录, 替代原计数法 - Greptile P1: 全部复查完成后 fs.rm 前仍有写入窗口 — 慢路径删除改 rename-then-remove: rename 后对 trash 目录做最终复查 (未识别 + 集合 对比), 命中即恢复原目录名, 压缩删除前窗口 新增 1 个 unit 用例 (23 total)。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7eab97f139
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…#2519) 存量会话在 findChangedAfterMerge 之后、rename 之前更新同名记忆时, trash 目录文件名集合不变但内容新 — rename 后最终复查只查未识别 + 集合, 会删掉包含新版本的 trash 目录, canonical 保留旧版本。 rename 后最终复查补 findChangedAfterMerge (trash vs target 逐字节对比), 任一命中即恢复原目录名并报告 'content appeared or changed before remove'。 新增 1 个 unit 用例 (24 total)。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d56cc03d2c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…cindy#2519 第八轮) Desktop 存正斜杠路径 (C:/repo), 但 resolver 内 path.normalize/join 在 Windows 上产反斜杠 (C:/repo)。两者 sanitize 到同一磁盘目录, 而 MakerMemoryManager 按 raw scope key 缓存 Store — 主 checkout 会话 (正斜杠 key) 与 worktree 会话 (反斜杠 key) 会开两个实例指向同一 SQLite/索引, 一侧写入后另一侧 MEMORY.md 缓存过期。 修复: canonicalizeLocalWorkdir 返回前 matchSeparatorStyle — 仅当输入是 Windows 盘符正斜杠形态 (C:/... Desktop 归一化拼写) 时输出转正斜杠; POSIX 风格路径与反斜杠输入保持 path.join 默认行为。 新增 1 个 unit 用例 + 更新 1 个缓存用例期望 (win32 分支)。 Signed-off-by: KingSystemHaiGo <kingsystemhaigo@users.noreply.github.com>
| // 无法进入待删目录 (与备份目录同层, 名字带后缀避免冲突)。 | ||
| const trashName = `${path.basename(shard.dir)}.trash-${now().replace(/[:.]/g, '-')}`; | ||
| const trashDir = path.join(path.dirname(shard.dir), trashName); | ||
| await fs.rename(shard.dir, trashDir); |
There was a problem hiding this comment.
如果存量会话在源分片被 rename 到 trash 后、迁移结束前写入记忆，常驻 store 仍会对原目录执行 fs.writeFile；由于该目录已不存在且写路径不会重新创建目录或重试，写入会以 ENOENT 失败。慢路径第 400 行的同类 rename 也存在相同窗口。
Context Used: 使用和PR描述相同的语言进行评论 (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/maker-core/src/memory/migrate.ts
Line: 307
Comment:
**活动分片写入被中断**
如果存量会话在源分片被 rename 到 trash 后、迁移结束前写入记忆，常驻 store 仍会对原目录执行 `fs.writeFile`；由于该目录已不存在且写路径不会重新创建目录或重试，写入会以 ENOENT 失败。慢路径第 400 行的同类 rename 也存在相同窗口。
**Context Used:** 使用和PR描述相同的语言进行评论 ([source](https://app.greptile.com/review/custom-context?memory=instruction-0))
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
维护者确认本 PR 为 memory 存量 worktree 分片迁移新功能（2251 行，触及 maker-core 核心路径），触发产品确认门。 确认方式：请维护者直接在 PR 上 Approve；如需修改请 Request Changes，作者改完后重新 Approve 即放行。 讨论 issue:#2529 |
2 participants
这次改了什么
摘要
修复 #2379 问题一的 P0 第二阶段：存量 worktree 分片迁移。#2399 已让新会话读写 canonical（主仓）分片，但 #2399 合入前产生的旧 worktree 独立分片（实测 105 个分片里 54 个 worktree 分片、38 个 dialogue 一次性目录）仍留在磁盘上不再被访问。本 PR 提供一次性迁移工具：把有内容的 legacy 分片合并进 canonical 分片，空分片直接删除，恢复「记忆写进读不到的地方」的最后一环。
改动：
packages/maker-core/src/memory/migrate.ts：planLegacyShardMigration(memoryRoot)— 纯只读扫描，产出迁移计划（emptyToDelete / mergeCandidates / skipped）runLegacyShardMigration(plan, opts)— 幂等执行（可重复跑）：<type>_<slug>.md）→ 直接删（零内容零风险，记忆系统改造：worktree 分片断裂 + 缺回合后复盘机制 #2379 正文）backupRoot：迁移前先把受影响分片复制备份meta.json.absPath经resolveMemoryScopeKey归一化后，canonical 目录名 ≠ 当前目录名 → 真正的旧 worktree 分片ssh-前缀，记忆系统改造：worktree 分片断裂 + 缺回合后复盘机制 #2379 约束 3）、无 meta.json 的残留目录（不猜不删）migrate.test.ts（11 个 unit 用例，fake resolver 不 spawn git）：计划生成 / 空分片删除 / rename 快路径 + meta.absPath 更新 / 同名同内容跳过 / 同名不同内容冲突保留源目录 / 备份 / 幂等 / SSH 与孤儿目录跳过migrate.git-integration.test.ts（2 个真实 git worktree 端到端用例，pnpm test:git-integration执行）：真实 resolver 全链路合并 + 幂等复扫scripts/migrate-maker-memory.mjsCLI：默认 dry-run 只出计划，--apply执行，--backup-dir备份，--json机器可读；遵循node --import tsx先例（同 browser-capability-benchmark）变更类型
feat新功能范围
buildMemoryScopeKey()/resolveMemoryScopeKey()的任何改动（fix(memory): worktree 会话共享主仓 memory store — scope key git 归一化 (#2379) #2399 已定契约，本 PR 只消费）UI 变化
不涉及：纯 maker-core 工具模块 + CLI 脚本，无 UI。
怎么验证的
自动验证
手工验证
不涉及（运行期行为验证需宿主 app；迁移工具行为已由 unit + 真实 git 端到端覆盖）。
未执行的验证
风险
风险分类
影响与回滚
--backup-dir可完整还原；无备份时，有冲突的分片源目录保留、无冲突分片删除后可用 git/磁盘恢复工具找回（删除前建议备份）；无数据格式变更、无 migration、无协议变更提交前检查
git commit -s，见 DCO）