diff --git a/.agents/docs/2026-10-12-windows-performance-plan.md b/.agents/docs/2026-10-12-windows-performance-plan.md new file mode 100644 index 00000000..f62955f5 --- /dev/null +++ b/.agents/docs/2026-10-12-windows-performance-plan.md @@ -0,0 +1,475 @@ +# Windows 专项性能方案：暖启动与编辑补全（2026-10-12） + +目标：在 Windows 上，**第二次打开项目**与**编辑时的补全反应**接近 Linux。基线是 mcppls 0.0.13 加 mcppls-clangd 0.0.14。 + +本文先说清楚一次补全、一次按键、一次暖启动在引擎里各做了什么，再说明 Windows 为什么把这些开销放大。然后逐个分析可选方案，给出分阶段计划和验收预算，最后从多个角度自审。冷启动（第一次构建全部模块）不是主线，只在相关处提到。 + +## 0. 结论 + +1. **Windows 上慢的主要不是编译，而是逐文件的元数据检查。** LLVM 在 Windows 上 stat 一个文件，要打开一个句柄（`CreateFileW` → `GetFileType` → `GetFileInformationByHandle` → `CloseHandle`），而且每次打开都要经过 Defender 等文件系统过滤驱动；Linux 上同样的操作只是一次 `stat`。C++ 模块的"能否复用"全靠逐文件确认：扫描缓存重放、BMI 校验缓存、preamble 输入。所以 Windows 上暖启动慢 2.8 倍，最慢的请求慢 4–5 倍，大项目上每次补全光检查就要约 1.2 s。 +2. **编辑器协议把这份开销按键次数放大。** clangd 默认一次最多返回 100 条补全。全局模块片段里有 ``、Qt 这类头文件时，几乎每个列表都是 `isIncomplete`，VS Code 就会在每次按键时重新请求。所以在 Windows 上打一个标识符，每个字符都要付一次完整的 clangd 补全。 +3. **暖启动没有可以复用的状态。** 扫描缓存和 BMI 校验缓存都只在 clangd 进程的内存里。第二次打开时，要把每个模块单元重新扫描一遍（预处理 ``），并把每个 BMI 的所有用户输入按内容重新算哈希。大项目上一个文件的 25 个前置模块全部复用，也要 28 s：复用检查 8.5 s，扫描与规划约 20 s。 +4. **被逐文件核对的头文件，大多来自 mcpp 管理的包。** Windows 上 mcpp 用 xlings 安装的 llvm-mingw，`` 来自它自带的 mingw-w64 头；第三方库也装在 mcpp 的包仓库里，按 `<包>/<版本>/` 组织，并带安装完成标记。这类文件不必逐个核对，**按包版本核对一次**就够了。 +5. 因此方案分五层： + - **产品层减少 clangd 请求**：让 clangd 一次给出完整的列表，由编辑器在本地过滤；同一个词的再次请求用完整列表本地作答。 + - **引擎层减少每次请求的检查**：去掉重复的 stat，系统头每个会话只核对一次，mcpp 管理的包按版本核对，Windows 上用快速 stat。 + - **持久化两个缓存**：暖启动只核对、不重新扫描。 + - **不保留进程的重开优化**：先准备上次活跃的文件，缓存异步加载。 + - **环境建议**：Dev Drive / Defender。 + + 除"系统头每会话一次"与 clang 自己的默认策略一致外，都不改变"认为什么是新鲜的"。 + +## 1. 现状数据 + +### 1.1 同一 fixture 三平台（CI，引擎 0.0.14，run 38206770230；runner 都是 4 vCPU，macOS 为 M1、3 vCPU） + +| 指标 | Linux x64 | Windows x64 | macOS arm64 | Win/Linux | +|---|---|---|---|---| +| `timing` 冷启动首次跳转（5 轮中位数） | 3.21 s | 7.06 s | 2.81 s | 2.2× | +| `timing` 暖启动首次跳转 | 0.75 s | 2.11 s | 0.54 s | 2.8× | +| STRESS1（103 文件）总耗时 | 2.0 s | 6.2 s | 1.7 s | 3.1× | +| 其中 补全 / 跳转定义 / 悬停的最慢一次 | 139 / 254 / 161 ms | 619 / 1218 / 827 ms | 133 / 366 / 165 ms | 4.5–5× | +| 稳定性 fixture（等待为主 → 编译为主） | — | — | — | 1.0–2.3× | + +Linux 冷启动的 3.2 s 里，有 2.7 s 是构建 std 的 BMI。Linux 暖启动的 0.75 s 里，约 0.46 s 是前置模块的复用检查。 + +### 1.2 Windows 交互场景（`ux-modules-windows`，全局模块片段 include ``） + +| 场景 | Windows | Linux 同类（`ux-modules-qt`，项目不同） | +|---|---|---| +| U3 空闲补全 p95 | 0.67 s（0.0.12 为 0.54 s） | 0.09 s | +| U5 在导入方里打字，补全 p95 | 1.00 s（顶到服务器的期限） | 0.30 s | +| U16 在模块单元里打字，补全 p95 | 0.76 s（引擎 0.0.13 时为 1.00–1.02 s） | 0.21 s | +| 带 `` 的 preamble | 28 MB，每次重建 2.4–3.9 s | — | + +### 1.3 大型 Windows 模块项目探针（引擎 0.0.13 候选；有 25 个前置模块的文件） + +| 指标 | 值 | +|---|---| +| 第二次打开一个文件 | 28 s：复用检查 8.5 s，扫描与规划约 20 s，构建 0 | +| 稳态每次补全的模块校验 | 约 1.2 s（25 个 BMI 的校验缓存，逐个 stat 输入） | +| 稳态一次补全 | 10–13 s，连续 3 次超时后文件被暂时转交 | +| 冷启动 | clangd 102 s 就绪，模块全部建完约 680 s | +| 缓存目录换位置 | 24 个模块全部重建（BMI 的键里有与位置有关的路径） | + +**缺口**：现有 fixture 每秒请求一次补全，没有模拟 VS Code 在 `isIncomplete` 时逐键重新请求的行为；Windows CI 也没有大项目的暖启动场景。阶段 0 补上这两项（§5）。 + +## 2. 成本模型：引擎里每一步做了什么 + +### 2.1 一次 stat 在 Windows 上的代价 + +LLVM `sys::fs::status(path)` 在 Windows 上的实现是 `CreateFileW(..., FILE_FLAG_BACKUP_SEMANTICS)` → `GetFileType` → `GetFileInformationByHandle` → 关闭句柄。打开句柄（IRP_MJ_CREATE）会经过所有文件系统过滤驱动，Defender 实时防护就是其中之一。clang 的 `FileManager` 需要文件的 UniqueID（卷序列号加文件索引），所以不能简单换成不开句柄的 `GetFileAttributesExW`。下文所有"逐文件"的开销，都是在这个单价上乘出来的。 + +### 2.2 一次补全（`CodeComplete.cpp` 的 module validation 段，加上之后的 Sema） + +| 步骤 | 做什么 | 文件系统操作 | +|---|---|---| +| a. 草稿的模块输入 | `getCompletionInputs` / `getRequiredModuleNames`：查扫描缓存（0029 之后，模块单元打字也能命中） | 命中时**重放**这次扫描记录的每一个观测：stat、open、fstat、对磁盘再 stat 一次、关闭。全局模块片段的 `` 就是几百个文件 | +| b. 新 import？ | `isBuildableModule`，有新模块时等最多 250 ms（0027） | 少量 | +| c. 依赖改了吗？ | `canReuse` → 每个 BMI 调一次 `IsModuleFileUpToDate` → `ModuleValidationCache::isFresh` | 每个 BMI stat 一次；每个用户输入 stat **两次**（经 VFS 一次、`sys::fs::status` 再一次）；**整段持有一把全进程锁** | +| d. Sema | 读 preamble 和 BMI，选择性加载（0025） | preamble 默认存在磁盘上（`--pch-storage` 默认 `disk`） | + +在大项目上，c 一步就要约 1.2 s（25 个 BMI）。a 一步在 Windows 上占多少还没有数据，阶段 0 去量。全进程锁还会让补全线程和其他 AST worker 互相等待。 + +c 里的"用户输入"在 clang 看来包括经 `-I` 引入的所有头文件，其中也包括 mcpp 包仓库里第三方库的头文件。clang 的系统头豁免（`ValidateSystemInputs=false`）只对 `-isystem` 和系统目录生效。 + +### 2.3 一次按键 + +- clangd 的文档更新（`TUScheduler`）：要判断 preamble 能不能跳过（`hasBuildableRequiredModules`、`getTextualPreambleInputs`），走的也是 a 那条重放路径；防抖之后重建 AST，又要走 c。 +- 编辑落在 preamble 区（`#include` 区或 import 区）时，要重建 preamble：带 `` 时是 28 MB、2.4–3.9 s，并写进 `%TEMP%`。 +- mcppls 自己的开销很轻：更新单词索引、记录结构变化、发布诊断，都在毫秒级。 + +### 2.4 编辑器协议的放大 + +clangd 默认 `--limit-results=100`，mcppls 没有改它，`isIncomplete` 原样交给编辑器。LSP 规定，列表标了 incomplete 时，后续输入要重新计算这个列表，所以 VS Code 每打一个字符就发一次新请求。全局模块片段里有大头文件时，第一个字母之后的匹配往往远超 100 条，于是**一个单词的每个字符都会付一次 2.2 节的完整开销**。mcppls 已有的 C-2（超时的 clangd 结果在同一个词内交出）只是在请求超时以后才起作用。 + +### 2.5 暖启动（第二次打开） + +| 部分 | 现在 | 原因 | +|---|---|---| +| 项目模型 | 内容相同即视为新鲜（0.0.13 W5） | 已解决 | +| 依赖扫描（每个模块单元） | 全部重做，按完整内容预处理 | 扫描缓存只在 clangd 进程内存里 | +| BMI 复用检查 | 完整校验，把每个用户输入按内容重算哈希 | 校验缓存只在内存里 | +| 打开文件的 preamble | 重建 | 不跨会话 | +| 后台索引 | 加载索引分片并核对文件，与上面几项争 I/O | 后台优先级，Windows 上也降了 I/O 优先级 | +| 准备顺序 | 等编辑器的 didOpen 到了才开始准备这个文件 | 服务器不知道上次在编辑哪些文件 | +| 缓存目录换位置 | 全部重建 | BMI 的键与位置有关 | + +### 2.6 输入文件的三类来源 + +| 类别 | 例子 | 会怎样变化 | 现在怎么核对 | +|---|---|---|---| +| 项目文件 | 工作区内的源码和头文件、路径依赖、mcpp 构建生成的目录 | 随时会被编辑、生成、检出 | 每次逐文件核对 | +| mcpp/xlings 管理的包 | `<仓库>/<包>/<版本>/` 下的一切：工具链（llvm-mingw 及其 mingw-w64 头、``）、第三方库 | 安装后只由 mcpp/xlings 改动（重装、修整），每个版本目录有安装完成标记（`.mcpp_ok`、`.xpkg-install.json`、可能有的 `.mcpp-fixup.json`） | 和项目文件一样逐文件核对 | +| 系统与外部工具链 | 系统 include 目录、外部安装的 Windows SDK 或 MSVC、kit、clang 资源目录 | 系统更新时才变 | BMI 校验已豁免（`-isystem`），扫描重放仍逐文件 | + +第二类数量最多（``、Qt、第三方库都在这里），又最可控：它的变化只来自 mcpp/xlings 自己，而且有标记文件可以确认。B2 就是按这三类分别处理（§4）。 + +## 3. 场景矩阵 + +| # | 场景 | 现在（Windows） | 主要开销 | 适用方案 | +|---|---|---|---|---| +| S1 | 打开文件后的第一次补全（暖启动） | 小项目约 2 s；大项目数十秒 | 2.5 节的扫描与完整校验、preamble | D1 D2 D3 B2 B3 G1 G2 | +| S2 | 什么都没改时的空闲补全 | 0.67 s（小）/ 约 1.2 s 以上（大） | 2.2 a、c | B1 B2 B3 B4 | +| S3 | 在普通 `.cpp` 正文里连续打一个标识符 | 每个字符一次完整补全 | 2.4，再乘以 2.2 | **A1** + B 组 | +| S4 | 成员访问 `obj.`、`ns::` | 一次完整补全（列表通常不大） | 2.2 | B 组；A1 覆盖之后的按键 | +| S5 | 在模块单元正文里打字（U16） | 0.76 s | 2.2，加上导入方重建失败模块的 CPU 争用 | A1、B 组、D3 | +| S6 | 在 import 区编辑或新增 import | 新模块最多等 250 ms，之后给部分结果 | 2.2 b、preamble 重建 | 已有（0027）；C1 | +| S7 | 另一个文件里改了被导入的模块，回来补全 | 等重建最多 250 ms | 构建 | 已有；D3 | +| S8 | 在 `#include` 区编辑 | preamble 重建 2.4–3.9 s，期间用旧 preamble 回答 | 编译与写 `%TEMP%` | C1、E1 | +| S9 | 切换文件（AST 被淘汰） | 悬停、跳转要重建 AST | `MaxRetainedASTs=3` | C2 | +| S10 | 项目级第二次打开 | 28 s/文件（大项目） | 2.5 节 | D1 D2 D3 D4 D5 G1 G2 | +| S11 | 冷启动、模块还在构建 | 补全有限等待（0027） | 构建 | 不在本方案主线 | + +## 4. 方案分析 + +收益估计都标明了依据，最终以阶段 0 的度量为准。 + +### A. 产品层：减少发给 clangd 的补全请求 + +**A2 让 clangd 一次给出完整的列表**（采纳，阶段 1，是这一组的主体） + +- 编辑器对**完整**的列表（`isIncomplete: false`）会在本地按后续输入过滤，不再请求；逐键请求只发生在列表不完整时。列表不完整，绝大多数是因为 clangd 的结果上限（`--limit-results`，默认 100）。 +- mcppls 给内置引擎传入更高的上限（默认 1000，由阶段 0 的度量确定）。匹配数在上限以内时列表就是完整的，同一个词之后的输入完全由编辑器处理，一个词只请求一次 clangd。 +- 代价：第一次回答的 JSON 变大（1000 项约几百 KB，本地管道传输与解析都在毫秒级），clangd 为更多项生成细节。阶段 0 用 `ux-modules-qt` 和大 fixture 量 100 与 1000 的单次耗时，增加超过 20% 就降低上限。 +- 第一个字母就匹配上几千个名字时（`` 的 `C`），列表仍然不完整，下一键照常请求 clangd；通常到第二、三个字母就完整了。 + +**A1 同一个词内的再次请求，用完整列表本地作答**（采纳，阶段 1；界面上不区分，只写进日志和 report） + +- 有的客户端每次按键都请求，不看 `isIncomplete`；VS Code 也会在某些情况下重新触发。对同一个词的再次请求，如果 clangd 上一次给这个词的回答是**完整的**，就用它在本地作答： + 1. 同一个文件，光标仍在这个词里（`word_key`/`typed_on`，C-2 已有）； + 2. 词起点之前的全部文本、光标之后的全部文本都与那次回答时相同（逐字比对两段的哈希）； + 3. 每一项的插入范围改写成"从词的起点到当前光标"（C-2 已有的 `retarget`），`filterText`、`sortText`、`score`、`additionalTextEdits` 不变，过滤和排序交给编辑器。 +- 上一次的回答不完整（包括 0027 的部分模块回答），就照常请求 clangd，不在本地作答。所以本地答案的候选集合与 clangd 当时给出的完全相同，不会漏项，也不需要后台刷新。 +- 它是 C-2（超时答案按词交出）的推广，复用同一套数据结构。 +- **退出开关**：命令行 `--no-completion-refilter`，供排查用，不做成用户设置。 + +### B. 引擎层：每次请求的文件系统检查 + +**B1 校验缓存：stat 时不持锁**（采纳，阶段 1，可上游） + +- 先在锁内把条目拷出来，在锁外做 stat，最后再加锁比对并删除。补全线程和 AST worker 不再串行等这把全进程锁。 +- 每个输入的两次 stat（经请求的文件系统一次、直接对磁盘一次）**保留**：`IsModuleFileUpToDate` 有多个调用点，传入的文件系统可能是带 stat 缓存的，对磁盘的那一次是防止旧状态的安全网。减少 stat 的工作交给 B2（按来源跳过）和 B3（更便宜的 stat）。 +- **风险**：无。判定条件不变。 + +**B2 按来源分类核对输入**（采纳，阶段 1；分类见 2.6） + +- **B2a 系统与外部工具链**：扫描缓存重放时，路径落在系统 include 根（`-isystem` 指向的系统目录、外部安装的 SDK 或 MSVC、kit、clang 资源目录）下的观测，本会话成功重放过一次后就不再重放。 + - 依据：clang 对模块输入本来就默认不检查系统头。mcppls 发现工具链变化时已经会重启 clangd。 + - 风险：会话中途系统更新头文件，要等 clangd 重启才生效，与 clang 的默认策略一致。 +- **B2b mcpp/xlings 管理的包**：以**包版本**为单位核对，不再逐文件。 + - **包版本的身份**：版本目录下安装标记文件的身份（`.mcpp_ok`、`.xpkg-install.json`，以及存在时的 `.mcpp-fixup.json`），加上 `.xpkg-install.json` 里的 revision。 + - **适用范围**：扫描缓存重放、BMI 校验缓存（这些头经 `-I` 进来，clang 当作用户输入），以及 D1、D2 的持久化条目。一次请求里，同一个包版本只核对它的标记文件，大约 3 次 stat，取代几百次。 + - **哪些路径算包**：由 mcppls 告诉引擎。mcppls 已经知道 mcpp 和 xlings 的包仓库位置，以新的引擎参数传入（例如 `--managed-input-root=<仓库>`，可重复）；引擎按 `<仓库>/<包>/<版本>/` 切出包版本。路径依赖、`[patch]` 指向本地目录的包，以及 mcpp 构建生成的目录（`/build-database/…/deps/<包>@<版本>/out/`），都不在仓库下，仍按项目文件处理。 + - **失效**：任何一个标记文件的身份变了，这个包版本的所有记录都失效，回到逐文件核对一次，然后记下新身份。另外，mcpp 的描述回答报告了安装类 effects 时，mcppls 让引擎丢掉所有包身份记录（新的引擎通知，或者重启引擎，取实现更简单的一种）。 + - **依据**：mcpp 安装的库是规范、可控的模型：一个版本装进自己的目录，装完写完成标记，之后只有 mcpp/xlings 会动它。 + - **风险**：如果有人手工改了包仓库里的头文件又不动标记，mcppls 看不到这个改动，直到包被重装或标记变化。这是使用包仓库时的预期行为，文档里写明。 +- **它与已撤掉的 UP-34 的区别**：UP-34 不分来源，对项目头也放宽，`module_preamble_mode` 用项目头测出了问题。B2 对项目文件什么都不改，那个测试的语义不受影响。 +- **收益**：Windows 上 ``（mingw-w64，在 llvm-mingw 包里）和第三方库头的逐文件核对，从每次请求几百次 open/stat，降到每个包版本几次 stat。按 2.2 节，这是 a、c 两步的大头；确切比例由阶段 0 量出。 + +**B3 Windows 快速 stat：`GetFileInformationByName`**（采纳，阶段 2，可上游到 LLVM Support） + +- Windows 11 24H2 和 Windows Server 2025 起，`GetFileInformationByName(FileStatBasicByNameInfo)` 不用打开句柄，就能拿到文件 ID、卷序列号、大小和时间，足够构造 UniqueID。运行时用 `GetProcAddress` 探测，没有这个 API 时回到现在的实现；符号链接和重解析点也走旧路径。 +- **收益**：每次 stat 从 4 次系统调用加一次过滤驱动的打开，变成一次调用，估计单价降到 1/2–1/4（需实测）。B2 之后剩下的都是项目文件，B3 让它们也变便宜。 +- **CI 限制**：GitHub 的 `windows-2022` 镜像是 Server 2022，没有这个 API，度量需要 `windows-2025` 镜像。 +- **风险**：API 较新，回退路径必须保留并测试。 + +**B4 按目录批量 stat**（阶段 3，取决于 B2、B3 之后的数据） + +- 同一个操作里要核对同一目录下的很多项目文件时，用 `NtQueryDirectoryFile(FileIdExtdDirectoryInformation)` 一次读出整个目录的文件 ID、大小和时间。 +- NTFS 目录项里的大小和时间是延迟更新的，修改时间离现在不到 30 s 的文件要回到逐文件 stat（与扫描缓存现有的 `unchangedOnDisk` 规则一致）。 +- B2 之后剩下的多是项目文件、目录分散，收益可能不大；数据不支持就不做。 + +**B5 以文件监视为准信任项目文件**（不进本期） + +- 正确性取决于编辑器的监视是否可靠（VS Code 在高负载时会丢事件），引擎的 e2e 测试也明确守着"没有事件也要看到改动"。B1–B3 做完后，项目文件的核对已经不是瓶颈，不需要冒这个险。 + +**B6 复用最近一次重放**（已否决） + +- 这就是 mcppls-clangd 0.0.14 试过又撤掉的 UP-34。B2 是它安全的部分：只放宽来源可控的那两类。 + +### C. preamble 与 AST + +**C1 Windows 上 preamble 放内存**（阶段 1，A/B 度量后决定默认值） + +- `--pch-storage=memory`：preamble 不写 `%TEMP%`，避免 Defender 扫描 28 MB 的临时文件，也省掉写盘再映射回来读。代价是常驻内存，约等于打开文件数乘以 preamble 大小。 +- 默认只在 Windows 上、物理内存不少于 16 GB 时开启。 +- 新设置 `mcppls.engine.preambleStorage`：`auto`（默认，按上面规则）、`disk`、`memory`。 + +**C2 AST 保留数可配置**（阶段 2，引擎加开关） + +- `MaxRetainedASTs` 写死为 3，没有命令行开关。Windows 上大 preamble 的文件来回切换时，悬停和跳转要重建 AST。 +- 增加 `--max-retained-asts`，mcppls 按物理内存给出默认值（16 GB 以上为 6），只传给声明了这个开关的引擎。 + +**C3 preamble 跨会话复用**（不进本期，D、G 之后的首选候选） + +- 收益在 S1、S10，但它的键（preamble 文本、命令、所有输入的身份）和体积管理都比较复杂。F1 不做之后，它是剩下的重开手段里最大的一项；D、G 落地后，用启动行和 cost 行看 preamble 在 S10 里的占比再定。 + +### D. 暖启动：持久化与扫描 + +**D1 持久化 BMI 校验缓存**（采纳，阶段 2，可上游） + +- 进程退出时（以及周期性地）把 `ModuleValidationCache` 写进工作区缓存目录，启动时读回。条目本来就带着 BMI 和每个用户输入的身份（大小、修改时间、文件 ID），读回后照样核对：项目文件逐个 stat，包内的输入按 B2b 的包版本身份核对，有任何不一致就回到完整校验。 +- **收益**：第二次打开时，复用检查从"按内容重算哈希"变成"核对身份"。大项目上那 8.5 s，加上 B1、B2、B3，估计降到 1 s 以内。 +- **风险**：文件 ID 在复制或恢复的目录上会变，这时回到完整校验，结果仍然正确。 + +**D2 持久化扫描缓存**（采纳，阶段 2） + +- 把 `ModuleScanManifest`（观测记录）和扫描结果写到磁盘，键与现在相同（文件、命令、主文件内容摘要、环境），读回后照样重放：项目文件逐个重放，包和系统根按 B2 处理。 +- **收益**：第二次打开时，每个模块单元从"预处理 ``"变成"核对"。大项目上约 20 s 的扫描与规划，是这一项的目标。 +- **风险**：重放规则不变，所以正确性不变。体积受现有的字节上限约束。 + +**D1、D2 的共同约定** + +- 文件放在 mcppls 的工作区缓存目录下（由 mcppls 以引擎参数传入），计入现有的缓存预算，"重置缓存"会清掉它们。 +- 格式带版本号和引擎身份（fork 版本、补丁序列哈希），不匹配就整体丢弃。 +- 写入用"写临时文件再改名"，只由持有工作区租约的实例写；损坏或读不懂就丢弃。 +- 读入在引擎启动时**异步**进行，读完之前的查询按缓存未命中处理，不阻塞第一个请求。 + +**D3 磁盘上的模块源也按前缀扫描**（采纳，阶段 2，引擎） + +- 0029 只对草稿（编辑器里的文本）生效。对磁盘上的模块源也做同样的切分（模块声明加紧随的 import），自动保存改正文时扫描缓存仍然命中。 +- **收益**：减少 S5、S7 里导入方重建前的扫描（Windows 上约占一次重建的一半），也减少 D2 的失效。 + +**D4 BMI 的键与缓存位置无关**（阶段 3） + +- 去掉命令键里与缓存目录有关的参数，按模块名和内容身份重新组织。缓存搬家、多个工作区副本之间就能复用。用户场景少，CI 和容器场景多。 + +**D5 暖启动时后台索引延后**（阶段 2，先度量） + +- 后台索引在打开的文件拿到第一次诊断之后再启动，避免暖启动时争 I/O。只有在度量显示它确实影响 S1 时才做。 + +### E. 环境 + +**E1 Dev Drive 与 Defender 提示**（采纳，阶段 1，产品与文档） + +- 启动时探测缓存目录所在卷是不是 Dev Drive（`GetVolumeInformationW` 加卷持久状态里的 Dev Drive 标志）。不是时，在 report 和 hub 概览的环境信息里写一行，并在文档里说明两种做法：把 `MCPPLS_CACHE_DIR` 放到 Dev Drive 上（Defender 对 Dev Drive 用异步的性能模式扫描），或者由用户自己为缓存目录添加 Defender 排除。 +- 不弹窗、不重复提示。**mcppls 不自动修改任何安全设置**，也不提示以管理员身份运行。 +- CI 上用 Dev Drive（VHD）做一次 A/B，把对 S1、S10 的影响写进文档。 + +### F. 编辑器关闭后让 clangd 保留一段时间（已否决） + +- 编辑器关闭后还留着语言服务进程，不符合用户习惯。重开的开销改用 D 组（持久化）和 G 组（不保留进程的重开优化）解决，C3 作为之后的候选。 + +### G. 不保留进程的重开优化 + +**G1 先准备上次在编辑的文件**（采纳，阶段 2，产品） + +- mcppls 在工作区状态里记下每次会话最后打开、最后活跃的几个文件（只记路径，存在本地缓存目录）。下次 `initialize` 之后、编辑器的 `didOpen` 到来之前，就按这个顺序让引擎准备它们的前置模块。这复用现有的 prime 机制，以及"有文件在等前置集合时，prime 只用剩下的 worker"的规则（R-1）。 +- 编辑器恢复窗口、插件激活本身就要几秒，这段时间用来校验和装载模块。 +- 做法：模型就绪、数据库写好后，用 mcppls 已有的"后台打开"机制（找定义和补索引时用的那一套）把这些文件在 clangd 里打开。编辑器随后打开同一个文件时，**接管**而不是关闭重开：发一次全文更新（编辑器的文本），clangd 沿用已经建好的 preamble，诊断照常发给编辑器。 +- 只准备上次真正打开过、现在仍存在、仍在计划里、编辑器还没打开的文件；最多 5 个，只记当前工作区。两分钟内编辑器没有打开的，就关掉。 +- 收益主要在恢复后的非前台标签页（切过去时 preamble 已经建好）和打开时机晚于模型就绪的情况。 + +**G2 BMI 预读**（阶段 2，先度量） + +- 重启或长时间不用之后，BMI 不在系统文件缓存里，第一次装载要真正读盘。G1 选出的文件，在后台以低 I/O 优先级顺序读一遍它们的 BMI，让之后的映射命中缓存。 +- 只有度量显示 S1 里冷读盘占比明显时才做。 + +### 方案总表 + +| 方案 | 场景 | 收益（估计，依据见上） | 正确性风险 | 成本 | 位置 | 结论 | +|---|---|---|---|---|---|---| +| A2 完整的列表 | S3 S4 S5 | 一个词只请求一次 clangd，之后由编辑器过滤 | 无 | 低 | 产品 | 阶段 1，度量定上限 | +| A1 完整列表本地作答 | S3 | 每键请求的客户端也不再打到 clangd | 无（候选集合与 clangd 当时的相同） | 低 | 产品 | 阶段 1 | +| B1 stat 不持锁 | S2–S5 | 去掉补全与 AST worker 之间的锁竞争 | 无 | 低 | 引擎 | 阶段 1 | +| B2a 系统根每会话一次 | S1–S5 | 系统头的重放降到每会话一次 | 低（与 clang 默认一致） | 低 | 引擎 | 阶段 1 | +| B2b 包按版本核对 | S1–S5 S10 | ``、第三方库：每次请求几百次 → 每包几次 stat | 低（依赖 mcpp 的安装模型） | 中 | 引擎 + 产品 | 阶段 1 | +| B3 快速 stat | 全部 | stat 单价估计降到 1/2–1/4 | 低（有回退） | 中 | LLVM Support | 阶段 2 | +| B4 目录批量 | S1 S2 | 视数据 | 中（延迟更新的目录项） | 中 | 引擎 | 阶段 3 | +| B5 监视信任 | 全部 | — | 高 | 中 | 引擎 + 产品 | 不进本期 | +| B6 复用最近重放 | — | — | 高（已被测试否决） | — | — | 否决 | +| C1 preamble 放内存 | S6 S8 | 免去写临时文件和扫描 | 无（内存代价） | 低 | 产品 | 阶段 1，A/B | +| C2 AST 保留数 | S9 | 切换文件不重建 | 无（内存代价） | 低 | 引擎 + 产品 | 阶段 2 | +| C3 preamble 跨会话 | S1 S10 | 中 | 中 | 高 | 引擎 | D、G 之后再定 | +| D1 持久化校验缓存 | S1 S10 | 复用检查从哈希变成核对（大项目 8.5 s → <1 s） | 无（照样核对） | 中 | 引擎 | 阶段 2 | +| D2 持久化扫描缓存 | S1 S10 | 扫描从预处理变成核对（约 20 s 那一段） | 无（照样重放） | 中 | 引擎 | 阶段 2 | +| D3 磁盘源按前缀扫描 | S5 S7 S10 | 导入方重建前的扫描 | 低 | 低 | 引擎 | 阶段 2 | +| D4 与位置无关的键 | 缓存搬家 | 免全部重建 | 中 | 中 | 引擎 | 阶段 3 | +| D5 后台索引延后 | S1 | 待度量 | 无 | 低 | 引擎 | 阶段 2，先度量 | +| E1 Dev Drive 提示 | 全部 | Defender 的那一份 | 无 | 低 | 产品 + 文档 | 阶段 1 | +| F1 关闭后保留 clangd | S10 | — | — | — | — | 否决（不符合习惯） | +| G1 先准备上次的文件 | S1 S10 | 与编辑器启动时间重叠 | 无 | 低 | 产品 | 阶段 2 | +| G2 BMI 预读 | S1 | 视数据 | 无 | 低 | 产品 | 阶段 2，先度量 | + +## 5. 计划 + +所有阶段在一个版本里交付：产品 0.0.14，引擎 mcppls-clangd 0.0.15。按惯例单 PR，以 commit 作为检查点；引擎 PR 先由产品 CI 通过 `engine-candidate.json` 测过再合入。 + +### 阶段 0：度量（先做，决定 A2、B3、B4、C1、D5、G2 的取舍和参数） + +1. **引擎的 cost 行**：慢的补全（超过 200 ms）在 info 级别说明时间花在哪，格式与 0026 的 "Prerequisites of" 一致：`Completion in : 412 ms (scan replay 40 ms over 380 inputs, module validation 280 ms over 25 BMIs / 1340 inputs, wait 0 ms, sema 90 ms)`。文档更新和 preamble 构建各加一行。 +2. **VFS 操作计数**：用一个计数用的 VFS 包装 clangd 的真实文件系统，统计每个操作的 stat、open、目录遍历次数，**按 2.6 节的三类来源分开**，写进上面的 cost 行。B2 的收益直接从这里读出来。 +3. **Windows 大项目 fixture `ux-modules-windows-large`**：在 prepare 阶段用脚本生成，仓库里只存生成器。30 个模块单元，全局模块片段都 include `` 和几个项目头，依赖链深 4，`main.cpp` import 25 个模块（贴近探针里那个文件），另有一个经 mcpp 依赖引入的第三方库头。场景： + - S10 暖启动：第一次跳转、第一次补全； + - S2 空闲补全； + - S3 逐键补全（新的 typing 模式：clangd 返回 incomplete 时每个字符都重新请求，与 VS Code 一致）； + - S5 在模块单元里打字（U16 式）； + - S8 编辑 `#include` 区； + - S9 在 4 个文件之间切换。 +4. **Windows runner**：`windows-2022` 与 `windows-2025` 各跑一次（B3 需要后者），另加一次 Dev Drive 的 A/B（E1）。 +5. **摘要**：conformance 的 measure JSON 按场景写出 p50/p95，CI 摘要分平台列表，不用下载大文件。 + +### 阶段 1：每次请求和每次按键 + +A1、A2、B1、B2a、B2b、C1（A/B 后定默认值）、E1。 + +### 阶段 2：暖启动和 Windows stat + +D1、D2、D3、B3、C2、G1，以及度量后决定的 D5、G2。 + +### 阶段 3：视数据而定 + +B4、D4；C3 是否立项。 + +## 6. 验收预算（Windows CI，`windows-2022`） + +| 场景 | 现在 | 目标 | +|---|---|---| +| U3 空闲补全 p95（小项目） | 0.67 s | ≤ 0.35 s | +| S3 打一个标识符时，每个词发给 clangd 的补全请求（按 VS Code 的方式：回答在路上时不再请求，回答不完整时下一键再请求） | 每个字符一次 | ≤ 2.0 次（冷启动时模块还在构建，回答会标为不完整，本地实测 1.6；构建好之后约 1.0）；其中因列表不完整的重请求只报告、不单设门槛（见 §11） | +| U5 在导入方里打字 p95 | 1.00 s | ≤ 0.6 s | +| U16 在模块单元里打字 p95 | 0.76 s | ≤ 0.5 s | +| `timing` 暖启动首次跳转 | 2.11 s | ≤ 1.2 s | +| 大 fixture S10：25 个前置模块全部复用时，这个文件的前置模块耗时 | 待量（探针上 28 s） | ≤ 5 s | +| 大 fixture S2：每次补全的模块校验 | 待量（探针上约 1.2 s） | ≤ 0.2 s | +| Linux、macOS | — | 不退步（各项 ±10% 以内） | + +预算的具体值在阶段 0 之后按实测确定一次，之后不再放宽。 + +## 7. 多视角自审 + +逐个角度检查方案，问题和据此做的修改如下（已并入上文）。 + +| 角度 | 检查的问题 | 结论与修改 | +|---|---|---| +| 正确性 | A1 本地过滤后，补全项的插入范围还对吗？ | clangd 给的 `textEdit` 范围只覆盖缓存时的前缀。已规定每次本地答案都把范围改写成"词的起点到当前光标"，并只在"每次变化都只是光标处的标识符字符"时才用本地答案（A1 第 2 条） | +| 正确性 | B2b 会不会把用户自己在改的代码当成包？ | 只认 mcpp/xlings 仓库下的 `<包>/<版本>/`。路径依赖、`[patch]` 本地目录、mcpp 构建生成的目录都不在仓库下，按项目文件处理（B2b"哪些路径算包"） | +| 正确性 | mcpp 原地修整包（`.mcpp-fixup.json` 晚于安装）怎么办？ | 包版本身份包含 fixup 标记；mcpp 回答里有安装类 effects 时整体失效（B2b"失效"） | +| 正确性 | 持久化的条目会不会让旧结论被直接采信？ | 不会。D1、D2 读回后都照样核对，核对规则与内存缓存相同 | +| 稳定性 | 持久化文件损坏、并发写、引擎中途崩溃 | 写临时文件再改名；只由持有工作区租约的实例写；读不懂就丢弃；异步读入不阻塞请求（D 共同约定） | +| 稳定性 | A1 的缓存会不会无限增长 | 每个打开的文档最多一份列表，文档关闭、词结束即释放 | +| 优雅简洁 | 新设置是否太多 | 只新增一个用户设置 `mcppls.engine.preambleStorage`。A1 只有命令行退出开关；B2 的仓库位置由 mcppls 自己知道，不需要用户配置 | +| 优雅简洁 | 是否有可删的方案 | B4、D4 放到阶段 3、视数据而定；B5、B6、F1 不做。A2 只是 A1 的一个参数，度量不支持就保持 100 | +| 用户体验 | 本地答案会不会让用户困惑 | 界面上不区分（已决定）；日志和 report 记录本地答案的比例 | +| 用户体验 | E1 的提示会不会吓人或打扰 | 不弹窗，只在 report 和概览的环境信息里写一行；明确说明不会改安全设置 | +| 兼容性 | 旧引擎、外部 clangd | 新引擎参数（`--managed-input-root`、持久化目录、`--max-retained-asts`）只传给声明了相应开关的引擎；外部 clangd 不受影响。A1 不依赖引擎版本 | +| 跨平台 | 哪些只在 Windows 上生效 | B3、C1 默认值、E1 只在 Windows；A1、B1、B2、D 组、G 组各平台都生效（Linux、macOS 预算要求不退步） | +| 一致性 | 日志和规范 | 新的 cost 行沿用 "Prerequisites of" 的格式；report 新字段写进 S3 规范和 traceability；`docs/50-troubleshooting.md` 的"What the log says"、`docs/30-settings.md` 及中文版同步更新；CHANGELOG | +| 无感升级 | 升级后第一次启动 | 没有持久化文件时行为与今天相同；格式不匹配就整体丢弃，用户不需要做任何事。持久化文件计入缓存预算，"重置缓存"会清掉 | +| 可观察性 | 出了问题能否从日志定位 | 启动时一行说明持久化缓存读入了多少条、多少条失效；B2 识别出的仓库在版本行里列出；每次慢补全的 cost 行按三类来源给出操作次数；report 给出本地答案比例和每个词的 clangd 请求数 | +| 安全与隐私 | 持久化内容与诊断包 | 持久化文件只含路径与文件身份，留在本地缓存目录；诊断包不收录它们（或按现有的"隐藏项目路径"规则处理）；不涉及任何网络 | +| 可测试性 | 每项怎么证明 | A1：范围改写与失效条件的单元测试，conformance 逐键模式，对比本地答案与 clangd 前 20 项。B1/B2：引擎单元测试，`module_preamble_mode` 必须照常通过，新增 e2e"包标记变了就看到包头的改动"。D1/D2：读写往返与损坏文件的单元测试，暖启动 e2e。G1：conformance 暖启动场景下，第一个请求到来前已开始准备 | +| 依赖外部约定 | B2b 依赖 mcpp 的安装模型 | 与 mcpp 确认并写进它的文档：版本目录安装后只由 mcpp/xlings 改动，改动会更新标记文件。长期更好的方式是 mcpp 在描述回答里直接给出包根和安装身份（见 §9） | + +## 8. 风险与已否决的方案 + +- **已否决**： + - B6（UP-34，补全看不到编辑器外的项目头改动）； + - F1（编辑器关闭后保留 clangd，不符合习惯）； + - 以任何形式自动添加 Defender 排除，或要求管理员权限。 +- **风险**： + - A1 的排序与 clangd 直接回答有细微差别。缓解：保留 `sortText` 和 `score`，最终排序交给编辑器。 + - B2b 依赖 mcpp 的安装模型。缓解：标记文件身份加上安装 effects 两道失效；与 mcpp 确认这条约定。 + - D1、D2 文件损坏或格式变化。缓解：带版本，坏了就丢弃，结果仍然正确。 + - B3 依赖新的系统 API。缓解：运行时探测，保留回退路径，并在两种 runner 上测试。 + +## 9. 决定与待定 + +**已决定**： + +1. 系统目录与 mcpp 安装的包分开处理：系统目录每会话核对一次（B2a），mcpp/xlings 管理的包按包版本核对（B2b）。 +2. C1 在 Windows 上默认开启的门槛是物理内存 16 GB。 +3. A1 的本地答案在界面上不区分，只写进日志和 report。 +4. 编辑器关闭后不保留 clangd（F1 否决），改用 D 组和 G 组优化重开，C3 作为之后的候选。 + +5. B2b 的包身份用标记文件（`.mcpp_ok`、`.xpkg-install.json`、`.mcpp-fixup.json`），不改 mcpp。原则是尽量解耦，只有像 emit build-database 这样必须两边联动的才配合开发。 +6. G1 记录最近活跃的 5 个文件，只记当前工作区。 + +**待定**：无。 + +## 10. 任务拆分与依赖关系 + +版本：产品 **0.0.14**，引擎 **mcppls-clangd 0.0.15**（`23.1.0-mcppls.3`）。产品单 PR、引擎单 PR，各自以 commit 作为检查点。产品 PR 先用引擎 PR 的构建（`engine-candidate.json`）跑通 CI，引擎发布后再锁定。 + +### 10.1 引擎与产品之间的约定（先定下来，两边就能并行） + +| 引擎提供 | 声明的 feature | 产品怎么用 | +|---|---|---| +| `--managed-input-root=<目录>`（可重复）：`<目录>/<包>/<版本>/` 按包版本核对（B2b） | `managed-input-roots` | 传入存在的 mcpp、xlings 包仓库 | +| 系统与外部工具链的输入每会话核对一次（B2a） | 同上 | 无需参数 | +| `--module-memo-dir=<目录>`：校验缓存与扫描缓存持久化（D1、D2） | `persistent-module-memos` | 传入工作区缓存目录下的 `clangd-memos/` | +| `--max-retained-asts=`（C2） | `retained-asts-option` | 物理内存 ≥ 16 GB 时传 6 | +| 慢补全、文档更新、preamble 构建的 cost 行，按三类来源计数（阶段 0） | `module-cost-lines` | 日志转出，report 汇总 | +| Windows 快速 stat（B3），启动时一行说明是否启用 | 无（不改语义） | 版本行旁边转出 | + +产品只把参数传给声明了相应 feature 的引擎，外部 clangd 什么都不传；`--limit-results`（A2）和 `--pch-storage`（C1）是 clangd 原有的参数，对外部 clangd 也成立，但只在用户没有自己给出时才传。 + +### 10.2 任务与依赖 + +| 编号 | 任务 | 位置 | 依赖 | 检查点 | +|---|---|---|---|---| +| E0 | cost 行与按来源的操作计数 | 引擎 | — | 补丁 0030 | +| E1 | B1 锁外 stat，B2a、B2b 按来源核对（扫描重放与校验缓存） | 引擎 | E0 的来源分类器 | 补丁 0031 | +| E2 | D1 持久化校验缓存 | 引擎 | E1（条目带包身份） | 补丁 0032 | +| E3 | D2 持久化扫描缓存 | 引擎 | E1 | 补丁 0033 | +| E4 | C2 `--max-retained-asts` | 引擎 | — | 补丁 0034 | +| E5 | B3 Windows 快速 stat（运行时探测，首次使用时与句柄路径对拍，不一致就关闭） | LLVM Support | — | 补丁 0035 | +| E6 | feature 声明、canary、e2e、账本与版本 | 引擎 CI | E0–E5 | 引擎 PR | +| P0 | conformance：逐键请求模式（按 VS Code 与"每键都请求"两种客户端）、大 fixture 生成器、度量摘要 | 产品 | — | commit | +| P1 | A2 结果上限、A1 完整列表本地作答 | 产品 | P0（度量定值） | commit | +| P2 | C1 `mcppls.engine.preambleStorage`，物理内存查询 | 产品 | — | commit | +| P3 | E1 Dev Drive 探测与提示 | 产品 | — | commit | +| P4 | G1 记录最近活跃文件、后台预打开与接管 | 产品 | — | commit | +| P5 | 按 feature 传入引擎参数、日志与 report 汇总 | 产品 | 10.1 的约定 | commit | +| P6 | 规范与文档：S3 report 字段、设置文档、"What the log says"、中文版、CHANGELOG、版本 0.0.14 | 产品 | P1–P5 | commit | +| P7 | 产品 CI 用引擎候选构建跑通；Windows 探针按场景给出分文件的摘要 | 产品 + 探针 | E6、P0–P6 | 候选 CI | +| P8 | 引擎发布、锁定、合入、发布产品、本地验证 | 两边 | P7 | 发布 | + +可以并行的：E0–E5 之间只有 E1→E2/E3 的依赖；产品的 P0–P5 与引擎完全并行，只靠 10.1 的约定对接。 + +D3（磁盘模块源按前缀扫描）、D5（后台索引延后）、G2（BMI 预读）、B4、D4、C3 不在本期：前三项要等阶段 0 的数据说明它们值得做，后三项见上文。 + +### 10.3 每个任务都要满足的约束 + +| 角度 | 约束 | +|---|---| +| 架构 | 引擎只按路径来源和身份做判断，不知道 mcpp；产品只传路径与目录，不重新实现引擎的核对 | +| 稳定性 | 持久化文件损坏、格式不符、写到一半崩溃，都只会导致回到完整核对；G1 的后台文档受现有的后台单元上限和淘汰规则约束 | +| 优雅简洁 | 只新增一个用户设置（`preambleStorage`）；其余都有合理默认值，按 feature 自动启用 | +| 用户体验 | 界面上没有新的弹窗；Dev Drive 提示只在 report 和概览的环境信息里出现一次 | +| 兼容性 | 旧引擎、外部 clangd 行为不变；没有持久化文件时行为与今天相同 | +| 跨平台 | B3、C1 默认值、E1 只在 Windows；其余各平台生效，Linux、macOS 的 CI 预算不退步 | +| 一致性 | 新日志行沿用 "Prerequisites of" 的格式；report 字段写进 S3 规范与 traceability；中英文文档同步 | +| 无感升级 | 用户不需要做任何事；持久化文件计入缓存预算，"重置缓存"会清掉 | + + +## 11. 实施状态与实施中的修正 + +交付：产品 0.0.14（本 PR）、引擎 mcppls-clangd 0.0.15（补丁 0030 UP-35、0031 UP-36、0032 UP-37，`23.1.0-mcppls.3`）。 + +| 项 | 状态 | 说明 | +|---|---|---| +| E0 cost 行 | 完成 | `Completion in : ms (module input checks … one by one, … by … package checks, … system already checked; waited …)`，200 ms 以上才打；`Module memos:`、`Module inputs:` 进入默认日志 | +| B1 | 完成（缩小） | 只做锁外 stat。两次 stat 保留：`IsModuleFileUpToDate` 的调用点不全是不带 stat 缓存的文件系统 | +| B2a、B2b | 完成（扩大） | 分层判定：包仓库 → 显式系统目录 → 用户目录（`-I`、`-iquote`、命令目录、源文件目录）→ 工具链根（项目在其中时不算）→ Windows SDK 与 Visual Studio 的头（路径含 `Windows Kits`、`Microsoft Visual Studio`）、`INCLUDE`/`EXTERNAL_INCLUDE` → 其余为项目。最后两类是实施中补上的：编辑器启动服务端时没有开发环境，MSVC 的头由驱动自己找到，既不在命令行也不在环境变量里。工具链根一开始包含了项目所在目录（Ubuntu 20.04 floor 检查里项目就在 clang 的构建目录下），由此改成分层 | +| B3、B4 | 不做 | LLVM 23 在 Windows 上的 UniqueID 是规范 NT 路径（`GetFinalPathNameByHandleW`）的哈希，不是文件索引：不开句柄就算不出同一个 ID，快速路径与句柄路径的 ID 会不一致，而 clang 依赖它判断是不是同一个文件。按目录批量也一样 | +| D1、D2 | 完成 | `--module-memo-dir`；读回只是候选，判定照样核对、扫描照样重放；扫描的持久化键用只含驱动会读的环境变量的摘要（完整环境每个会话都不同） | +| C1、C2 | 完成 | `mcppls.engine.preambleStorage`；`--max-retained-asts=6`（16 GiB 以上）。物理内存原先只在 Linux 上读得到，Windows 上补上了 | +| A2、A1 | 完成 | `--limit-results=1000`；同一个词的再次请求，在上次回答完整且词外文本完全相同时本地作答。`ux-modules-qt` U17 本地实测：一个词的请求 127 → 79 次（-38%），其中因列表不完整的只有 13 → 10，减少的主要是"上一次还没回来又请求"，因为请求少了、回答更快 | +| E1 | 完成 | Dev Drive 探测（`FSCTL_QUERY_PERSISTENT_VOLUME_STATE`），日志、`about.cacheDevDrive`（S3-4-34）、概览；CI 上的 Dev Drive A/B 没有做（要引入第三方 action 创建 VHD），只作为文档里的建议 | +| G1 | 完成 | 最近 5 个文件；后台预打开，编辑器打开时接管（同文本沿用诊断，不同文本发一次全文更新），两分钟未用即关闭 | +| 阶段 0 度量 | 完成 | conformance 的 `reask`（VS Code 与"每键都请求"两种客户端），U17（Qt、``），`ux-modules-windows-large`（生成，MSVC 构建）在发布检查的性能 job 里冷、暖各一次，每轮的检查、cost 行与日志分文件 | +| D3、D5、G2、C3 | 延后 | 等 cost 行与大 fixture 的 Windows 数据说明它们值得做 | +| H1 reduced BMI（新增） | 完成 | E0 的 cost 行在大 fixture 上说明了剩下的时间不在输入检查：0030 之后检查只剩约 10 ms（831 个系统输入本会话已查过），`::` 后的补全仍要约 2 s，`m29::` 约 0.4 s。根因在 clang：reduced BMI 会省略全局模块片段里没被用到的声明，namespace 里的会延后到所有声明写完再记录，`extern "C" { }`/`extern "C++" { }` 块却没有，块里一个声明被用到，整个块的声明就全被写进 BMI；``（以及 C 库头文件的 `__BEGIN_DECLS`）几乎全在这种块里。补全对每个导入的模块都会加载其可见声明，于是导入 25 个这样的模块就加载 25 份。补丁 0032 让这种块与 namespace 一样延后。Linux 上同形状的复现（2 万个 `extern "C"` 声明的头文件，30 个模块）：BMI 9.5 MB → 20 KB，`::` 补全 2250 → 277 ms，30 个模块的构建加一次检查 90 → 7.5 s | + +仍未解决、留给下一轮的（来自 Windows 上一个真实中型项目的探针数据，补全每次仍要数秒，语义答案赶不上 1 s 预算）： + +- 项目目录里由第三方包管理器装出来的头文件树（例如 vcpkg 的 `vcpkg_installed//include`，以 `-I` 传入）按项目文件逐个检查，一次补全一万五千多个，约 1.5 s。这类树也是由包管理器整体改写的（vcpkg 有 `vcpkg/status`），可以像 B2b 那样按它的状态文件检查，但它不是 mcpp/xlings 的规范模型，要单独设计。 +- 一个文件导入十几个封装大型第三方库（boost、abseil 等）的模块时，每次补全都要重新加载这些 BMI（UP-25 的"每个请求重载模块上下文"），检查之外还有约 3 s。要么让 clangd 在请求之间保留模块上下文，要么减小这些 BMI；都在引擎里，规模比本轮任何一项都大。 + +实施中顺带发现并修正的问题： + +- nlohmann `Json x { y }` 在本项目使用的编译器（clang 22，CWG 2137）下会构造只含一个元素的数组。除了 G1 接管时把诊断包成 `[[…]]` 导致服务端异常退出之外，又找出三处既有问题：C-2 用 `[id]` 去放开或取消超时的补全请求、`mcppls.sweepCache` 按类别清理时一个类别都不会执行、缓存报告里实例的 `at` 成了数组。全部改为 `=` 初始化。 +- 原先给"因列表不完整的重请求"单设的门槛（每词 ≤ 0.6）去掉了，只保留每词总请求数 ≤ 2.0。0032 之后 `ux-modules-qt` 的补全 p95 从 0.20 s 降到 0.13 s，回答在路上时敲下的键从 42 个降到 16 个，于是"上一次不完整、下一键再问"从 0.55 升到 0.74 次/词，总数仍是 1.74：这个分量随回答变快而变大，单独设门槛会把变快判成退步。它照样在报告里。 +- `test_workarounds`、`test_engineidentity` 的 `main` 没有 `return report();`，断言失败从不让测试失败；补上后修正了那条把打包引擎版本写死的断言。 diff --git a/.github/workflows/release-checks.yml b/.github/workflows/release-checks.yml index 71301dd9..19876d34 100644 --- a/.github/workflows/release-checks.yml +++ b/.github/workflows/release-checks.yml @@ -184,6 +184,37 @@ jobs: done "stage/mcppls-devtools${{ matrix.exe }}" measure summary "$results" --max-cold 12 --max-warm 5 \ | tee "$results/summary.md" >> "$GITHUB_STEP_SUMMARY" + # 0.0.14 Windows plan, phase 0: a project the size where module checks cost (thirty module units with + # , built by MSVC), opened cold and then warm with the same cache. Each run's measures and the + # server's log are files of their own, and the step summary says what the two runs measured. + - name: The MSVC developer environment, for the large fixture's own build + if: runner.os == 'Windows' + shell: pwsh + run: | + $vs = & "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath + Set-Content -Path "$env:RUNNER_TEMP\capture-msvc.cmd" -Value "@call `"$vs\VC\Auxiliary\Build\vcvars64.bat`" >nul`r`n@set" + cmd /d /c "$env:RUNNER_TEMP\capture-msvc.cmd" | Out-File -Encoding utf8 "$env:RUNNER_TEMP\msvc.env" + - name: A large module project, cold and warm + if: runner.os == 'Windows' + run: | + set -euo pipefail + large="$RUNNER_TEMP/timing/large" + mkdir -p "$large" + for start in cold warm; do + warm=(); [ "$start" = warm ] && warm=(--expect-warm) + status=0 + "stage/mcppls-conformance${{ matrix.exe }}" run --server "payload/bin/mcppls${{ matrix.exe }}" --payload payload \ + --fixture conformance/fixtures/ux-modules-windows-large --msvc-env "$RUNNER_TEMP/msvc.env" \ + --workspace-dir "$RUNNER_TEMP/large/workspace" --cache-dir "$RUNNER_TEMP/large/cache" \ + --timeout 900 --measure "$large/$start.json" --verbose ${warm[@]+"${warm[@]}"} > "$large/$start.log" 2>&1 || status=$? + grep -E '^(PASS|FAIL) |failure\(s\)' "$large/$start.log" > "$large/$start-checks.txt" || true + grep -E "Prerequisites of |Completion in .*: [0-9]+ ms|Module memos: |Module inputs: |ahead of the editor|ready in clangd|startup \(" "$large/$start.log" \ + | sed -E 's/^.*\[(info|warning)\] //' > "$large/$start-costs.txt" || true + { echo "### Large module project, $start"; echo; echo '```'; cat "$large/$start-checks.txt"; echo '```'; } >> "$large/summary.md" + [ "$status" = 0 ] || echo "::warning::the large fixture's $start run failed: see prerelease-timing-${{ matrix.platform }}/large/$start-checks.txt" + done + cat "$large/summary.md" >> "$GITHUB_STEP_SUMMARY" + ! grep -q '^FAIL ' "$large/cold-checks.txt" "$large/warm-checks.txt" - uses: actions/upload-artifact@v7 if: always() with: diff --git a/CHANGELOG.md b/CHANGELOG.md index e64aba77..8a85f275 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,52 @@ release's notes are that section. Versions are three-part semantic versions, `MAJOR.MINOR.PATCH`, and every editor plugin carries the product version unchanged. +## [0.0.14] — 2026-10-13 + +The bundled engine is [mcppls-clangd 0.0.15](https://github.com/Sunrisepeak/mcppls-clangd/releases/tag/v0.0.15) +(`23.1.0-mcppls.3`): 32 patches, three of them new. This release is about Windows: opening a project again and completion +as you type, where every check that a module is still good is a file operation that opens the file. + +### Completion as you type + +- **A module that includes `` no longer carries all of it.** A module's BMI kept every declaration of an + `extern "C"` block its global module fragment includes, used or not, and nearly all of `` -- and of a C + library's headers -- is in such blocks. The engine leaves the unused ones out, as it already did for a namespace's: a + module unit that includes `` builds a BMI of kilobytes instead of megabytes, and a completion in a file + that imports many of them reads only what they use (`::` in a file importing 25 of them: 2.2 s before, 0.3 s now). +- **A completion list comes whole.** clangd gives up to 1000 items instead of 100, so a list whose matches fit is + complete and the editor filters it as you type instead of asking again at every key. A client that asks at every key + anyway is answered from the word's complete list at once. In a file whose modules include Qt, the requests while a + word is typed went down by about two fifths. +- **Checking a module costs less where its headers come from a package or the system.** A header of an mcpp or xlings + package version -- the toolchain's own, such as llvm-mingw's ``, or a library's -- is checked by the + version's install markers instead of one by one; system headers, the Windows SDK's and Visual Studio's included, once + per session, as clang itself skips them. Your project's files are checked at every request, as before. +- **On Windows, preambles are kept in memory** with 16 GiB or more, instead of being written to the temporary + directory and scanned there. New setting `mcppls.engine.preambleStorage` (`auto`, `disk`, `memory`). + +### Opening a project again + +- **What the last session learned is kept.** Which BMIs were up to date and what each module unit imports are written to + the workspace cache and read back by the next session, each checked again before it is used, so a second opening + checks instead of hashing every input and preprocessing every module unit again. +- **The files you last worked in are ready first.** The five files last opened or edited are opened in clangd as soon as + it starts, before the editor asks; the editor takes them over as they are. With 16 GiB or more, six ASTs of files you + switch between are kept instead of three. + +### Seeing what happens + +- A slow completion (200 ms or more) in a module importer says in the log where its time went; the log says what the + last session left and which package stores are checked by their markers. On Windows, a cache that is not on a Dev + Drive is said once, in the log and the overview (`about.cacheDevDrive`); see "Windows: a cache on a Dev Drive" in the + troubleshooting guide. mcppls changes no security setting. + +### Fixes + +- A cache sweep asked for by category (`mcppls.sweepCache` with `categories`) ran none of them. +- A completion clangd answered late was not let go of when its word was given up. +- Two test programs never reported their failures; they do now. + ## [0.0.13] — 2026-10-12 The bundled engine is [mcppls-clangd 0.0.14](https://github.com/Sunrisepeak/mcppls-clangd/releases/tag/v0.0.14) diff --git a/conformance/fixtures/ux-modules-qt/scenario.json b/conformance/fixtures/ux-modules-qt/scenario.json index 1a387ec5..9706776b 100644 --- a/conformance/fixtures/ux-modules-qt/scenario.json +++ b/conformance/fixtures/ux-modules-qt/scenario.json @@ -165,6 +165,35 @@ "diagnosticsRefresh": 6.0 } }, + { + "id": "U17-words-as-typed", + "kind": "typing", + "file": "src/main.cpp", + "seconds": 45, + "hz": 10, + "completion-every-ms": 0, + "reask": "incomplete", + "description": "0.0.14 A1, A2: completion asked as VS Code asks it while a word is typed in a file whose module units include Qt headers: the word's first character asks; the keys typed while that answer is on its way wait for it, and once it is in the next key asks again only when it was marked incomplete. With clangd's lists complete (--limit-results), a word costs one or two requests, and each answers in time.", + "scripts": [ + { + "line": 12, + "text": "QStringList names = inventory.itemNames(); QString joined = names.join(QStringLiteral(\",\")); window.setWindowTitle(joined);" + }, + { + "line": 12, + "text": "QTimer timer; timer.setInterval(100); QObject::connect(&timer, &QTimer::timeout, &app, [&] { window.showCount(inventory.rowCount()); });" + } + ], + "budget": { + "firstOpenSeconds": 120, + "p95": 1.0, + "max": 3.0, + "timeouts": 0, + "restarts": 0, + "filesSetAside": 0, + "askedAsTypedPerWord": 2.0 + } + }, { "id": "U16-module-autosave", "kind": "typing", diff --git a/conformance/fixtures/ux-modules-windows-large/CMakeLists.txt b/conformance/fixtures/ux-modules-windows-large/CMakeLists.txt new file mode 100644 index 00000000..7118b3d7 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/CMakeLists.txt @@ -0,0 +1,13 @@ +cmake_minimum_required(VERSION 3.28) +project(bigwin LANGUAGES CXX) + +set(CMAKE_CXX_STANDARD 20) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_library(big) +target_sources(big PUBLIC FILE_SET CXX_MODULES FILES src/m00.cppm src/m01.cppm src/m02.cppm src/m03.cppm src/m04.cppm src/m05.cppm src/m06.cppm src/m07.cppm src/m08.cppm src/m09.cppm src/m10.cppm src/m11.cppm src/m12.cppm src/m13.cppm src/m14.cppm src/m15.cppm src/m16.cppm src/m17.cppm src/m18.cppm src/m19.cppm src/m20.cppm src/m21.cppm src/m22.cppm src/m23.cppm src/m24.cppm src/m25.cppm src/m26.cppm src/m27.cppm src/m28.cppm src/m29.cppm) +target_compile_definitions(big PUBLIC UNICODE _UNICODE NOMINMAX WIN32_LEAN_AND_MEAN) + +add_executable(app src/main.cpp) +target_link_libraries(app PRIVATE big) diff --git a/conformance/fixtures/ux-modules-windows-large/scenario.json b/conformance/fixtures/ux-modules-windows-large/scenario.json new file mode 100644 index 00000000..874ec03b --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/scenario.json @@ -0,0 +1,131 @@ +{ + "name": "ux-modules-windows-large", + "description": "0.0.14 Windows plan, phase 0: a Windows program the size where module checks cost -- thirty module units whose global module fragments include , in chains four deep, and a main.cpp that includes and imports twenty-five of them -- built by CMake with MSVC (cl.exe) and Ninja. Run cold and then warm with the same cache directory (the release checks' performance job): first navigation, idle latency, completion asked as a word is typed, and on the warm run the module files reused. Budgets are the first measurements' with room; they hold what the plan's work must not undo.", + "prepare-environment": "msvc", + "prepare": [ + [ + "cmake", + "-S", + ".", + "-B", + "build", + "-G", + "Ninja", + "-DCMAKE_CXX_COMPILER=cl", + "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON" + ], + [ + "cmake", + "--build", + "build" + ] + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "source": "cmake", + "profile-kind": "build-toolchain", + "state": "ready", + "timeout": 600 + }, + { + "id": "C1", + "kind": "diagnostics-empty", + "file": "src/main.cpp", + "timeout": 900 + }, + { + "id": "T1-definition-into-module", + "kind": "definition", + "file": "src/main.cpp", + "at": [ + 29, + 24 + ], + "expect": "src/m29.cppm", + "timeout": 120 + }, + { + "id": "W1-modules-reused", + "kind": "module-cache-reused", + "module": "big.m29" + }, + { + "id": "U3-idle", + "kind": "latency", + "phase": "idle", + "rounds": 20, + "requests": [ + { + "kind": "completion", + "name": "module-member", + "file": "src/main.cpp", + "at": [ + 29, + 23 + ] + }, + { + "kind": "completion", + "name": "windows-api", + "file": "src/main.cpp", + "at": [ + 32, + 6 + ] + }, + { + "kind": "hover", + "file": "src/main.cpp", + "at": [ + 29, + 24 + ] + }, + { + "kind": "definition", + "file": "src/main.cpp", + "at": [ + 29, + 24 + ], + "expect": "src/m29.cppm" + } + ], + "budget": { + "completion.p95": 3.0, + "hover.p95": 1.5, + "definition.p95": 2.0, + "engineShare": 0.8 + } + }, + { + "id": "U17-words-as-typed", + "kind": "typing", + "file": "src/main.cpp", + "seconds": 45, + "hz": 10, + "completion-every-ms": 0, + "reask": "incomplete", + "scripts": [ + { + "line": 33, + "text": "HANDLE handle = CreateFileW(nullptr, GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, nullptr);" + }, + { + "line": 33, + "text": "total += m28::value() + m27::value(); m26::Window26 other; if (other.visible()) total += m25::ticks() % 3;" + } + ], + "budget": { + "firstOpenSeconds": 300, + "p95": 2.0, + "max": 5.0, + "timeouts": 0, + "restarts": 0, + "filesSetAside": 0 + } + } + ] +} diff --git a/conformance/fixtures/ux-modules-windows-large/src/m00.cppm b/conformance/fixtures/ux-modules-windows-large/src/m00.cppm new file mode 100644 index 00000000..46e899f1 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m00.cppm @@ -0,0 +1,9 @@ +module; +#include +export module big.m00; + +export namespace m00 { +inline DWORD value() { return ::GetCurrentProcessId() % 7; } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window00 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m00 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m01.cppm b/conformance/fixtures/ux-modules-windows-large/src/m01.cppm new file mode 100644 index 00000000..cfda4712 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m01.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m01; +import big.m00; + +export namespace m01 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m00::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window01 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m01 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m02.cppm b/conformance/fixtures/ux-modules-windows-large/src/m02.cppm new file mode 100644 index 00000000..ca002799 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m02.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m02; +import big.m01; + +export namespace m02 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m01::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window02 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m02 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m03.cppm b/conformance/fixtures/ux-modules-windows-large/src/m03.cppm new file mode 100644 index 00000000..179ed1de --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m03.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m03; +import big.m02; + +export namespace m03 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m02::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window03 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m03 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m04.cppm b/conformance/fixtures/ux-modules-windows-large/src/m04.cppm new file mode 100644 index 00000000..b9528f23 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m04.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m04; +import big.m03; + +export namespace m04 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m03::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window04 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m04 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m05.cppm b/conformance/fixtures/ux-modules-windows-large/src/m05.cppm new file mode 100644 index 00000000..25d2474a --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m05.cppm @@ -0,0 +1,9 @@ +module; +#include +export module big.m05; + +export namespace m05 { +inline DWORD value() { return ::GetCurrentProcessId() % 7; } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window05 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m05 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m06.cppm b/conformance/fixtures/ux-modules-windows-large/src/m06.cppm new file mode 100644 index 00000000..71483f51 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m06.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m06; +import big.m05; + +export namespace m06 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m05::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window06 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m06 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m07.cppm b/conformance/fixtures/ux-modules-windows-large/src/m07.cppm new file mode 100644 index 00000000..a6307263 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m07.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m07; +import big.m06; + +export namespace m07 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m06::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window07 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m07 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m08.cppm b/conformance/fixtures/ux-modules-windows-large/src/m08.cppm new file mode 100644 index 00000000..ac0285c1 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m08.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m08; +import big.m07; + +export namespace m08 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m07::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window08 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m08 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m09.cppm b/conformance/fixtures/ux-modules-windows-large/src/m09.cppm new file mode 100644 index 00000000..a666296e --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m09.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m09; +import big.m08; + +export namespace m09 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m08::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window09 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m09 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m10.cppm b/conformance/fixtures/ux-modules-windows-large/src/m10.cppm new file mode 100644 index 00000000..05079f53 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m10.cppm @@ -0,0 +1,9 @@ +module; +#include +export module big.m10; + +export namespace m10 { +inline DWORD value() { return ::GetCurrentProcessId() % 7; } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window10 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m10 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m11.cppm b/conformance/fixtures/ux-modules-windows-large/src/m11.cppm new file mode 100644 index 00000000..65717551 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m11.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m11; +import big.m10; + +export namespace m11 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m10::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window11 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m11 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m12.cppm b/conformance/fixtures/ux-modules-windows-large/src/m12.cppm new file mode 100644 index 00000000..295008d6 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m12.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m12; +import big.m11; + +export namespace m12 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m11::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window12 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m12 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m13.cppm b/conformance/fixtures/ux-modules-windows-large/src/m13.cppm new file mode 100644 index 00000000..75fe4d89 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m13.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m13; +import big.m12; + +export namespace m13 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m12::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window13 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m13 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m14.cppm b/conformance/fixtures/ux-modules-windows-large/src/m14.cppm new file mode 100644 index 00000000..1daca3b8 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m14.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m14; +import big.m13; + +export namespace m14 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m13::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window14 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m14 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m15.cppm b/conformance/fixtures/ux-modules-windows-large/src/m15.cppm new file mode 100644 index 00000000..841715cb --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m15.cppm @@ -0,0 +1,9 @@ +module; +#include +export module big.m15; + +export namespace m15 { +inline DWORD value() { return ::GetCurrentProcessId() % 7; } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window15 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m15 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m16.cppm b/conformance/fixtures/ux-modules-windows-large/src/m16.cppm new file mode 100644 index 00000000..744cca88 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m16.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m16; +import big.m15; + +export namespace m16 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m15::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window16 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m16 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m17.cppm b/conformance/fixtures/ux-modules-windows-large/src/m17.cppm new file mode 100644 index 00000000..f7dbeb11 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m17.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m17; +import big.m16; + +export namespace m17 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m16::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window17 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m17 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m18.cppm b/conformance/fixtures/ux-modules-windows-large/src/m18.cppm new file mode 100644 index 00000000..f8939c26 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m18.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m18; +import big.m17; + +export namespace m18 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m17::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window18 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m18 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m19.cppm b/conformance/fixtures/ux-modules-windows-large/src/m19.cppm new file mode 100644 index 00000000..77786d1b --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m19.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m19; +import big.m18; + +export namespace m19 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m18::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window19 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m19 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m20.cppm b/conformance/fixtures/ux-modules-windows-large/src/m20.cppm new file mode 100644 index 00000000..771f4237 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m20.cppm @@ -0,0 +1,9 @@ +module; +#include +export module big.m20; + +export namespace m20 { +inline DWORD value() { return ::GetCurrentProcessId() % 7; } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window20 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m20 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m21.cppm b/conformance/fixtures/ux-modules-windows-large/src/m21.cppm new file mode 100644 index 00000000..4581ddcf --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m21.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m21; +import big.m20; + +export namespace m21 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m20::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window21 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m21 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m22.cppm b/conformance/fixtures/ux-modules-windows-large/src/m22.cppm new file mode 100644 index 00000000..55eaba22 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m22.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m22; +import big.m21; + +export namespace m22 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m21::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window22 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m22 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m23.cppm b/conformance/fixtures/ux-modules-windows-large/src/m23.cppm new file mode 100644 index 00000000..c96d8ba6 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m23.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m23; +import big.m22; + +export namespace m23 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m22::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window23 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m23 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m24.cppm b/conformance/fixtures/ux-modules-windows-large/src/m24.cppm new file mode 100644 index 00000000..85daa702 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m24.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m24; +import big.m23; + +export namespace m24 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m23::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window24 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m24 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m25.cppm b/conformance/fixtures/ux-modules-windows-large/src/m25.cppm new file mode 100644 index 00000000..fbe2f28a --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m25.cppm @@ -0,0 +1,9 @@ +module; +#include +export module big.m25; + +export namespace m25 { +inline DWORD value() { return ::GetCurrentProcessId() % 7; } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window25 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m25 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m26.cppm b/conformance/fixtures/ux-modules-windows-large/src/m26.cppm new file mode 100644 index 00000000..0f148290 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m26.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m26; +import big.m25; + +export namespace m26 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m25::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window26 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m26 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m27.cppm b/conformance/fixtures/ux-modules-windows-large/src/m27.cppm new file mode 100644 index 00000000..0d2d3bc6 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m27.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m27; +import big.m26; + +export namespace m27 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m26::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window27 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m27 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m28.cppm b/conformance/fixtures/ux-modules-windows-large/src/m28.cppm new file mode 100644 index 00000000..001b21d2 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m28.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m28; +import big.m27; + +export namespace m28 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m27::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window28 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m28 diff --git a/conformance/fixtures/ux-modules-windows-large/src/m29.cppm b/conformance/fixtures/ux-modules-windows-large/src/m29.cppm new file mode 100644 index 00000000..0807ba23 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/m29.cppm @@ -0,0 +1,10 @@ +module; +#include +export module big.m29; +import big.m28; + +export namespace m29 { +inline DWORD value() { return ::GetCurrentProcessId() % 7 + m28::value(); } +inline ULONGLONG ticks() { return ::GetTickCount64(); } +struct Window29 { HWND handle = nullptr; bool visible() const { return handle != nullptr; } }; +} // namespace m29 diff --git a/conformance/fixtures/ux-modules-windows-large/src/main.cpp b/conformance/fixtures/ux-modules-windows-large/src/main.cpp new file mode 100644 index 00000000..17929ff7 --- /dev/null +++ b/conformance/fixtures/ux-modules-windows-large/src/main.cpp @@ -0,0 +1,35 @@ +#include + +import big.m05; +import big.m06; +import big.m07; +import big.m08; +import big.m09; +import big.m10; +import big.m11; +import big.m12; +import big.m13; +import big.m14; +import big.m15; +import big.m16; +import big.m17; +import big.m18; +import big.m19; +import big.m20; +import big.m21; +import big.m22; +import big.m23; +import big.m24; +import big.m25; +import big.m26; +import big.m27; +import big.m28; +import big.m29; + +int main() { + DWORD total = m29::value(); + m29::Window29 window; + if (window.visible()) total += 1; + ::Sleep(0); + return static_cast(total); +} diff --git a/conformance/fixtures/ux-modules-windows/scenario.json b/conformance/fixtures/ux-modules-windows/scenario.json index ab48ce40..4bd2b77b 100644 --- a/conformance/fixtures/ux-modules-windows/scenario.json +++ b/conformance/fixtures/ux-modules-windows/scenario.json @@ -163,6 +163,39 @@ "diagnosticsRefresh": 4.0 } }, + { + "id": "U17-words-as-typed", + "kind": "typing", + "file": "src/main.cpp", + "seconds": 45, + "hz": 10, + "completion-every-ms": 0, + "reask": "incomplete", + "description": "0.0.14 A1, A2: completion asked as VS Code asks it while a word is typed in a file whose module units include : the word's first character asks; the keys typed while that answer is on its way wait for it, and once it is in the next key asks again only when it was marked incomplete. With clangd's lists complete (--limit-results), a word costs one or two requests, and each answers in time.", + "scripts": [ + { + "line": 11, + "text": "HANDLE handle = CreateFileW(nullptr, GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, nullptr);" + }, + { + "line": 11, + "text": "DWORD written = 0; WriteConsoleW(GetStdHandle(STD_OUTPUT_HANDLE), L\"x\", 1, &written, nullptr); CloseHandle(handle);" + }, + { + "line": 11, + "text": "SYSTEMTIME now {}; GetLocalTime(&now); OutputDebugStringW(std::to_wstring(now.wHour).c_str());" + } + ], + "budget": { + "firstOpenSeconds": 120, + "p95": 1.1, + "max": 3.0, + "timeouts": 0, + "restarts": 0, + "filesSetAside": 0, + "askedAsTypedPerWord": 2.0 + } + }, { "id": "U16-module-autosave", "kind": "typing", diff --git a/conformance/traceability.json b/conformance/traceability.json index eb8d6956..6ab0eeb4 100644 --- a/conformance/traceability.json +++ b/conformance/traceability.json @@ -508,7 +508,7 @@ "S1-11.2-7": [ { "script": "editors/vscode/src/status.ts", - "contains": "status.project.source} \u00b7 L${tier}" + "contains": "status.project.source} · L${tier}" }, { "check": "mcpp-emit/S1" @@ -2236,5 +2236,15 @@ { "check": "build-program-manual/S2-held" } + ], + "S3-4-34": [ + { + "script": "src/orchestrator/workspace.cpp", + "contains": "if (cacheOnDevDrive) about[\"cacheDevDrive\"] = *cacheOnDevDrive;" + }, + { + "script": "editors/vscode/src/cacheHub.ts", + "contains": "about?.cacheDevDrive === false" + } ] } diff --git a/docs/30-settings.md b/docs/30-settings.md index b269ba63..97edf763 100644 --- a/docs/30-settings.md +++ b/docs/30-settings.md @@ -54,6 +54,7 @@ either wrapped in a top-level `mcppls` object or not. |---|---|---|---|---|---| | `mcppls.engine.name` | `clangd`, `none` | `clangd` | `--engine` | restart | The core semantic engine. mcppls's own module engine always runs beside it; `none` means module-level features only. | | `mcppls.engine.workers` | a string | `auto` | `--engine-workers` | restart | How many files clangd builds at once (its `-j`), shared by module preparation, indexing and the files you edit; one is always kept for what you are waiting on. `auto`: one fewer than the hardware threads, between two and eight, and no more than half the memory in gigabytes. A number sets it. | +| `mcppls.engine.preambleStorage` | `auto`, `disk`, `memory` | `auto` | `--engine-preamble-storage` | restart | Where clangd keeps the preambles of the files you edit (its `--pch-storage`). `auto`: in memory on Windows with 16 GiB or more, where a preamble written to the temporary directory is scanned when written and when read; on disk elsewhere. `memory` costs about one preamble per open file (tens of MB with ``). | | `mcppls.compiler` | a string | *(empty)* | `--compiler` | reload | Use this compiler for module semantics instead of what was detected: an absolute path, a name on `PATH`, or `kit` to force the bundled semantic kit. Empty means discovered automatically. | | `mcppls.semanticKit` | `auto`, `off` | `auto` | `--semantic-kit` | reload | Whether the bundled standard library kit may be used at all: `auto`, when no compiler is found; `off`, never (without a compiler, only module-level features remain). | | `mcppls.requestTimeout` | a non-negative number of seconds | `60` | `--request-timeout` | restart | How long an engine request may take before it is answered without the engine. A request a person waits for (hover, definition, completion and the like) waits at most 30s in all, including while clangd starts or prepares its modules, and is then answered by mcppls's own engine. | diff --git a/docs/50-troubleshooting.md b/docs/50-troubleshooting.md index 5272a7eb..e13cbe70 100644 --- a/docs/50-troubleshooting.md +++ b/docs/50-troubleshooting.md @@ -38,6 +38,11 @@ lines have a fixed shape; the report's `events` carry the same facts. | `clangd (): I[…] Rebuilding module : ` | mcppls-clangd 0.0.13: why a module was built rather than reused — none for this command and prerequisites (and how many were built with others), or the input that changed | | `clangd (): I[…] Prerequisites of : modules, reused, built, failed, in (reuse checks , builds , the rest scanning and planning)` | What one file's modules cost | | `clangd (): I[…] Preamble of is not compatible: ` | Why a file's preamble was built again | +| `clangd (): I[…] Completion in : ms (module input checks : one by one, by package checks, system already checked; waited for modules)` | mcppls-clangd 0.0.15: where a completion of 200 ms or more in a module importer spent its time. Inputs under an mcpp or xlings package version are checked by the version's markers, system headers once per session, the project's files one by one | +| `clangd (): I[…] Module memos: BMI verdicts and scans kept from the last session …` | mcppls-clangd 0.0.15, at start: what the last session left to check instead of redoing it (each is checked before it is used) | +| `clangd (): I[…] Module inputs: managed package store(s) checked by their markers` | Which package stores (mcpp's, xlings') the engine checks by version | +| `the last files in use (): of them, opened in clangd ahead of the editor once it takes files` | At start: the files last worked in are opened in clangd before the editor asks; ` was ready in clangd ms before the editor opened it` says it paid off, `... is still being built there; the editor takes it as it is` that the editor came while clangd was at it, and one the editor did not open within two minutes is closed again | +| `the cache is not on a Dev Drive: …` | Windows only, once: see [Windows](#windows-a-cache-on-a-dev-drive) | | `restarting clangd (): ()` | Every restart, and whether it counts against clangd | | `the timers of took ms, ms of them in ` | What held the event loop | @@ -339,6 +344,23 @@ payload. An editor that starts `mcppls` itself can give it a clangd of your own, (`proot`, say). An engine that cannot be started there is the status issue `engine-start-failed`, with what went wrong; see [Android, under Termux](00-install.md#android-under-termux). +## Windows: a cache on a Dev Drive + +On Windows every check that a module or a preamble is still good is a file operation that opens the +file, and Microsoft Defender scans each file clangd writes and reads in its cache as it happens. The +server says once in the log, and the overview of the editor says too, when its cache is not on a +[Dev Drive](https://learn.microsoft.com/windows/dev-drive/), where Defender scans asynchronously. +Two ways help, both yours to choose; mcppls changes no security setting: + +- Put the cache on a Dev Drive: set the environment variable `MCPPLS_CACHE_DIR` to a folder on it. +- Or exclude the cache folder from real-time scanning in Windows Security. + +What else makes completion slow on Windows, and what this version does about it, is in the log's +`Completion in … ms (…)` lines: they say how much of a completion went to checking module inputs. +Lists come whole from clangd (up to 1000 items, `MCPPLS_ENGINE_ARGUMENTS="--limit-results=100"` puts +clangd's own limit back) so the editor filters them as you type; a client that asks again at every key +is answered from the word's complete list (`MCPPLS_COMPLETION_REFILTER=off` turns that off). + ## Resetting a workspace's cache When preparation never finishes, the engine keeps crashing or a model looks wrong, reset the cache of diff --git a/docs/specs/CHANGELOG.md b/docs/specs/CHANGELOG.md index 209883f0..4d45e5f9 100644 --- a/docs/specs/CHANGELOG.md +++ b/docs/specs/CHANGELOG.md @@ -1,5 +1,11 @@ # Changelog +## 2026-10-12 — S3: whether a Windows cache is on a Dev Drive + +`about` may carry `cacheDevDrive` on Windows (S3-4-34): whether the server's cache directory is on a Dev Drive, where +Microsoft Defender scans the files an engine writes and reads asynchronously. A client may show `false` as a fact, never +changes a security setting for the person. Additive: protocol version stays 1. + ## 2026-10-12 — S3: what runs, and a build program that does not compile `cxxModules/status` may carry `about` (S3-4-31): the server, the editor package, the engine (and how often it was restarted), diff --git a/docs/specs/s3-lsp-extensions.md b/docs/specs/s3-lsp-extensions.md index 942fef81..a46596f8 100644 --- a/docs/specs/s3-lsp-extensions.md +++ b/docs/specs/s3-lsp-extensions.md @@ -90,6 +90,7 @@ interface About { kit?: string; // the semantic kit in use mcpp?: string; // the mcpp that describes the project, when it is an mcpp project ("" when it did not say) mismatch?: string; // the editor package started a server other than its own (S3-4-32) + cacheDevDrive?: boolean; // Windows: whether the server's cache directory is on a Dev Drive (S3-4-34) } interface CacheStatus { @@ -175,7 +176,7 @@ A server **SHOULD NOT** send `degraded` for a condition that ends by itself with A `producer-needs-download` issue says that the build tool, run without the network as a server runs it on its own, cannot describe the project until something is downloaded. A server **MAY** set `askOnline` on it when, asked by `workspace/executeCommand` with the command `mcppls.describeOnline`, it will describe the project once with the network allowed; every later description is without it again. S3-4-16 Until the client asks, and while the download runs, the server **MUST** go on serving the root from what it has (its sources, a partial description) S3-4-17, and **MUST NOT** reach the network on its own. S3-4-18 A client that offers the download **MUST NOT** block anything on the question: no modal dialog, and no request, activation or startup waits for the answer. S3-4-19 It **SHOULD** ask at most once per root and set of missing things. S3-4-20 It **MUST NOT** act on an answer that comes after the root's status no longer carries the issue: the person may have built the project in their own terminal meanwhile, and the server's own offline retries find that by themselves. S3-4-21 A client that does not know `askOnline` sees an issue with a command, as before. A `producer-install-failed` issue is what that description answers when the network was allowed (the client asked, or the person set the build tool online) and the install failed: it names what failed and carries the build tool's own error lines, it carries no `askOnline` (asking again would repeat the failure), and its command is the same terminal action. -A server that described a root with the network because a client asked (`mcppls.describeOnline`) **SHOULD** say how that ended in `onlineRun`, and keep it in the status of that root until another such run ends. A server **MAY** put the cache's own numbers in the optional `cache` field of a root's status, for a client that declared `status: true`, and only then. S3-4-29 The status carries the cache at a coarse grain -- a size rounded up to the next 100 MB, a fill level, the two counts a person compares at a glance -- and **MUST NOT** carry the per-file, per-module or per-instance detail: the detail is what `cxxModules/cache` (5.7) answers, and a field that changes with every module clangd builds would turn every status into a new one. S3-4-30 A server **SHOULD** say in `about` which server, engine and build tool serve the root, with no path in it: a client shows it and copies it into a report as it is. It changes only when one of them does (a restart counted in `engine.restarts`, a producer that answers with another version), so it costs no extra notification. S3-4-31 When the client's `initializationOptions` carry `extensionVersion` and it is not the server's version, the server **MUST** say so in `about.mismatch` and in a `version-mismatch` notice: a report then describes a version nobody thinks is running. S3-4-32 A notice `build-program-failed` says that a build program (mcpp's `build.mcpp`) did not compile or run and the root keeps the description it had; `build-program-held` says that a changed build program waits to be applied (the editor shows errors in it, or `buildProgram.apply` is `manual`). Neither makes the state `degraded`: the targets are served as before. S3-4-33 A client that does not know `cache` ignores it, as with every optional field. S3-4-26 A client **SHOULD** tell the person each run once, told apart by `at`, without blocking anything (S3-4-18); a failed run's `message` says what failed, and the root's issues say what is still missing. S3-4-27 A client **MAY** remember, per workspace and only on the person's say-so, that every download the server offers is to be fetched, and then call `mcppls.describeOnline` without asking; it **MUST** offer a way to take that back. S3-4-28 +A server that described a root with the network because a client asked (`mcppls.describeOnline`) **SHOULD** say how that ended in `onlineRun`, and keep it in the status of that root until another such run ends. A server **MAY** put the cache's own numbers in the optional `cache` field of a root's status, for a client that declared `status: true`, and only then. S3-4-29 The status carries the cache at a coarse grain -- a size rounded up to the next 100 MB, a fill level, the two counts a person compares at a glance -- and **MUST NOT** carry the per-file, per-module or per-instance detail: the detail is what `cxxModules/cache` (5.7) answers, and a field that changes with every module clangd builds would turn every status into a new one. S3-4-30 A server **SHOULD** say in `about` which server, engine and build tool serve the root, with no path in it: a client shows it and copies it into a report as it is. It changes only when one of them does (a restart counted in `engine.restarts`, a producer that answers with another version), so it costs no extra notification. S3-4-31 When the client's `initializationOptions` carry `extensionVersion` and it is not the server's version, the server **MUST** say so in `about.mismatch` and in a `version-mismatch` notice: a report then describes a version nobody thinks is running. S3-4-32 A notice `build-program-failed` says that a build program (mcpp's `build.mcpp`) did not compile or run and the root keeps the description it had; `build-program-held` says that a changed build program waits to be applied (the editor shows errors in it, or `buildProgram.apply` is `manual`). Neither makes the state `degraded`: the targets are served as before. S3-4-33 On Windows a server **MAY** say in `about.cacheDevDrive` whether its cache directory is on a Dev Drive, where Microsoft Defender scans the module and preamble files an engine writes and reads asynchronously; it is absent when the server cannot tell or the platform has no such volumes. A client **MAY** show `false` as a fact about the environment, once and without blocking anything, and **MUST NOT** change any security setting on the person's behalf. S3-4-34 A client that does not know `cache` ignores it, as with every optional field. S3-4-26 A client **SHOULD** tell the person each run once, told apart by `at`, without blocking anything (S3-4-18); a failed run's `message` says what failed, and the root's issues say what is still missing. S3-4-27 A client **MAY** remember, per workspace and only on the person's say-so, that every download the server offers is to be fetched, and then call `mcppls.describeOnline` without asking; it **MUST** offer a way to take that back. S3-4-28 An issue the server cannot recover from without the person — an engine that keeps exiting, cannot be started or cannot run on the machine, a corrupt installation, preparation that stopped making progress — is what a bug report is written about, and what it needs is gone once the editor is restarted. For such an issue a server **SHOULD** write a diagnostic bundle by itself when the issue first appears, and name it in `bundle`. S3-4-22 The bundle **MUST** be redacted as a report is (S3-5.5-3), and stay on the machine it was written on: neither the server nor the client sends it anywhere. S3-4-23 A server **SHOULD** keep only the few newest bundles it wrote by itself. S3-4-24 A client that presents `bundle` **SHOULD** offer, once per issue and without blocking anything, to report the problem with the file attached by the person, to restart and to leave the server off for the workspace. S3-4-25 A client that does not know `bundle` sees the issue as before. diff --git a/docs/zh-CN/30-settings.md b/docs/zh-CN/30-settings.md index d4420a5d..8cd8c64e 100644 --- a/docs/zh-CN/30-settings.md +++ b/docs/zh-CN/30-settings.md @@ -49,6 +49,7 @@ VS Code 扩展已经会这样做）；`重新加载模型` 只重新加载项目 |---|---|---|---|---|---| | `mcppls.engine.name` | `clangd`, `none` | `clangd` | `--engine` | 重启 | 核心引擎。无论如何，mcppls 自己的模块引擎都会运行；`none` 表示只提供模块相关功能。 | | `mcppls.engine.workers` | 字符串 | `auto` | `--engine-workers` | 重启 | clangd 同时构建的文件数（它的 `-j`），由模块预建、索引和你正在编辑的文件共用；总会留一个给你正在等的东西。`auto`：硬件线程数减一，介于二到八之间，且不超过内存 GB 数的一半。写数字则按数字。 | +| `mcppls.engine.preambleStorage` | `auto`, `disk`, `memory` | `auto` | `--engine-preamble-storage` | 重启 | clangd 把你正在编辑的文件的 preamble 放在哪里（它的 `--pch-storage`）。`auto`：Windows 上物理内存不少于 16 GiB 时放内存——那里写进临时目录的 preamble 在写入和读出时都会被扫描；其他情况放磁盘。`memory` 的代价大约是每个打开的文件一份 preamble（带 `` 时几十 MB）。 | | `mcppls.compiler` | 字符串 | （空） | `--compiler` | 重新加载模型 | 为模块语义使用这个编译器，而不是检测到的那个：可以是绝对路径、`PATH` 上的名字，或 `kit`（强制使用内置的语义工具包）。空表示自动检测。 | | `mcppls.semanticKit` | `auto`, `off` | `auto` | `--semantic-kit` | 重新加载模型 | 内置的标准库工具包是否可以被使用：`auto` 在没有找到编译器时使用；`off` 从不使用（没有编译器时只剩模块相关功能）。 | | `mcppls.requestTimeout` | 非负整数（秒） | `60` | `--request-timeout` | 重启 | 一个引擎请求最多等待多久，超时后不经该引擎就给出答复。用户在等的请求（悬停、跳转、补全等）总共最多等 30 秒，clangd 启动或准备模块期间也算在内，之后由 mcppls 自己的引擎答复。 | diff --git a/docs/zh-CN/50-troubleshooting.md b/docs/zh-CN/50-troubleshooting.md index 16491f99..9abe920b 100644 --- a/docs/zh-CN/50-troubleshooting.md +++ b/docs/zh-CN/50-troubleshooting.md @@ -37,6 +37,11 @@ | `clangd (): I[…] Rebuilding module : <原因>` | mcppls-clangd 0.0.13：一个模块为什么重建而不是复用——这个命令和这些前置模块下没有 BMI（以及有几份是用别的命令构建的），或者是哪个输入变了 | | `clangd (): I[…] Prerequisites of : modules, reused, built, failed, in (reuse checks , builds , …)` | 一个文件的前置模块花了多少 | | `clangd (): I[…] Preamble of is not compatible: <原因>` | 一个文件的 preamble 为什么重建 | +| `clangd (): I[…] Completion in : ms (module input checks : one by one, by package checks, system already checked; waited for modules)` | mcppls-clangd 0.0.15：模块导入方里一次 200 ms 以上的补全，时间花在了哪里。mcpp 或 xlings 包版本里的输入按该版本的标记核对，系统头每次会话核对一次，项目文件逐个核对 | +| `clangd (): I[…] Module memos: BMI verdicts and scans kept from the last session …` | mcppls-clangd 0.0.15，启动时：上次会话留下了多少条可以核对而不必重做的结果（每条使用前都会再核对） | +| `clangd (): I[…] Module inputs: managed package store(s) checked by their markers` | 引擎按版本核对的包仓库（mcpp 的、xlings 的）有几个 | +| `the last files in use (): of them, opened in clangd ahead of the editor once it takes files` | 启动时：上次在编辑的文件会在编辑器请求之前就在 clangd 里打开；` was ready in clangd ms before the editor opened it` 说明提前准备起了作用，`... is still being built there; the editor takes it as it is` 说明编辑器打开时 clangd 还在构建它、直接接着用，两分钟内编辑器没打开的会再关掉 | +| `the cache is not on a Dev Drive: …` | 只在 Windows 上，只说一次：见 [Windows：缓存放在 Dev Drive 上](#windows缓存放在-dev-drive-上) | | `restarting clangd (): <原因> ()` | 每次重启，以及是否计入 clangd 的重启预算 | | `the timers of took ms, ms of them in ` | 是什么占住了事件循环 | @@ -133,6 +138,20 @@ mcppls report --bundle problem.zip --root path/to/project # 可加 --hide-proj **在 Termux 或 PRoot 里。** `mcppls report` 把服务端运行所在的沙箱显示为 `server.sandbox`（比如 `proot`）。引擎在那里起不来时，状态里是 `engine-start-failed` 并写明原因；见 [Android 上的 Termux](00-install.md#android-上的-termux)。 +## Windows：缓存放在 Dev Drive 上 + +在 Windows 上，确认一个模块或 preamble 是否仍然可用，每一步都是要打开文件的文件操作；Microsoft Defender 还会在 clangd +读写缓存里的每个文件时实时扫描。缓存不在 [Dev Drive](https://learn.microsoft.com/windows/dev-drive/) 上时（Dev Drive 上 +Defender 异步扫描），服务端会在日志里说一次，编辑器的概览里也会显示。下面两种做法都有帮助，由你决定；mcppls 不会修改任何 +安全设置： + +- 把缓存放到 Dev Drive 上：把环境变量 `MCPPLS_CACHE_DIR` 设成 Dev Drive 上的一个文件夹。 +- 或者在 Windows 安全中心里把缓存文件夹排除在实时扫描之外。 + +Windows 上补全还慢在哪里、这一版做了什么，看日志里的 `Completion in … ms (…)` 行：它们说明一次补全有多少时间花在核对模块 +输入上。clangd 一次给出完整的列表（最多 1000 项，`MCPPLS_ENGINE_ARGUMENTS="--limit-results=100"` 恢复 clangd 自己的上限）， +编辑器在你打字时自己过滤；每键都请求的客户端，用这个词的完整列表直接作答（`MCPPLS_COMPLETION_REFILTER=off` 关闭）。 + ## 重置工作区缓存 准备一直完不成、引擎反复崩溃，或者模型看起来不对时，重置这一个工作区的缓存，不用再手动删目录： diff --git a/editors/claude-code/.claude-plugin/marketplace.json b/editors/claude-code/.claude-plugin/marketplace.json index ad1f595d..cb268751 100644 --- a/editors/claude-code/.claude-plugin/marketplace.json +++ b/editors/claude-code/.claude-plugin/marketplace.json @@ -11,7 +11,7 @@ "displayName": "C++ Modules Language Server", "source": "./mcppls-lsp", "description": "Registers mcppls as the language server for C and C++ sources, including C++20/23 named modules.", - "version": "0.0.13", + "version": "0.0.14", "author": { "name": "Sunrisepeak", "url": "https://github.com/Sunrisepeak/mcpp-language-server" diff --git a/editors/claude-code/mcppls-lsp/.claude-plugin/plugin.json b/editors/claude-code/mcppls-lsp/.claude-plugin/plugin.json index ae6c9ce0..f11d0696 100644 --- a/editors/claude-code/mcppls-lsp/.claude-plugin/plugin.json +++ b/editors/claude-code/mcppls-lsp/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "mcppls-lsp", "displayName": "C++ Modules Language Server", - "version": "0.0.13", + "version": "0.0.14", "description": "Registers mcppls as the language server for C and C++ sources, including C++20/23 named modules, and its MCP tools (symbols, references, modules, verification, review). Replaces clangd-lsp for a project; do not enable both at once.", "author": { "name": "Sunrisepeak", diff --git a/editors/clion/gradle.properties b/editors/clion/gradle.properties index b69b8218..d9c0c190 100644 --- a/editors/clion/gradle.properties +++ b/editors/clion/gradle.properties @@ -2,7 +2,7 @@ # ones within the same major line; sinceBuild/untilBuild in plugin.xml is what actually gates it. platformType = CL platformVersion = 2026.2.3 -pluginVersion = 0.0.13 +pluginVersion = 0.0.14 org.gradle.jvmargs = -Xmx2g # The IDE ships the Kotlin standard library; bundling a second copy in the plugin is what JetBrains # asks plugins not to do. diff --git a/editors/vscode/l10n/bundle.l10n.json b/editors/vscode/l10n/bundle.l10n.json index ab75c143..5624b89e 100644 --- a/editors/vscode/l10n/bundle.l10n.json +++ b/editors/vscode/l10n/bundle.l10n.json @@ -100,5 +100,7 @@ "modules {0}/{1}": "modules {0}/{1}", "restarts {0}": "restarts {0}", "{0} issues": "{0} issues", - "Versions copied": "Versions copied" + "Versions copied": "Versions copied", + "Cache is not on a Dev Drive": "Cache is not on a Dev Drive", + "Defender scans module files as they are written and read; the log says what helps": "Defender scans module files as they are written and read; the log says what helps" } diff --git a/editors/vscode/l10n/bundle.l10n.zh-cn.json b/editors/vscode/l10n/bundle.l10n.zh-cn.json index 38e2311b..528e5f7a 100644 --- a/editors/vscode/l10n/bundle.l10n.zh-cn.json +++ b/editors/vscode/l10n/bundle.l10n.zh-cn.json @@ -100,5 +100,7 @@ "modules {0}/{1}": "模块 {0}/{1}", "restarts {0}": "重启 {0} 次", "{0} issues": "{0} 个问题", - "Versions copied": "版本信息已复制" + "Versions copied": "版本信息已复制", + "Cache is not on a Dev Drive": "缓存不在 Dev Drive 上", + "Defender scans module files as they are written and read; the log says what helps": "Defender 会在写入和读出时扫描模块文件；日志里写了怎么改善" } diff --git a/editors/vscode/package.json b/editors/vscode/package.json index 65c7c849..3e0f3f32 100644 --- a/editors/vscode/package.json +++ b/editors/vscode/package.json @@ -2,7 +2,7 @@ "name": "mcpp-language-server", "displayName": "%displayName%", "description": "%description%", - "version": "0.0.13", + "version": "0.0.14", "publisher": "sunrisepeak", "license": "Apache-2.0", "icon": "icon.png", @@ -295,6 +295,21 @@ "patternErrorMessage": "%config.mcppls.engine.workers.patternErrorMessage%", "description": "%config.mcppls.engine.workers.description%" }, + "mcppls.engine.preambleStorage": { + "type": "string", + "enum": [ + "auto", + "disk", + "memory" + ], + "default": "auto", + "enumDescriptions": [ + "%config.mcppls.engine.preambleStorage.enum.0%", + "%config.mcppls.engine.preambleStorage.enum.1%", + "%config.mcppls.engine.preambleStorage.enum.2%" + ], + "description": "%config.mcppls.engine.preambleStorage.description%" + }, "mcppls.engine.name": { "type": "string", "enum": [ diff --git a/editors/vscode/package.nls.json b/editors/vscode/package.nls.json index e11ba74e..285afa68 100644 --- a/editors/vscode/package.nls.json +++ b/editors/vscode/package.nls.json @@ -75,5 +75,9 @@ "command.mcppls.applyBuildProgram.title": "Apply build.mcpp", "config.mcppls.buildProgram.apply.description": "When a changed mcpp build program (build.mcpp) is applied. Applying runs mcpp to describe the project again, which compiles and runs the build program. Either way a build program that does not compile leaves the project its last complete description.", "config.mcppls.buildProgram.apply.enum.0": "Once it is saved and left alone for two seconds, and not while the editor shows errors in it.", - "config.mcppls.buildProgram.apply.enum.1": "Only with the command C++ Modules: Apply build.mcpp." + "config.mcppls.buildProgram.apply.enum.1": "Only with the command C++ Modules: Apply build.mcpp.", + "config.mcppls.engine.preambleStorage.description": "Where clangd keeps the preambles of the files you edit. A change restarts the server.", + "config.mcppls.engine.preambleStorage.enum.0": "In memory on Windows with 16 GiB or more, where a preamble written to the temporary directory is scanned when written and when read; on disk elsewhere.", + "config.mcppls.engine.preambleStorage.enum.1": "On disk, in the temporary directory (clangd's default).", + "config.mcppls.engine.preambleStorage.enum.2": "In memory: about one preamble per open file, tens of MB with ." } diff --git a/editors/vscode/package.nls.zh-cn.json b/editors/vscode/package.nls.zh-cn.json index 1a8169b4..0a4a50c8 100644 --- a/editors/vscode/package.nls.zh-cn.json +++ b/editors/vscode/package.nls.zh-cn.json @@ -75,5 +75,9 @@ "command.mcppls.applyBuildProgram.title": "应用 build.mcpp", "config.mcppls.buildProgram.apply.description": "修改后的 mcpp 构建程序（build.mcpp）何时生效。生效会运行 mcpp 重新描述项目，即编译并运行构建程序。无论哪种方式，构建程序编译不过时，项目都保留上一份完整的描述。", "config.mcppls.buildProgram.apply.enum.0": "保存后两秒内没有再改动、且编辑器里它没有错误时生效。", - "config.mcppls.buildProgram.apply.enum.1": "只在执行命令 C++ Modules: Apply build.mcpp 时生效。" + "config.mcppls.buildProgram.apply.enum.1": "只在执行命令 C++ Modules: Apply build.mcpp 时生效。", + "config.mcppls.engine.preambleStorage.description": "clangd 把你正在编辑的文件的 preamble 放在哪里。修改后服务端会重启。", + "config.mcppls.engine.preambleStorage.enum.0": "Windows 上物理内存不少于 16 GiB 时放内存（那里写进临时目录的 preamble 在写入和读出时都会被扫描）；其他情况放磁盘。", + "config.mcppls.engine.preambleStorage.enum.1": "放磁盘，在临时目录里（clangd 的默认值）。", + "config.mcppls.engine.preambleStorage.enum.2": "放内存：大约每个打开的文件一份 preamble，带 时几十 MB。" } diff --git a/editors/vscode/src/cacheHub.ts b/editors/vscode/src/cacheHub.ts index 20dee79a..989c4ba9 100644 --- a/editors/vscode/src/cacheHub.ts +++ b/editors/vscode/src/cacheHub.ts @@ -102,6 +102,12 @@ export function runningEntries(status: CxxModulesStatus | undefined, now: number const problems = (status.issues ?? []).length; if (problems > 0) state.push(t('{0} issues', problems)); entries.push({ icon: '$(pulse)', label: t('Running'), description: state.join(' · '), behavior: 'command', action: { command: 'mcppls.showLogs' } }); + // 0.0.14 E1: on Windows, a cache off a Dev Drive is scanned by Microsoft Defender as clangd writes and reads it. + if (about?.cacheDevDrive === false) { + entries.push({ icon: '$(shield)', label: t('Cache is not on a Dev Drive'), + description: t('Defender scans module files as they are written and read; the log says what helps'), + behavior: 'command', action: { command: 'mcppls.showLogs' } }); + } return entries; } diff --git a/editors/vscode/src/commands.ts b/editors/vscode/src/commands.ts index f013939b..513a9fd3 100644 --- a/editors/vscode/src/commands.ts +++ b/editors/vscode/src/commands.ts @@ -14,7 +14,7 @@ import { COPY_ABOUT_COMMAND } from './cacheHub'; import { REPOSITORY, feedbackIssueUrl, IssueContext } from './issueUrl'; import { turnOffInWorkspace, turnOnInWorkspace } from './enable'; import { sourceOf } from './quickSuggestions'; -import { RENAMED_SETTINGS, resolveRenamed, workersSetting } from './settingsRead'; +import { preambleStorageSetting, RENAMED_SETTINGS, resolveRenamed, workersSetting } from './settingsRead'; import { redactJson, Who } from './redact'; import { CxxModulesStatus, describeProfile, SemanticProfile } from './status'; import { t } from './strings'; @@ -247,6 +247,7 @@ function mcpplsSettings(): Record { 'engine.name': resolveRenamed(settings, RENAMED_SETTINGS[0], 'clangd'), 'format.fallbackStyle': settings.get('format.fallbackStyle') || 'auto', 'engine.workers': workersSetting(settings.get('engine.workers')), + 'engine.preambleStorage': preambleStorageSetting(settings.get('engine.preambleStorage')), 'buildDiscovery.mode': resolveRenamed(settings, RENAMED_SETTINGS[1], 'auto'), buildTool: settings.get('buildTool'), toolEnvironment: settings.get('toolEnvironment'), diff --git a/editors/vscode/src/extension.ts b/editors/vscode/src/extension.ts index 699798a6..8ab9afe6 100644 --- a/editors/vscode/src/extension.ts +++ b/editors/vscode/src/extension.ts @@ -709,7 +709,8 @@ export function activate(context: vscode.ExtensionContext): TestApi { // S-1, S-2 (plan 0.0.9): `mcppls.engine` and `mcppls.buildDiscovery` still match, as the old // names a hand edit may touch; the new ones are named so the list says what it restarts for. || event.affectsConfiguration('mcppls.engine') || event.affectsConfiguration('mcppls.engine.name') - || event.affectsConfiguration('mcppls.format.fallbackStyle') || event.affectsConfiguration('mcppls.engine.workers') || event.affectsConfiguration('mcppls.buildTool') + || event.affectsConfiguration('mcppls.format.fallbackStyle') || event.affectsConfiguration('mcppls.engine.workers') + || event.affectsConfiguration('mcppls.engine.preambleStorage') || event.affectsConfiguration('mcppls.buildTool') || event.affectsConfiguration('mcppls.toolEnvironment') || event.affectsConfiguration('mcppls.semanticTokens.modules') || event.affectsConfiguration('mcppls.completion.triggerOnSpace') // 0.0.6 plan §3.7 B-7, §2.6/§9 T5: new settings, same treatment as the ones above. diff --git a/editors/vscode/src/settingsRead.ts b/editors/vscode/src/settingsRead.ts index 60a579e7..6ccaed66 100644 --- a/editors/vscode/src/settingsRead.ts +++ b/editors/vscode/src/settingsRead.ts @@ -85,6 +85,11 @@ export const BUILD_DISCOVERY_PROVIDERS = ['mcpp', 'cmake', 'xmake', 'meson', 'co const WORKERS = /^(auto|[1-9][0-9]?)?$/; +/** `engine.preambleStorage` as the server takes it: `auto`, `disk` or `memory`; anything else is `auto`. */ +export function preambleStorageSetting(value: string | undefined): 'auto' | 'disk' | 'memory' { + return value === 'disk' || value === 'memory' ? value : 'auto'; +} + /** `engine.workers` as the server takes it: `auto` or a whole number from 1 to 99; empty means `auto`. */ export function workersSetting(value: string | undefined): string { const trimmed = (value ?? '').trim(); @@ -105,6 +110,8 @@ export function buildInitializationOptions(configuration: SettingsReader, compil 'engine.name': resolveRenamed(configuration, RENAMED_SETTINGS[0], 'clangd') === 'none' ? 'none' : 'clangd', // S-2 (plan 0.0.9): how many files clangd builds at once; a change restarts the server. 'engine.workers': workersSetting(configuration.get('engine.workers')), + // 0.0.14 C1: where clangd keeps preambles; a change restarts the server. + 'engine.preambleStorage': preambleStorageSetting(configuration.get('engine.preambleStorage')), // The formatting fallback without a project .clang-format; a change restarts the server. 'format.fallbackStyle': configuration.get('format.fallbackStyle') || 'auto', // build description design 4.4: how the user's build tool may be run. diff --git a/editors/vscode/src/status.ts b/editors/vscode/src/status.ts index 924213bb..ace65caf 100644 --- a/editors/vscode/src/status.ts +++ b/editors/vscode/src/status.ts @@ -82,6 +82,8 @@ export interface CxxAbout { mcpp?: string; // Set when the editor package started a server other than its own. mismatch?: string; + // Windows only (0.0.14 E1): whether the cache is on a Dev Drive. + cacheDevDrive?: boolean; } // What the status bar shows for each state. diff --git a/editors/vscode/test/unit/hubOverview.test.ts b/editors/vscode/test/unit/hubOverview.test.ts index f527f286..1e04ecfe 100644 --- a/editors/vscode/test/unit/hubOverview.test.ts +++ b/editors/vscode/test/unit/hubOverview.test.ts @@ -53,6 +53,16 @@ suite('hub overview: versions and running state (0.0.13 W7)', () => { assert.ok(entries[0].description?.includes('0.0.11')); }); + test('a cache off a Dev Drive is said once, after the running state; on one, or unknown, nothing is said', () => { + const off: CxxModulesStatus = { ...status, about: { ...status.about!, cacheDevDrive: false } }; + const entries = runningEntries(off); + assert.strictEqual(entries[entries.length - 1].icon, '$(shield)'); + assert.strictEqual(entries[entries.length - 1].label, 'Cache is not on a Dev Drive'); + const on: CxxModulesStatus = { ...status, about: { ...status.about!, cacheDevDrive: true } }; + assert.ok(!runningEntries(on).some((entry) => entry.icon === '$(shield)')); + assert.ok(!runningEntries(status).some((entry) => entry.icon === '$(shield)')); + }); + test('an older server without `about` still shows what the status has; no status shows nothing', () => { const old: CxxModulesStatus = { ...status, about: undefined }; assert.strictEqual(aboutText(old), 'mcppls · clangd 23.1.0-mcppls.1'); diff --git a/editors/zed/extension.toml b/editors/zed/extension.toml index f01d61a4..6d55c88e 100644 --- a/editors/zed/extension.toml +++ b/editors/zed/extension.toml @@ -1,6 +1,6 @@ id = "mcppls" name = "C++ Modules Language Server" -version = "0.0.13" +version = "0.0.14" schema_version = 1 description = "mcppls - C++20/23 named modules that just work: navigation, completion, hover and diagnostics across modules for any compiler" repository = "https://github.com/Sunrisepeak/mcpp-language-server" diff --git a/mcpp.toml b/mcpp.toml index 93ff2ecf..061eee75 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -37,7 +37,7 @@ libarchive = "3.8.7" [package] name = "mcpp-language-server" -version = "0.0.13" +version = "0.0.14" description = "Compiler-agnostic C++ modules language server" license = "Apache-2.0" authors = ["Sunrisepeak"] diff --git a/modules/base/src/version.cppm b/modules/base/src/version.cppm index d619a8cf..ea966087 100644 --- a/modules/base/src/version.cppm +++ b/modules/base/src/version.cppm @@ -9,9 +9,9 @@ export namespace mcppls::base { // checked against mcpp.toml (the one source) by `mcppls-devtools version --check`, not kept in step by // hand. Three constants that lived here and nothing read were removed rather than left to drift: // the S1 profile version is spec::PROFILE_VERSION, the kit manifest version is spec::KIT_VERSION. -inline constexpr std::string_view VERSION { "0.0.13" }; +inline constexpr std::string_view VERSION { "0.0.14" }; // The clangd the payload ships (mcppls-clangd). Checked against packaging/payload.lock.json by the same command. -inline constexpr std::string_view CLANGD_VERSION { "23.1.0-mcppls.2" }; +inline constexpr std::string_view CLANGD_VERSION { "23.1.0-mcppls.3" }; // Maintained-engine identity stays intact in logs and caches. Its LLVM base // decides libc++ kit compatibility; unknown vendor suffixes remain unchanged. inline std::string_view llvm_base_version(std::string_view engineVersion) { diff --git a/modules/platform/build.mcpp b/modules/platform/build.mcpp index 8b3d5099..b3e9e2cf 100644 --- a/modules/platform/build.mcpp +++ b/modules/platform/build.mcpp @@ -1,16 +1,16 @@ import mcpp; import std; -// THE PROCESS IDENTITY'S THREE NAMES, AS AN IMPORT LIBRARY OF THEIR OWN (X-6). +// THE WIN32 NAMES OPENKAL-WINDOWS DOES NOT CARRY, AS AN IMPORT LIBRARY OF THEIR OWN (X-6). // -// process.cpp declares GetProcessTimes, OpenProcess and GetCurrentProcessId +// process.cpp declares GetProcessTimes, OpenProcess and GetCurrentProcessId, system.cpp GlobalMemoryStatusEx and DeviceIoControl, // itself -- this build's windows target has no vendor SDK to include, and // openkal-windows' win32.h showed the way. The LINK then needs import entries // for them, and the only kernel32 import library on this link line is // openkal-windows' own generated one, which lists exactly the forty-five names // it calls and deliberately no more (its build.mcpp says why, and what a // consumer that calls a forty-sixth is told). So this package carries its own -// port/winprocess.def -- three names, still bound to KERNEL32.dll -- and turns +// port/winprocess.def -- the names it needs, still bound to KERNEL32.dll -- and turns // it into libwinprocess.a the same way openkal-windows turns its .def files: // llvm-dlltool under clang, into this package's out directory, which // mcpp::link_search puts on the link line. A name of its own cannot shadow diff --git a/modules/platform/mcpp.toml b/modules/platform/mcpp.toml index 61c84cc0..242f87d3 100644 --- a/modules/platform/mcpp.toml +++ b/modules/platform/mcpp.toml @@ -30,8 +30,9 @@ mcppls-os-windows = { path = "../os/windows" } # X-6 (plan 2026-10-03): the process identity asks this system's own tables (GetProcessTimes, # OpenProcess, GetCurrentProcessId -- declared in process.cpp the way openkal-windows declares -# its Win32 set, without a vendor SDK). `winprocess` is this package's own import library for -# those three names, generated from port/winprocess.def by build.mcpp (see there for why it is +# its Win32 set, without a vendor SDK), and since 0.0.14 system.cpp asks for the memory and a +# volume's state (GlobalMemoryStatusEx, DeviceIoControl). `winprocess` is this package's own +# import library for those names, generated from port/winprocess.def by build.mcpp (see there for why it is # not called kernel32); `kernel32` itself is already on the line through openkal-windows, which # carries the rest (CloseHandle, GetExitCodeProcess, GetCurrentProcess). The predicate is the # object ABI (openkal-windows/mcpp.toml explains why `env`). diff --git a/modules/platform/port/winprocess.def b/modules/platform/port/winprocess.def index 83db07e6..fdb52ba1 100644 --- a/modules/platform/port/winprocess.def +++ b/modules/platform/port/winprocess.def @@ -1,6 +1,7 @@ -; The three Win32 names the process identity needs that openkal-windows' own -; list does not carry (X-6, plan 2026-10-03): GetProcessTimes, OpenProcess, -; GetCurrentProcessId. Generated into an import library by build.mcpp; the +; The Win32 names this package calls that openkal-windows' own list does not +; carry: GetProcessTimes, OpenProcess, GetCurrentProcessId for the process +; identity (X-6, plan 2026-10-03); GlobalMemoryStatusEx and DeviceIoControl for +; what the machine is (system.cpp, 0.0.14). Generated into an import library by build.mcpp; the ; library is named `winprocess` -- NOT `kernel32` -- so it cannot shadow ; openkal-windows' generated kernel32 (its -L is searched first, and a second ; libkernel32.a there would hide the forty-five names it lists). @@ -9,6 +10,8 @@ ; ours to choose, the DLL its entries bind is a property of the names. LIBRARY KERNEL32.dll EXPORTS +DeviceIoControl GetCurrentProcessId GetProcessTimes +GlobalMemoryStatusEx OpenProcess diff --git a/modules/platform/src/platform.cppm b/modules/platform/src/platform.cppm index 3674d138..8d2bb114 100644 --- a/modules/platform/src/platform.cppm +++ b/modules/platform/src/platform.cppm @@ -11,6 +11,7 @@ export import mcppls.platform.preopen; export import mcppls.platform.process; export import mcppls.platform.sandbox; export import mcppls.platform.stdio; +export import mcppls.platform.system; export import mcppls.platform.task; export import mcppls.platform.toolenv; export import mcppls.platform.toolrun; diff --git a/modules/platform/src/system.cpp b/modules/platform/src/system.cpp new file mode 100644 index 00000000..9c5cbd48 --- /dev/null +++ b/modules/platform/src/system.cpp @@ -0,0 +1,124 @@ +module mcppls.platform.system; + +import std; +import mcppls.base.text; +import mcppls.platform.fs; + +// The Windows calls are declared the way process.cpp declares its own: exactly what is used, `dllimport` +// and `__stdcall`, since this package reaches the system through openkal and includes no vendor header. +#if defined(__MCPP_TARGET_WINDOWS__) +extern "C" { +using KalHandle = void*; +using KalDword = unsigned long; +using KalBool = int; +struct KalMemoryStatusEx { + KalDword length; + KalDword memoryLoad; + unsigned long long totalPhysical; + unsigned long long availablePhysical; + unsigned long long totalPageFile; + unsigned long long availablePageFile; + unsigned long long totalVirtual; + unsigned long long availableVirtual; + unsigned long long availableExtendedVirtual; +}; +// FILE_FS_PERSISTENT_VOLUME_INFORMATION +struct KalPersistentVolumeInformation { + KalDword volumeFlags; + KalDword flagMask; + KalDword version; + KalDword reserved; +}; +__declspec(dllimport) KalBool __stdcall GlobalMemoryStatusEx(KalMemoryStatusEx*); +__declspec(dllimport) KalHandle __stdcall CreateFileW(const wchar_t*, KalDword, KalDword, void*, KalDword, KalDword, KalHandle); +__declspec(dllimport) KalBool __stdcall DeviceIoControl(KalHandle, KalDword, void*, KalDword, void*, KalDword, KalDword*, void*); +__declspec(dllimport) KalBool __stdcall CloseHandle(KalHandle); +} +namespace { +constexpr KalDword KAL_FILE_READ_ATTRIBUTES { 0x0080 }; +constexpr KalDword KAL_FILE_SHARE_ALL { 0x1 | 0x2 | 0x4 }; +constexpr KalDword KAL_OPEN_EXISTING { 3 }; +constexpr KalDword KAL_FILE_FLAG_BACKUP_SEMANTICS { 0x02000000 }; +constexpr KalDword KAL_FSCTL_QUERY_PERSISTENT_VOLUME_STATE { 0x0009023C }; +constexpr KalDword KAL_PERSISTENT_VOLUME_STATE_DEV_VOLUME { 0x00002000 }; +const KalHandle KAL_INVALID_HANDLE { reinterpret_cast(static_cast(-1)) }; + +// UTF-8 to UTF-16, enough for a path; nullopt on a malformed sequence. +std::optional widen(std::string_view text) { + std::wstring out; + for (std::size_t i = 0; i < text.size();) { + const auto lead = static_cast(text[i]); + std::uint32_t code { 0 }; + std::size_t length { 1 }; + if (lead < 0x80) code = lead; + else if ((lead & 0xE0) == 0xC0) { code = lead & 0x1F; length = 2; } + else if ((lead & 0xF0) == 0xE0) { code = lead & 0x0F; length = 3; } + else if ((lead & 0xF8) == 0xF0) { code = lead & 0x07; length = 4; } + else return std::nullopt; + if (i + length > text.size()) return std::nullopt; + for (std::size_t k = 1; k < length; ++k) { + const auto next = static_cast(text[i + k]); + if ((next & 0xC0) != 0x80) return std::nullopt; + code = (code << 6) | (next & 0x3F); + } + if (code >= 0x10000) { + code -= 0x10000; + out.push_back(static_cast(0xD800 + (code >> 10))); + out.push_back(static_cast(0xDC00 + (code & 0x3FF))); + } else { + out.push_back(static_cast(code)); + } + i += length; + } + return out; +} +} // namespace +#endif + +namespace mcppls::platform::system { + +std::optional physical_memory_bytes() { +#if defined(__MCPP_TARGET_WINDOWS__) + KalMemoryStatusEx status {}; + status.length = sizeof(status); + if (!GlobalMemoryStatusEx(&status)) return std::nullopt; + return static_cast(status.totalPhysical); +#else + const auto info = fs::read_file("/proc/meminfo"); + if (!info) return std::nullopt; + for (const auto line : base::split_lines(*info)) { + if (!line.starts_with("MemTotal:")) continue; + std::uint64_t kib { 0 }; + const auto digits = base::trim(line.substr(9)); + const auto number = digits.substr(0, digits.find(' ')); + if (std::from_chars(number.data(), number.data() + number.size(), kib).ec != std::errc {}) return std::nullopt; + return kib * 1024; + } + return std::nullopt; +#endif +} + +std::optional on_dev_drive(std::string_view directory) { +#if defined(__MCPP_TARGET_WINDOWS__) + auto wide = widen(directory); + if (!wide) return std::nullopt; + // The FSCTL goes to the file system of whatever is opened: the directory itself will do. + const KalHandle handle { CreateFileW(wide->c_str(), KAL_FILE_READ_ATTRIBUTES, KAL_FILE_SHARE_ALL, nullptr, KAL_OPEN_EXISTING, + KAL_FILE_FLAG_BACKUP_SEMANTICS, nullptr) }; + if (handle == KAL_INVALID_HANDLE || handle == nullptr) return std::nullopt; + KalPersistentVolumeInformation info {}; + info.version = 1; + info.flagMask = KAL_PERSISTENT_VOLUME_STATE_DEV_VOLUME; + KalDword returned { 0 }; + const bool asked { DeviceIoControl(handle, KAL_FSCTL_QUERY_PERSISTENT_VOLUME_STATE, &info, sizeof(info), &info, sizeof(info), &returned, + nullptr) != 0 }; + CloseHandle(handle); + if (!asked) return std::nullopt; + return (info.volumeFlags & KAL_PERSISTENT_VOLUME_STATE_DEV_VOLUME) != 0; +#else + (void)directory; + return std::nullopt; +#endif +} + +} // namespace mcppls::platform::system diff --git a/modules/platform/src/system.cppm b/modules/platform/src/system.cppm new file mode 100644 index 00000000..8ced0040 --- /dev/null +++ b/modules/platform/src/system.cppm @@ -0,0 +1,16 @@ +// What the machine is: its memory, and what the volume a directory lives on is. Asked once at startup and +// cheap where the platform answers directly; nullopt where it does not. +export module mcppls.platform.system; + +import std; + +export namespace mcppls::platform::system { + +// The machine's physical memory in bytes: /proc/meminfo on Linux, GlobalMemoryStatusEx on Windows. +std::optional physical_memory_bytes(); + +// Windows: whether `directory` is on a Dev Drive (a ReFS volume Microsoft Defender scans asynchronously, in its +// performance mode). nullopt elsewhere, or when the volume cannot be asked. +std::optional on_dev_drive(std::string_view directory); + +} // namespace mcppls::platform::system diff --git a/packaging/README.md b/packaging/README.md index 1625f1cb..3ff07db9 100644 --- a/packaging/README.md +++ b/packaging/README.md @@ -55,7 +55,7 @@ and the kit installed by xlings. | Entry | What | Used by | |---|---|---| -| `mcppls-clangd-` | the maintained engine, mcppls-clangd 0.0.14's `clangd-23.1.0-mcppls.2-.tar.gz`: clangd, its builtin headers, licenses and `engine.json` (source and binary identity), taken byte for byte | `mcppls.pack.clangd` | +| `mcppls-clangd-` | the maintained engine, mcppls-clangd 0.0.15's `clangd-23.1.0-mcppls.3-.tar.gz`: clangd, its builtin headers, licenses and `engine.json` (source and binary identity), taken byte for byte | `mcppls.pack.clangd` | | `llvm-project-src` | llvm-project 23.1.0 source archive | `mcppls.pack.kit`, recipe `libcxx-source` | | `llvm-mingw` | llvm-mingw 20260826 (LLVM 23.1.0), UCRT, Linux x86_64 host | `mcppls.pack.kit`, recipe `llvm-mingw` | diff --git a/packaging/payload.lock.json b/packaging/payload.lock.json index dc27b2fe..a9ea6441 100644 --- a/packaging/payload.lock.json +++ b/packaging/payload.lock.json @@ -1,35 +1,35 @@ { "lock-version": 1, "comment": "Every upstream input of the bundled payload, and the one table of platforms. Change versions here only; CI fetches, verifies and assembles. Each sha256 was checked against two independent downloads and against the digest the release publishes beside the asset. The server version is read from mcpp.toml. A platform is named - as VS Code names extension targets; the VS Code extension, the release manifest and CI's matrices list the same platforms, which `mcppls-devtools check platforms` holds them to.", - "clangd-version": "23.1.0-mcppls.2", + "clangd-version": "23.1.0-mcppls.3", "libcxx-version": "23.1.0", "entries": { "mcppls-clangd-linux-x64": { - "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.14/clangd-23.1.0-mcppls.2-linux-x64.tar.gz", - "file": "clangd-23.1.0-mcppls.2-linux-x64.tar.gz", - "size": 20911488, - "sha256": "4d46cc3e1b83c2d479d1ad0327b65acaefd582c54c08afa761b9f36fe3046a12", + "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.15/clangd-23.1.0-mcppls.3-linux-x64.tar.gz", + "file": "clangd-23.1.0-mcppls.3-linux-x64.tar.gz", + "size": 20950855, + "sha256": "39d44f2a9694bfaf47b085f03420e9d90043c75dbaedc44229f8e366d8332bcb", "license": "Apache-2.0 WITH LLVM-exception" }, "mcppls-clangd-linux-arm64": { - "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.14/clangd-23.1.0-mcppls.2-linux-arm64.tar.gz", - "file": "clangd-23.1.0-mcppls.2-linux-arm64.tar.gz", - "size": 19870899, - "sha256": "0e55c9e4cbc363b184d7dcf859744e6cb36d8caed2293827b25b568ce35efcc3", + "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.15/clangd-23.1.0-mcppls.3-linux-arm64.tar.gz", + "file": "clangd-23.1.0-mcppls.3-linux-arm64.tar.gz", + "size": 19905599, + "sha256": "6a690c1ab6ffdec33d9e4c63e122a8201f9c138e638d9142e28e91a30269a3f3", "license": "Apache-2.0 WITH LLVM-exception" }, "mcppls-clangd-darwin-arm64": { - "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.14/clangd-23.1.0-mcppls.2-darwin-arm64.tar.gz", - "file": "clangd-23.1.0-mcppls.2-darwin-arm64.tar.gz", - "size": 19818655, - "sha256": "10685730a79212150108dc4c58f32117584ce3b3414dd4093fd84f26f1d77cd9", + "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.15/clangd-23.1.0-mcppls.3-darwin-arm64.tar.gz", + "file": "clangd-23.1.0-mcppls.3-darwin-arm64.tar.gz", + "size": 19855622, + "sha256": "9bff5958c4e79bdafd319c99213c9e4cceb2744991f014bb7240bfaa8da93144", "license": "Apache-2.0 WITH LLVM-exception" }, "mcppls-clangd-win32-x64": { - "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.14/clangd-23.1.0-mcppls.2-win32-x64.tar.gz", - "file": "clangd-23.1.0-mcppls.2-win32-x64.tar.gz", - "size": 20205399, - "sha256": "bd405020c212e26f52a8713b6dd76d7d048d75eed1b2e024f4b06af0c3366beb", + "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.15/clangd-23.1.0-mcppls.3-win32-x64.tar.gz", + "file": "clangd-23.1.0-mcppls.3-win32-x64.tar.gz", + "size": 20241419, + "sha256": "1b5345af310ecbda5580d02f1c75c807c26f14158a6f4b0821b7c0da78118727", "license": "Apache-2.0 WITH LLVM-exception" }, "llvm-project-src": { diff --git a/src/bin/conformance.cpp b/src/bin/conformance.cpp index 9af2b7ed..87cabff0 100644 --- a/src/bin/conformance.cpp +++ b/src/bin/conformance.cpp @@ -428,7 +428,7 @@ class Client { Clock::time_point sent; std::optional answered; bool isError { false }; - Json result { nullptr }; + Json result = nullptr; }; std::map asyncRequests; // The server's OS pid where the platform can say; read by the clangd watcher's thread, so atomic. @@ -491,7 +491,7 @@ class Client { std::int64_t send_async(std::string_view method, Json params) { const std::int64_t id { nextId_++ }; - asyncRequests[id] = { std::string { method }, Clock::now(), std::nullopt, false, Json { nullptr } }; + asyncRequests[id] = { std::string { method }, Clock::now(), std::nullopt, false, Json(nullptr) }; (void)connection_->send(lsp::make_request(id, method, std::move(params))); return id; } @@ -514,10 +514,10 @@ class Client { // empty answer — both of which `request` below collapses to `Json(nullptr)`, which is fine for // every check that only asks "did it answer", but not for one that counts errors on their own. struct RequestOutcome { - Json result { nullptr }; + Json result = nullptr; bool timedOut { false }; bool isError { false }; - Json error { nullptr }; // the error answer, when isError + Json error = nullptr; // the error answer, when isError }; RequestOutcome request_full(std::string_view method, Json params, std::chrono::seconds timeout) { @@ -2377,6 +2377,18 @@ class Scenario { const auto period = std::chrono::duration(1.0 / hz); const std::chrono::seconds duration { check.value("seconds", 120) }; const std::chrono::milliseconds completionEvery { check.value("completion-every-ms", 1000) }; + // 0.0.14: completions asked as an editor asks them while a word is typed, besides the periodic ones. "incomplete" + // is VS Code: the word's first character asks; while that answer is on its way, the keys typed meanwhile wait for + // it (the editor filters it by them when it comes), and once it is in, the next key asks again only when it was + // marked incomplete. "always" is a client that asks at every key whatever the list says. + const std::string reask { check.value("reask", std::string {}) }; + if (!reask.empty() && reask != "incomplete" && reask != "always") return { false, std::format("unknown reask '{}'", reask) }; + std::optional wordRequest; // the last completion asked in the word being typed + std::size_t askedAsTyped { 0 }; + std::size_t waitedForPending { 0 }; // keys typed while the word's answer was on its way + std::size_t askedForIncomplete { 0 }; // the last one was answered incomplete + std::size_t words { 0 }; + const auto identifier_char = [](char c) { return std::isalnum(static_cast(c)) != 0 || c == '_'; }; const std::chrono::milliseconds autosaveEvery { check.value("autosave-ms", 0) }; // files.autoSave "afterDelay": the buffer is saved once this long has passed since the last keystroke. const std::chrono::milliseconds autosaveIdle { check.value("autosave-idle-ms", 0) }; @@ -2523,10 +2535,34 @@ class Scenario { lines.insert(lines.begin() + static_cast(line), std::string {}); // a fresh line to type on send(); cursor = { static_cast(line), 0 }; + wordRequest.reset(); for (const char c : text) { lines[line].push_back(c); ++cursor.character; send(); + if (!reask.empty()) { + if (!identifier_char(c)) { + wordRequest.reset(); + } else { + if (!wordRequest) ++words; + bool ask { reask == "always" || !wordRequest }; + if (!ask) { + const auto& last { client_.asyncRequests[*wordRequest] }; + if (!last.answered) { + ++waitedForPending; // VS Code waits for it, and filters it by what was typed since + } else if (last.result.is_object() && last.result.value("isIncomplete", false)) { + ask = true; + ++askedForIncomplete; + } + } + if (ask) { + wordRequest = client_.send_async("textDocument/completion", Json { { "textDocument", Json { { "uri", documentUri } } }, + { "position", Json { { "line", cursor.line }, { "character", cursor.character } } } }); + completions.push_back(*wordRequest); + ++askedAsTyped; + } + } + } elapse(period * uniform(0.6, 1.4)); if (Clock::now() > end) break; } @@ -2652,7 +2688,22 @@ class Scenario { for (const auto& never : check.value("states-never", Json::array())) { if (statesSeen.contains(never.get())) failures.push_back(std::format("the status was {} while typing", never.get())); } + // 0.0.14 A1, A2: what the editor's way of asking cost -- the completions asked as the word was typed, and those the + // server gave from the word's complete core answer without asking the engine again. + const auto fromWordOf = [](const Json& root) { + return root.is_object() && root.contains("completion") && root["completion"].is_object() ? root["completion"].value("fromWord", 0L) : 0L; + }; + const long fromWord { fromWordOf(after) - fromWordOf(before) }; + enforce_max(check, "askedAsTypedPerWord", + words > 0 ? static_cast(askedAsTyped) / static_cast(words) : 0.0, "completions asked per typed word", "", failures); + // What complete lists (A2) remove: asking again because the answer was cut short. Reported, and a budget only + // where a fixture sets one: the faster the engine answers, the fewer keys wait for an answer on its way and the + // more of them ask again after an incomplete one, so the per-word total above is what a fixture bounds. + enforce_max(check, "incompletePerWord", + words > 0 ? static_cast(askedForIncomplete) / static_cast(words) : 0.0, "completions asked again per word for an incomplete answer", "", failures); Json summary { { "seconds", typingSeconds }, { "hz", hz }, { "autosaveMs", autosaveEvery.count() }, { "autosaveIdleMs", autosaveIdle.count() }, { "completions", completions.size() }, + { "reask", reask }, { "askedAsTyped", askedAsTyped }, { "keysWhilePending", waitedForPending }, + { "askedForIncomplete", askedForIncomplete }, { "typedWords", words }, { "fromWord", fromWord }, { "answered", answered.size() }, { "clientTimeouts", timeouts }, { "serverTimeouts", serverTimeouts }, { "errors", errors }, { "completion", latency }, { "engineShare", share ? Json(*share) : Json(nullptr) }, { "answeredBy", detail }, { "restarts", restarts }, { "filesSetAside", setAside }, { "diagnosticsRefresh", refresh ? Json(*refresh) : Json(nullptr) }, @@ -2666,9 +2717,12 @@ class Scenario { importerEmpty, importerModuleFailed); } return finish_measure(std::move(failures), std::move(summary), - std::format("{}{:.0f} s at {:.0f} Hz, {} completions p95 {:.2f}s max {:.2f}s{}, {} restart(s), {} set aside, {} doomed, {} module-failed diagnostic(s), {} stalled sample(s), {} timeout(s), diagnostics after {}", + std::format("{}{:.0f} s at {:.0f} Hz, {} completions p95 {:.2f}s max {:.2f}s{}{}, {} restart(s), {} set aside, {} doomed, {} module-failed diagnostic(s), {} stalled sample(s), {} timeout(s), diagnostics after {}", firstOpen ? std::format("first clangd answer {:.1f}s after opening; then ", *firstOpen) : std::string {}, - typingSeconds, hz, completions.size(), latency["p95"].get(), latency["max"].get(), importerBrief, restarts, setAside, doomed, + typingSeconds, hz, completions.size(), latency["p95"].get(), latency["max"].get(), + reask.empty() ? std::string {} : std::format(" ({} asked as {} words were typed, {} again for an incomplete answer; {} keys typed while an answer was on its way; {} answered from a word's complete list)", + askedAsTyped, words, askedForIncomplete, waitedForPending, fromWord), + importerBrief, restarts, setAside, doomed, moduleFailedDiagnostics, stalledSamples, allTimeouts, refresh ? std::format("{:.1f}s", *refresh) : std::string { "never" })); } diff --git a/src/cli/options.cpp b/src/cli/options.cpp index 92474bcf..f65ffa4a 100644 --- a/src/cli/options.cpp +++ b/src/cli/options.cpp @@ -56,6 +56,7 @@ orchestrator::EngineFactories engine_factories(const orchestrator::SessionOption else base::log::warning("mcppls.engine.workers is '{}', neither auto nor a number; using auto", options.engineWorkers); } clangd.disabledWorkarounds = options.disabledWorkarounds; + clangd.preambleStorage = options.preambleStorage; clangd.primeImplementationUnits = options.primeImplementationUnits != "off"; return engine::clangd::make_engine(std::move(clangd)); }; @@ -98,6 +99,7 @@ orchestrator::SessionOptions session_options(const cmdline::ParsedArgs& args) { options.requestTimeout = std::chrono::duration_cast(settings.seconds_value("requestTimeout")); options.formatFallbackStyle = settings.string_value("format.fallbackStyle"); options.engineWorkers = settings.string_value("engine.workers"); + options.preambleStorage = settings.string_value("engine.preambleStorage"); // This very program, for the reviews an editor asks for: named as the process started it, else found on PATH. if (const auto arguments = platform::env::arguments(); !arguments.empty()) { const std::string started { arguments.front() }; diff --git a/src/config/settings.cpp b/src/config/settings.cpp index 1e2ea9ee..1499762b 100644 --- a/src/config/settings.cpp +++ b/src/config/settings.cpp @@ -149,6 +149,18 @@ const std::vector& shipped_registry() { "硬件线程数减一，介于二到八之间，且不超过内存 GB 数的一半。写数字则按数字。", .clientConfigurable = true, }, + Setting { + .key = "engine.preambleStorage", .kind = Kind::enumeration, .values = { "auto", "disk", "memory" }, .defaultValue = "auto", + .commandLine = "--engine-preamble-storage", .surface = Surface::server, .applies = Applies::restart, .category = "engines", + .since = "0.0.14", + .summary = "Where clangd keeps the preambles of the files you edit (its `--pch-storage`). `auto`: in memory on Windows with " + "16 GiB or more, where a preamble written to the temporary directory is scanned when written and when read; " + "on disk elsewhere. `memory` costs about one preamble per open file (tens of MB with ``).", + .summaryZh = "clangd 把你正在编辑的文件的 preamble 放在哪里（它的 `--pch-storage`）。`auto`：Windows 上物理内存不少于 16 GiB " + "时放内存——那里写进临时目录的 preamble 在写入和读出时都会被扫描；其他情况放磁盘。`memory` 的代价大约是每个打开的" + "文件一份 preamble（带 `` 时几十 MB）。", + .clientConfigurable = true, + }, Setting { .key = "compiler", .kind = Kind::string, .defaultValue = "", .commandLine = "--compiler", .surface = Surface::server, .applies = Applies::reload, .category = "engines", .since = "0.0.1", diff --git a/src/engine/clangd.cpp b/src/engine/clangd.cpp index 8282e3b1..638c2ba0 100644 --- a/src/engine/clangd.cpp +++ b/src/engine/clangd.cpp @@ -10,6 +10,7 @@ import mcppls.base.sha256; import mcppls.base.text; import mcppls.base.uri; import mcppls.base.version; +import mcppls.platform.dirs; import mcppls.platform.env; import mcppls.platform.fs; import mcppls.platform.process; @@ -47,6 +48,24 @@ bool fixed_by_verified_features(std::string_view id, std::span///, each version +// with its markers. A header there -- a toolchain's ( of llvm-mingw), a library's -- is checked by its +// version's markers rather than one by one (mcppls-clangd --managed-input-root). Only the stores that exist. +std::vector managed_package_stores() { + const std::string home { platform::dirs::home_directory() }; + const auto under = [&](std::string_view variable, std::string_view fallback, std::string_view store) { + const auto set = platform::env::get(variable); + const std::string base { set && !set->empty() ? *set : base::join_path(home, fallback) }; + return base::join_path(base, store); + }; + std::vector stores; + for (auto store : { under("MCPP_HOME", ".mcpp", "registry/data/xpkgs"), under("XLINGS_HOME", ".xlings", "data/xpkgs") }) { + if (home.empty() && !base::is_absolute_path(store)) continue; + if (platform::fs::is_directory(store) && std::ranges::find(stores, store) == stores.end()) stores.push_back(std::move(store)); + } + return stores; +} } // namespace EngineTraits traits_for_version(std::string_view version, std::span disabled, @@ -467,7 +486,18 @@ class ClangdEngine final : public Engine { bool built { false }; std::string state; // clangd's last fileStatus state for it bool priming { false }; // opened to put its definitions in clangd's index (N-7), not for a waiting request + // 0.0.14 G1: opened ahead of the editor, for a file the person last worked in; the editor's open adopts it. + bool warm { false }; + std::size_t textHash { 0 }; // of the text clangd was given + Json diagnostics; // what clangd last published for it, for the editor that adopts it }; + static constexpr std::int64_t BACKGROUND_VERSION { 1 }; // a unit opened without the editor is at this version in clangd + static constexpr std::size_t WARM_FILES { 5 }; + static constexpr std::chrono::seconds WARM_WAIT { 60 }; // for the database to have them + static constexpr std::chrono::seconds WARM_UNUSED { 120 }; // closed when the editor has not opened them by then + std::vector warmPending_; + std::optional warmPumpAt_; + Clock::time_point warmGiveUpAt_ {}; std::map> background_; // path key std::set> closedBackground_; // path keys whose closing diagnostics are still to come std::map, std::less<>> backgroundRefused_; // units that did not build in time, as they were @@ -960,6 +990,7 @@ class ClangdEngine final : public Engine { accept_traffic_if_ready_(); open_held_files_(appliedAt); prepare_modules_(); + if (accepting_) warm_now_(); // the plan may have given clangd more of them (G1) } if (unresolvedForgot || doomForgot) host_->request_replan(); } @@ -974,22 +1005,36 @@ class ClangdEngine final : public Engine { note_edit_during_preparation_(document.path); } switch (event.change) { - case DocumentChange::opened: + case DocumentChange::opened: { touch_(document.path); - // clangd has it open without the editor: it is opened again as the editor's, so its diagnostics come again. - if (!document.path.empty()) close_background_(base::path_key(document.path)); - if (const std::string key { document.path.empty() ? std::string {} : base::path_key(document.path) }; !key.empty()) { - if (const auto doom = doomedFiles_.find(key); doom != doomedFiles_.end()) { + // clangd has it open without the editor: it is opened again as the editor's, so its diagnostics come again -- + // unless it was opened ahead of the editor (G1), which then adopts it below. + const std::string openedKey { document.path.empty() ? std::string {} : base::path_key(document.path) }; + const bool warmUnit { !openedKey.empty() && warm_unit_(openedKey) }; + if (!openedKey.empty() && !warmUnit) close_background_(openedKey); + const auto closeWarm = [&] { if (warmUnit) close_background_(openedKey); }; + if (!openedKey.empty()) { + if (const auto doom = doomedFiles_.find(openedKey); doom != doomedFiles_.end()) { + closeWarm(); // Never sent to clangd: it does not know it, so there is nothing to close there. publish_doom_diagnostic_(document, doom->second); break; } } - if (excluded_path_(document.path) || quarantined_(document.path)) break; + if (excluded_path_(document.path) || quarantined_(document.path)) { + closeWarm(); + break; + } // Fix plan F16: what is on disk is what clangd reads the file's imports from. - if (check_disk_(document.path, &document)) break; + if (check_disk_(document.path, &document)) { + closeWarm(); + break; + } if (accepting_) { - open_or_hold_(document, false); + if (!(warmUnit && adopt_warm_(document, Clock::now()))) { + closeWarm(); + open_or_hold_(document, false); + } prepare_imports_of_(document); queue_implementations_of_(document.path); pump_implementations_(Clock::now()); @@ -1000,6 +1045,7 @@ class ClangdEngine final : public Engine { held_.try_emplace(base::path_key(document.path), HeldFile { Clock::now(), false }); } break; + } case DocumentChange::changed: touch_(document.path); // clangd is given the whole text when it is given the file. @@ -1395,6 +1441,10 @@ class ClangdEngine final : public Engine { consider(restartAt_); consider(stuckCheckAt_); consider(doomRetryAt_); + consider(warmPumpAt_); + for (const auto& [key, unit] : background_) { + if (unit.warm) consider(unit.openedAt + WARM_UNUSED); + } if (accepting_) consider(spinSampleAt_); if (pendingExit_) consider(pendingExit_->at + EXIT_CONTEXT_WAIT); if (upSince_) consider(*upSince_ + RECOVERED_AFTER); @@ -1440,6 +1490,7 @@ class ClangdEngine final : public Engine { } if (implementationRestAt_ && *implementationRestAt_ <= now) queue_rest_of_implementations_(now); else if (!implementationQueue_.empty() && !primer_.busy()) pump_implementations_(now); // preparation ended another way + pump_warm_(now); if (pendingExit_ && now >= pendingExit_->at + EXIT_CONTEXT_WAIT) settle_exit_(); if (!closingAfterBuild_.empty()) settle_disk_builds_(now); if (diskRecheckAt_ && *diskRecheckAt_ <= now) recheck_disk_(now); @@ -1719,6 +1770,14 @@ class ClangdEngine final : public Engine { if (base::to_lower_ascii(options_.formatFallbackStyle) == "mcpp" && !options_.mcppFormatStyle) add_issue_(Issue { "format-style-unavailable", "the selected engine does not declare the mcpp formatting style", "mcppls.showLogs", "environment" }); config.modulesSupport = modulesSupport_; // WA-CLANGD-009 + // 0.0.14 Windows plan. A2: lists complete in one answer; C1: preambles in memory where that is the cheaper store. + if (options_.completionResultLimit > 0) config.resultLimit = options_.completionResultLimit; + config.preambleStorage = preamble_storage(options_.preambleStorage, mcppls::os::FAMILY == mcppls::os::Family::windows, total_memory_bytes()); + // By the features the verified engine declares (an external clangd is given none of these). + const auto declares = [&](std::string_view feature) { return std::ranges::find(options_.verifiedFeatures, feature) != options_.verifiedFeatures.end(); }; + if (declares("managed-input-roots")) config.managedInputRoots = managed_package_stores(); + if (declares("persistent-module-memos")) config.moduleMemoDirectory = base::join_path(host_->cache_directory(), "clangd-memos"); + if (declares("retained-asts-option")) config.maxRetainedAsts = retained_asts(total_memory_bytes()); config.extraArguments = options_.extraArguments; // Extra engine arguments for troubleshooting, e.g. MCPPLS_ENGINE_ARGUMENTS="-j=8 --background-index-priority=background". if (auto extra = platform::env::get("MCPPLS_ENGINE_ARGUMENTS")) { @@ -2350,6 +2409,7 @@ class ClangdEngine final : public Engine { } prepare_modules_(); pump_implementations_(Clock::now()); + warm_now_(); host_->status_changed(); } @@ -2452,11 +2512,13 @@ class ClangdEngine final : public Engine { if (!diagnosedKey.empty() && doomedFiles_.contains(diagnosedKey)) return; // A unit opened without the editor: its diagnostics are nobody's. if (const auto unit = background_.find(diagnosedKey); unit != background_.end() && !host_->has_document(uri)) { + if (unit->second.warm) unit->second.diagnostics = params.value("diagnostics", Json::array()); if (!unit->second.built) { unit->second.built = true; lastBackgroundBuiltAt_ = Clock::now(); const auto took = std::chrono::duration_cast(Clock::now() - unit->second.openedAt).count(); if (unit->second.priming) log::debug("{} built in clangd in {} ms for its index ({})", unit->second.path, took, host_->root_directory()); + else if (unit->second.warm) log::info("{} built in clangd in {} ms ahead of the editor ({})", unit->second.path, took, host_->root_directory()); else log::info("{} built in clangd in {} ms to find definitions ({})", unit->second.path, took, host_->root_directory()); } // Every unit built here, for a search or for the index, is now in clangd's index (N-7). @@ -4084,6 +4146,107 @@ class ClangdEngine final : public Engine { return std::nullopt; } + // ---- files the person last worked in, ready before the editor opens them (0.0.14 G1) ---------------- + + void warm(std::span paths) override { + warmPending_.clear(); + for (const auto& path : paths) { + if (warmPending_.size() >= WARM_FILES) break; + if (!path.empty() && project::is_cxx_source_name(path)) warmPending_.push_back(path); + } + if (warmPending_.empty()) return; + warmGiveUpAt_ = Clock::now() + WARM_WAIT; + warmPumpAt_ = Clock::now(); + } + + // As soon as clangd takes files, and whenever a plan may have given it more of the pending ones. + void warm_now_() { + if (warmPending_.empty()) return; + warmPumpAt_ = Clock::now(); + pump_warm_(Clock::now()); + } + + // Opens the pending ones the database has, once clangd takes files; closes those the editor never opened. + void pump_warm_(Clock::time_point now) { + if (warmPumpAt_ && *warmPumpAt_ <= now) { + warmPumpAt_.reset(); + if (accepting_) { + // By path: the editor's URI for a file may be spelled otherwise (Windows: a drive letter's case, an escaped ':'). + std::set> openInEditor; + for (const auto& document : host_->documents()) { + if (!document.path.empty()) openInEditor.insert(base::path_key(document.path)); + } + std::erase_if(warmPending_, [&](const std::string& path) { + const std::string key { base::path_key(path) }; + if (openInEditor.contains(key) || background_.contains(key) || doomed_path_(path)) return true; + if (!writtenArguments_.contains(key)) return false; // not in the database (yet) + if (open_in_background_(path, {}, now)) background_[key].warm = true; + return true; + }); + } + if (now >= warmGiveUpAt_) warmPending_.clear(); + if (!warmPending_.empty()) warmPumpAt_ = now + std::chrono::seconds { 1 }; + } + std::vector unused; + for (const auto& [key, unit] : background_) { + if (unit.warm && now - unit.openedAt >= WARM_UNUSED) unused.push_back(key); // one the editor opened was adopted, and is no longer here + } + for (const auto& key : unused) { + log::info("closing {} in clangd: the editor did not open it ({})", background_.at(key).path, host_->root_directory()); + close_background_(key); + } + } + + // The editor opens a file clangd has open ahead of it: it is the editor's now, with what clangd built for it. The + // same text keeps clangd's build and its diagnostics, or the build still under way when clangd has the editor's + // version too; another text (an unsaved edit the editor restored) or version is sent as a change, which reuses the + // preamble clangd built. + bool adopt_warm_(const DocumentView& document, Clock::time_point now) { + const std::string key { base::path_key(document.path) }; + const auto found = background_.find(key); + if (found == background_.end() || !found->second.warm || !accepting_) return false; + const BackgroundUnit unit { std::move(found->second) }; + background_.erase(found); + const std::string text { engine_text_(document.uri, document.text) }; + spin_.sent(document.uri, text_hash_(text), now); + note_database_read_(); + const auto ahead = std::chrono::duration_cast(now - unit.openedAt).count(); + const bool sameText { text_hash_(text) == unit.textHash }; + const auto awaitDiagnostics = [&] { + if (diagnosed_.contains(document.uri)) return; + awaitingDiagnostics_.insert(document.uri); + awaitingSince_[document.uri] = now; + schedule_stuck_check_(now + GENERAL_PATIENCE); + }; + if (sameText && unit.built && unit.diagnostics.is_array()) { + diagnosed_.insert(document.uri); + Json diagnostics = unit.diagnostics; // `=`: braces would make an array holding it + rewrite_diagnostics_(document.uri, diagnostics); + host_->publish_engine_diagnostics(ENGINE_ID, document.uri, std::move(diagnostics), document.version); + lastDiagnosticsAt_ = now; + log::info("{} was ready in clangd {} ms before the editor opened it ({})", document.path, ahead, host_->root_directory()); + } else if (sameText && !unit.built && document.version == BACKGROUND_VERSION) { + // clangd is building this very text and version: sending it again would only start that build over. + awaitDiagnostics(); + log::info("{} was opened in clangd {} ms before the editor and is still being built there; the editor takes it as it is ({})", + document.path, ahead, host_->root_directory()); + } else { + Json params { { "textDocument", Json { { "uri", document.uri }, { "version", document.version } } }, + { "contentChanges", Json::array({ Json { { "text", text } } }) } }; + if (!send_(lsp::make_notification("textDocument/didChange", std::move(params)))) return false; + awaitDiagnostics(); + log::info("{} was opened in clangd {} ms before the editor, which has {}; clangd takes it as a change ({})", document.path, + ahead, sameText ? "another version of the same text" : "other text", host_->root_directory()); + } + host_->record_event("warm-adopted", Json { { "file", document.path }, { "aheadMs", ahead }, { "sameText", sameText }, { "built", unit.built } }); + return true; + } + + bool warm_unit_(std::string_view key) const { + const auto found = background_.find(key); + return found != background_.end() && found->second.warm; + } + bool open_in_background_(const std::string& path, std::string_view module, Clock::time_point now, bool priming = false) { const std::string key { base::path_key(path) }; if (const auto refused = backgroundRefused_.find(key); refused != backgroundRefused_.end()) { @@ -4111,12 +4274,15 @@ class ClangdEngine final : public Engine { if (traits_.hangsOnTrailingDotModuleName) { if (auto sanitized = sanitize_module_names(*text); sanitized.changed()) *text = std::move(sanitized.text); } - Json params { { "textDocument", Json { { "uri", uri }, { "languageId", "cpp" }, { "version", 1 }, { "text", std::move(*text) } } } }; + const std::size_t textHash { text_hash_(*text) }; // before the text moves into the message + Json params { { "textDocument", Json { { "uri", uri }, { "languageId", "cpp" }, { "version", BACKGROUND_VERSION }, { "text", std::move(*text) } } } }; if (!send_(lsp::make_notification("textDocument/didOpen", std::move(params)))) return false; note_database_read_(); closedBackground_.erase(key); background_[key] = BackgroundUnit { path, uri, now, now, false, {}, priming }; + background_[key].textHash = textHash; if (priming) log::debug("opening {} in clangd to index its definitions (module {}, {})", path, module, host_->root_directory()); + else if (module.empty()) log::info("opening {} in clangd ahead of the editor: it was among the last files in use ({})", path, host_->root_directory()); else log::info("opening {} in clangd to find definitions in module {} ({})", path, module, host_->root_directory()); return true; } diff --git a/src/engine/clangd.cppm b/src/engine/clangd.cppm index 30cb1d61..ca745fbc 100644 --- a/src/engine/clangd.cppm +++ b/src/engine/clangd.cppm @@ -47,6 +47,10 @@ struct Options { // mcppls.engine.workers (R-2, plan 2026-09-30): clangd's workers; nullopt or 0 is the automatic value (engine_workers). std::optional workers; std::vector disabledWorkarounds; // registered workarounds turned off (import-hang plan §9) + // mcppls.engine.preambleStorage (0.0.14 C1): auto (memory on Windows with 16 GiB or more), disk or memory. + std::string preambleStorage { "auto" }; + // 0.0.14 A2: clangd's --limit-results; 0 leaves clangd's own (100). + std::size_t completionResultLimit { 1000 }; // mcppls.index.primeImplementationUnits (plan 2026-09-27 N-7): implementation units are built through clangd's // foreground in the background, so go-to-definition reaches them; off leaves only the search a definition request starts. bool primeImplementationUnits { true }; diff --git a/src/engine/clangd/guard.cpp b/src/engine/clangd/guard.cpp index a36b5a97..9461e8ee 100644 --- a/src/engine/clangd/guard.cpp +++ b/src/engine/clangd/guard.cpp @@ -3,6 +3,7 @@ module mcppls.engine.clangd.guard; import std; import mcppls.base.text; import mcppls.platform.fs; +import mcppls.platform.system; namespace mcppls::engine::clangd { @@ -335,18 +336,16 @@ std::size_t preparation_limit(std::size_t workers, bool waitingOnPreparation) { return waitingOnPreparation ? background : std::max(1, background / 2); } -std::optional total_memory_bytes() { - const auto info = platform::fs::read_file("/proc/meminfo"); - if (!info) return std::nullopt; - for (const auto line : base::split_lines(*info)) { - if (!line.starts_with("MemTotal:")) continue; - std::uint64_t kib { 0 }; - const auto digits = base::trim(line.substr(9)); - const auto end = digits.find(' '); - const auto number = digits.substr(0, end); - if (std::from_chars(number.data(), number.data() + number.size(), kib).ec != std::errc {}) return std::nullopt; - return kib * 1024; - } +std::optional total_memory_bytes() { return platform::system::physical_memory_bytes(); } + +std::string preamble_storage(std::string_view setting, bool windows, std::optional memoryBytes) { + if (setting == "memory" || setting == "disk") return std::string { setting }; + if (windows && memoryBytes && *memoryBytes >= PREAMBLES_IN_MEMORY_FROM) return "memory"; + return {}; +} + +std::optional retained_asts(std::optional memoryBytes) { + if (memoryBytes && *memoryBytes >= PREAMBLES_IN_MEMORY_FROM) return 6; return std::nullopt; } diff --git a/src/engine/clangd/guard.cppm b/src/engine/clangd/guard.cppm index c34b7c36..59f7df91 100644 --- a/src/engine/clangd/guard.cppm +++ b/src/engine/clangd/guard.cppm @@ -233,7 +233,15 @@ std::size_t engine_workers(std::size_t hardwareThreads, std::optional total_memory_bytes(); +// 0.0.14 Windows plan C1, C2: the memory from which preambles are kept in memory and more ASTs are retained. +inline constexpr std::uint64_t PREAMBLES_IN_MEMORY_FROM { std::uint64_t { 16 } << 30 }; +// clangd's --pch-storage for mcppls.engine.preambleStorage: "memory" or "disk" as asked; for "auto", memory on +// Windows with PREAMBLES_IN_MEMORY_FROM or more -- a preamble written to the temporary directory there is scanned on +// write and on read -- and clangd's own (empty) elsewhere. +std::string preamble_storage(std::string_view setting, bool windows, std::optional memoryBytes); +// mcppls-clangd's --max-retained-asts (retained-asts-option): 6 with PREAMBLES_IN_MEMORY_FROM or more, else clangd's own. +std::optional retained_asts(std::optional memoryBytes); } // namespace mcppls::engine::clangd diff --git a/src/engine/clangd/process.cpp b/src/engine/clangd/process.cpp index c6d6f0e0..fefce93d 100644 --- a/src/engine/clangd/process.cpp +++ b/src/engine/clangd/process.cpp @@ -65,6 +65,21 @@ std::vector clangd_arguments(const ProcessConfig& config) { } else if (config.ownedCachePayloadMiB && !payloadGiven) { arguments.push_back(std::format("--modules-builder-owned-cache-payload-mib={}", *config.ownedCachePayloadMiB)); } + // The rest are given unless the person's own arguments name them. + const auto given = [&](std::string_view option) { + return std::ranges::any_of(config.extraArguments, [&](const std::string& argument) { + const std::string_view word { argument }; + return word == option || word == option.substr(1) || word.starts_with(std::string { option } + "=") + || word.starts_with(std::string { option.substr(1) } + "="); + }); + }; + if (config.resultLimit && !given("--limit-results")) arguments.push_back(std::format("--limit-results={}", *config.resultLimit)); + if (!config.preambleStorage.empty() && !given("--pch-storage")) arguments.push_back("--pch-storage=" + config.preambleStorage); + if (!given("--managed-input-root")) { + for (const auto& root : config.managedInputRoots) arguments.push_back("--managed-input-root=" + root); + } + if (!config.moduleMemoDirectory.empty() && !given("--module-memo-dir")) arguments.push_back("--module-memo-dir=" + config.moduleMemoDirectory); + if (config.maxRetainedAsts && !given("--max-retained-asts")) arguments.push_back(std::format("--max-retained-asts={}", *config.maxRetainedAsts)); arguments.insert(arguments.end(), config.extraArguments.begin(), config.extraArguments.end()); return arguments; } @@ -390,11 +405,13 @@ std::optional parse_ast_build(std::string_view message) { bool telling_message(std::string_view message) { // mcppls-clangd 0.0.13 says why a module is rebuilt ("Rebuilding module M: ..."), what a file's prerequisites cost // ("Prerequisites of F: ..."), why a preamble is not reused ("Preamble of F is not compatible: ...") and when - // completion answers before its modules ("Completion in F answer..."); the rest are clangd's own. - static constexpr std::array PREFIXES { + // completion answers before its modules ("Completion in F answer..."); 0.0.15 what a slow completion cost + // ("Completion in F: N ms (...)"), what was kept from the last session ("Module memos: ...") and which package + // stores are checked by their markers ("Module inputs: ..."); the rest are clangd's own. + static constexpr std::array PREFIXES { "Built prerequisite modules for file", "Building first preamble for", "Rebuilding invalidated preamble", "Built preamble of size", "Built module ", "Rebuilding module ", "Prerequisites of ", "Preamble of ", - "Completion in ", "Failed to build", + "Completion in ", "Failed to build", "Module memos: ", "Module inputs: ", }; return std::ranges::any_of(PREFIXES, [&](std::string_view prefix) { return message.starts_with(prefix); }); } diff --git a/src/engine/clangd/process.cppm b/src/engine/clangd/process.cppm index b5a1c1a2..e76b951b 100644 --- a/src/engine/clangd/process.cppm +++ b/src/engine/clangd/process.cppm @@ -25,6 +25,15 @@ struct ProcessConfig { // --experimental-modules-support. WA-CLANGD-009: off for a project that uses no modules, where it only costs every // request a scan of the file's module dependencies. bool modulesSupport { true }; + // 0.0.14 (Windows plan A2): completions clangd gives at most (--limit-results). Above its 100, a list whose + // matches fit is complete, and the editor filters it as the person types instead of asking again per key. + std::optional resultLimit; + std::string preambleStorage; // "memory" or "disk" (--pch-storage); empty: clangd's own (disk) + // mcppls-clangd 0.0.15, by declared feature: package stores checked by their markers (managed-input-roots), + // where module memos are kept across sessions (persistent-module-memos), ASTs kept (retained-asts-option). + std::vector managedInputRoots; + std::string moduleMemoDirectory; + std::optional maxRetainedAsts; }; class Process { diff --git a/src/engine/engine.cppm b/src/engine/engine.cppm index a7481e78..45ff3a42 100644 --- a/src/engine/engine.cppm +++ b/src/engine/engine.cppm @@ -200,6 +200,9 @@ public: virtual void document(const DocumentEvent& event) = 0; virtual void notify(const Json& message) = 0; // any other client notification virtual void sources_changed() = 0; // sources or build descriptions changed on disk + // 0.0.14 G1: the files the person last worked in, most recent first, to get ready before the editor opens them + // again. An engine without such work ignores it. + virtual void warm(std::span paths) { (void)paths; } virtual bool claims(const RequestView& request) const = 0; // Answers the client's request `message`; `reply` is called exactly once, on the event loop. diff --git a/src/orchestrator/cache.cpp b/src/orchestrator/cache.cpp index 0be1424e..43f561f6 100644 --- a/src/orchestrator/cache.cpp +++ b/src/orchestrator/cache.cpp @@ -379,7 +379,7 @@ Json report(std::string_view workspaceDirectory, const Budget& budget, std::chro instanceList.push_back(Json { { "token", described ? described->token : std::string { base::file_name(entry) } }, { "version", described ? described->version : std::string {} }, { "root", described ? described->root : std::string {} }, - { "at", Json { described ? described->at : std::int64_t { 0 } } }, + { "at", Json(described ? described->at : std::int64_t { 0 }) }, { "bytes", bytes }, { "shared", described ? described->shared : true }, { "alive", instance_alive(described, now) }, diff --git a/src/orchestrator/completion.cpp b/src/orchestrator/completion.cpp index fc25debd..b259ad79 100644 --- a/src/orchestrator/completion.cpp +++ b/src/orchestrator/completion.cpp @@ -191,6 +191,21 @@ std::optional word_key(std::string_view text, base::Position position) return WordKey { position.line, std::string { text.substr(lineStart, start - lineStart) }, std::string { text.substr(start, *offset - start) } }; } +std::optional outside_word(std::string_view text, base::Position position) { + const auto offset = base::offset_at(text, position); + if (!offset) return std::nullopt; + std::size_t start { *offset }; + while (start > 0 && base::is_identifier_char(text[start - 1])) --start; + const std::uint64_t before { std::hash {}(text.substr(0, start)) }; + const std::uint64_t after { std::hash {}(text.substr(*offset)) }; + return before ^ (after + 0x9E3779B97F4A7C15ull + (before << 6) + (before >> 2)); +} + +bool is_complete(const Json& result) { + if (result.is_array()) return true; + return result.is_object() && result.contains("items") && !result.value("isIncomplete", false); +} + bool typed_on(const WordKey& earlier, const WordKey& later) { return earlier.line == later.line && earlier.before == later.before && later.typed.starts_with(earlier.typed); } diff --git a/src/orchestrator/completion.cppm b/src/orchestrator/completion.cppm index 81c42ad5..52e63f14 100644 --- a/src/orchestrator/completion.cppm +++ b/src/orchestrator/completion.cppm @@ -83,6 +83,13 @@ std::optional word_key(std::string_view text, base::Position position); // filters it by what was typed since. A word typed back (a backspace), or another word, is not. bool typed_on(const WordKey& earlier, const WordKey& later); +// 0.0.14 A1: the document outside the word at `position` -- all of it before the word's start, all of it after the +// cursor -- as one value. A complete answer for a word still holds for a later request in it while this is the same: +// what the engine answered from is unchanged but for the word, which the client filters by. +std::optional outside_word(std::string_view text, base::Position position); +// A completion result the client filters by itself as the person types: a list not marked incomplete. +bool is_complete(const Json& result); + // C-2: a completion answered for an earlier position in the same word, for `position`: each item's edit that replaced // the word up to the old cursor replaces it up to this one (a client drops an item whose range does not contain the // position it asked at). Items without an edit are kept as they are; an edit on another line or starting after diff --git a/src/orchestrator/workspace.cpp b/src/orchestrator/workspace.cpp index 3a1556c3..198e48bc 100644 --- a/src/orchestrator/workspace.cpp +++ b/src/orchestrator/workspace.cpp @@ -13,8 +13,10 @@ import mcppls.base.version; import mcppls.orchestrator.cache; import mcppls.engine.clangd.process; import mcppls.platform.dirs; +import mcppls.platform.env; import mcppls.platform.fs; import mcppls.platform.process; +import mcppls.platform.system; import mcppls.platform.toolenv; import mcppls.platform.toolrun; import mcppls.platform.task; @@ -183,6 +185,54 @@ struct Workspace::Impl final : engine::Host { bool kitEnabled { true }; std::string cacheDirectory; + std::optional cacheOnDevDrive; // Windows only (0.0.14 E1) + // 0.0.14 G1: the files last worked in, most recent first, kept in the cache directory: the next session has the core + // engine get them ready before the editor opens them again. + static constexpr std::size_t RECENT_FILES { 5 }; + static constexpr std::chrono::seconds RECENT_FILES_WRITTEN { 15 }; + std::deque recentFiles; + bool recentFilesChanged { false }; + std::optional recentFilesWrittenAt; + + void note_recent_file(const std::string& path) { + if (path.empty() || !base::is_within(path, root) || !project::is_cxx_source_name(path)) return; + if (!recentFiles.empty() && base::same_path(recentFiles.front(), path)) return; + std::erase_if(recentFiles, [&](const std::string& recent) { return base::same_path(recent, path); }); + recentFiles.push_front(path); + while (recentFiles.size() > RECENT_FILES) recentFiles.pop_back(); + recentFilesChanged = true; + } + + // A file the person closed is not among the next session's: an editor that quits closes nothing, so what it + // restores stays, and what was put away goes. + void forget_recent_file(const std::string& path) { + if (std::erase_if(recentFiles, [&](const std::string& recent) { return base::same_path(recent, path); }) > 0) recentFilesChanged = true; + } + + void save_recent_files(bool now) { + if (!recentFilesChanged) return; + if (!now && recentFilesWrittenAt && Clock::now() - *recentFilesWrittenAt < RECENT_FILES_WRITTEN) return; + Json list = Json::array(); + for (const auto& path : recentFiles) list.push_back(path); + (void)platform::fs::write_file_atomic(base::join_path(cacheDirectory, "recent-files.json"), list.dump()); + recentFilesChanged = false; + recentFilesWrittenAt = Clock::now(); + } + + std::vector read_recent_files() { + std::vector paths; + const auto text = platform::fs::read_file(base::join_path(cacheDirectory, "recent-files.json")); + if (!text) return paths; + const Json list = Json::parse(*text, nullptr, false); + if (!list.is_array()) return paths; + for (const auto& item : list) { + if (!item.is_string() || paths.size() >= RECENT_FILES) continue; + const std::string path { item.get() }; + if (base::is_within(path, root) && platform::fs::is_regular_file(path)) paths.push_back(path); + } + recentFiles.assign(paths.begin(), paths.end()); + return paths; + } // overall design 6.3: the lease on /workspaces/; a second instance works in a private directory. std::optional lease; // ---- the cache mcppls owns (0.0.10 plan C-7, C-8, C-9, C-13.1) ---------------------- @@ -360,6 +410,20 @@ struct Workspace::Impl final : engine::Host { // elsewhere), so it serves the next requests of that word and not a return to it minutes later. static constexpr std::chrono::seconds LATE_ANSWER_KEPT { 5 }; std::map> lateAnswers; // document URI -> the last late answer, for its word + // 0.0.14 A1: the core engine's last complete answer in a word, with the document outside the word as it was then. A + // client that asks again as the word is typed on (some ask at every key, whatever the list says) is given it at once: + // the engine's candidates for the word are the same while nothing outside it changed. Only a complete list: one the + // engine cut short may lack what the longer word matches. MCPPLS_COMPLETION_REFILTER=off turns it off. + struct WordAnswer { + completion::WordKey key; + std::uint64_t outside { 0 }; + Json result; + Clock::time_point at; + }; + static constexpr std::chrono::seconds WORD_ANSWER_KEPT { 30 }; + std::map> wordAnswers; // document URI -> its last complete core answer + const bool wordAnswersOn { platform::env::get("MCPPLS_COMPLETION_REFILTER").value_or("") != "off" }; + std::size_t completionsFromWord { 0 }; // Project model and plan. std::shared_ptr model; @@ -509,6 +573,13 @@ struct Workspace::Impl final : engine::Host { // and the database live here, and clangd answers for them under the name it was given. (void)platform::fs::create_directories(cacheDirectory); cacheDirectory = platform::fs::canonical_path(cacheDirectory); + // 0.0.14 E1: on Windows, a cache off a Dev Drive has every file clangd writes and reads there scanned by + // Microsoft Defender as it happens. Said once, in the log and the overview; nothing is changed for the person. + cacheOnDevDrive = platform::system::on_dev_drive(cacheDirectory); + if (cacheOnDevDrive == false) { + log::info("the cache {} is not on a Dev Drive: Microsoft Defender scans the module and preamble files clangd writes and reads " + "there as it does; a Dev Drive scans them asynchronously (docs/50-troubleshooting.md, \"Windows\")", cacheDirectory); + } // usable plan W9.3: taken now, before this root is even started, so the client cannot // possibly have created or changed a watched file yet (see start_watch_polling below). watchBaseline = watched_files_snapshot(root); @@ -932,6 +1003,7 @@ struct Workspace::Impl final : engine::Host { if (const auto position = position_of(job.params)) { job.keywords = index::keyword_completion(job.text, *position, index.scan_of(job.path), index::KeywordOptions { .suggestModulesAfterImport = vscodeLike }); + if (answered_from_word(jobId, uri, *position)) return; if (answered_late(jobId, uri, *position)) return; } } @@ -1098,7 +1170,7 @@ struct Workspace::Impl final : engine::Host { // timeouts and watchdogs still see the request until clangd lets go of it. void answer_without_core(std::uint64_t jobId) { Job& job = jobs.at(jobId); - const Json clientId { job.clientId }; + const Json clientId = job.clientId; // `=`: braces make an array holding it (CWG 2137) if (job.keywords.is_array() && !job.keywords.empty()) ++keywordsWithoutEngine; job.answeredBy = "mcppls"; Json result; @@ -1123,6 +1195,42 @@ struct Workspace::Impl final : engine::Host { } } + // 0.0.14 A1: a complete core answer, kept for its word. + void note_word_answer(const Job& job, const Json& result) { + if (!wordAnswersOn || !completion::is_complete(result) || completion::is_empty(result)) return; + const auto position = position_of(job.params); + if (!position) return; + const std::string uri { uri_of_params(job.params) }; + auto key = completion::word_key(job.text, *position); + const auto outside = completion::outside_word(job.text, *position); + if (!key || !outside) { + wordAnswers.erase(uri); + return; + } + wordAnswers[uri] = WordAnswer { std::move(*key), *outside, result, Clock::now() }; + } + + // 0.0.14 A1: a request typed on in the word of a complete core answer, with the document outside the word as it + // was, is given that answer for its position. + bool answered_from_word(std::uint64_t jobId, const std::string& uri, base::Position position) { + if (!wordAnswersOn || coreEngine == nullptr) return false; + const auto answer = wordAnswers.find(uri); + if (answer == wordAnswers.end()) return false; + Job& job = jobs.at(jobId); + const auto key = completion::word_key(job.text, position); + const auto outside = completion::outside_word(job.text, position); + if (!key || !outside || !completion::typed_on(answer->second.key, *key) || *outside != answer->second.outside + || Clock::now() - answer->second.at > WORD_ANSWER_KEPT) { + wordAnswers.erase(answer); + return false; + } + job.coreAnswered = true; + job.answeredBy = std::string { coreEngine->id() }; + ++completionsFromWord; + finish_job(jobId, completion::retarget(answer->second.result, position)); + return true; + } + // C-2: a completion in a word the core engine answered late is given that answer, at once; one in a word the core // engine is still working on waits for it, within its own budget. A completion in another word ends the core // engine's work on the one before: nobody is typing it any more. An import directive's module names are mcppls's own. @@ -1204,6 +1312,7 @@ struct Workspace::Impl final : engine::Host { const bool typing { current->second.method == lsp::method::TEXT_DOCUMENT_COMPLETION || current->second.method == lsp::method::TEXT_DOCUMENT_SIGNATURE_HELP }; if (typing) note_core_completion(current->second.path, current->second.started); + if (current->second.method == lsp::method::TEXT_DOCUMENT_COMPLETION) note_word_answer(current->second, answer.value); } if (!answer.value.is_null()) { current->second.engineRepliedAt = Clock::now(); @@ -2386,6 +2495,7 @@ struct Workspace::Impl final : engine::Host { if (kit) about["kit"] = kit->name; if (detectedSource == project::SourceKind::mcpp) about["mcpp"] = producerVersion; if (const std::string mismatch { version_mismatch() }; !mismatch.empty()) about["mismatch"] = mismatch; + if (cacheOnDevDrive) about["cacheDevDrive"] = *cacheOnDevDrive; return about; } @@ -2891,6 +3001,7 @@ struct Workspace::Impl final : engine::Host { } void handle_timers() { + save_recent_files(false); // 0.0.13 W10 (#52-8): a turn of the timers that holds the loop says which part held it; one once took 15 minutes // and the log could say only "its timers". const auto timersBegan = Clock::now(); @@ -3064,6 +3175,13 @@ void Workspace::start(Json clientParams, bool clientSupportsStatus, bool usePoll about.contains("extension") ? std::format(" · extension {}", about["extension"].get()) : std::string {}, editor); if (about.contains("mismatch")) log::warning("version mismatch ({}): {}", root_, about["mismatch"].get()); } + // 0.0.14 G1: the files of the last session, ready before the editor opens them again. + if (impl_->coreEngine != nullptr) { + if (const auto recent = impl_->read_recent_files(); !recent.empty()) { + log::info("the last files in use ({}): {} of them, opened in clangd ahead of the editor once it takes files", root_, recent.size()); + impl_->coreEngine->warm(recent); + } + } if (!impl_->payloadCorrupt && impl_->kitEnabled && !impl_->payload.kit.empty()) { if (auto kit = spec::load_kit(impl_->payload.kit)) { impl_->kit = std::move(*kit); @@ -3108,6 +3226,7 @@ void Workspace::allow_status_notifications() { } void Workspace::shut_down() { + impl_->save_recent_files(true); if (impl_->reviewProcess) impl_->reviewProcess->kill(); for (const auto& engine : impl_->engines) engine->shut_down(); if (impl_->lease) impl_->lease->release(); @@ -3122,6 +3241,7 @@ void Workspace::did_open(const Json& params) { const Document& document = impl_->documents_.open(uri, path, item->value("languageId", std::string { "cpp" }), item->value("version", std::int64_t { 0 }), item->value("text", std::string {})); if (!path.empty()) { + impl_->note_recent_file(path); impl_->index.update(path, document.text); impl_->note_structure_change(path); impl_->note_opened(path); @@ -3141,6 +3261,7 @@ void Workspace::did_change(const Json& message, const Json& params) { const Document* document { impl_->documents_.find(uri) }; if (!document->path.empty()) { impl_->editedAt[base::path_key(document->path)] = Clock::now(); + impl_->note_recent_file(document->path); impl_->index.update(document->path, document->text); impl_->note_structure_change(document->path, true); } @@ -3155,6 +3276,7 @@ void Workspace::did_close(const Json& message, const Json& params) { const Document document { *found }; impl_->documents_.close(uri); if (!document.path.empty()) { + impl_->forget_recent_file(document.path); impl_->editedAt.erase(base::path_key(document.path)); impl_->completionPace_.erase(document.path); // the file's pace says how its engine answered while it was open } @@ -3432,7 +3554,8 @@ Json Workspace::report() const { Json completionCosts { { "spaceTrigger", Json { { "count", impl.spaceTrigger.count }, { "passed", impl.spaceTrigger.passed }, { "maxMicros", impl.spaceTrigger.maxMicros }, { "totalMicros", impl.spaceTrigger.totalMicros } } }, { "keywordsWithoutEngine", impl.keywordsWithoutEngine }, { "wordsWithoutCore", impl.wordsWithoutCore }, - { "late", Json { { "arrived", impl.lateCompletionsArrived }, { "served", impl.lateCompletionsServed } } } }; + { "late", Json { { "arrived", impl.lateCompletionsArrived }, { "served", impl.lateCompletionsServed } } }, + { "fromWord", impl.completionsFromWord } }; Json startups = Json::array(); if (auto text = platform::fs::read_file(base::join_path(impl.cacheDirectory, "startups.json"))) { startups = Json::parse(*text, nullptr, false); @@ -3589,7 +3712,8 @@ Json Workspace::cache_report() const { SweepStart Workspace::prepare_sweep(const Json& params) { auto& impl = *impl_; SweepStart start; - const Json given { params.value("categories", Json::array()) }; + // `=`: braces would make an array holding the list, and no category would ever be found in it. + const Json given = params.value("categories", Json::array()); const auto wants = [&](std::string_view category, bool byDefault) { if (!given.is_array() || given.empty()) return byDefault; return std::ranges::find(given, Json(std::string { category })) != given.end(); diff --git a/src/orchestrator/workspace.cppm b/src/orchestrator/workspace.cppm index 857cdacf..28e4c6ed 100644 --- a/src/orchestrator/workspace.cppm +++ b/src/orchestrator/workspace.cppm @@ -52,6 +52,7 @@ struct SessionOptions { std::chrono::milliseconds requestTimeout { std::chrono::seconds { 60 } }; std::string formatFallbackStyle { "auto" }; std::string engineWorkers { "auto" }; // mcppls.engine.workers (R-2): "auto" or a number + std::string preambleStorage { "auto" }; // mcppls.engine.preambleStorage (0.0.14 C1): auto, disk or memory // Registered workarounds turned off (import-hang plan §9): to see whether one is still needed. std::vector disabledWorkarounds; // initializationOptions.semanticTokens (design doc 2026-09-25 K/§7, contract T0): native @@ -130,7 +131,7 @@ std::map watched_files_snapshot(std::strin struct SweepStart { bool started { false }; bool dryRun { false }; - Json immediate { Json::object() }; + Json immediate = Json::object(); // `=`: braces make an array holding it // Valid when started; every path and limit the walks need, with no Workspace state in it. bool copies { true }, instances { true }, trash { true }, staleCommands { false }, budget { true }; bool engineLive { false }; // S3 5.8: staleCommands only with no live engine diff --git a/src/server/session.cpp b/src/server/session.cpp index f611af6d..d7062d60 100644 --- a/src/server/session.cpp +++ b/src/server/session.cpp @@ -463,6 +463,7 @@ class Session { options_.requestTimeout = std::chrono::duration_cast(settings.seconds_value("requestTimeout")); options_.formatFallbackStyle = settings.string_value("format.fallbackStyle"); options_.engineWorkers = settings.string_value("engine.workers"); + options_.preambleStorage = settings.string_value("engine.preambleStorage"); options_.disabledWorkarounds = settings.list_value("disableWorkaround"); options_.semanticTokensModules = settings.bool_value("semanticTokens.modules"); options_.semanticTokensModuleType = settings.bool_value("semanticTokens.moduleType"); diff --git a/tests/test_completion.cpp b/tests/test_completion.cpp index a3d9e938..5cb75919 100644 --- a/tests/test_completion.cpp +++ b/tests/test_completion.cpp @@ -298,6 +298,22 @@ int main() { expect(completion::retarget(Json(nullptr), Position { 1, 19 })["items"].empty()); }; + "a complete answer holds for its word while the document outside the word is the same (0.0.14 A1)"_test = [] { + const std::string text { "int main() {\n auto v = obj.s\n}\n" }; + const std::string typedOn { "int main() {\n auto v = obj.si\n}\n" }; + const std::string elsewhere { "int main() {\n auto v = obj.si\n int w;\n}\n" }; + const auto at = completion::outside_word(text, Position { 1, 18 }); + expect(at.has_value()); + expect(completion::outside_word(typedOn, Position { 1, 19 }) == at) << "the word typed on: the same outside it"; + expect(completion::outside_word(elsewhere, Position { 1, 19 }) != at) << "a line added after the word"; + expect(completion::outside_word("int x;\n" + typedOn, Position { 2, 19 }) != at) << "a line added before it"; + expect(completion::outside_word(text, Position { 9, 0 }) == std::nullopt); + expect(completion::is_complete(Json { { "isIncomplete", false }, { "items", Json::array() } })); + expect(!completion::is_complete(Json { { "isIncomplete", true }, { "items", Json::array() } })); + expect(completion::is_complete(Json::array())) << "a bare array is complete"; + expect(!completion::is_complete(Json(nullptr))); + }; + "module names for completion are kept until a declaration changes"_test = [] { idx::ModuleIndex index; index.update("/p/a.cppm", "export module a;\n"); diff --git a/tests/test_engineidentity.cpp b/tests/test_engineidentity.cpp index b9b435fd..20e4c2f2 100644 --- a/tests/test_engineidentity.cpp +++ b/tests/test_engineidentity.cpp @@ -43,4 +43,6 @@ int main() { changed["features"] = Json::array({ 42 }); expect(!base::decode_engine_identity(changed)); }; + + return report(); } diff --git a/tests/test_server.cpp b/tests/test_server.cpp index 66ef8080..183aa111 100644 --- a/tests/test_server.cpp +++ b/tests/test_server.cpp @@ -157,12 +157,16 @@ class BuildingProcess : public cld::Process { std::mutex mutex; std::vector sent; LogHandler log; // what the running process writes to stderr, for a test to say something as clangd + bool diagnose { false }; // each opened file comes back with one warning, not none + std::set building; // URIs whose build goes on until finish() is called for them + std::function finish; }; explicit BuildingProcess(std::shared_ptr shared) : shared_ { std::move(shared) } {} mcppls::base::Result start(const cld::ProcessConfig&, MessageHandler onMessage, ClosedHandler, LogHandler onLog) override { onMessage_ = std::move(onMessage); shared_->log = std::move(onLog); + shared_->finish = [this](const std::string& uri) { publish_(uri); }; running_ = true; return {}; } @@ -176,8 +180,7 @@ class BuildingProcess : public cld::Process { onMessage_(Json { { "jsonrpc", "2.0" }, { "id", message["id"] }, { "result", Json { { "capabilities", Json::object() } } } }); } else if (method == "textDocument/didOpen") { const std::string uri { message["params"]["textDocument"]["uri"].get() }; - onMessage_(Json { { "jsonrpc", "2.0" }, { "method", "textDocument/publishDiagnostics" }, - { "params", Json { { "uri", uri }, { "diagnostics", Json::array() } } } }); + if (!shared_->building.contains(uri)) publish_(uri); } return {}; } @@ -196,6 +199,16 @@ class BuildingProcess : public cld::Process { } private: + void publish_(const std::string& uri) { + Json diagnostics = Json::array(); + if (shared_->diagnose) { + diagnostics.push_back(Json { { "range", Json { { "start", Json { { "line", 0 }, { "character", 0 } } }, { "end", Json { { "line", 0 }, { "character", 3 } } } } }, + { "severity", 2 }, { "message", "unused variable" }, { "source", "clang" } }); + } + onMessage_(Json { { "jsonrpc", "2.0" }, { "method", "textDocument/publishDiagnostics" }, + { "params", Json { { "uri", uri }, { "diagnostics", std::move(diagnostics) } } } }); + } + std::shared_ptr shared_; MessageHandler onMessage_; bool running_ { false }; @@ -258,6 +271,23 @@ class RecordingHost : public eng::Host { Json params_ = Json::object(); }; +// A host whose editor has documents open, and which keeps what the engine publishes for them (0.0.14 G1). +class EditorHost : public RecordingHost { +public: + using RecordingHost::RecordingHost; + std::vector open; + std::vector>> published; // uri, version + std::map diagnostics; // uri -> the last list published + std::vector documents() const override { return open; } + bool has_document(std::string_view uri) const override { + return std::ranges::any_of(open, [&](const eng::DocumentView& document) { return document.uri == uri; }); + } + void publish_engine_diagnostics(std::string_view, const std::string& uri, Json list, std::optional version) override { + published.emplace_back(uri, version); + diagnostics[uri] = std::move(list); + } +}; + idx::ModuleIndex fixture_index() { idx::ModuleIndex index; index.update("/p/src/main.cpp", "import std;\nimport hello.greet;\n\nint main() {\n return 0;\n}\n"); @@ -1457,6 +1487,102 @@ int main() { fs::remove_all(root); }; + "the files of the last session are opened ahead of the editor, which adopts them (0.0.14 G1)"_test = [] { + namespace fs = mcppls::platform::fs; + using mcppls::base::join_path; + const std::string root { join_path(mcppls::platform::dirs::temp_directory(), + std::format("mcppls-test-warm-{}", std::chrono::steady_clock::now().time_since_epoch().count())) }; + (void)fs::create_directories(join_path(root, "src")); + const std::string executable { join_path(root, "clangd") }; + (void)fs::write_file(executable, "pretend-clangd"); + // A project with modules: one without would have clangd restarted without their support first (WA-CLANGD-009). + const std::string mainText { "import hello.greet;\nint main() { return hello::add(1, 2); }\n" }; + const std::string otherText { "int other() { return 1; }\n" }; + const std::string slowText { "int slow() { return 2; }\n" }; + const std::string main { join_path(root, "src/main.cpp") }; + const std::string other { join_path(root, "src/other.cpp") }; + const std::string slow { join_path(root, "src/slow.cpp") }; + const std::string interface { join_path(root, "src/greet.cppm") }; + (void)fs::write_file(main, mainText); + (void)fs::write_file(other, otherText); + (void)fs::write_file(slow, slowText); + (void)fs::write_file(interface, "export module hello.greet;\nexport namespace hello { int add(int a, int b); }\n"); + + auto shared = std::make_shared(); + shared->diagnose = true; // what is handed to the editor is a real list, rewritten as any other + shared->building.insert(mcppls::base::path_to_uri(slow)); // still being built when the editor opens it + cld::Options options; + options.executable = executable; + options.version = "23.1.0"; + options.processFactory = [shared] { return std::make_unique(shared); }; + EditorHost host { root }; + auto engine = cld::make_engine(std::move(options)); + engine->start(host); + host.pump(*engine); + const std::vector recent { main, other, slow, join_path(root, "src/gone.cpp") }; + engine->warm(recent); + + mcppls::normalize::EnginePlan plan; + plan.entries.push_back(mcppls::normalize::EngineEntry { root, main, { "clang++", "-std=c++23", "-c", main }, "", "", { "hello.greet" }, {} }); + plan.entries.push_back(mcppls::normalize::EngineEntry { root, interface, { "clang++", "-std=c++23", "-c", interface }, "hello.greet", "hello.greet", {}, {} }); + plan.entries.push_back(mcppls::normalize::EngineEntry { root, other, { "clang++", "-std=c++23", "-c", other }, "", "", {}, {} }); + plan.entries.push_back(mcppls::normalize::EngineEntry { root, slow, { "clang++", "-std=c++23", "-c", slow }, "", "", {}, {} }); + engine->apply(&plan); + host.pump(*engine); + engine->handle_timers(); + for (int turn { 0 }; turn < 3; ++turn) host.pump(*engine); + auto count = [&](std::string_view method, const std::string& path) { + return std::ranges::count(BuildingProcess::files(*shared, method), path); + }; + expect(count("textDocument/didOpen", main) == 1 && count("textDocument/didOpen", other) == 1) + << "both are opened before the editor asks: it was the last session's"; + + // The editor opens main.cpp with the text clangd has: adopted, with what clangd built and published for it. + const eng::DocumentView mainView { mcppls::base::path_to_uri(main), main, "cpp", 1, mainText }; + host.open.push_back(mainView); + engine->document(eng::DocumentEvent { eng::DocumentChange::opened, mainView }); + host.pump(*engine); + expect(count("textDocument/didOpen", main) == 1) << "not opened a second time"; + expect(count("textDocument/didClose", main) == 0) << "nor closed first"; + expect(std::ranges::any_of(host.published, [&](const auto& entry) { return entry.first == mainView.uri; })) + << "the diagnostics clangd published for it reach the editor"; + const Json& handed { host.diagnostics[mainView.uri] }; + expect(handed.is_array() && handed.size() == 1u && handed[0].is_object() && handed[0].value("message", std::string {}) == "unused variable") + << "as the list clangd published: " << handed.dump(); + + // other.cpp comes with an edit the editor restored: clangd takes it as a change of the file it has. + const eng::DocumentView otherView { mcppls::base::path_to_uri(other), other, "cpp", 3, otherText + "// restored edit\n" }; + host.open.push_back(otherView); + engine->document(eng::DocumentEvent { eng::DocumentChange::opened, otherView }); + host.pump(*engine); + expect(count("textDocument/didOpen", other) == 1 && count("textDocument/didClose", other) == 0); + bool changed { false }; + { + const std::lock_guard lock { shared->mutex }; + for (const auto& message : shared->sent) { + if (message.value("method", std::string {}) != "textDocument/didChange") continue; + changed |= message["params"]["textDocument"]["uri"] == otherView.uri && message["params"]["textDocument"]["version"] == 3; + } + } + expect(changed) << "the editor's text and version, as a change"; + + // slow.cpp is opened with the text and version clangd is still building: nothing is sent again, and what that + // build publishes is the editor's. + const eng::DocumentView slowView { mcppls::base::path_to_uri(slow), slow, "cpp", 1, slowText }; + host.open.push_back(slowView); + engine->document(eng::DocumentEvent { eng::DocumentChange::opened, slowView }); + host.pump(*engine); + expect(count("textDocument/didOpen", slow) == 1 && count("textDocument/didClose", slow) == 0 && count("textDocument/didChange", slow) == 0) + << "the build under way is not started over"; + expect(!host.diagnostics.contains(slowView.uri)) << "nothing published before clangd finishes"; + shared->finish(slowView.uri); + host.pump(*engine); + const Json& built { host.diagnostics[slowView.uri] }; + expect(built.is_array() && built.size() == 1u && built[0].is_object()) << "clangd's list reaches the editor: " << built.dump(); + engine->shut_down(); + fs::remove_all(root); + }; + "what clangd could not find under the provisional model is not carried into the build tool's plan"_test = [] { namespace fs = mcppls::platform::fs; using mcppls::base::join_path; @@ -1871,6 +1997,10 @@ int main() { expect(reader.read("I[00:56:20.713] Built preamble of size 274032 for file /p/math2.cppm version 1 in 0.02 seconds").telling); expect(reader.read("I[00:56:20.790] Built module math to /c/math.pcm").telling); expect(reader.read("I[00:56:20.790] Rebuilding module math: input changed /p/math.cppm").telling); + expect(reader.read("I[00:56:21.000] Completion in /p/main.cpp: 412 ms (module input checks 280 ms: 12 one by one, 900 by 3 package checks, 40 system already checked; waited 0 ms for modules)").telling) + << "0.0.15: what a slow completion cost"; + expect(reader.read("I[00:56:21.000] Module memos: 25 BMI verdicts and 31 scans kept from the last session (0 unreadable), read in 12 ms; each is checked again before it is used").telling); + expect(reader.read("I[00:56:21.000] Module inputs: 2 managed package store(s) checked by their markers").telling); expect(reader.read("I[00:57:01.000] Prerequisites of /p/main.cpp: 25 modules, 25 reused, 0 built, 0 failed, in 3.10 s").telling); expect(reader.read("I[00:57:02.000] Preamble of /p/main.cpp is not compatible: the compile command changed").telling); expect(reader.read("I[00:57:03.000] Completion in /p/main.cpp answered before its first preamble").telling); diff --git a/tests/test_workarounds.cpp b/tests/test_workarounds.cpp index 7cc6461b..461d14ac 100644 --- a/tests/test_workarounds.cpp +++ b/tests/test_workarounds.cpp @@ -1,10 +1,12 @@ import std; import mcppls.testing; +import mcppls.base.version; import mcppls.engine; import mcppls.engine.clangd; import mcppls.engine.clangd.workarounds; import mcppls.normalize.plan; import mcppls.engine.clangd.process; +import mcppls.engine.clangd.guard; import mcppls.engine.payload; import mcppls.orchestrator.workspace; import mcppls.cli.options; @@ -46,7 +48,7 @@ int main() { "the traits are the registry's"_test = [] { const auto pinned = cld::traits_for_version("23.1.0"); expect(pinned.tested && pinned.hangsOnTrailingDotModuleName && pinned.hangsOnUnresolvedImports); - const auto bundled = cld::traits_for_version("23.1.0-mcppls.0"); + const auto bundled = cld::traits_for_version(mcppls::base::CLANGD_VERSION); expect(bundled.tested && bundled.kitStdlibVersion == "23.1.0") << "the bundled maintained engine is the tested one, its kit the LLVM base"; expect(!cld::traits_for_version("23.1.0-mcppls.7").tested) << "another maintained build is not"; expect(pinned.needsModulePreparation && pinned.needsModuleHints && pinned.msvcStlNeedsNoAlignedAllocation); @@ -272,4 +274,38 @@ int main() { expect(std::ranges::find(arguments, std::string { "--experimental-modules-support" }) == arguments.end()); expect(std::ranges::find(arguments, std::string { "--background-index" }) != arguments.end()) << "the rest stays"; }; + + "the Windows plan's arguments are given unless the person's own name them (0.0.14 A2, B2b, C1, C2, D1)"_test = [] { + const auto has = [](const std::vector& arguments, std::string_view argument) { + return std::ranges::find(arguments, std::string { argument }) != arguments.end(); + }; + cld::ProcessConfig config; + expect(!has(cld::clangd_arguments(config), "--limit-results=1000")) << "nothing unless asked"; + config.resultLimit = 1000; + config.preambleStorage = "memory"; + config.managedInputRoots = { "/stores/mcpp", "/stores/xlings" }; + config.moduleMemoDirectory = "/cache/clangd-memos"; + config.maxRetainedAsts = 6; + auto arguments = cld::clangd_arguments(config); + for (const auto argument : { "--limit-results=1000", "--pch-storage=memory", "--managed-input-root=/stores/mcpp", + "--managed-input-root=/stores/xlings", "--module-memo-dir=/cache/clangd-memos", "--max-retained-asts=6" }) { + expect(has(arguments, argument)) << argument; + } + config.extraArguments = { "--limit-results=100", "-pch-storage=disk" }; + arguments = cld::clangd_arguments(config); + expect(!has(arguments, "--limit-results=1000") && has(arguments, "--limit-results=100")) << "the person's limit wins"; + expect(!has(arguments, "--pch-storage=memory") && has(arguments, "-pch-storage=disk")) << "a single dash names it too"; + + constexpr std::uint64_t GIB { std::uint64_t { 1 } << 30 }; + expect(cld::preamble_storage("auto", true, 32 * GIB) == "memory") << "Windows with enough memory"; + expect(cld::preamble_storage("auto", true, 16 * GIB) == "memory") << "the threshold included"; + expect(cld::preamble_storage("auto", true, 8 * GIB).empty()) << "too little memory: clangd's own"; + expect(cld::preamble_storage("auto", true, std::nullopt).empty()) << "unknown memory: clangd's own"; + expect(cld::preamble_storage("auto", false, 64 * GIB).empty()) << "elsewhere: clangd's own"; + expect(cld::preamble_storage("disk", true, 64 * GIB) == "disk" && cld::preamble_storage("memory", false, std::nullopt) == "memory"); + expect(cld::retained_asts(32 * GIB) == std::optional { 6 }); + expect(cld::retained_asts(8 * GIB) == std::nullopt && cld::retained_asts(std::nullopt) == std::nullopt); + }; + + return report(); }