diff --git a/README.md b/README.md index 8022497..af18943 100644 --- a/README.md +++ b/README.md @@ -8,8 +8,8 @@ Gives dsh agents a persistent evolution memory and a native bridge to the [EvoMap network](https://evomap.ai): behind each prompt it injects one reusable strategy -that fits, records how every turn ended, and adds `evolver_*` tools and `/evolver-*` -commands. +that fits, reports back to the network how that strategy fared, and adds `evolver_*` tools +and `/evolver-*` commands. ## Install @@ -76,22 +76,63 @@ dsh plugin --profile dsh-tui add -w @evomap/dsh-evolver@latest ## Connect the EvoMap network (optional) -Local memory works without an account. To reuse strategies from the network: - -```bash -npm install -g @evomap/evolver -``` - -1. Run `evolver` once inside a git repository. It starts the local Proxy and prints a - claim link. -2. Open the link while signed in to [evomap.ai](https://evomap.ai). -3. Run `/evolver-status` in dsh to confirm the Proxy and node. - -Network recall needs `@evomap/evolver` 2.0.39 or newer. +Local memory works without an account. Network reuse runs through the local Evolver +Proxy, which the plugin sets up on its own once per dsh process, in the background: + +- `evolver` missing, or older than 2.0.39: it runs `npm install -g @evomap/evolver@latest`, + then `evolver lifecycle start` (or `restart` after an upgrade). +- `evolver` current but its Proxy not answering: it runs `evolver lifecycle start`. + +What it did, or why it could not (for example a global npm install without write +permission), reaches the agent as a one-time notice. To finish connecting: + +1. Open the claim link the Proxy writes to `~/.evomap/claim_url` while signed in to + [evomap.ai](https://evomap.ai). +2. Run `/evolver-status` in dsh to confirm the Proxy and node. + +Network recall needs `@evomap/evolver` 2.0.39 or newer. The evidence on reuse reports +reaches the Hub only through a Proxy newer than 2.0.42; an older one drops it, and the Hub +counts those reports as unevidenced. + +## Reuse reports + +Every strategy injected into a turn is reported through the Proxy's +`POST /asset/reuse-result` as `{asset_id, outcome, task_id, reason, evidence}`. The Hub +weighs each report by its evidence: + +| Report | When | `task_id` | `evidence` | +|---|---|---|---| +| Turn end | the turn ends | `dsh::` | `tier` `L1` when the turn ran a test, build, lint or type check, else `none`; `applied`; `check` `{kind, passed, fail_then_pass}` of the last check; `aborted`; `stage: immediate` | +| Correction | the next prompt says the answer did not hold, or approves it | `…:correction` | `tier: L3`, `applied`, `stage: correction` | +| Retention | the next session in the same project finds the turn's edits kept or reverted | `…:retention` | `tier: L2`, `applied`, `stage: retention` | +| `evolver_asset_reuse_result` | the model reports on its own | the model's own | `tier: L5`, `applied`, `stage: immediate` | + +- **`applied`** compares the strategy's numbered steps with the turn's shell commands, + touched paths, the code its edits wrote (200,000 characters a turn, kept in memory only) + and the final answer: `used` when half the steps show up, `partial` when fewer do, + `unused` when none do. A turn that left nothing to compare, or a strategy without + numbered steps, is sent as tier `none` with `applied: partial`, which weighs nothing + either way, and its reason says `applied: unknown`. +- **Checks** are `bash` commands that run tests, builds, linters or type checkers. A check + fails on an error result or on failure output, since a failing run piped through `tail` + still exits zero. `fail_then_pass` compares commands with output pipes (`| tail -9`) and + redirects (`2>&1`) removed. +- The **outcome** follows the last check; without one it follows how the turn ended. +- **Delivery.** Reports wait in `~/.evolver/state/reports-outbox.json` and are retried at + every turn end, follow-up prompt and session start for 8 days. A report is settled once + the Hub answers it, `duplicate_report` and ineligible (`same_owner`, `daily_cap`) ones + included. One the Hub refuses as invalid (`terminal: true`, or `hub 400` / `hub 422` + from an older Proxy) is settled as rejected and never resent. `hub 404`, `hub 5xx` and + network failures are retried. +- `/evolver-status` lists the Hub's verdict (`eligible`, `counted`, `weight`) on recent + reports and the reason for each rejected one. Reports with evidence are also written to + the local root event log once. +- Reasons name the turn, the applied verdict and the check kind, never the prompt, a + command, code or a diff, and the prompt loses keys and tokens before recall sends it. ## Requirements -Node.js 22.13+, git (for turn capture), dsh 0.1.5+. +Node.js 22.13+, dsh 0.1.5+. Settings and their defaults are in [`src/config.js`](src/config.js). diff --git a/README.zh.md b/README.zh.md index 7e02ade..f5307b6 100644 --- a/README.zh.md +++ b/README.zh.md @@ -6,7 +6,7 @@

English · 简体中文

-为 dsh 智能体提供持久的进化记忆,并原生接入 [EvoMap 网络](https://evomap.ai):在每条提示词后面注入一条合适的可复用策略,记录每一轮的结果,并提供 `evolver_*` 工具和 `/evolver-*` 命令。 +为 dsh 智能体提供持久的进化记忆,并原生接入 [EvoMap 网络](https://evomap.ai):在每条提示词后面注入一条合适的可复用策略,把这条策略的效果回报给网络,并提供 `evolver_*` 工具和 `/evolver-*` 命令。 ## 安装 @@ -65,21 +65,39 @@ dsh plugin --profile dsh-tui add -w @evomap/dsh-evolver@latest ## 连接 EvoMap 网络(可选) -本地记忆不需要账号就能用。要复用网络上的策略: +本地记忆不需要账号就能用。网络复用经由本地 Evolver Proxy,插件会在每个 dsh 进程里于后台自动准备一次: -```bash -npm install -g @evomap/evolver -``` +- 没装 `evolver`,或版本低于 2.0.39:执行 `npm install -g @evomap/evolver@latest`,然后执行 `evolver lifecycle start`(升级后用 `restart`)。 +- `evolver` 版本够新但 Proxy 没有响应:执行 `evolver lifecycle start`。 + +做了什么、或者为什么没做成(比如没有全局 npm 安装的写权限),都会以一次性通知告诉智能体。完成连接: + +1. 登录 [evomap.ai](https://evomap.ai) 后,打开 Proxy 写在 `~/.evomap/claim_url` 里的认领链接。 +2. 在 dsh 里执行 `/evolver-status`,确认 Proxy 和节点状态正常。 + +网络召回需要 `@evomap/evolver` 2.0.39 或更新版本。复用上报里的证据只有经 2.0.42 之后的 Proxy 才能送到 Hub;更旧的 Proxy 会丢掉证据,Hub 把这些上报按无证据计。 + +## 复用上报 + +每一轮注入过的策略都会经 Proxy 的 `POST /asset/reuse-result` 上报,请求体为 `{asset_id, outcome, task_id, reason, evidence}`,Hub 按证据给上报加权: -1. 在任意 git 仓库里运行一次 `evolver`。它会启动本地 Proxy,并打印一个认领链接。 -2. 登录 [evomap.ai](https://evomap.ai) 后打开这个链接。 -3. 在 dsh 里执行 `/evolver-status`,确认 Proxy 和节点状态正常。 +| 上报 | 时机 | `task_id` | `evidence` | +|---|---|---|---| +| 回合结束 | 回合结束时 | `dsh:<会话>:<回合>` | 本回合跑过测试、构建、lint 或类型检查时 `tier` 为 `L1`,否则为 `none`;`applied`;最后一次检查的 `check` `{kind, passed, fail_then_pass}`;`aborted`;`stage: immediate` | +| 追问补报 | 下一条提示词表示结果不对,或表示认可 | `…:correction` | `tier: L3`、`applied`、`stage: correction` | +| 留存回访 | 同一项目的下一个会话发现本回合的改动被保留或被撤销 | `…:retention` | `tier: L2`、`applied`、`stage: retention` | +| `evolver_asset_reuse_result` | 模型自己上报 | 模型自带 | `tier: L5`、`applied`、`stage: immediate` | -网络召回需要 `@evomap/evolver` 2.0.39 或更新版本。 +- **`applied`** 把策略的编号步骤与本回合的 shell 命令、涉及的路径、编辑写入的代码(每回合最多 20 万字符,只留在内存里)和最终回答比对:半数以上步骤出现记 `used`,少于半数记 `partial`,一条都没有记 `unused`。回合没留下可比对的内容,或策略没有编号步骤时,按 tier `none` 加 `applied: partial` 上报,两边都不计权重,reason 里写 `applied: unknown`。 +- **检查**指运行测试、构建、lint 或类型检查的 `bash` 命令。结果报错或输出里有失败信息都算失败,因为经 `tail` 管道的失败运行退出码仍是 0。判断 `fail_then_pass` 时会先去掉输出管道(`| tail -9`)和重定向(`2>&1`)再比较命令。 +- **outcome** 跟随最后一次检查;没有检查时跟随回合的结束方式。 +- **投递**:上报先存进 `~/.evolver/state/reports-outbox.json`,在每次回合结束、追问和会话开始时重试,保留 8 天。Hub 只要给出答复就算送达,包括 `duplicate_report` 和不计分的回执(`same_owner`、`daily_cap`)。Hub 判为无效的(`terminal: true`,或旧版 Proxy 返回的 `hub 400` / `hub 422`)记为被拒,不再重发。`hub 404`、`hub 5xx` 和网络失败会重试。 +- `/evolver-status` 会列出最近上报的 Hub 判定(`eligible`、`counted`、`weight`)以及每条被拒上报的原因。带证据的上报还会在本地 root event 日志里记一次。 +- reason 只写回合、采用判定和检查类型,不含提示词、命令、代码或 diff;召回前会先去掉提示词里的密钥和令牌。 ## 环境要求 -Node.js 22.13+、git(用于记录每轮结果)、dsh 0.1.5+。 +Node.js 22.13+、dsh 0.1.5+。 配置项及默认值见 [`src/config.js`](src/config.js)。 diff --git a/assets/commands/evolve.md b/assets/commands/evolve.md index 30e7ed8..58da080 100644 --- a/assets/commands/evolve.md +++ b/assets/commands/evolve.md @@ -4,8 +4,8 @@ description: Run an evolution checkpoint — recall relevant past outcomes, refl Take a deliberate evolution step for the current task. -1. **Recall.** Look at the evolution memory this plugin injected at session start, or read - the tail of the memory graph — `~/.evolver/memory/evolution/memory_graph.jsonl`, or the +1. **Recall.** Look at the evolution memory this plugin injected behind the prompt, or read + the tail of the full engine's memory graph — `~/.evolver/memory/evolution/memory_graph.jsonl`, or the project's `memory/evolution/memory_graph.jsonl` when it exists. Summarize any recent outcome — success or failure — relevant to what we're working on. Search the network too, with `evolver_search_assets`, when the task looks like something others have hit. @@ -13,8 +13,8 @@ Take a deliberate evolution step for the current task. 2. **Reflect.** Given the current diff and task state, say in one or two lines: what worked, what didn't, and what the durable lesson is. -3. **Record.** Outcomes are captured automatically at every turn end, so nothing is needed - for the local graph. If the lesson is worth sharing, distill it with +3. **Record.** Reuse of an injected strategy is reported automatically at every turn end. + If the lesson is worth sharing, distill it with `evolver_distill_conversation` (see `/evolver-distill`). To run the full engine now, use `/evolver-run`. diff --git a/assets/commands/status.md b/assets/commands/status.md index 71411a8..8d09c2c 100644 --- a/assets/commands/status.md +++ b/assets/commands/status.md @@ -1,5 +1,5 @@ --- -description: Show Evolver health — Proxy, running engine version, recent strategy recalls, evolution memory, workspace id, network claim, and full-engine availability. +description: Show Evolver health — Proxy, running engine version, recent strategy recalls and reuse reports, evolution memory, network claim, and full-engine availability. --- Report Evolver health as a checklist grouped under the headings below. Use the available @@ -8,19 +8,19 @@ token values. Call `evolver_status` once and use its fields throughout; it returns `proxy` (the Proxy's own status, or `reachable: false` with the error) and the plugin's view (`plugin`, `runtime`, -`proxy_endpoint`, `engine`, `recall`). +`proxy_endpoint`, `engine`, `recall`, `reuse_reports`). 1. **Plugin and runtime** — plugin version, platform/arch, Node version. 2. **Proxy** — reachable or not. When reachable: `status`, `node_id`, `hub_mode`, `hub_auth_status` (flag anything but `ok`), `last_sync_at` as a local time, `last_sync_error`, pending inbound/outbound counts, and any `reauth_backoff_until` / `hello_rate_limit_until` in the future. When unreachable: show the error, the endpoint it tried (`proxy_endpoint.url` - and `url_source`), and say that running `evolver` once in a git repository starts the Proxy; - local recall and capture remain available. + and `url_source`), and say that `evolver lifecycle start` starts the Proxy (the plugin also tries this once per dsh process); + reuse reports wait and are resent once it is up. 3. **Engine version** — `engine.running_version` (the Proxy actually serving, with `proxy_endpoint.pid` and `started_at`) and `engine.path_version` (the `evolver` on PATH). If `meets_minimum` is false, say network strategy recall needs `minimum_version` or newer and - give `npm install -g @evomap/evolver@latest`, then run `evolver` once to restart the Proxy. + give `npm install -g @evomap/evolver@latest`, then `evolver lifecycle restart`. If `path_differs_from_running` is true, say the running Proxy is a different install from the one on PATH and should be restarted from the upgraded one. If both versions are null, say the version could not be determined. @@ -33,18 +33,16 @@ status, or `reachable: false` with the error) and the plugin's view (`plugin`, ` null, no prompt has been recalled in this dsh process yet. 5. **Network claim** — check `~/.evomap/claim_url`. If it contains an HTTPS `evomap.ai` link, report that the node is awaiting claim and show the link without modifying it. -6. **Turn capture** — report whether the working directory is a git repository. Turn - outcomes are derived from git diffs, so outside one they are not recorded; strategy - injection from the network is unaffected either way. +6. **Reuse reports** — from `reuse_reports`, the recent `delivered` reports with the Hub's + verdict (`eligible`, `counted`, `weight`, and `reason` when not eligible, e.g. `daily_cap` + or `same_owner`) and every `rejected` one with its `reason` and `status`. A rejected + report means the Hub refused it as invalid; show it as a problem worth reporting. Both + lists are empty until a turn with an injected strategy ends in this dsh process. 7. **Evolution memory** — use an existing `/memory/evolution/memory_graph.jsonl` when present; otherwise inspect `~/.evolver/memory/evolution/memory_graph.jsonl`. Report the selected path and count of non-empty JSONL rows without printing their contents. -8. **Workspace id** — find the git root, and the `workspace/` directory inside it when one - exists; that path is the workspace root. The id is stored outside the repository, at - `~/.evolver/state/workspace-`. Report only - present/missing, never the id value. -9. **Full engine** — if `engine.path_version` is null, say that `npm install -g @evomap/evolver` +8. **Full engine** — if `engine.path_version` is null, say that `npm install -g @evomap/evolver` enables `/evolver-run`, `/evolver-review`, and `/evolver-solidify`. Finish with one line on overall readiness and the single most useful next action, if any. diff --git a/assets/skills/capability-evolver/SKILL.md b/assets/skills/capability-evolver/SKILL.md index a844850..3259001 100644 --- a/assets/skills/capability-evolver/SKILL.md +++ b/assets/skills/capability-evolver/SKILL.md @@ -12,29 +12,28 @@ sessions. ## What runs on its own -Three seams work without you invoking anything: +These seams work without you invoking anything: - **Behind the first prompt** (`agent/pre-step`, once per agent) — with `claimNudgeEnabled`, - surfaces a trusted pending node-claim link; outside a git repository, says once per - directory that recording is inactive. + surfaces a trusted pending node-claim link. +- **Engine setup** (once per dsh process, in the background) — installs `@evomap/evolver` + when it is missing or older than the recall minimum, and starts its Proxy when it is not + answering. One `[Evolver]` notice says what was done, or the manual steps when it failed. - **Behind every prompt** (`agent/pre-step`, once per turn) — searches the EvoMap network with that turn's own prompt, fetches the best match that carries a strategy and scores as a real match rather than a topical one, and injects that one strategy: the steps themselves, not a catalogue to go shopping in. Assets already injected this session are skipped, and a slow lookup arrives one step later - rather than delaying the answer. Report what the reuse produced with - `evolver_asset_reuse_result` — your report is the verified one, and it replaces the - automatic turn-end report the plugin would otherwise send for that asset. -- **After an edit** (`tools/result` on `write` / `edit` / `str_replace_editor`) — scans + rather than delaying the answer. +- **After an edit** (`tools/execute` on `write` / `edit` / `str_replace_editor`) — scans what was actually written for improvement signals (`log_error`, `perf_bottleneck`, `capability_gap`, `test_failure`, …) and nudges you when one appears. -- **Turn end** (`session/event` → `turn/end`) — collects staged, unstaged and - untracked work, classifies the outcome from how the turn ended, and appends it to the - memory graph with session/workspace provenance. Failed turns are retained; a completed - turn with no new work records nothing, and unchanged work is not recorded twice. - -Memory lands in `~/.evolver/memory/evolution/memory_graph.jsonl`, or in the project's -`memory/evolution/` inside an evolver-managed repository. +- **Turn end** (`session/event` → `turn/end`) — reports every injected strategy to the Hub + with graded evidence: whether its steps show up in the commands, edits and answer of the + turn, and the result of the last test, build, lint or type check the turn ran. +- **Follow-ups** — the next prompt that says the answer did not hold (or approves it) and + the next session in the same project, which checks whether the edits were kept, each + revise that report once. ## What you should do @@ -46,13 +45,14 @@ For any **substantive** task — a feature, a non-trivial fix, a refactor: nothing was injected and others plausibly hit the problem before, call `evolver_search_assets`. 2. **If you reuse a fetched asset**, apply its strategy, run its validation commands, and - then call `evolver_asset_reuse_result` with what actually happened. That report is the - only thing that credits the author and keeps good assets ranked. -3. **Do the work**, and verify it. -4. **After finishing**, the turn-end seam records the local outcome for you. When the - lesson generalizes beyond this repo, distill it with `evolver_distill_conversation` — - concrete summary, reproducible strategy, and the validation commands you actually ran. - Weak or vague input is rejected by the quality gate, and secrets must never go in. + then call `evolver_asset_reuse_result` with what actually happened. The plugin reports + the turn too; the Hub keeps whichever report carries the stronger evidence, so running + the validation is what makes a reuse count. +3. **Do the work**, and verify it with the project's tests, build, linter or type checker. +4. **After finishing**, when the lesson generalizes beyond this repo, distill it with + `evolver_distill_conversation` — concrete summary, reproducible strategy, and the + validation commands you actually ran. Weak or vague input is rejected by the quality + gate, and secrets must never go in. Trivial or purely conversational turns don't need any of this — skip it. @@ -68,8 +68,8 @@ Trivial or purely conversational turns don't need any of this — skip it. | `deployment_issue` | build / CI / pipeline / rollback | | `recurring_error` | "same error" / "still failing" / "keeps failing" | -Signals are descriptive tags, not a verdict: whether a turn is recorded as a success or a -failure comes from how the turn itself ended. +Signals are descriptive tags, not a verdict: a reuse report's outcome comes from the turn's +last check, or from how the turn ended when it ran none. ## Tools diff --git a/package.json b/package.json index 34c204a..82c6cd8 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "@evomap/dsh-evolver", "version": "0.1.3", - "description": "Evolver for DeepSeek Harness — persistent, auditable evolution memory: recalls what worked at session start, detects improvement signals while you edit, records outcomes at turn end, and exposes the EvoMap network's genes and capsules as native dsh tools.", + "description": "Evolver for DeepSeek Harness — persistent, auditable evolution memory: recalls what worked behind each prompt, detects improvement signals while you edit, reports each reuse back with graded evidence, and exposes the EvoMap network's genes and capsules as native dsh tools.", "type": "module", "main": "src/index.js", "exports": { diff --git a/src/applied.js b/src/applied.js new file mode 100644 index 0000000..e29181f --- /dev/null +++ b/src/applied.js @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +const STEP_PREFIX = /^\s*\d+\.\s+/; +const BACKTICKED = /`([^`\n]{2,120})`/g; +const PATH_LIKE = /(?:^|[\s("'])((?:\.{0,2}\/)?[\w@.-]+(?:\/[\w@.-]+)+|[\w-]+\.(?:js|mjs|cjs|ts|tsx|jsx|py|go|rs|java|kt|rb|php|json|ya?ml|toml|md|sh|sql|css|html|vue|svelte))(?=$|[\s)"',.:;])/g; +const COMMAND_LIKE = /\b((?:npm|npx|pnpm|yarn|bun|node|git|go|cargo|pytest|python3?|pip|make|docker|kubectl|tsc|eslint|jest|vitest|dsh|gh)\s+[\w:@./=-]+)/g; +const WORD = /[\p{L}\p{N}_]{4,}/gu; +const STOPWORDS = new Set([ + 'that', 'this', 'with', 'from', 'into', 'then', 'when', 'each', 'only', 'before', 'after', 'step', 'steps', 'make', 'sure', + 'should', 'must', 'never', 'always', 'use', 'using', 'used', 'have', 'has', 'were', 'will', 'your', 'their', 'there', 'these', + 'those', 'what', 'which', 'while', 'until', 'other', 'every', 'more', 'than', 'also', 'apply', 'where', 'fits', 'report', + 'outcome', 'evolver_asset_reuse_result', 'safety', 'boundary', 'optional', 'otherwise', 'first', 'next', 'last', 'same', +]); + +const USED_RATIO = 0.5; +export const UNKNOWN = 'unknown'; +const TERM_COVERAGE = 0.5; +const MIN_STEP_TERMS = 3; + +export function strategyStepsOf(injectedText) { + return String(injectedText ?? '') + .split('\n') + .filter((line) => STEP_PREFIX.test(line)) + .map((line) => line.replace(STEP_PREFIX, '').trim()); +} + +function matchesOf(text, pattern) { + return [...text.matchAll(pattern)].map((match) => match[1].trim().toLowerCase()); +} + +export function concreteFeaturesOf(step) { + const features = [...matchesOf(step, BACKTICKED), ...matchesOf(step, PATH_LIKE), ...matchesOf(step, COMMAND_LIKE)]; + return [...new Set(features.map((feature) => feature.replace(/[.,:;]+$/, '')).filter((feature) => feature.length >= 3))]; +} + +function termsOf(text) { + return [...new Set((String(text).toLowerCase().match(WORD) ?? []).filter((term) => !STOPWORDS.has(term)))]; +} + +function turnHaystack({ commands = [], paths = [], edits = [], answer = '' }) { + return [...commands, ...paths, ...edits, answer].join('\n').toLowerCase(); +} + +function termsWereFollowed(step, haystack) { + const terms = termsOf(step); + if (terms.length < MIN_STEP_TERMS) return false; + return terms.filter((term) => haystack.includes(term)).length / terms.length >= TERM_COVERAGE; +} + +function stepWasFollowed(step, haystack) { + return concreteFeaturesOf(step).some((feature) => haystack.includes(feature)) || termsWereFollowed(step, haystack); +} + +function leftNothingToCompare({ commands = [], paths = [], edits = [] }) { + return commands.length + paths.length + edits.length === 0; +} + +function verdictOf(turn, steps, hits) { + if (hits > 0) return hits / steps >= USED_RATIO ? 'used' : 'partial'; + return steps === 0 || leftNothingToCompare(turn) ? UNKNOWN : 'unused'; +} + +export function appliedVerdictOf({ strategyText, ...turn }) { + const steps = strategyStepsOf(strategyText); + const haystack = turnHaystack(turn); + const hits = steps.filter((step) => stepWasFollowed(step, haystack)).length; + const ratio = steps.length === 0 ? 0 : hits / steps.length; + return { verdict: verdictOf(turn, steps.length, hits), hits, steps: steps.length, ratio: Math.round(ratio * 1000) / 1000 }; +} diff --git a/src/capture-coordinator.js b/src/capture-coordinator.js deleted file mode 100644 index 411ae59..0000000 --- a/src/capture-coordinator.js +++ /dev/null @@ -1,37 +0,0 @@ -// SPDX-License-Identifier: MIT -// Copyright (c) 2026 EvoMap - -import { commitPreparedCapture, prepareCapture } from './capture.js'; -import { sessionKeyOf } from './session-key.js'; - -export function createCaptureCoordinator({ - prepare = prepareCapture, - commit = commitPreparedCapture, -} = {}) { - const pending = new Map(); - - const schedule = (options) => { - const key = sessionKeyOf(options.sessionId, options.projectDir); - const snapshot = Promise.resolve().then(() => prepare(options)); - const previous = pending.get(key) ?? Promise.resolve(); - const current = previous - .catch(() => null) - .then(async () => commit(await snapshot)) - .catch(() => null); - pending.set(key, current); - current.finally(() => { - if (pending.get(key) === current) pending.delete(key); - }); - return current; - }; - - const flush = async (sessionId, projectDir) => { - await pending.get(sessionKeyOf(sessionId, projectDir)); - }; - - const flushAll = async () => { - await Promise.all([...pending.values()]); - }; - - return { schedule, flush, flushAll }; -} diff --git a/src/capture.js b/src/capture.js deleted file mode 100644 index d4375ef..0000000 --- a/src/capture.js +++ /dev/null @@ -1,476 +0,0 @@ -// SPDX-License-Identifier: MIT -// Copyright (c) 2026 EvoMap - -import crypto from 'node:crypto'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; - -import { git, gitText } from './git.js'; -import { isLoopbackUrl } from './proxy.js'; -import { detectSignalsInDiff } from './signals.js'; -import { appendMemoryGraph, captureStatePath, resolveWorkspaceId } from './workspace.js'; - -const DEFAULT_HUB_TIMEOUT_MS = 8_000; -const DEFAULT_DEDUPE_TTL_MS = 24 * 60 * 60 * 1000; -const DEFAULT_UNTRACKED_HASH_MAX_BYTES = 1024 * 1024; -const DEFAULT_CAPTURE_LOCK_STALE_MS = 60_000; -const DEFAULT_CAPTURE_LOCK_WAIT_MS = 65_000; -const CAPTURE_LOCK_RETRY_MS = 25; - -const TURN_OUTCOMES = { - completed: { status: 'success', score: 0.8 }, - 'max-tokens': { status: 'failed', score: 0.5 }, - aborted: { status: 'failed', score: 0.4 }, - blocked: { status: 'failed', score: 0.3 }, - error: { status: 'failed', score: 0.2 }, -}; - -const TURN_SIGNALS = { - 'max-tokens': 'max_tokens', - aborted: 'turn_aborted', - blocked: 'turn_blocked', - error: 'log_error', -}; - -const PLUGIN_STATE = /(^|\/)\.evolver\/|memory_graph\.jsonl$/; - -export function outcomeOfReason(reasonKind) { - return TURN_OUTCOMES[reasonKind] ?? null; -} - -async function diffCommands(projectDir, gitOptions) { - const hasHead = (await git(['rev-parse', '--verify', '--quiet', 'HEAD'], projectDir, gitOptions)).ok; - return hasHead ? [['diff', 'HEAD']] : [['diff', '--cached'], ['diff']]; -} - -function pathInside(parent, candidate) { - const relative = path.relative(parent, candidate); - return relative === '' || (!relative.startsWith(`..${path.sep}`) && relative !== '..' && !path.isAbsolute(relative)); -} - -async function untrackedMark(projectDir, relativePath, maxBytes) { - const root = path.resolve(projectDir); - const absolute = path.resolve(root, relativePath); - if (!pathInside(root, absolute)) return `${relativePath}\0outside`; - - let fileHandle; - try { - const initial = await fs.promises.lstat(absolute); - if (initial.isSymbolicLink()) { - return `${relativePath}\0symlink\0${await fs.promises.readlink(absolute)}`; - } - if (!initial.isFile()) return `${relativePath}\0other\0${initial.size}\0${initial.mtimeMs}`; - - const noFollow = fs.constants.O_NOFOLLOW ?? 0; - fileHandle = await fs.promises.open(absolute, fs.constants.O_RDONLY | noFollow); - const current = await fileHandle.stat(); - if (!current.isFile()) return `${relativePath}\0other\0${current.size}\0${current.mtimeMs}`; - - const sampleSize = Math.min(current.size, maxBytes); - const firstSize = Math.ceil(sampleSize / 2); - const lastSize = sampleSize - firstSize; - const hash = crypto.createHash('sha256'); - if (firstSize > 0) { - const first = Buffer.alloc(firstSize); - const { bytesRead } = await fileHandle.read(first, 0, first.length, 0); - hash.update(first.subarray(0, bytesRead)); - } - if (lastSize > 0) { - const last = Buffer.alloc(lastSize); - const position = Math.max(0, current.size - lastSize); - const { bytesRead } = await fileHandle.read(last, 0, last.length, position); - hash.update(last.subarray(0, bytesRead)); - } - return `${relativePath}\0file\0${current.size}\0${current.mtimeMs}\0${hash.digest('hex')}`; - } catch { - return `${relativePath}\0unreadable`; - } finally { - await fileHandle?.close().catch(() => {}); - } -} - -export async function collectDiff(projectDir, options = {}) { - const gitOptions = { - signal: options.signal, - timeoutMs: options.gitTimeoutMs, - maxBuffer: options.gitMaxBufferBytes, - }; - const insideTree = await git(['rev-parse', '--is-inside-work-tree'], projectDir, gitOptions); - if (!insideTree.ok || insideTree.stdout.trim() !== 'true') { - return { isRepo: false, statText: '', body: '', untracked: [], untrackedMarks: [] }; - } - - const commands = await diffCommands(projectDir, gitOptions); - const [statTexts, bodies, untrackedText] = await Promise.all([ - Promise.all(commands.map((args) => gitText([...args, '--stat', '--'], projectDir, gitOptions))), - Promise.all(commands.map((args) => gitText([...args, '--no-color', '--'], projectDir, gitOptions))), - gitText(['ls-files', '--others', '--exclude-standard', '-z'], projectDir, gitOptions), - ]); - - const untracked = untrackedText - .split('\0') - .filter((relativePath) => relativePath.length > 0 && !PLUGIN_STATE.test(relativePath)); - const maxBytes = options.untrackedHashMaxBytes ?? DEFAULT_UNTRACKED_HASH_MAX_BYTES; - const untrackedMarks = []; - for (const relativePath of untracked) { - if (options.signal?.aborted) break; - untrackedMarks.push(await untrackedMark(projectDir, relativePath, maxBytes)); - } - - return { - isRepo: true, - statText: statTexts.join('\n'), - body: bodies.join('\n'), - untracked, - untrackedMarks, - }; -} - -export function parseStat(statText) { - const sum = (pattern) => { - let total = 0; - for (const match of statText.matchAll(pattern)) total += Number.parseInt(match[1], 10); - return total; - }; - return { - files: sum(/(\d+)\s+files?\s+changed/g), - insertions: sum(/(\d+)\s+insertions?\(\+\)/g), - deletions: sum(/(\d+)\s+deletions?\(-\)/g), - }; -} - -export function isEmpty(diff) { - return diff.body.trim().length === 0 && diff.untracked.length === 0; -} - -function untrackedNote(untracked) { - if (untracked.length === 0) return ''; - const shown = untracked.slice(0, 5).join(', '); - const rest = untracked.length > 5 ? `, +${untracked.length - 5} more` : ''; - return ` New untracked files: ${shown}${rest}.`; -} - -export function summarize(diff, reasonKind, observedSignals = []) { - const stats = parseStat(diff.statText); - const detected = [...observedSignals, ...detectSignalsInDiff(diff.body)]; - const reasonSignal = TURN_SIGNALS[reasonKind]; - if (reasonSignal) detected.push(reasonSignal); - const signals = [...new Set(detected)].sort(); - if (signals.length === 0) signals.push('stable_success_plateau'); - const { status, score } = outcomeOfReason(reasonKind) ?? TURN_OUTCOMES.completed; - - return { - signals, - status, - score, - note: - `Turn ${reasonKind}: ${stats.files + diff.untracked.length} files changed, ` + - `+${stats.insertions}/-${stats.deletions}. Signals: [${signals.join(', ')}]` + - `.${untrackedNote(diff.untracked)}`, - }; -} - -export function fingerprint(diff, reasonKind) { - return crypto - .createHash('sha256') - .update(reasonKind) - .update('\0') - .update(diff.statText) - .update('\0') - .update(diff.body) - .update('\0') - .update((diff.untrackedMarks ?? diff.untracked).join('\n')) - .digest('hex'); -} - -export function appendEvolutionLog(line) { - try { - const dir = process.env.EVOLVER_HOOK_LOG_DIR || path.join(os.homedir(), '.evolver', 'logs'); - fs.mkdirSync(dir, { recursive: true, mode: 0o700 }); - fs.appendFileSync(path.join(dir, 'evolution.log'), `${new Date().toISOString()} ${line}\n`, { mode: 0o600 }); - } catch { - } -} - -function buildEntry(outcome, context) { - return { - timestamp: new Date().toISOString(), - gene_id: 'ad_hoc', - signals: outcome.signals, - outcome: { status: outcome.status, score: outcome.score, note: outcome.note }, - cwd: context.projectDir, - workspace_id: context.workspaceId, - session_id: context.sessionId, - turn: context.turn, - turn_reason: context.reasonKind, - diff_hash: context.diffHash, - diff_scope: 'working_tree', - source: 'dsh:turn-end', - }; -} - -export function normalizeHubUrl(raw) { - try { - const url = new URL(raw); - if (url.username || url.password) return null; - if (url.protocol === 'https:') return url; - if (url.protocol === 'http:' && isLoopbackUrl(url.toString())) return url; - return null; - } catch { - return null; - } -} - -async function recordToHub(payload, options = {}) { - const rawUrl = process.env.EVOMAP_HUB_URL || process.env.A2A_HUB_URL; - const apiKey = process.env.EVOMAP_API_KEY || process.env.A2A_NODE_SECRET; - if (!rawUrl || !apiKey) return false; - const hubUrl = normalizeHubUrl(rawUrl); - if (!hubUrl) return false; - - try { - const endpoint = new URL('/a2a/evolution/record', hubUrl); - const timeout = AbortSignal.timeout(options.hubTimeoutMs ?? DEFAULT_HUB_TIMEOUT_MS); - const signal = options.signal ? AbortSignal.any([options.signal, timeout]) : timeout; - const response = await fetch(endpoint, { - method: 'POST', - headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` }, - body: JSON.stringify(payload), - signal, - }); - return response.ok; - } catch { - return false; - } -} - -function readCaptureState(statePath) { - try { - const stat = fs.lstatSync(statePath); - if (stat.isSymbolicLink() || !stat.isFile()) return { records: {} }; - const state = JSON.parse(fs.readFileSync(statePath, 'utf8')); - if (!state || typeof state !== 'object' || Array.isArray(state)) return { records: {} }; - return { records: state.records && typeof state.records === 'object' ? state.records : {} }; - } catch { - return { records: {} }; - } -} - -function writeCaptureState(statePath, state) { - let temporary; - try { - fs.mkdirSync(path.dirname(statePath), { recursive: true, mode: 0o700 }); - temporary = `${statePath}.${process.pid}.${crypto.randomBytes(6).toString('hex')}.tmp`; - fs.writeFileSync(temporary, JSON.stringify(state), { mode: 0o600, flag: 'wx' }); - try { - fs.renameSync(temporary, statePath); - } catch (error) { - if (!['EEXIST', 'EPERM'].includes(error?.code)) throw error; - fs.rmSync(statePath, { force: true }); - fs.renameSync(temporary, statePath); - } - return true; - } catch { - if (temporary) { - try { - fs.rmSync(temporary, { force: true }); - } catch { - } - } - return false; - } -} - -function captureKeys(context) { - const scope = context.sessionId || context.workspaceId || context.projectDir; - const keys = []; - if (context.sessionId && Number.isSafeInteger(context.turn)) { - keys.push(`turn:${context.sessionId}:${context.turn}`); - } - if (context.reasonKind === 'completed') keys.push(`diff:${scope}:${context.diffHash}`); - if (keys.length === 0) keys.push(`diff:${scope}:${context.reasonKind}:${context.diffHash}`); - return keys; -} - -function pruneCaptureState(state, now, ttlMs) { - for (const [key, timestamp] of Object.entries(state.records)) { - if (typeof timestamp !== 'number' || now - timestamp > ttlMs) delete state.records[key]; - } -} - -function wait(ms, signal) { - if (signal?.aborted) return Promise.reject(signal.reason); - return new Promise((resolve, reject) => { - const cleanup = () => signal?.removeEventListener('abort', abort); - const timer = setTimeout(() => { - cleanup(); - resolve(); - }, ms); - const abort = () => { - clearTimeout(timer); - cleanup(); - reject(signal.reason); - }; - signal?.addEventListener('abort', abort, { once: true }); - }); -} - -async function withCaptureLock(projectDir, signal, callback, options = {}) { - const statePath = captureStatePath(projectDir); - const lockPath = `${statePath}.lock`; - const staleMs = options.captureLockStaleMs ?? DEFAULT_CAPTURE_LOCK_STALE_MS; - const waitMs = options.captureLockWaitMs ?? DEFAULT_CAPTURE_LOCK_WAIT_MS; - const deadline = Date.now() + waitMs; - let lock; - - while (Date.now() <= deadline) { - if (signal?.aborted) return null; - try { - fs.mkdirSync(path.dirname(statePath), { recursive: true, mode: 0o700 }); - lock = fs.openSync(lockPath, fs.constants.O_WRONLY | fs.constants.O_CREAT | fs.constants.O_EXCL, 0o600); - break; - } catch (error) { - if (error?.code !== 'EEXIST') return callback({ statePath, state: readCaptureState(statePath), locked: false }); - try { - const age = Date.now() - fs.statSync(lockPath).mtimeMs; - if (age >= staleMs) { - fs.rmSync(lockPath, { force: true }); - appendEvolutionLog(`[Evolution] Recovered a stale capture lock after ${Math.round(age)}ms.`); - continue; - } - } catch { - } - const remaining = deadline - Date.now(); - if (remaining <= 0) break; - await wait(Math.min(CAPTURE_LOCK_RETRY_MS, remaining), signal).catch(() => {}); - } - } - if (lock === undefined) { - appendEvolutionLog(`[Evolution] Turn end: capture lock wait expired after ${waitMs}ms; outcome not recorded.`); - return null; - } - - try { - return await callback({ statePath, state: readCaptureState(statePath), locked: true }); - } finally { - try { - fs.closeSync(lock); - } catch { - } - try { - fs.rmSync(lockPath, { force: true }); - } catch { - } - } -} - -function normalizeCaptureOptions(projectDirOrOptions, reasonKind) { - if (typeof projectDirOrOptions === 'string') { - return { projectDir: projectDirOrOptions, reasonKind }; - } - return { ...projectDirOrOptions }; -} - -export function forgetCaptures(projectDir) { - for (const suffix of ['', '.lock']) { - try { - fs.rmSync(`${captureStatePath(projectDir)}${suffix}`, { force: true }); - } catch { - } - } -} - -export async function prepareCapture(projectDirOrOptions, legacyReasonKind = 'completed') { - const options = normalizeCaptureOptions(projectDirOrOptions, legacyReasonKind); - const { - projectDir, - reasonKind = 'completed', - sessionId = null, - turn = null, - observedSignals = [], - signal, - } = options; - if (!outcomeOfReason(reasonKind) || signal?.aborted) return null; - - const diff = await collectDiff(projectDir, options); - if (!diff.isRepo) { - appendEvolutionLog('[Evolution] Turn end: nothing recorded (not a git workspace).'); - return null; - } - if (isEmpty(diff) && reasonKind === 'completed') { - appendEvolutionLog('[Evolution] Turn end: nothing recorded (no changes detected this turn).'); - return null; - } - - const workspaceId = resolveWorkspaceId(projectDir); - const diffHash = fingerprint(diff, reasonKind); - const context = { projectDir, workspaceId, sessionId, turn, reasonKind, diffHash }; - return { - options, - context, - keys: captureKeys(context), - outcome: summarize(diff, reasonKind, observedSignals), - }; -} - -export async function commitPreparedCapture(prepared) { - if (!prepared) return null; - const { options, context, keys, outcome } = prepared; - const { - projectDir, - workspaceId, - sessionId, - turn, - reasonKind, - diffHash, - } = context; - const { - signal, - captureDedupeTtlMs = DEFAULT_DEDUPE_TTL_MS, - } = options; - - const hubPayload = { - gene_id: 'ad_hoc', - signals: outcome.signals, - status: outcome.status, - score: outcome.score, - summary: outcome.note, - sender_id: process.env.EVOMAP_NODE_ID || process.env.A2A_NODE_ID, - workspace_id: workspaceId, - session_id: sessionId, - turn, - turn_reason: reasonKind, - diff_hash: diffHash, - diff_scope: 'working_tree', - }; - - const recorded = await withCaptureLock(projectDir, signal, async ({ statePath, state }) => { - const now = Date.now(); - pruneCaptureState(state, now, captureDedupeTtlMs); - if (keys.some((key) => typeof state.records[key] === 'number')) { - appendEvolutionLog('[Evolution] Turn end: nothing recorded (duplicate turn or unchanged result).'); - return { duplicate: true, localOk: false, hubOk: false, hubAttempted: false }; - } - - const localOk = appendMemoryGraph(projectDir, buildEntry(outcome, context)); - const hubOk = localOk ? false : await recordToHub(hubPayload, options); - if (localOk || hubOk) { - for (const key of keys) state.records[key] = now; - writeCaptureState(statePath, state); - } - return { duplicate: false, localOk, hubOk, hubAttempted: !localOk }; - }, options); - if (!recorded || recorded.duplicate) return null; - - const hubOk = recorded.hubAttempted ? recorded.hubOk : await recordToHub(hubPayload, options); - const { localOk } = recorded; - const destination = hubOk && localOk ? 'Hub and local memory' : hubOk ? 'Hub' : localOk ? 'local memory' : 'nowhere'; - const receipt = `[Evolution] Turn outcome recorded to ${destination}: ${outcome.note}`; - appendEvolutionLog(receipt); - return receipt; -} - -export async function captureOutcome(projectDirOrOptions, legacyReasonKind = 'completed') { - return commitPreparedCapture(await prepareCapture(projectDirOrOptions, legacyReasonKind)); -} diff --git a/src/checks.js b/src/checks.js new file mode 100644 index 0000000..6662b67 --- /dev/null +++ b/src/checks.js @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +const CHECK_KINDS = [ + ['typecheck', /\btsc\b|\btype-?check\b|\bmypy\b|\bcargo\s+check\b|\bgo\s+vet\b/], + ['lint', /\blint\b|\beslint\b|\bruff\b|\bclippy\b|\bvalidate\b/], + ['test', /\btest\b|\bjest\b|\bvitest\b|\bmocha\b|\bpytest\b|\brspec\b|\bphpunit\b/], +]; + +export function checkKindOf(command) { + return CHECK_KINDS.find(([, pattern]) => pattern.test(command))?.[0] ?? 'build'; +} + +const OUTPUT_PIPE = /\s*(?&?\s*(?:\d|\S+)/g; + +export function checkIdentityOf(command) { + return String(command).replace(OUTPUT_PIPE, '').replace(STREAM_REDIRECT, '').replace(/\s+/g, ' ').trim(); +} + +function failedEarlierThenPassed(checks, last) { + const identity = checkIdentityOf(last.command); + return !last.failed && checks.slice(0, -1).some(({ command, failed }) => failed && checkIdentityOf(command) === identity); +} + +export function checkEvidenceOf(checks) { + const last = checks?.at(-1); + if (!last) return undefined; + return { kind: checkKindOf(last.command), passed: !last.failed, fail_then_pass: failedEarlierThenPassed(checks, last) }; +} diff --git a/src/config.js b/src/config.js index 4b6e0b8..5a714e7 100644 --- a/src/config.js +++ b/src/config.js @@ -11,18 +11,11 @@ export const Config = Schema.object({ projectDir: Schema.string(), proxyPort: Schema.number().step(1).min(1).max(65_535), editToolNames: Schema.array(Schema.string()).default([...EDIT_TOOL_NAMES]), - gitTimeoutMs: Schema.number().step(1).min(1).max(MAX_TIMER_MS).default(5_000), - gitMaxBufferBytes: Schema.number().step(1).min(1).max(128 * 1024 * 1024).default(10 * 1024 * 1024), proxyTimeoutMs: Schema.number().step(1).min(1).max(MAX_TIMER_MS).default(8_000), - hubTimeoutMs: Schema.number().step(1).min(1).max(MAX_TIMER_MS).default(8_000), assetPrimeEnabled: Schema.boolean().default(true), assetPrimeMinSimilarity: Schema.number().min(0).max(1).default(0.3), assetPrimeWaitMs: Schema.number().step(1).min(1).max(MAX_TIMER_MS).default(6_000), assetPrimeTimeoutMs: Schema.number().step(1).min(1).max(MAX_TIMER_MS).default(8_000), claimNudgeEnabled: Schema.boolean().default(false), claimNudgeTtlMs: Schema.number().step(1).min(1).max(MAX_TIMER_MS).default(12 * 60 * 60 * 1000), - captureDedupeTtlMs: Schema.number().step(1).min(1).max(MAX_TIMER_MS).default(24 * 60 * 60 * 1000), - captureLockStaleMs: Schema.number().step(1).min(1).max(MAX_TIMER_MS).default(60_000), - captureLockWaitMs: Schema.number().step(1).min(1).max(MAX_TIMER_MS).default(65_000), - untrackedHashMaxBytes: Schema.number().step(1).min(1).max(64 * 1024 * 1024).default(1024 * 1024), }); diff --git a/src/diagnostics.js b/src/diagnostics.js index 04642f1..ced34d6 100644 --- a/src/diagnostics.js +++ b/src/diagnostics.js @@ -7,6 +7,7 @@ import os from 'node:os'; import { MIN_EVOLVER_VERSION, isOlderThan } from './engine-version.js'; import { STRATEGY_MAX_CHARS } from './prime.js'; import { describeProxyEndpoint } from './proxy.js'; +import { REJECTED } from './report-outbox.js'; const PLUGIN_VERSION = JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8')).version; @@ -24,6 +25,28 @@ export function createRecallLog(now = () => Date.now()) { }; } +const MAX_LOGGED_REPORTS = 10; + +function keepLast(list, entry) { + list.push(entry); + if (list.length > MAX_LOGGED_REPORTS) list.shift(); +} + +export function createReportLog(now = () => Date.now()) { + const delivered = []; + const rejected = []; + return { + record({ verdict, taskId, assetId, receipt, reason, status }) { + const entry = { at: new Date(now()).toISOString(), task_id: taskId, asset_id: assetId }; + if (verdict === REJECTED) keepLast(rejected, { ...entry, reason, status }); + else keepLast(delivered, { ...entry, ...receipt }); + }, + snapshot() { + return { delivered: [...delivered], rejected: [...rejected] }; + }, + }; +} + function engineReport(pathVersion, runningVersion) { const effective = runningVersion ?? pathVersion; return { @@ -35,14 +58,14 @@ function engineReport(pathVersion, runningVersion) { }; } -export async function pluginDiagnostics({ config, engineVersion, recallLog, port }) { +export async function pluginDiagnostics({ config, engineVersion, recallLog, reportLog, port }) { const endpoint = describeProxyEndpoint(port); const runningVersion = typeof endpoint.running_version === 'string' ? endpoint.running_version : null; return { plugin: { name: '@evomap/dsh-evolver', version: PLUGIN_VERSION }, runtime: { platform: process.platform, arch: process.arch, node: process.version, os_release: os.release() }, proxy_endpoint: endpoint, - engine: engineReport(await engineVersion, runningVersion), + engine: engineReport(await engineVersion(), runningVersion), recall: { enabled: config.assetPrimeEnabled !== false, wait_ms: config.assetPrimeWaitMs ?? 6_000, @@ -51,5 +74,6 @@ export async function pluginDiagnostics({ config, engineVersion, recallLog, port max_strategy_chars: STRATEGY_MAX_CHARS, ...recallLog.snapshot(), }, + reuse_reports: reportLog.snapshot(), }; } diff --git a/src/engine-setup.js b/src/engine-setup.js new file mode 100644 index 0000000..a2f3c86 --- /dev/null +++ b/src/engine-setup.js @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +import { execFile } from 'node:child_process'; + +import { MIN_EVOLVER_VERSION, isOlderThan } from './engine-version.js'; + +const INSTALL_TIMEOUT_MS = 5 * 60 * 1000; +const START_TIMEOUT_MS = 60 * 1000; +const ERROR_MAX_CHARS = 300; +const INSTALL_ARGS = ['install', '-g', '@evomap/evolver@latest']; + +export function runCommand(command, args, timeoutMs) { + return new Promise((resolve) => { + execFile(command, args, { timeout: timeoutMs, encoding: 'utf8', shell: process.platform === 'win32', windowsHide: true }, + (error, stdout, stderr) => resolve({ ok: !error, error: error ? String(stderr || error.message).trim().slice(-ERROR_MAX_CHARS) : null })); + }); +} + +async function installLatest({ probeVersion, run }, previous) { + const installed = await run('npm', INSTALL_ARGS, INSTALL_TIMEOUT_MS); + const version = installed.ok ? await probeVersion() : null; + if (!version || isOlderThan(version, MIN_EVOLVER_VERSION)) { + return { status: 'install_failed', previous, version, error: installed.error ?? `npm installed ${version ?? 'no evolver on PATH'}` }; + } + const started = await run('evolver', ['lifecycle', previous ? 'restart' : 'start'], START_TIMEOUT_MS); + return { status: previous ? 'upgraded' : 'installed', previous, version, ...(!started.ok && { startError: started.error }) }; +} + +export async function prepareEngine({ probeVersion, isProxyReachable, run = runCommand }) { + const version = await probeVersion(); + if (!version || isOlderThan(version, MIN_EVOLVER_VERSION)) return installLatest({ probeVersion, run }, version); + if (await isProxyReachable()) return { status: 'ready', version }; + const started = await run('evolver', ['lifecycle', 'start'], START_TIMEOUT_MS); + return started.ok ? { status: 'started', version } : { status: 'start_failed', version, error: started.error }; +} + +const MANUAL_INSTALL = 'Run `npm install -g @evomap/evolver@latest`, then `evolver lifecycle start`.'; + +function startPhrase(outcome, verb) { + return outcome.startError + ? `, but its Proxy did not start (${outcome.startError}); run \`evolver lifecycle start\`.` + : ` and ${verb} its Proxy.`; +} + +export function setupNoticeText(outcome) { + switch (outcome.status) { + case 'installed': + return `[Evolver] Evolver was not installed, so this plugin installed ${outcome.version}${startPhrase(outcome, 'started')}`; + case 'upgraded': + return `[Evolver] Evolver ${outcome.previous} is older than the ${MIN_EVOLVER_VERSION} network recall needs, so this plugin ` + + `upgraded it to ${outcome.version}${startPhrase(outcome, 'restarted')}`; + case 'started': + return `[Evolver] The Evolver Proxy was not running, so this plugin started it (Evolver ${outcome.version}).`; + case 'install_failed': + return `[Evolver] ${outcome.previous ? `Evolver ${outcome.previous} is older than ${MIN_EVOLVER_VERSION}` : 'Evolver is not installed'} ` + + `and installing the latest release failed (${outcome.error}). ${MANUAL_INSTALL} Local memory keeps working meanwhile.`; + case 'start_failed': + return `[Evolver] The Evolver Proxy is not running and \`evolver lifecycle start\` failed (${outcome.error}). ` + + 'Network recall and reuse reports wait until it runs.'; + default: + return null; + } +} diff --git a/src/engine-version.js b/src/engine-version.js index 49522e3..6c5de34 100644 --- a/src/engine-version.js +++ b/src/engine-version.js @@ -38,9 +38,3 @@ export function installedEvolverVersion() { }); } -export function upgradeNoticeText(version) { - if (!version || !isOlderThan(version, MIN_EVOLVER_VERSION)) return null; - return `[Evolver] Evolver ${version} is installed, but this plugin needs ${MIN_EVOLVER_VERSION} or newer to recall ` - + 'network strategies. Upgrade with `npm install -g @evomap/evolver@latest`, then run `evolver` once to restart ' - + 'the Proxy. Local memory keeps working meanwhile.'; -} diff --git a/src/feedback-label.js b/src/feedback-label.js new file mode 100644 index 0000000..742c2e8 --- /dev/null +++ b/src/feedback-label.js @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +import { looksLikeCorrection } from './dissatisfaction.js'; + +const PRAISE_PATTERNS = [ + /\b(?:thanks|thank you|thx|perfect|great|awesome|nice|excellent|works now|it works|that works|lgtm|looks good)\b/i, + /谢谢|感谢|完美|好了|可以了|搞定|没问题了|成功了|不错|很好|太好了|牛/, +]; + +export function feedbackLabelOf(text) { + if (typeof text !== 'string' || !text.trim()) return null; + if (looksLikeCorrection(text)) return 'correction'; + return PRAISE_PATTERNS.some((pattern) => pattern.test(text)) ? 'praise' : null; +} diff --git a/src/git.js b/src/git.js deleted file mode 100644 index ad921e8..0000000 --- a/src/git.js +++ /dev/null @@ -1,39 +0,0 @@ -// SPDX-License-Identifier: MIT -// Copyright (c) 2026 EvoMap - -import { execFile } from 'node:child_process'; - -const DEFAULT_GIT_TIMEOUT_MS = 5_000; -const DEFAULT_GIT_MAX_BUFFER = 10 * 1024 * 1024; - -export function git(args, cwd, options = {}) { - const { - signal, - timeoutMs = DEFAULT_GIT_TIMEOUT_MS, - maxBuffer = DEFAULT_GIT_MAX_BUFFER, - } = options; - - return new Promise((resolve) => { - execFile( - 'git', - args, - { - cwd, - shell: false, - timeout: timeoutMs, - maxBuffer, - encoding: 'utf8', - signal, - }, - (error, stdout) => resolve({ - ok: !error, - stdout: typeof stdout === 'string' ? stdout : '', - error: error ?? null, - }), - ); - }); -} - -export async function gitText(args, cwd, options) { - return (await git(args, cwd, options)).stdout; -} diff --git a/src/index.js b/src/index.js index 7c02a54..c18b290 100644 --- a/src/index.js +++ b/src/index.js @@ -3,31 +3,35 @@ import { boundContextSummary, createUserMessage } from '@deepseek-ai/dsh-llm'; -import { createCaptureCoordinator } from './capture-coordinator.js'; -import { outcomeOfReason } from './capture.js'; +import { appliedVerdictOf } from './applied.js'; import { evolverCommands } from './commands.js'; import { Config } from './config.js'; +import { createRecallLog, createReportLog, pluginDiagnostics } from './diagnostics.js'; import { EDIT_TOOL_NAMES, editedContent, editedPath } from './edited-content.js'; -import { MIN_EVOLVER_VERSION, installedEvolverVersion, upgradeNoticeText } from './engine-version.js'; +import { prepareEngine, setupNoticeText } from './engine-setup.js'; +import { installedEvolverVersion } from './engine-version.js'; +import { feedbackLabelOf } from './feedback-label.js'; +import { forgetSession, rememberInjected, takeUnreported } from './injected-assets.js'; +import { recordReuseLocally } from './local-ledger.js'; import { noticeDue, pendingClaimUrl } from './onboarding.js'; import { hubGene, promptTextOf } from './prime.js'; -import { createRecallLog, pluginDiagnostics } from './diagnostics.js'; +import { resolveProjectDir, sessionDir } from './project-dir.js'; import { createProxyClient } from './proxy.js'; -import { looksLikeCorrection } from './dissatisfaction.js'; -import { correctableAssets, forgetSession, markCorrected, markReported, rememberInjected, unreportedAssets } from './injected-assets.js'; -import { reportInjectedReuse, reportReuseCorrection } from './reuse.js'; +import { createReportQueue } from './report-queue.js'; +import { rememberForRetention, retentionReportsOf, takeDueRetentions } from './retention-pending.js'; +import { correctionReportOf, immediateReportOf, reportedReuseOf } from './reuse.js'; +import { sessionKeyOf } from './session-key.js'; import { detectSignals } from './signals.js'; import { evolverSkillProvider } from './skill.js'; -import { sessionKeyOf } from './session-key.js'; import { evolverTools } from './tools.js'; -import { resolveProjectDir, sessionDir } from './workspace.js'; +import { createTurnRecords, noteAnswer, noteFileBeforeEdit, noteToolResult } from './turn-record.js'; +import { outcomeOfReason } from './turn-outcomes.js'; export const name = 'evolver'; export { Config }; export const inject = ['tools']; const DEFAULT_PRIME_WAIT_MS = 6_000; -const UPGRADE_NOTICE_TTL_MS = 24 * 60 * 60 * 1000; function pluginMessage(text, formed) { return createUserMessage({ @@ -36,65 +40,38 @@ function pluginMessage(text, formed) { }); } -function createTurnTracker() { - const records = new Map(); - const recordFor = (agent) => { - const key = sessionKeyOf(agent); - if (!key) return null; - let record = records.get(key); - if (!record) { - record = { signals: new Set(), notices: new Set() }; - records.set(key, record); - } - return { key, record }; - }; +function isSubagent(session) { + return session?.header?.origin === 'subagent'; +} + +function sessionMessages(config) { + const claimUrl = config.claimNudgeEnabled ? pendingClaimUrl() : null; + if (!claimUrl || !noticeDue(claimUrl, config.claimNudgeTtlMs)) return []; + const text = + `[Evolver] Your local node is not connected to the EvoMap network yet. Open ${claimUrl} ` + + 'while signed in to evomap.ai. Local memory already works; claiming enables network reuse.'; + return [pluginMessage(text, { form: 'notice', summary: 'Claim the local Evolver node.' })]; +} +// Installing or upgrading Evolver takes up to minutes, so it runs once per dsh +// process in the background and its outcome rides the first step after it. +function createEngineSetup(prepare) { + let started = null; + let notice = null; return { - add(agent, signals, noticeKey) { - const owned = recordFor(agent); - if (!owned) return true; - for (const signal of signals) owned.record.signals.add(signal); - if (owned.record.notices.has(noticeKey)) return false; - owned.record.notices.add(noticeKey); - return true; - }, - takeSignals(session) { - const key = sessionKeyOf(session); - if (!key) return []; - const record = records.get(key); - if (!record) return []; - const signals = [...record.signals]; - record.signals.clear(); - record.notices.clear(); - return signals; + start() { + started ??= prepare() + .then((outcome) => { notice = setupNoticeText(outcome); }) + .catch(() => {}); }, - clear(session) { - const key = sessionKeyOf(session); - if (key) records.delete(key); + takeNotice() { + const text = notice; + notice = null; + return text ? [pluginMessage(text, { form: 'notice', summary: boundContextSummary(text) })] : []; }, }; } -async function sessionMessages(config, engineVersion) { - const messages = []; - - const version = await engineVersion; - const upgrade = upgradeNoticeText(version); - if (upgrade && noticeDue(`evolver-version:${version}`, UPGRADE_NOTICE_TTL_MS)) { - messages.push(pluginMessage(upgrade, { form: 'notice', summary: `Upgrade Evolver to ${MIN_EVOLVER_VERSION} or newer.` })); - } - - const claimUrl = config.claimNudgeEnabled ? pendingClaimUrl() : null; - if (claimUrl && noticeDue(claimUrl, config.claimNudgeTtlMs)) { - const text = - `[Evolver] Your local node is not connected to the EvoMap network yet. Open ${claimUrl} ` - + 'while signed in to evomap.ai. Local memory already works; claiming enables network reuse.'; - messages.push(pluginMessage(text, { form: 'notice', summary: 'Claim the local Evolver node.' })); - } - - return messages; -} - function afterWait(ms) { return new Promise((resolve) => { const timer = setTimeout(() => resolve(null), ms); @@ -108,7 +85,7 @@ function afterWait(ms) { // that carries both. Workspace memory is a session fact and seeds once; the Hub // is re-queried per turn, because each prompt is a different task — bounded by // the ids already listed, so a repeat search adds nothing the model has seen. -function primeSteps(ctx, config, primeFetch, engineVersion, recallLog) { +function primeSteps(ctx, config, primeFetch, engineSetup, recallLog, onPrompt) { const seeded = new WeakSet(); const searchedTurn = new WeakMap(); const listedAssets = new WeakMap(); @@ -123,7 +100,7 @@ function primeSteps(ctx, config, primeFetch, engineVersion, recallLog) { }; const strategyMessage = (agent, turn, listed, { ids, text }) => { - for (const id of ids) rememberInjected(sessionKeyOf(agent), id, turn); + for (const id of ids) rememberInjected(sessionKeyOf(agent), id, { turn, strategyText: text }); for (const id of ids) listed.add(id); return text ? pluginMessage(text, { form: 'recall' }) : null; }; @@ -159,34 +136,17 @@ function primeSteps(ctx, config, primeFetch, engineVersion, recallLog) { return []; }; - // The prompt that opens the next turn is the only verdict available without - // asking anyone: when it plainly says the last answer did not hold, whatever - // was injected before it is revised. `correctableAssets` yields each asset at - // most once, because the Hub counts every report it receives. - const correctEarlier = (agent, claimed, signal) => { - const sessionKey = sessionKeyOf(agent); - if (!sessionKey || !looksLikeCorrection(promptTextOf(claimed))) return; - const assets = correctableAssets(sessionKey); - if (assets.length === 0) return; - reportReuseCorrection(primeFetch, { - assets, - sessionId: sessionKey, - signal, - onCorrected: (assetId) => markCorrected(sessionKey, assetId), - }).catch(() => {}); - }; - ctx.on('agent/pre-step', async (payload, next) => { const decision = await next(); const { agent, signal, turn } = payload; if (decision.kind === 'reject' || signal?.aborted || decision.messages.length === 0) return decision; - correctEarlier(agent, decision.messages, signal); + onPrompt(agent, promptTextOf(decision.messages)); - const messages = []; + const messages = engineSetup.takeNotice(); if (!seeded.has(agent)) { seeded.add(agent); - messages.push(...await sessionMessages(config, engineVersion)); + messages.push(...sessionMessages(config)); } messages.push(...await hubMessages(agent, turn, decision.messages, signal)); @@ -195,114 +155,104 @@ function primeSteps(ctx, config, primeFetch, engineVersion, recallLog) { }); } -const REUSE_RESULT_TOOL = 'evolver_asset_reuse_result'; - -function reuseReportReachedALedger(result) { - return result?.value?.recorded !== false; +function signalNotice(record, args) { + const signals = detectSignals(editedContent(args)); + if (signals.length === 0) return null; + const where = editedPath(args) || 'edited file'; + const noticeKey = `${where}\0${signals.join(',')}`; + if (record.notices.has(noticeKey)) return null; + record.notices.add(noticeKey); + return pluginMessage( + `[Evolution Signal] Detected: [${signals.join(', ')}] in ${where}.`, + { form: 'notice', summary: boundContextSummary(`Evolution signal: ${signals.join(', ')}`) }, + ); } -function nudgeOnSignals(ctx, editToolNames, tracker) { - ctx.on('tools/result', (exec, result) => { - if (!exec.agent || result?.isError) return; - if (exec.name === REUSE_RESULT_TOOL) { - const assetId = exec.arguments?.asset_id; - const reported = exec.arguments?.outcome; - if (reuseReportReachedALedger(result) && typeof assetId === 'string' && assetId) { - markReported(sessionKeyOf(exec.agent), assetId, typeof reported === 'string' && reported ? reported : 'success'); - } - return; - } - if (!editToolNames.includes(exec.name)) return; - - const signals = detectSignals(editedContent(exec.arguments)); - if (signals.length === 0) return; - - const where = editedPath(exec.arguments) || 'edited file'; - const noticeKey = `${where}\0${signals.join(',')}`; - if (!tracker.add(exec.agent, signals, noticeKey)) return; - exec.agent.inject( - pluginMessage( - `[Evolution Signal] Detected: [${signals.join(', ')}] in ${where}. This will be attached to the turn outcome.`, - { form: 'notice', summary: boundContextSummary(`Evolution signal: ${signals.join(', ')}`) }, - ), - ); - }); +function observeResult(record, exec, result, isEdit) { + try { + noteToolResult(record, exec, result, { isEdit }); + const notice = isEdit && !result?.isError ? signalNotice(record, exec.arguments) : null; + if (notice) exec.agent.inject(notice); + } catch { + } } -function assetsByTurn(injected) { - const byTurn = new Map(); - for (const { turn, assetId } of injected) { - const ids = byTurn.get(turn) ?? []; - ids.push(assetId); - byTurn.set(turn, ids); - } - return byTurn; +// `tools/execute` wraps only calls that were allowed to run, so a check the +// person denied is not taken for a failed one. +function observeTools(ctx, editToolNames, records, projectDirOf) { + ctx.on('tools/execute', async (exec, next) => { + const sessionKey = sessionKeyOf(exec.agent); + const isEdit = editToolNames.includes(exec.name); + if (sessionKey && isEdit) noteFileBeforeEdit(records.of(sessionKey), projectDirOf(exec.agent.session), exec.arguments); + const result = await next(); + if (sessionKey) observeResult(records.of(sessionKey), exec, result, isEdit); + return result; + }); } -function captureOnTurnEnd(ctx, fallbackDir, config, tracker, coordinator, primeFetch) { - ctx.on('session/event', (session, event) => { - if (event.type !== 'turn/end') return; - const reasonKind = event.data.reason?.kind; - const outcome = outcomeOfReason(reasonKind); - if (!outcome) return; - const projectDir = sessionDir(session?.header?.cwd, fallbackDir); +function reportTurns(ctx, records, deliver, labelable, projectDirOf) { + const closeTurn = (session, { turn, reason }) => { const sessionId = sessionKeyOf(session); - const signals = tracker.takeSignals(session); - const assets = unreportedAssets(sessionId).filter((entry) => entry.turn <= event.data.turn); - - for (const [injectedTurn, assetIds] of assetsByTurn(assets)) { - reportInjectedReuse(primeFetch, { - assetIds, - outcome, - turn: injectedTurn, - reasonKind, - sessionId, - onReported: (assetId, status) => markReported(sessionId, assetId, status), - }).catch(() => {}); - } - - coordinator.schedule({ - projectDir, - reasonKind, + const finished = records.take(sessionId); + const reasonKind = reason?.kind; + if (!outcomeOfReason(reasonKind)) return; + const reports = takeUnreported(sessionId, turn).map((asset) => immediateReportOf({ + asset, sessionId, - turn: event.data.turn, - observedSignals: signals, - gitTimeoutMs: config.gitTimeoutMs, - gitMaxBufferBytes: config.gitMaxBufferBytes, - hubTimeoutMs: config.hubTimeoutMs, - captureDedupeTtlMs: config.captureDedupeTtlMs, - captureLockStaleMs: config.captureLockStaleMs, - captureLockWaitMs: config.captureLockWaitMs, - untrackedHashMaxBytes: config.untrackedHashMaxBytes, - }); + turn, + reasonKind, + checks: finished.checks, + applied: appliedVerdictOf({ strategyText: asset.strategyText, ...finished }).verdict, + })); + const reuses = reports.map(reportedReuseOf); + labelable.set(sessionId, reuses); + deliver(reports.map((report) => ({ report, sessionId }))).catch(() => {}); + rememberForRetention({ sessionId, projectDir: projectDirOf(session), reuses, editedBefore: finished.editedBefore }); + }; + + ctx.on('session/event', (session, event) => { + if (isSubagent(session)) return; + if (event.type === 'assistant/message') noteAnswer(records.of(sessionKeyOf(session)), event.data.message); + if (event.type === 'turn/end') closeTurn(session, event.data); }); +} - ctx.on('session/flush', (session) => coordinator.flush(sessionKeyOf(session), sessionDir(session?.header?.cwd, fallbackDir))); - ctx.on('session/disposed', (session) => { - tracker.clear(session); - forgetSession(sessionKeyOf(session)); +function onSessionStart(ctx, engineSetup, deliver, projectDirOf) { + ctx.on('agent/session-start', ({ agent }) => { + engineSetup.start(); + if (isSubagent(agent.session)) return; + const due = takeDueRetentions({ sessionId: sessionKeyOf(agent), projectDir: projectDirOf(agent.session) }); + deliver(due.flatMap(retentionReportsOf)).catch(() => {}); }); } +function followUpReporter(deliver, labelable) { + return (agent, prompt) => { + const sessionId = sessionKeyOf(agent); + if (!prompt || !labelable.has(sessionId)) return; + const reuses = labelable.get(sessionId); + labelable.delete(sessionId); + const label = feedbackLabelOf(prompt); + const reports = reuses.map((reuse) => correctionReportOf(reuse, label)).filter((report) => report !== null); + deliver(reports.map((report) => ({ report, sessionId }))).catch(() => {}); + }; +} + export function apply(ctx, config = {}) { const explicitDir = config.projectDir ? sessionDir(config.projectDir, null) : null; if (config.projectDir && !explicitDir) throw new Error(`Evolver projectDir is not a directory: ${config.projectDir}`); - if ( - Number.isFinite(config.captureLockWaitMs) - && Number.isFinite(config.captureLockStaleMs) - && config.captureLockWaitMs <= config.captureLockStaleMs - ) { - throw new Error('Evolver captureLockWaitMs must be greater than captureLockStaleMs.'); - } const fallbackDir = explicitDir ?? resolveProjectDir(); + const projectDirOf = (session) => sessionDir(session?.header?.cwd, fallbackDir); const proxyFetch = createProxyClient({ port: config.proxyPort, timeoutMs: config.proxyTimeoutMs }); const primeFetch = createProxyClient({ port: config.proxyPort, timeoutMs: config.assetPrimeTimeoutMs ?? 8_000 }); - const tracker = createTurnTracker(); - const coordinator = createCaptureCoordinator(); - const engineVersion = installedEvolverVersion(); + const engineSetup = createEngineSetup(() => prepareEngine({ + probeVersion: installedEvolverVersion, + isProxyReachable: async () => !(await proxyFetch('GET', '/proxy/status')).unreachable, + })); const recallLog = createRecallLog(); - const diagnostics = () => pluginDiagnostics({ config, engineVersion, recallLog, port: config.proxyPort }); + const reportLog = createReportLog(); + const diagnostics = () => pluginDiagnostics({ config, engineVersion: installedEvolverVersion, recallLog, reportLog, port: config.proxyPort }); for (const tool of evolverTools(proxyFetch, { diagnostics })) ctx.tools.register(tool); ctx.inject(['skills'], (scoped) => { @@ -313,7 +263,19 @@ export function apply(ctx, config = {}) { for (const command of evolverCommands()) scoped.commands.register(command); }); - primeSteps(ctx, config, primeFetch, engineVersion, recallLog); - nudgeOnSignals(ctx, config.editToolNames ?? EDIT_TOOL_NAMES, tracker); - captureOnTurnEnd(ctx, fallbackDir, config, tracker, coordinator, primeFetch); + const deliver = createReportQueue({ proxyFetch, recordLocally: recordReuseLocally, onSettled: reportLog.record }); + const records = createTurnRecords(); + const labelable = new Map(); + + primeSteps(ctx, config, primeFetch, engineSetup, recallLog, followUpReporter(deliver, labelable)); + observeTools(ctx, config.editToolNames ?? EDIT_TOOL_NAMES, records, projectDirOf); + reportTurns(ctx, records, deliver, labelable, projectDirOf); + onSessionStart(ctx, engineSetup, deliver, projectDirOf); + + ctx.on('session/disposed', (session) => { + const sessionKey = sessionKeyOf(session); + records.forget(sessionKey); + labelable.delete(sessionKey); + forgetSession(sessionKey); + }); } diff --git a/src/injected-assets.js b/src/injected-assets.js index fe574fa..e1c6134 100644 --- a/src/injected-assets.js +++ b/src/injected-assets.js @@ -1,110 +1,50 @@ // SPDX-License-Identifier: MIT // Copyright (c) 2026 EvoMap -import crypto from 'node:crypto'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; +import { hashedName, readState, removeState, statePath, writeState } from './state-file.js'; const ENTRY_PRUNE_MS = 7 * 24 * 60 * 60 * 1000; -// The Hub aggregates reuse per asset id and counts every report it receives: -// `summarizeReuseOutcomes` keys on assetId alone and increments, with no -// de-duplication anywhere upstream. Exactly-once is therefore this file's job, -// and the asset id is the only key that means anything — a task id is neither -// an aggregation key nor one of the audit anchors (assetId + cycleId). -function storePath(sessionKey) { - const key = crypto.createHash('sha256').update(String(sessionKey)).digest('hex').slice(0, 16); - return path.join(os.homedir(), '.evolver', 'state', `injected-${key}.json`); -} - -function readStore(sessionKey) { - try { - const parsed = JSON.parse(fs.readFileSync(storePath(sessionKey), 'utf8')); - return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? parsed : {}; - } catch { - return {}; - } -} - -function writeStore(sessionKey, store) { - const target = storePath(sessionKey); - try { - fs.mkdirSync(path.dirname(target), { recursive: true, mode: 0o700 }); - const temporary = `${target}.${process.pid}.${crypto.randomBytes(6).toString('hex')}.tmp`; - fs.writeFileSync(temporary, JSON.stringify(store), { mode: 0o600, flag: 'wx' }); - try { - fs.renameSync(temporary, target); - } catch (error) { - if (!['EEXIST', 'EPERM'].includes(error?.code)) throw error; - fs.rmSync(target, { force: true }); - fs.renameSync(temporary, target); - } - } catch { - } -} - -function pruned(store, now) { - const kept = {}; - for (const [assetId, entry] of Object.entries(store)) { +function pruned(ledger, now) { + return Object.fromEntries(Object.entries(ledger ?? {}).filter(([, entry]) => { const at = Number(entry?.injectedAt); - if (Number.isFinite(at) && now - at <= ENTRY_PRUNE_MS) kept[assetId] = entry; - } - return kept; + return Number.isFinite(at) && now - at <= ENTRY_PRUNE_MS; + })); } -function mutate(sessionKey, change, now = Date.now()) { - if (!sessionKey) return; - const store = pruned(readStore(sessionKey), now); - if (change(store) === false) return; - writeStore(sessionKey, store); +export function withInjected(ledger, assetId, { turn, strategyText }, now) { + const kept = pruned(ledger, now); + if (kept[assetId]) return kept; + return { ...kept, [assetId]: { turn, injectedAt: now, strategyText } }; } -// An injected id is written before the turn it belongs to can end, so a report -// survives the process dying between the two — and a lookup that lands late -// records itself the same way, from whatever step it arrived in. -export function rememberInjected(sessionKey, assetId, turn, now = Date.now()) { - mutate(sessionKey, (store) => { - if (store[assetId]) return false; - store[assetId] = { turn, injectedAt: now }; - }, now); +export function unreportedOf(ledger, turn) { + return Object.entries(ledger ?? {}) + .filter(([, entry]) => !entry?.reportedAt && Number(entry?.turn) <= turn) + .map(([assetId, entry]) => ({ assetId, strategyText: entry.strategyText ?? '' })); } -export function unreportedAssets(sessionKey) { - return Object.entries(readStore(sessionKey)) - .filter(([, entry]) => !entry?.outcome) - .map(([assetId, entry]) => ({ assetId, turn: Number(entry?.turn) })); +export function withReported(ledger, assetIds, now) { + const kept = pruned(ledger, now); + for (const assetId of assetIds) if (kept[assetId]) kept[assetId] = { ...kept[assetId], reportedAt: now }; + return kept; } -// A correction can only revise a verdict this plugin actually sent, and only -// once: the Hub counts each report, so a second correction would be a second -// negative for one reuse. -export function correctableAssets(sessionKey) { - return Object.entries(readStore(sessionKey)) - .filter(([, entry]) => entry?.outcome === 'success' && !entry?.corrected) - .map(([assetId, entry]) => ({ assetId, turn: Number(entry?.turn) })); -} +const ledgerPath = (sessionKey) => statePath(hashedName('injected', sessionKey)); -export function markReported(sessionKey, assetId, outcome, now = Date.now()) { - mutate(sessionKey, (store) => { - const entry = store[assetId] ?? { turn: 0, injectedAt: now }; - if (entry.outcome) return false; - store[assetId] = { ...entry, outcome, reportedAt: now }; - }, now); +export function rememberInjected(sessionKey, assetId, injection, now = Date.now()) { + if (!sessionKey) return; + writeState(ledgerPath(sessionKey), withInjected(readState(ledgerPath(sessionKey), {}), assetId, injection, now)); } -export function markCorrected(sessionKey, assetId, now = Date.now()) { - mutate(sessionKey, (store) => { - const entry = store[assetId]; - if (!entry || entry.corrected) return false; - store[assetId] = { ...entry, corrected: true, correctedAt: now }; - }, now); +export function takeUnreported(sessionKey, turn, now = Date.now()) { + if (!sessionKey) return []; + const ledger = readState(ledgerPath(sessionKey), {}); + const due = unreportedOf(ledger, turn); + if (due.length > 0) writeState(ledgerPath(sessionKey), withReported(ledger, due.map((asset) => asset.assetId), now)); + return due; } export function forgetSession(sessionKey) { - if (!sessionKey) return; - try { - fs.rmSync(storePath(sessionKey), { force: true }); - } catch { - } + if (sessionKey) removeState(ledgerPath(sessionKey)); } diff --git a/src/prime.js b/src/prime.js index 16808b0..d81da7a 100644 --- a/src/prime.js +++ b/src/prime.js @@ -1,6 +1,8 @@ // SPDX-License-Identifier: MIT // Copyright (c) 2026 EvoMap +import { withoutSecrets } from './redact.js'; + // One round trip, not two: since evolver 2.0.39 `/asset/fetch` recalls by text // when it is given no ids, so selecting a candidate and materialising it are the // same call. No `limit` is sent, so the Hub's own default decides how many come @@ -48,7 +50,7 @@ export function promptTextOf(messages) { if (block?.type === 'text' && typeof block.text === 'string' && block.text.trim()) parts.push(block.text.trim()); } } - return parts.join('\n').trim().slice(0, PROMPT_MAX_CHARS); + return withoutSecrets(parts.join('\n')).trim().slice(0, PROMPT_MAX_CHARS); } function recalledAssets(data) { diff --git a/src/project-dir.js b/src/project-dir.js new file mode 100644 index 0000000..c16662c --- /dev/null +++ b/src/project-dir.js @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +import fs from 'node:fs'; + +function isDirectory(candidate) { + if (typeof candidate !== 'string' || candidate.length === 0) return false; + try { + return fs.statSync(candidate).isDirectory(); + } catch { + return false; + } +} + +function realPath(dir) { + try { + return fs.realpathSync(dir); + } catch { + return dir; + } +} + +export function resolveProjectDir() { + for (const fromEnv of [process.env.DSH_PROJECT_DIR, process.env.CLAUDE_PROJECT_DIR]) { + if (isDirectory(fromEnv)) return realPath(fromEnv); + } + return realPath(process.cwd()); +} + +export function sessionDir(candidate, fallback = resolveProjectDir()) { + return isDirectory(candidate) ? realPath(candidate) : fallback; +} diff --git a/src/proxy.js b/src/proxy.js index fc3b894..9fdbb03 100644 --- a/src/proxy.js +++ b/src/proxy.js @@ -68,7 +68,7 @@ export function describeProxyEndpoint(port = process.env.EVOMAP_PROXY_PORT || DE } const START_HINT = - 'Start it by running `evolver` once inside a git repo (the CLI launches the Proxy). ' + + 'Start it with `evolver lifecycle start`. ' + 'Set EVOMAP_PROXY_PORT if you use a non-default port.'; function httpErrorHint(status, base, token) { @@ -104,7 +104,7 @@ export function createProxyClient({ } catch (error) { if (callerSignal?.aborted) return { ok: false, error: 'Proxy request cancelled.' }; const what = error?.name === 'TimeoutError' ? 'Proxy request timed out' : `Proxy connection failed: ${error?.message}`; - return { ok: false, error: `${what}. Evolver Proxy not reachable at ${base}. ${START_HINT}` }; + return { ok: false, unreachable: true, error: `${what}. Evolver Proxy not reachable at ${base}. ${START_HINT}` }; } const text = await response.text(); diff --git a/src/redact.js b/src/redact.js new file mode 100644 index 0000000..26211a0 --- /dev/null +++ b/src/redact.js @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +const SECRET_PATTERNS = [ + /-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?(?:-----END [A-Z ]*PRIVATE KEY-----|$)/g, + /\b(?:sk|pk|rk)-(?:ant-|proj-|live-|test-)?[A-Za-z0-9_-]{16,}/g, + /\bgh[pousr]_[A-Za-z0-9]{20,}/g, + /\bgithub_pat_[A-Za-z0-9_]{20,}/g, + /\bglpat-[A-Za-z0-9_-]{20,}/g, + /\bxox[abposr]-[A-Za-z0-9-]{10,}/g, + /\b(?:AKIA|ASIA)[A-Z0-9]{16}\b/g, + /\bAIza[A-Za-z0-9_-]{30,}/g, + /\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}/g, + /\bBearer\s+[A-Za-z0-9._~+/-]{16,}=*/gi, + /\b[a-z][a-z0-9+.-]*:\/\/[^\s:/@]+:[^\s@/]+@/gi, + /\b(?:api[_-]?key|secret|token|passw(?:or)?d|pwd|access[_-]?key|private[_-]?key|client[_-]?secret)\b\s*[:=]\s*["']?[^\s"',;]{6,}/gi, +]; + +const REDACTED = '[redacted]'; + +export function withoutSecrets(text) { + if (typeof text !== 'string') return ''; + return SECRET_PATTERNS.reduce((cleaned, pattern) => cleaned.replace(pattern, REDACTED), text); +} diff --git a/src/report-outbox.js b/src/report-outbox.js new file mode 100644 index 0000000..2b321dc --- /dev/null +++ b/src/report-outbox.js @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +export const OUTBOX_TTL_MS = 8 * 24 * 60 * 60 * 1000; + +const keyOf = (report) => `${report.task_id}\0${report.asset_id}`; + +function pruned(outbox, now) { + return Object.fromEntries(Object.entries(outbox ?? {}).filter(([, entry]) => now - Number(entry?.queuedAt) <= OUTBOX_TTL_MS)); +} + +export function withQueued(outbox, queued, now) { + const kept = pruned(outbox, now); + for (const { report, sessionId } of queued) { + kept[keyOf(report)] ??= { report, sessionId, queuedAt: now, attempts: 0 }; + } + return kept; +} + +export const SENT = 'sent'; +export const REJECTED = 'rejected'; +export const RETRY = 'retry'; + +export function isSettled(entry) { + return Boolean(entry.sentAt || entry.rejectedAt); +} + +function settlementOf({ verdict, reason, status, receipt }, now) { + if (verdict === SENT) return { sentAt: now, ...(receipt && { receipt }) }; + if (verdict === REJECTED) return { rejectedAt: now, rejection: { reason, ...(status !== undefined && { status }) } }; + return {}; +} + +export function withAttempt(outbox, key, attempt, now) { + const entry = outbox[key]; + if (!entry) return outbox; + const local = attempt.isLocal ? { localAt: now } : {}; + return { ...outbox, [key]: { ...entry, attempts: entry.attempts + 1, ...local, ...settlementOf(attempt, now) } }; +} diff --git a/src/report-queue.js b/src/report-queue.js new file mode 100644 index 0000000..1b0d526 --- /dev/null +++ b/src/report-queue.js @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +import { withQueued } from './report-outbox.js'; +import { deliverPending } from './reuse.js'; +import { readState, statePath, writeState } from './state-file.js'; + +const outboxPath = () => statePath('reports-outbox.json'); + +function attemptedEntries(before, after) { + return Object.fromEntries(Object.entries(after).filter(([key, entry]) => before[key] !== entry)); +} + +// Another dsh process may queue reports while this one is delivering, so the +// attempts are merged into the file as it stands afterwards, not over the +// snapshot that was delivered. +export function createReportQueue(ledgers, clock = () => Date.now()) { + let work = Promise.resolve(); + return function deliver(queued = []) { + work = work.catch(() => {}).then(async () => { + const now = clock(); + const outbox = withQueued(readState(outboxPath(), {}), queued, now); + writeState(outboxPath(), outbox); + const delivered = await deliverPending(ledgers, outbox, now); + writeState(outboxPath(), { ...withQueued(readState(outboxPath(), {}), [], now), ...attemptedEntries(outbox, delivered) }); + }); + return work; + }; +} diff --git a/src/retention-pending.js b/src/retention-pending.js new file mode 100644 index 0000000..3da1501 --- /dev/null +++ b/src/retention-pending.js @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +import crypto from 'node:crypto'; +import fs from 'node:fs'; + +import { OUTBOX_TTL_MS } from './report-outbox.js'; +import { ABSENT, retentionOf } from './retention.js'; +import { retentionReportOf } from './reuse.js'; +import { readState, statePath, writeState } from './state-file.js'; + +const pendingPath = () => statePath('retention-pending.json'); + +export function contentHashOf(file) { + try { + return crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex'); + } catch { + return ABSENT; + } +} + +export function rememberForRetention({ sessionId, projectDir, reuses, editedBefore, now = Date.now() }) { + if (reuses.length === 0 || editedBefore.size === 0) return; + const files = [...editedBefore].map(([file, before]) => ({ path: file, before, after: contentHashOf(file) })); + writeState(pendingPath(), [...readState(pendingPath(), []), { sessionId, projectDir, reuses, files, atMs: now }]); +} + +export function takeDueRetentions({ sessionId, projectDir, now = Date.now() }) { + const pending = readState(pendingPath(), []); + if (pending.length === 0) return []; + const isDue = (entry) => entry.projectDir === projectDir && entry.sessionId !== sessionId; + writeState(pendingPath(), pending.filter((entry) => !isDue(entry) && now - entry.atMs <= OUTBOX_TTL_MS)); + return pending.filter((entry) => isDue(entry) && now - entry.atMs <= OUTBOX_TTL_MS); +} + +export function retentionReportsOf(entry) { + const verdict = retentionOf(entry.files.map((file) => ({ ...file, now: contentHashOf(file.path) }))).class; + return entry.reuses + .map((reuse) => retentionReportOf(reuse, verdict)) + .filter((report) => report !== null) + .map((report) => ({ report, sessionId: entry.sessionId })); +} diff --git a/src/retention.js b/src/retention.js new file mode 100644 index 0000000..70f4c22 --- /dev/null +++ b/src/retention.js @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +export const ABSENT = 'absent'; + +function fileFateOf({ before, after, now }) { + if (now === after) return 'kept'; + if (now === before) return 'reverted'; + return 'rewritten'; +} + +export function retentionOf(files) { + const changed = (files ?? []).filter(({ before, after }) => before !== after); + if (changed.length === 0) return { class: 'no_change', files: 0 }; + const fates = new Set(changed.map(fileFateOf)); + const verdict = fates.size === 1 ? [...fates][0] : 'rewritten'; + return { class: verdict, files: changed.length }; +} diff --git a/src/reuse.js b/src/reuse.js index 53966af..7ea5510 100644 --- a/src/reuse.js +++ b/src/reuse.js @@ -1,76 +1,143 @@ // SPDX-License-Identifier: MIT // Copyright (c) 2026 EvoMap -import { recordReuseLocally } from './local-ledger.js'; - -// An automatic report is weaker evidence than one the model made after -// validating its own work: the strategy was put in front of the model, but -// nothing proves it was followed. The reason says so, so the Hub can weigh it -// against a reported reuse rather than mistaking it for one. -function automaticReason(turn, reasonKind) { - return `Injected by Evolver into dsh turn ${turn}; not confirmed as applied. ` - + `Outcome derived from how the turn ended (${reasonKind}).`; -} - -function correctionReason(turn) { - return `Revising the automatic verdict for dsh turn ${turn}: the next prompt ` - + 'in the same session read as a correction, so the earlier reuse did not hold.'; -} - -// The asset id is what the report is about — `summarizeReuseOutcomes` -// aggregates on it alone. The task id is something else: the Hub hashes it with -// the asset and the reporting node into the event id that makes a retry -// idempotent, and its schema requires one, so a report without it is rejected -// with 400 and reaches no Hub ledger at all. One id per turn per session means -// re-sending the same verdict is recognised, while a later correction carries -// its own and is not mistaken for that retry. -// `ok` only says the Proxy answered. The Proxy forwards to the Hub and reports -// the real fate in the body. Treating the transport as the outcome would mark -// an asset reported that no ledger ever saw, and the on-disk record would make -// that mistake permanent. -async function postToHub(proxyFetch, { assetId, outcome, reason, taskId, signal }) { +import { UNKNOWN } from './applied.js'; +import { checkEvidenceOf } from './checks.js'; +import { REJECTED, RETRY, SENT, isSettled, withAttempt } from './report-outbox.js'; +import { outcomeOfReason } from './turn-outcomes.js'; + +const LABEL_OUTCOMES = { correction: 'failed', praise: 'success' }; +const RETENTION_OUTCOMES = { kept: 'success', reverted: 'failed' }; +const RECEIPT_FIELDS = ['eligible', 'counted', 'weight', 'reason']; + +function turnTaskId(sessionId, turn) { + return `dsh:${sessionId || 'unknown'}:${turn ?? 'unknown'}`; +} + +// The Hub has no `unknown`, and `unused` carries a soft negative there; a +// partial reuse at tier none weighs nothing either way. +const UNKNOWN_APPLIED_AS = 'partial'; + +function immediateEvidenceOf({ checks, applied, reasonKind }) { + const check = checkEvidenceOf(checks); + const isKnown = applied !== UNKNOWN; + return { + tier: check && isKnown ? 'L1' : 'none', + applied: isKnown ? applied : UNKNOWN_APPLIED_AS, + ...(check && { check }), + aborted: reasonKind === 'aborted', + stage: 'immediate', + }; +} + +function immediateOutcomeOf(evidence, reasonKind) { + if (evidence.check) return evidence.check.passed ? 'success' : 'failed'; + return outcomeOfReason(reasonKind) ?? 'failed'; +} + +function checkPhrase(check) { + if (!check) return 'no check ran'; + const result = check.passed ? 'passed' : 'failed'; + return `last ${check.kind} ${result}${check.fail_then_pass ? ' after failing earlier' : ''}`; +} + +function reportOf({ assetId, taskId, outcome, reason, evidence }) { + return { asset_id: assetId, outcome, task_id: taskId, reason, evidence }; +} + +export function immediateReportOf({ asset, sessionId, turn, reasonKind, checks, applied }) { + const evidence = immediateEvidenceOf({ checks, applied, reasonKind }); + return reportOf({ + assetId: asset.assetId, + taskId: turnTaskId(sessionId, turn), + outcome: immediateOutcomeOf(evidence, reasonKind), + reason: `Injected by Evolver into dsh turn ${turn}; applied: ${applied}; ${checkPhrase(evidence.check)}.`, + evidence, + }); +} + +export function reportedReuseOf(report) { + return { assetId: report.asset_id, applied: report.evidence.applied, taskId: report.task_id }; +} + +export function correctionReportOf(reuse, label) { + if (!LABEL_OUTCOMES[label]) return null; + return reportOf({ + ...reuse, + taskId: `${reuse.taskId}:correction`, + outcome: LABEL_OUTCOMES[label], + reason: `The person's next prompt read as ${label} of the turn that reused this strategy.`, + evidence: { tier: 'L3', applied: reuse.applied, stage: 'correction' }, + }); +} + +export function retentionReportOf(reuse, retentionClass) { + if (!RETENTION_OUTCOMES[retentionClass]) return null; + return reportOf({ + ...reuse, + taskId: `${reuse.taskId}:retention`, + outcome: RETENTION_OUTCOMES[retentionClass], + reason: `By the next session the files the turn edited were ${retentionClass}.`, + evidence: { tier: 'L2', applied: reuse.applied, stage: 'retention' }, + }); +} + +export function selfReportedEvidenceOf(outcome) { + return { tier: 'L5', applied: outcome === 'mismatched' ? 'unused' : 'used', stage: 'immediate' }; +} + +// A Proxy older than `terminal` says `hub 400` or `hub 422` for a request the +// Hub refused as invalid. `hub 404` stays retryable: the Hub may not serve the +// route yet. +const REJECTED_BY_HUB = /^hub (?:400|422)\b/; + +function isRejection(data) { + return data?.terminal === true || REJECTED_BY_HUB.test(String(data?.reason ?? '')); +} + +function isTaken(data) { + return data?.recorded !== false || data?.reason === 'duplicate_report'; +} + +function receiptOf(data) { + return Object.fromEntries(RECEIPT_FIELDS.filter((field) => data?.[field] !== undefined).map((field) => [field, data[field]])); +} + +export function deliveryOf(result) { + if (result?.ok !== true) return { verdict: RETRY }; + if (isRejection(result.data)) return { verdict: REJECTED, reason: String(result.data.reason ?? 'rejected'), status: result.data.status }; + if (!isTaken(result.data)) return { verdict: RETRY }; + return { verdict: SENT, receipt: receiptOf(result.data) }; +} + +async function postToHub(proxyFetch, report) { try { - const result = await proxyFetch('POST', '/asset/reuse-result', { asset_id: assetId, outcome, reason, task_id: taskId }, signal); - return result?.ok === true && result.data?.recorded !== false; + return deliveryOf(await proxyFetch('POST', '/asset/reuse-result', report)); } catch { - return false; + return { verdict: RETRY }; } } -function reportId(sessionId, turn) { - return `dsh:${sessionId || 'unknown'}:${turn ?? 'unknown'}`; +function isLocalRecordDue({ report, localAt }, delivery) { + return !localAt && delivery.verdict !== REJECTED && report.evidence.tier !== 'none'; } -// The local root_event is the half that pays off today: the actuator that -// re-orders candidates reads root_events, not the Hub. A verdict counts as -// delivered when either ledger took it, so one of them being down does not -// keep the asset pending forever. -async function postOutcome(proxyFetch, { assetId, outcome, reason, taskId, sessionId, signal }) { - const hub = await postToHub(proxyFetch, { assetId, outcome, reason, taskId, signal }); - const local = await recordReuseLocally({ assetId, outcome, sessionId }); - return hub || local; +async function recordedLocally(recordLocally, { report, sessionId }) { + return recordLocally({ assetId: report.asset_id, outcome: report.outcome, sessionId }).catch(() => false); } -export async function reportInjectedReuse(proxyFetch, options) { - const { assetIds, outcome, turn, reasonKind, sessionId, signal, onReported } = options; - if (!outcome || assetIds.length === 0) return []; - - const reported = []; - for (const assetId of assetIds) { - if (!await postOutcome(proxyFetch, { assetId, outcome: outcome.status, reason: automaticReason(turn, reasonKind), taskId: reportId(sessionId, turn), sessionId, signal })) continue; - reported.push(assetId); - onReported?.(assetId, outcome.status); - } - return reported; +async function attemptOf(ledgers, entry) { + const delivery = await postToHub(ledgers.proxyFetch, entry.report); + const isLocal = isLocalRecordDue(entry, delivery) && (await recordedLocally(ledgers.recordLocally, entry)); + if (delivery.verdict !== RETRY) ledgers.onSettled({ taskId: entry.report.task_id, assetId: entry.report.asset_id, ...delivery }); + return { ...delivery, isLocal }; } -export async function reportReuseCorrection(proxyFetch, options) { - const { assets, sessionId, signal, onCorrected } = options; - const corrected = []; - for (const { assetId, turn } of assets) { - if (!await postOutcome(proxyFetch, { assetId, outcome: 'failed', reason: correctionReason(turn), taskId: `${reportId(sessionId, turn)}:correction`, sessionId, signal })) continue; - corrected.push(assetId); - onCorrected?.(assetId); +export async function deliverPending(ledgers, outbox, now) { + let delivered = outbox; + for (const [key, entry] of Object.entries(outbox)) { + if (isSettled(entry)) continue; + delivered = withAttempt(delivered, key, await attemptOf(ledgers, entry), now); } - return corrected; + return delivered; } diff --git a/src/session-key.js b/src/session-key.js index 5af66a2..00ea189 100644 --- a/src/session-key.js +++ b/src/session-key.js @@ -1,9 +1,7 @@ // SPDX-License-Identifier: MIT // Copyright (c) 2026 EvoMap -export function sessionKeyOf(subject, projectDir) { - if (typeof subject === 'string' && subject.length > 0) return subject; +export function sessionKeyOf(subject) { const identity = subject?.session?.id ?? subject?.id; - if (identity !== null && identity !== undefined && String(identity).length > 0) return String(identity); - return projectDir ? `workspace:${projectDir}` : null; + return identity !== null && identity !== undefined && String(identity).length > 0 ? String(identity) : null; } diff --git a/src/signals.js b/src/signals.js index 6f6e4f8..b3220b3 100644 --- a/src/signals.js +++ b/src/signals.js @@ -48,17 +48,3 @@ export function detectSignals(text) { return [...found].sort(); } -// Only added lines describe this turn's intent; headers, hunk markers and -// removed lines describe what the repository used to be. -export function addedLines(diffBody) { - if (typeof diffBody !== 'string') return ''; - return diffBody - .split('\n') - .filter((line) => line.startsWith('+') && !line.startsWith('+++')) - .map((line) => line.slice(1)) - .join('\n'); -} - -export function detectSignalsInDiff(diffBody) { - return detectSignals(addedLines(diffBody)); -} diff --git a/src/state-file.js b/src/state-file.js new file mode 100644 index 0000000..90b5a6a --- /dev/null +++ b/src/state-file.js @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +import crypto from 'node:crypto'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; + +export function statePath(name) { + return path.join(os.homedir(), '.evolver', 'state', name); +} + +export function hashedName(prefix, key) { + return `${prefix}-${crypto.createHash('sha256').update(String(key)).digest('hex').slice(0, 16)}.json`; +} + +export function readState(target, fallback) { + try { + const parsed = JSON.parse(fs.readFileSync(target, 'utf8')); + return parsed && typeof parsed === 'object' && Array.isArray(parsed) === Array.isArray(fallback) ? parsed : fallback; + } catch { + return fallback; + } +} + +export function writeState(target, value) { + try { + fs.mkdirSync(path.dirname(target), { recursive: true, mode: 0o700 }); + const temporary = `${target}.${process.pid}.${crypto.randomBytes(6).toString('hex')}.tmp`; + fs.writeFileSync(temporary, JSON.stringify(value), { mode: 0o600, flag: 'wx' }); + try { + fs.renameSync(temporary, target); + } catch (error) { + if (!['EEXIST', 'EPERM'].includes(error?.code)) throw error; + fs.rmSync(target, { force: true }); + fs.renameSync(temporary, target); + } + } catch { + } +} + +export function removeState(target) { + try { + fs.rmSync(target, { force: true }); + } catch { + } +} diff --git a/src/tools.js b/src/tools.js index 9cb253c..33dcfb1 100644 --- a/src/tools.js +++ b/src/tools.js @@ -3,6 +3,8 @@ import { defineTool } from '@deepseek-ai/dsh-tools'; +import { selfReportedEvidenceOf } from './reuse.js'; + const MAX_RENDERED_FIELD_CHARS = 16_000; const jsonOutput = { @@ -215,7 +217,7 @@ export function evolverTools(proxyFetch, { diagnostics = async () => null } = {} proxyTool(proxyFetch, { name: 'evolver_asset_reuse_result', description: - 'Report the verified outcome after reusing a fetched asset. This closes the feedback loop, credits its author, and improves future ranking.', + 'Report the verified outcome after reusing a fetched asset. This closes the feedback loop, credits its author, and improves future ranking. The response carries the Hub verdict: eligible, counted and weight.', parameters: { asset_id: { type: 'string', required: true, description: 'The reused asset id.' }, outcome: { type: 'string', required: true, enum: ['success', 'failed', 'mismatched', 'stale', 'unsafe'] }, @@ -238,6 +240,7 @@ export function evolverTools(proxyFetch, { diagnostics = async () => null } = {} reason: args.reason, time_saved_seconds: args.time_saved_seconds, task_id: args.task_id, + evidence: selfReportedEvidenceOf(args.outcome), }, }), }), diff --git a/src/turn-outcomes.js b/src/turn-outcomes.js new file mode 100644 index 0000000..e9ec902 --- /dev/null +++ b/src/turn-outcomes.js @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +const TURN_OUTCOMES = { + completed: 'success', + 'max-tokens': 'failed', + aborted: 'failed', + blocked: 'failed', + error: 'failed', +}; + +export function outcomeOfReason(reasonKind) { + return TURN_OUTCOMES[reasonKind] ?? null; +} diff --git a/src/turn-record.js b/src/turn-record.js new file mode 100644 index 0000000..81a5744 --- /dev/null +++ b/src/turn-record.js @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +import path from 'node:path'; + +import { editedContent, editedPath } from './edited-content.js'; +import { contentHashOf } from './retention-pending.js'; +import { checkCommandOf, verificationFailed } from './verification.js'; + +const TURN_EDITED_MAX_CHARS = 200_000; +const CHECK_COMMAND_MAX_CHARS = 200; +const PATH_KEYS = ['path', 'file_path']; + +function freshTurnRecord() { + return { notices: new Set(), checks: [], commands: [], paths: [], edits: [], editedChars: 0, editedBefore: new Map(), answer: '' }; +} + +export function createTurnRecords() { + const records = new Map(); + return { + of(sessionKey) { + let record = records.get(sessionKey); + if (!record) { + record = freshTurnRecord(); + records.set(sessionKey, record); + } + return record; + }, + take(sessionKey) { + const record = records.get(sessionKey) ?? freshTurnRecord(); + records.delete(sessionKey); + return record; + }, + forget(sessionKey) { + records.delete(sessionKey); + }, + }; +} + +export function noteFileBeforeEdit(record, projectDir, args) { + const edited = editedPath(args); + if (!edited) return; + const file = path.resolve(projectDir, edited); + if (!record.editedBefore.has(file)) record.editedBefore.set(file, contentHashOf(file)); +} + +function noteEditedContent(record, args) { + const kept = editedContent(args).slice(0, TURN_EDITED_MAX_CHARS - record.editedChars); + if (!kept) return; + record.edits.push(kept); + record.editedChars += kept.length; +} + +export function noteToolResult(record, exec, result, { isEdit }) { + const args = exec.arguments ?? {}; + if (typeof args.command === 'string') record.commands.push(args.command); + for (const key of PATH_KEYS) if (typeof args[key] === 'string') record.paths.push(args[key]); + const check = checkCommandOf(exec); + if (check) record.checks.push({ command: check.slice(0, CHECK_COMMAND_MAX_CHARS), failed: verificationFailed(result) }); + if (isEdit && !result?.isError) noteEditedContent(record, args); +} + +export function noteAnswer(record, message) { + const text = (message?.content ?? []) + .filter((block) => block?.type === 'text' && typeof block.text === 'string') + .map((block) => block.text) + .join('') + .trim(); + if (text) record.answer = text; +} diff --git a/src/verification.js b/src/verification.js new file mode 100644 index 0000000..a4d4368 --- /dev/null +++ b/src/verification.js @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 EvoMap + +import { detectSignals } from './signals.js'; + +const VERIFICATION_COMMANDS = [ + /\b(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?(?:test|build|lint|typecheck|type-check|check)\b/, + /\bnode\s+--test\b/, + /\b(?:jest|vitest|mocha|pytest|rspec|phpunit|tsc|eslint|ruff|mypy)\b/, + /\bgo\s+(?:test|build|vet)\b/, + /\bcargo\s+(?:test|build|check|clippy)\b/, + /\bdotnet\s+(?:test|build)\b/, + /\b(?:mvn|gradle|gradlew)\b/, + /\bmake\b/, +]; + +const FAILURE_SIGNALS = new Set(['test_failure', 'log_error']); + +// Summaries of a passing run still name failures by count ("0 failed", +// "failures: 0"); those lines are not evidence of one. +const ZERO_COUNT = /\b0\s+(?:failed|failures?|errors?)\b|\b(?:failed|failures?|errors?)\s*[:=]\s*0\b/i; + +const SHELL_TOOL_NAMES = ['bash']; + +export function isVerificationCommand(command) { + return typeof command === 'string' && VERIFICATION_COMMANDS.some((pattern) => pattern.test(command)); +} + +function textOf(result) { + return (result?.content ?? []) + .filter((block) => block?.type === 'text' && typeof block.text === 'string') + .map((block) => block.text) + .join('\n'); +} + +// dsh's bash reports a failing command piped through `tail` as a success, so +// the output is read too, not only `isError`. +export function verificationFailed(result) { + if (result?.isError) return true; + const evidence = textOf(result) + .split('\n') + .filter((line) => !ZERO_COUNT.test(line)) + .join('\n'); + return detectSignals(evidence).some((signal) => FAILURE_SIGNALS.has(signal)); +} + +export function checkCommandOf(exec) { + if (!SHELL_TOOL_NAMES.includes(exec?.name)) return null; + const command = exec.arguments?.command; + return isVerificationCommand(command) ? command : null; +} diff --git a/src/workspace.js b/src/workspace.js deleted file mode 100644 index 1bed6cf..0000000 --- a/src/workspace.js +++ /dev/null @@ -1,277 +0,0 @@ -// SPDX-License-Identifier: MIT -// Copyright (c) 2026 EvoMap - -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import crypto from 'node:crypto'; -import { spawnSync } from 'node:child_process'; - -const WORKSPACE_ID_PATTERN = /^[a-f0-9]{32,}$/i; -const GIT_TIMEOUT_MS = 5000; - -function isDirectory(candidate) { - if (typeof candidate !== 'string' || candidate.length === 0) return false; - try { - return fs.statSync(candidate).isDirectory(); - } catch { - return false; - } -} - -function realPath(dir) { - try { - return fs.realpathSync(dir); - } catch { - return dir; - } -} - -function isPathInside(parent, candidate) { - const relative = path.relative(parent, candidate); - return relative === '' || (!relative.startsWith(`..${path.sep}`) && relative !== '..' && !path.isAbsolute(relative)); -} - -export function resolveProjectDir() { - for (const fromEnv of [process.env.DSH_PROJECT_DIR, process.env.CLAUDE_PROJECT_DIR]) { - if (isDirectory(fromEnv)) return realPath(fromEnv); - } - return realPath(process.cwd()); -} - -export function sessionDir(candidate, fallback = resolveProjectDir()) { - return isDirectory(candidate) ? realPath(candidate) : fallback; -} - -export function isGitWorkspace(dir) { - try { - const result = spawnSync('git', ['rev-parse', '--is-inside-work-tree'], { - cwd: isDirectory(dir) ? dir : undefined, - shell: false, - timeout: GIT_TIMEOUT_MS, - encoding: 'utf8', - stdio: ['ignore', 'pipe', 'pipe'], - }); - return result.status === 0 && String(result.stdout).trim() === 'true'; - } catch { - return false; - } -} - -export function projectMemoryGraphPath(projectDir) { - return path.join(projectDir, 'memory', 'evolution', 'memory_graph.jsonl'); -} - -function safeProjectMemoryGraph(projectDir) { - if (!isDirectory(projectDir)) return null; - try { - const root = realPath(projectDir); - const memoryDir = path.join(root, 'memory'); - const evolutionDir = path.join(memoryDir, 'evolution'); - const graphPath = path.join(evolutionDir, 'memory_graph.jsonl'); - const memoryStat = fs.lstatSync(memoryDir); - const evolutionStat = fs.lstatSync(evolutionDir); - const graphStat = fs.lstatSync(graphPath); - if (memoryStat.isSymbolicLink() || !memoryStat.isDirectory()) return null; - if (evolutionStat.isSymbolicLink() || !evolutionStat.isDirectory()) return null; - if (graphStat.isSymbolicLink() || !graphStat.isFile()) return null; - return isPathInside(root, fs.realpathSync(graphPath)) ? graphPath : null; - } catch { - return null; - } -} - -function userMemoryGraphPath() { - return path.join(os.homedir(), '.evolver', 'memory', 'evolution', 'memory_graph.jsonl'); -} - -export function findMemoryGraph(projectDir) { - const override = process.env.MEMORY_GRAPH_PATH; - if (typeof override === 'string' && override.length > 0) return override; - - const projectPath = safeProjectMemoryGraph(projectDir); - if (projectPath) return projectPath; - - const userPath = userMemoryGraphPath(); - try { - fs.mkdirSync(path.dirname(userPath), { recursive: true, mode: 0o700 }); - } catch { - } - return userPath; -} - -export function appendMemoryGraph(projectDir, entry) { - const graphPath = findMemoryGraph(projectDir); - let fd; - try { - try { - const existing = fs.lstatSync(graphPath); - if (existing.isSymbolicLink() || !existing.isFile()) return false; - } catch (error) { - if (error?.code !== 'ENOENT') return false; - } - fs.mkdirSync(path.dirname(graphPath), { recursive: true, mode: 0o700 }); - const noFollow = fs.constants.O_NOFOLLOW ?? 0; - const flags = fs.constants.O_WRONLY | fs.constants.O_CREAT | fs.constants.O_APPEND | noFollow; - fd = fs.openSync(graphPath, flags, 0o600); - if (!fs.fstatSync(fd).isFile()) return false; - fs.writeSync(fd, `${JSON.stringify(entry)}\n`); - return true; - } catch { - return false; - } finally { - if (fd !== undefined) { - try { - fs.closeSync(fd); - } catch { - } - } - } -} - -export function captureStatePath(projectDir) { - const key = crypto.createHash('sha256').update(projectDir).digest('hex').slice(0, 16); - return path.join(os.homedir(), '.evolver', 'state', `capture-${key}.json`); -} - -function findRepoRoot(start) { - let current = path.resolve(start); - for (let depth = 0; depth < 256; depth += 1) { - try { - if (fs.existsSync(path.join(current, '.git'))) return current; - } catch { - } - const parent = path.dirname(current); - if (parent === current) return null; - current = parent; - } - return null; -} - -function computeWorkspaceRoot(projectDir) { - const explicit = process.env.OPENCLAW_WORKSPACE; - if (typeof explicit === 'string' && explicit.length > 0) return explicit; - - const repoRoot = findRepoRoot(projectDir); - if (!repoRoot) return projectDir; - - const nested = path.join(repoRoot, 'workspace'); - return isDirectory(nested) ? nested : repoRoot; -} - -function readWorkspaceIdFile(dotEvolverDir, idFile) { - let dirStat; - try { - dirStat = fs.lstatSync(dotEvolverDir); - } catch { - return { ok: false, missing: true }; - } - if (dirStat.isSymbolicLink()) return { ok: false, missing: false }; - - let fileStat; - try { - fileStat = fs.lstatSync(idFile); - } catch { - return { ok: false, missing: true }; - } - if (fileStat.isSymbolicLink() || !fileStat.isFile()) return { ok: false, missing: false }; - - let raw; - try { - raw = fs.readFileSync(idFile, 'utf8'); - } catch { - return { ok: false, missing: false }; - } - const value = raw.trim(); - return WORKSPACE_ID_PATTERN.test(value) ? { ok: true, id: value } : { ok: false, missing: false }; -} - -function createWorkspaceIdFile(dotEvolverDir, idFile) { - try { - if (fs.lstatSync(dotEvolverDir).isSymbolicLink()) return null; - } catch { - } - try { - fs.mkdirSync(dotEvolverDir, { recursive: true, mode: 0o700 }); - } catch { - return null; - } - - const fresh = crypto.randomBytes(16).toString('hex'); - let fd; - try { - const noFollow = fs.constants.O_NOFOLLOW ?? 0; - const flags = fs.constants.O_WRONLY | fs.constants.O_CREAT | fs.constants.O_EXCL | noFollow; - fd = fs.openSync(idFile, flags, 0o600); - fs.writeSync(fd, fresh); - } catch (error) { - if (error && error.code === 'EEXIST') { - const raced = readWorkspaceIdFile(dotEvolverDir, idFile); - return raced.ok ? raced.id : null; - } - return null; - } finally { - if (fd !== undefined) { - try { - fs.closeSync(fd); - } catch { - } - } - } - - try { - fs.chmodSync(idFile, 0o600); - } catch { - } - return fresh; -} - -/** - * Where a workspace's id lives now: beside the capture state, under the user's home, - * keyed by the workspace root. The id is our bookkeeping, not the user's source — writing - * it into their repository made every Evolver user carry an untracked `.evolver/` they - * had to gitignore. - */ -export function workspaceIdPath(workspaceRoot) { - const key = crypto.createHash('sha256').update(workspaceRoot).digest('hex').slice(0, 16); - return path.join(os.homedir(), '.evolver', 'state', `workspace-${key}`); -} - -export function resolveWorkspaceId(projectDir) { - try { - const fromEnv = process.env.EVOLVER_WORKSPACE_ID; - if (typeof fromEnv === 'string' && fromEnv.length > 0) return fromEnv; - - const workspaceRoot = computeWorkspaceRoot(projectDir); - const homeFile = workspaceIdPath(workspaceRoot); - const homeDir = path.dirname(homeFile); - - const stored = readWorkspaceIdFile(homeDir, homeFile); - if (stored.ok) return stored.id; - if (!stored.missing) return null; - - // An id minted by an earlier version still keys that workspace's memory rows. Adopt it - // rather than minting a fresh one, or every existing user's recorded history silently detaches. - const legacyDir = path.join(workspaceRoot, '.evolver'); - const legacy = readWorkspaceIdFile(legacyDir, path.join(legacyDir, 'workspace-id')); - if (legacy.ok) { - adoptWorkspaceId(homeDir, homeFile, legacy.id); - return legacy.id; - } - - return createWorkspaceIdFile(homeDir, homeFile); - } catch { - return null; - } -} - -/** Copy a legacy id to its new home. Best effort: the legacy file still answers if this fails. */ -function adoptWorkspaceId(homeDir, homeFile, id) { - try { - fs.mkdirSync(homeDir, { recursive: true, mode: 0o700 }); - fs.writeFileSync(homeFile, id, { mode: 0o600, flag: 'wx' }); - return true; - } catch { - return false; - } -} diff --git a/test/applied.test.js b/test/applied.test.js new file mode 100644 index 0000000..eca22fb --- /dev/null +++ b/test/applied.test.js @@ -0,0 +1,95 @@ +import assert from 'node:assert/strict'; +import { readFileSync } from 'node:fs'; +import { test } from 'node:test'; + +import { appliedVerdictOf, concreteFeaturesOf, strategyStepsOf } from '../src/applied.js'; + +const STRATEGY = [ + '[Evolution Memory] Fix flaky Redis connection in tests (EvoMap network):', + '1. Set `maxRetriesPerRequest` to null in lib/redis.js', + '2. Run npm test to confirm the suite passes', + '3. Wrap the client in a singleton factory module', + '4. Close the connection in the global teardown hook', + '', + 'Apply it where it fits, then report the outcome with evolver_asset_reuse_result for sha256:abc.', +].join('\n'); + +test('only the numbered steps of an injected strategy are matched', () => { + assert.equal(strategyStepsOf(STRATEGY).length, 4); +}); + +test('backticked names, paths and commands are concrete features', () => { + assert.deepEqual(concreteFeaturesOf('Set `maxRetriesPerRequest` to null in lib/redis.js'), ['maxretriesperrequest', 'lib/redis.js']); + assert.deepEqual(concreteFeaturesOf('Run npm test to confirm'), ['npm test']); +}); + +test('a turn that touched the named file and ran the named command used the strategy', () => { + const verdict = appliedVerdictOf({ + strategyText: STRATEGY, + commands: ['npm test'], + paths: ['/repo/lib/redis.js'], + answer: 'I wrapped the client in a singleton factory module.', + }); + assert.equal(verdict.verdict, 'used'); + assert.equal(verdict.hits, 3); + assert.equal(verdict.steps, 4); +}); + +test('one matching step out of four is partial', () => { + const verdict = appliedVerdictOf({ strategyText: STRATEGY, commands: ['npm test'], paths: [], answer: 'Done.' }); + assert.equal(verdict.verdict, 'partial'); + assert.equal(verdict.ratio, 0.25); +}); + +test('a turn sharing nothing with the strategy left it unused', () => { + const verdict = appliedVerdictOf({ strategyText: STRATEGY, commands: ['ls'], paths: ['/repo/README.md'], answer: 'Renamed the heading.' }); + assert.equal(verdict.verdict, 'unused'); + assert.equal(verdict.hits, 0); +}); + +test('generic words alone do not count as following a prose step', () => { + const prose = '1. Make sure you always check the result before you report the outcome'; + assert.equal(appliedVerdictOf({ strategyText: prose, commands: ['ls'], paths: [], answer: 'make sure always check before report outcome' }).verdict, 'unused'); +}); + +test('a strategy with no steps cannot be judged', () => { + assert.deepEqual(appliedVerdictOf({ strategyText: '', commands: ['npm test'], paths: [], answer: '' }), { verdict: 'unknown', hits: 0, steps: 0, ratio: 0 }); +}); + +const SESSION = JSON.parse(readFileSync(new URL('./fixtures/fair-play-session.json', import.meta.url), 'utf8')); + +function turnOf({ strategyText, tools, answer }) { + const args = tools.map((tool) => tool.arguments); + return { + strategyText, + commands: args.map((input) => input.command).filter((command) => typeof command === 'string'), + paths: args.flatMap((input) => [input.path, input.file_path]).filter((value) => typeof value === 'string'), + edits: tools.filter((tool) => ['write', 'edit'].includes(tool.name)).map((tool) => tool.arguments.content ?? tool.arguments.new_string), + answer, + }; +} + +const BUILD_TURN = turnOf(SESSION.turns[0]); + +test('a strategy carried out in the code the turn wrote is used', () => { + const verdict = appliedVerdictOf(BUILD_TURN); + assert.equal(verdict.verdict, 'used'); + assert.equal(verdict.steps, 4); + assert.ok(verdict.hits >= 3, `hits ${verdict.hits}`); +}); + +test('a prose word that reads like a command does not stop a step from matching on its terms', () => { + const step = '1. On get, move the accessed node to the head of the list to mark it as most recently used'; + const edits = ['moveToHead(node) { if (node === this.head) return } // most recently used']; + assert.equal(appliedVerdictOf({ strategyText: step, commands: [], paths: [], edits, answer: '' }).verdict, 'used'); +}); + +test('work that shares nothing with the strategy is unused', () => { + const verdict = appliedVerdictOf({ ...BUILD_TURN, edits: ['export const greeting = "hello";\n'], commands: ['ls'], paths: ['/tmp/x/greeting.js'], answer: '好了。' }); + assert.equal(verdict.verdict, 'unused'); +}); + +test('a turn that left only an answer, or a strategy without steps, cannot be judged', () => { + assert.equal(appliedVerdictOf(turnOf(SESSION.turns[1])).verdict, 'unknown'); + assert.equal(appliedVerdictOf({ strategyText: 'no numbered steps here', commands: ['npm test'], paths: [], edits: [], answer: '' }).verdict, 'unknown'); +}); diff --git a/test/apply.test.js b/test/apply.test.js index 58bc8ec..d9a3a47 100644 --- a/test/apply.test.js +++ b/test/apply.test.js @@ -1,110 +1,14 @@ import assert from 'node:assert/strict'; -import { execFileSync } from 'node:child_process'; -import { chmodSync, existsSync, mkdtempSync, readFileSync, realpathSync, writeFileSync } from 'node:fs'; +import { mkdtempSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; -import { delimiter, join } from 'node:path'; +import { join } from 'node:path'; import { createServer } from 'node:http'; -import { after, test } from 'node:test'; +import { test } from 'node:test'; -import { outcomeOfReason } from '../src/capture.js'; import { Config, apply, inject, name } from '../src/index.js'; - -const missingClaimFile = join(mkdtempSync(join(tmpdir(), 'evolver-claim-')), 'missing'); -process.env.EVOLVER_CLAIM_URL_PATH = missingClaimFile; -const sharedStateDir = mkdtempSync(join(tmpdir(), 'evolver-notice-state-')); -process.env.EVOLVER_SESSION_STATE_DIR = sharedStateDir; -const originalPath = process.env.PATH; - -function evolverOnPath(version) { - const dir = mkdtempSync(join(tmpdir(), 'evolver-bin-')); - const bin = join(dir, 'evolver'); - writeFileSync(bin, `#!/bin/sh\necho ${version}\n`); - chmodSync(bin, 0o755); - process.env.PATH = `${dir}${delimiter}${originalPath}`; -} - -evolverOnPath('2.0.39'); -after(() => { - delete process.env.EVOLVER_CLAIM_URL_PATH; - delete process.env.EVOLVER_SESSION_STATE_DIR; - process.env.PATH = originalPath; -}); - -function fakeContext() { - const registered = { tools: [], skills: [], commands: [] }; - const listeners = new Map(); - const ctx = { - tools: { register: (definition) => registered.tools.push(definition) }, - skills: { registerProvider: (create) => registered.skills.push(create({})) }, - commands: { register: (definition) => registered.commands.push(definition) }, - on: (event, listener) => { - listeners.set(event, listener); - return () => listeners.delete(event); - }, - inject: (_deps, callback) => callback(ctx), - }; - return { ctx, registered, listeners }; -} - -function fakeAgent({ id = 'agent-test', sessionId = id, cwd } = {}) { - const injected = []; - const session = { id: sessionId, header: cwd ? { cwd } : {} }; - return { - agent: { id, session, inject: (message) => injected.push(message), followup: () => {} }, - injected, - }; -} - -function userMessage(text) { - return { content: [{ type: 'text', text }], source: { kind: 'user' } }; -} - -async function primedBy(listeners, agent, prompt = 'add a retry to the uploader', turn = 1, step = 1) { - const claimed = [userMessage(prompt)]; - const decision = await listeners.get('agent/pre-step')( - { agent, messages: claimed, turn, step, signal: new AbortController().signal }, - async () => ({ kind: 'enter', messages: claimed }), - ); - assert.equal(decision.messages[0], claimed[0]); - return decision.messages.slice(1); -} - -// A watchdog against a hang, not an assertion: each of these returns the moment -// its condition holds, so the budget only has to outlast the slowest runner. Two -// seconds did not -- the correction case takes ~2.0s on an idle laptop, and the -// dsh 0.1.5-rc.3 matrix leg failed on it while every other leg passed. -async function untilCount(items, predicate, count, deadlineMs = 15_000) { - const start = Date.now(); - while (items.filter(predicate).length < count) { - if (Date.now() - start > deadlineMs) throw new Error(`only ${items.filter(predicate).length} of ${count} expected entries arrived`); - await new Promise((resolve) => { setTimeout(resolve, 10); }); - } - return items.filter(predicate); -} - -async function untilRequest(requests, path, deadlineMs = 15_000) { - const start = Date.now(); - while (!requests.some((request) => request.path === path)) { - if (Date.now() - start > deadlineMs) throw new Error(`no ${path} request arrived`); - await new Promise((resolve) => { setTimeout(resolve, 10); }); - } - return requests; -} - -async function untilInjected(injected, count, deadlineMs = 15_000) { - const start = Date.now(); - while (injected.length < count) { - if (Date.now() - start > deadlineMs) throw new Error(`only ${injected.length} of ${count} messages were injected`); - await new Promise((resolve) => { setTimeout(resolve, 10); }); - } - return injected; -} - -function gitDirectory(prefix = 'evolver-') { - const directory = realpathSync(mkdtempSync(join(tmpdir(), prefix))); - execFileSync('git', ['init', '--quiet'], { cwd: directory }); - return directory; -} +import { + evolverOnPath, fakeAgent, fakeContext, freshHome, missingClaimFile, primedBy, projectDirectory, sharedStateDir, startProxy, untilCount, untilInjected, +} from './support/host.js'; test('plugin metadata matches the Cordis v4 contract', () => { assert.equal(name, 'evolver'); @@ -122,13 +26,8 @@ test('configuration applies defaults and rejects invalid values', () => { configured.assetPrimeWaitMs < configured.assetPrimeTimeoutMs, 'a step must give up waiting before the Proxy call it is waiting on does', ); - assert.ok(configured.captureLockWaitMs > configured.captureLockStaleMs); assert.throws(() => Config({ proxyPort: 70_000 }), /proxyPort/); assert.throws(() => Config({ editToolNames: 'write' }), /editToolNames/); - assert.throws( - () => apply(fakeContext().ctx, Config({ captureLockStaleMs: 100, captureLockWaitMs: 100 })), - /captureLockWaitMs/, - ); assert.throws(() => apply(fakeContext().ctx, { projectDir: join(tmpdir(), 'definitely-missing-evolver-dir') }), /not a directory/); }); @@ -167,10 +66,10 @@ test('apply registers every surface and lifecycle listener', () => { assert.ok(registered.commands.every((command) => command.description.length > 0)); assert.deepEqual([...listeners.keys()].sort(), [ 'agent/pre-step', + 'agent/session-start', 'session/disposed', 'session/event', - 'session/flush', - 'tools/result', + 'tools/execute', ]); }); @@ -188,7 +87,7 @@ test('command invocation preserves arguments without Claude placeholders', () => }); test('every turn looks the Hub up with its own prompt, without repeating assets', async () => { - const projectDir = gitDirectory('evolver-prime-'); + const projectDir = projectDirectory('evolver-prime-'); const requests = []; const server = createServer((request, response) => { let body = ''; @@ -236,7 +135,7 @@ test('every turn looks the Hub up with its own prompt, without repeating assets' }); test('claim guidance is opt-in', async () => { - const projectDir = gitDirectory('evolver-claim-opt-in-'); + const projectDir = projectDirectory('evolver-claim-opt-in-'); const claimDir = mkdtempSync(join(tmpdir(), 'evolver-claim-enabled-')); const claimPath = join(claimDir, 'claim_url'); const stateDir = mkdtempSync(join(tmpdir(), 'evolver-claim-state-')); @@ -257,26 +156,99 @@ test('claim guidance is opt-in', async () => { } }); -test('an Evolver older than 2.0.39 is asked to upgrade once a day', { skip: process.platform === 'win32' }, async () => { - const projectDir = gitDirectory('evolver-old-engine-'); - const stateDir = mkdtempSync(join(tmpdir(), 'evolver-upgrade-state-')); - process.env.EVOLVER_SESSION_STATE_DIR = stateDir; - evolverOnPath('2.0.38'); +function startSession(listeners, id, projectDir) { + listeners.get('agent/session-start')({ agent: fakeAgent({ id, cwd: projectDir }).agent, source: 'startup' }); +} + +async function setupNotice(listeners, projectDir) { + startSession(listeners, 'setup-session', projectDir); + const deadline = Date.now() + 15_000; + for (let turn = 1; Date.now() < deadline; turn += 1) { + const notices = await primedBy(listeners, fakeAgent({ id: `setup-${turn}`, cwd: projectDir }).agent, 'add a retry to the uploader', turn); + if (notices.length > 0) return notices; + await new Promise((resolve) => { setTimeout(resolve, 20); }); + } + throw new Error('no setup notice arrived'); +} + +const unixOnly = { skip: process.platform === 'win32' }; +test('an Evolver older than 2.0.39 is upgraded to the latest release and its Proxy restarted', unixOnly, async () => { + const projectDir = projectDirectory('evolver-old-engine-'); + const calls = evolverOnPath('2.0.38', { npmInstalls: '2.0.42' }); try { const { ctx, listeners } = fakeContext(); apply(ctx, Config({ projectDir, assetPrimeEnabled: false })); - const [notice] = await primedBy(listeners, fakeAgent({ id: 'old-engine-a', cwd: projectDir }).agent); - assert.match(notice.content[0].text, /Evolver 2\.0\.38 is installed.*npm install -g @evomap\/evolver@latest/s); - assert.deepEqual(await primedBy(listeners, fakeAgent({ id: 'old-engine-b', cwd: projectDir }).agent), []); + const [notice] = await setupNotice(listeners, projectDir); + assert.equal(notice.content[0].text, '[Evolver] Evolver 2.0.38 is older than the 2.0.39 network recall needs, so this plugin upgraded it to 2.0.42 and restarted its Proxy.'); + assert.deepEqual(calls(), ['evolver --version', 'npm install -g @evomap/evolver@latest', 'evolver --version', 'evolver lifecycle restart']); + assert.deepEqual(await primedBy(listeners, fakeAgent({ id: 'setup-again', cwd: projectDir }).agent, 'add a retry to the uploader', 99), []); + startSession(listeners, 'setup-session-again', projectDir); + assert.equal(calls().length, 4, 'setup runs once per dsh process'); + } finally { + evolverOnPath('2.0.39'); + } +}); + +test('a missing Evolver is installed and its Proxy started', unixOnly, async () => { + const projectDir = projectDirectory('evolver-no-engine-'); + const calls = evolverOnPath(null, { npmInstalls: '2.0.42' }); + try { + const { ctx, listeners } = fakeContext(); + apply(ctx, Config({ projectDir, assetPrimeEnabled: false })); + const [notice] = await setupNotice(listeners, projectDir); + assert.equal(notice.content[0].text, '[Evolver] Evolver was not installed, so this plugin installed 2.0.42 and started its Proxy.'); + assert.deepEqual(calls(), ['evolver --version', 'npm install -g @evomap/evolver@latest', 'evolver --version', 'evolver lifecycle start']); + } finally { + evolverOnPath('2.0.39'); + } +}); + +test('an install that fails says how to do it by hand', unixOnly, async () => { + const projectDir = projectDirectory('evolver-install-fails-'); + evolverOnPath('2.0.38'); + try { + const { ctx, listeners } = fakeContext(); + apply(ctx, Config({ projectDir, assetPrimeEnabled: false })); + const [notice] = await setupNotice(listeners, projectDir); + assert.match(notice.content[0].text, /Evolver 2\.0\.38 is older than 2\.0\.39 and installing the latest release failed \(EACCES: permission denied\)\. Run `npm install -g @evomap\/evolver@latest`/); } finally { - process.env.EVOLVER_SESSION_STATE_DIR = sharedStateDir; + evolverOnPath('2.0.39'); + } +}); + +test('a current Evolver whose Proxy is down is started; one already serving is left alone', unixOnly, async () => { + const projectDir = projectDirectory('evolver-start-'); + const calls = evolverOnPath('2.0.42'); + const restoreHome = freshHome('evolver-start-home-'); + try { + const down = fakeContext(); + apply(down.ctx, Config({ projectDir, assetPrimeEnabled: false, proxyPort: 1 })); + const [notice] = await setupNotice(down.listeners, projectDir); + assert.equal(notice.content[0].text, '[Evolver] The Evolver Proxy was not running, so this plugin started it (Evolver 2.0.42).'); + assert.deepEqual(calls(), ['evolver --version', 'evolver lifecycle start']); + + const proxy = await startProxy(() => ({ status: 'running' })); + const served = evolverOnPath('2.0.42'); + try { + const up = fakeContext(); + apply(up.ctx, Config({ projectDir, assetPrimeEnabled: false, proxyPort: proxy.port })); + startSession(up.listeners, 'served', projectDir); + await untilCount(proxy.requests, (request) => request.path === '/proxy/status', 1); + await new Promise((resolve) => { setTimeout(resolve, 100); }); + assert.deepEqual(served(), ['evolver --version']); + assert.deepEqual(await primedBy(up.listeners, fakeAgent({ id: 'served-step', cwd: projectDir }).agent), []); + } finally { + proxy.close(); + } + } finally { + restoreHome(); evolverOnPath('2.0.39'); } }); test('a search slower than the wait budget injects itself instead of holding the step', async () => { - const projectDir = gitDirectory('evolver-slow-prime-'); + const projectDir = projectDirectory('evolver-slow-prime-'); const server = createServer((request, response) => { request.on('data', () => {}); request.on('end', () => { @@ -309,168 +281,25 @@ test('a search slower than the wait budget injects itself instead of holding the } }); -test('a non-git workspace is primed like any other, and lectured about nothing', async () => { - const projectDir = realpathSync(mkdtempSync(join(tmpdir(), 'evolver-nongit-'))); - const graph = join(projectDir, 'graph.jsonl'); - writeFileSync( - graph, - `${JSON.stringify({ - timestamp: new Date().toISOString(), - signals: ['perf_bottleneck'], - outcome: { status: 'success', score: 0.9, note: 'foreign memory' }, - cwd: projectDir, - })}\n`, - ); - process.env.MEMORY_GRAPH_PATH = graph; - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, assetPrimeEnabled: false })); - const primed = await primedBy(listeners, fakeAgent({ cwd: projectDir }).agent); - - assert.deepEqual(primed, [], 'git is what turn capture needs; the model is not the one who can fix that'); - } finally { - delete process.env.MEMORY_GRAPH_PATH; - } -}); - -test('an edit carrying a signal nudges the agent once', () => { +test('an edit carrying a signal nudges the agent once', async () => { const { ctx, listeners } = fakeContext(); apply(ctx, { projectDir: mkdtempSync(join(tmpdir(), 'evolver-')) }); const { agent, injected } = fakeAgent(); - const onResult = listeners.get('tools/result'); - onResult({ name: 'write', agent, arguments: { path: '/a.ts', content: 'the deploy failed again' } }, { isError: false }); - onResult({ name: 'write', agent, arguments: { path: '/a.ts', content: 'the deploy failed again' } }, { isError: false }); - onResult({ name: 'write', agent, arguments: { path: '/b.ts', content: 'all good' } }, { isError: false }); - onResult({ name: 'read', agent, arguments: { path: '/c.ts', content: 'the deploy failed' } }, { isError: false }); - onResult({ name: 'write', agent, arguments: { path: '/d.ts', content: 'the deploy failed' } }, { isError: true }); + const run = (name, args, result) => listeners.get('tools/execute')({ name, agent, arguments: args }, async () => result); + await run('write', { path: '/a.ts', content: 'the deploy failed again' }, { isError: false }); + await run('write', { path: '/a.ts', content: 'the deploy failed again' }, { isError: false }); + await run('write', { path: '/b.ts', content: 'all good' }, { isError: false }); + await run('read', { path: '/c.ts', content: 'the deploy failed' }, { isError: false }); + await run('write', { path: '/d.ts', content: 'the deploy failed' }, { isError: true }); assert.equal(injected.length, 1); - assert.match(injected[0].content[0].text, /deployment_issue.*\/a\.ts/); + assert.equal(injected[0].content[0].text, '[Evolution Signal] Detected: [deployment_issue, log_error] in /a.ts.'); assert.equal(injected[0].source.kind, 'plugin:evolver'); assert.equal(injected[0].source.form, 'notice'); assert.ok(injected[0].source.summary.length > 0 && injected[0].source.summary.length <= 120); }); -test('edit signals and turn capture share the session key seam', async () => { - const projectDir = gitDirectory('evolver-signal-session-'); - writeFileSync(join(projectDir, 'app.js'), 'export const deploy = "failed";\n'); - const graph = join(mkdtempSync(join(tmpdir(), 'evolver-signal-graph-')), 'graph.jsonl'); - process.env.MEMORY_GRAPH_PATH = graph; - - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir })); - const { agent } = fakeAgent({ id: 'agent-fixture-id', sessionId: 'session-shared', cwd: projectDir }); - listeners.get('tools/result')( - { name: 'write', agent, arguments: { path: join(projectDir, 'app.js'), content: 'the deploy failed' } }, - { isError: false }, - ); - listeners.get('session/event')(agent.session, { - type: 'turn/end', - data: { turn: 1, reason: { kind: 'completed' } }, - }); - await listeners.get('session/flush')(agent.session); - - delete process.env.MEMORY_GRAPH_PATH; - const [recorded] = readFileSync(graph, 'utf8').trim().split('\n').map((line) => JSON.parse(line)); - assert.equal(recorded.session_id, 'session-shared'); - assert.ok(recorded.signals.includes('deployment_issue')); -}); - -test('an injected strategy is reported back when the turn ends', async () => { - const projectDir = gitDirectory('evolver-reuse-'); - const requests = []; - const server = createServer((request, response) => { - let body = ''; - request.on('data', (chunk) => { body += chunk; }); - request.on('end', () => { - const parsed = JSON.parse(body); - requests.push({ path: request.url, body: parsed }); - response.setHeader('Content-Type', 'application/json'); - response.end(JSON.stringify( - request.url === '/asset/search' - ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:used', has_strategy: true, similarity: 0.9 }] } - : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:used', strategy: ['Reuse this.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } - : { ok: true }, - )); - }); - }); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-reuse-home-')); - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, proxyPort: server.address().port })); - const { agent } = fakeAgent({ sessionId: 'session-reuse', cwd: projectDir }); - const primed = await primedBy(listeners, agent); - assert.match(primed.at(-1).content[0].text, /evolver_asset_reuse_result for sha256:used\./); - - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); - await untilRequest(requests, '/asset/reuse-result'); - - const [report] = requests.filter((request) => request.path === '/asset/reuse-result'); - assert.equal(report.body.asset_id, 'sha256:used'); - assert.equal(report.body.outcome, 'success'); - assert.equal(report.body.task_id, 'dsh:session-reuse:1', 'the Hub rejects a report with no dedupe key, and hashes this one with the asset to recognise a retry'); - assert.match(report.body.reason, /not confirmed as applied/); - } finally { - process.env.HOME = home; - server.close(); - } -}); - -test('a strategy the model reported itself is not reported again', async () => { - const projectDir = gitDirectory('evolver-reuse-own-'); - const requests = []; - const server = createServer((request, response) => { - let body = ''; - request.on('data', (chunk) => { body += chunk; }); - request.on('end', () => { - requests.push({ path: request.url, body: JSON.parse(body) }); - response.setHeader('Content-Type', 'application/json'); - response.end(JSON.stringify( - request.url === '/asset/search' - ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:owned', has_strategy: true, similarity: 0.9 }] } - : { assets: [{ asset_id: 'sha256:owned', strategy: ['Reuse this.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] }, - )); - }); - }); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-reuse-own-home-')); - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, proxyPort: server.address().port })); - const { agent } = fakeAgent({ sessionId: 'session-owned', cwd: projectDir }); - await primedBy(listeners, agent); - - listeners.get('tools/result')( - { name: 'evolver_asset_reuse_result', agent, arguments: { asset_id: 'sha256:owned', outcome: 'success' } }, - { isError: false }, - ); - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); - await new Promise((resolve) => { setTimeout(resolve, 150); }); - - assert.deepEqual(requests.filter((request) => request.path === '/asset/reuse-result'), []); - } finally { - process.env.HOME = home; - server.close(); - } -}); - -test('every live turn ending is a capturable outcome, but a synthesized one is not', () => { - assert.equal(outcomeOfReason('completed').status, 'success'); - assert.equal(outcomeOfReason('error').status, 'failed'); - assert.equal(outcomeOfReason('aborted').status, 'failed'); - assert.equal(outcomeOfReason('blocked').status, 'failed'); - assert.equal(outcomeOfReason('max-tokens').status, 'failed'); - assert.equal(outcomeOfReason('interrupted'), null); -}); - test('a fetched asset renders as reusable prose, not the raw envelope', async () => { const { evolverTools } = await import('../src/tools.js'); const envelope = { @@ -499,317 +328,3 @@ test('a fetched asset renders as reusable prose, not the raw envelope', async () assert.doesNotMatch(text, /signals_match|source_node_id|gdi_score/); }); -test('a strategy that arrived after its own turn ended is still reported, under that turn', async () => { - const projectDir = gitDirectory('evolver-late-report-'); - const requests = []; - const server = createServer((request, response) => { - let body = ''; - request.on('data', (chunk) => { body += chunk; }); - request.on('end', () => { - const parsed = JSON.parse(body); - requests.push({ path: request.url, body: parsed }); - const payload = request.url === '/asset/search' - ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:late', has_strategy: true, similarity: 0.9 }] } - : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:late', strategy: ['Arrived after the turn.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } - : { ok: true }; - const delay = request.url === '/asset/reuse-result' ? 0 : 150; - setTimeout(() => { - response.setHeader('Content-Type', 'application/json'); - response.end(JSON.stringify(payload)); - }, delay); - }); - }); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-late-report-home-')); - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, proxyPort: server.address().port, assetPrimeWaitMs: 20 })); - const { agent, injected } = fakeAgent({ sessionId: 'session-late', cwd: projectDir }); - - assert.deepEqual(await primedBy(listeners, agent, 'add a retry to the uploader', 1, 1), []); - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); - - await untilInjected(injected, 1); - assert.match(injected[0].content[0].text, /evolver_asset_reuse_result for sha256:late\./); - assert.deepEqual(requests.filter((request) => request.path === '/asset/reuse-result'), []); - - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 2, reason: { kind: 'completed' } } }); - await untilRequest(requests, '/asset/reuse-result'); - - const [report] = requests.filter((request) => request.path === '/asset/reuse-result'); - assert.equal(report.body.asset_id, 'sha256:late'); - assert.equal(report.body.task_id, 'dsh:session-late:1', 'a late strategy is reported under the turn it was selected for, not the one that ended'); - assert.match(report.body.reason, /into dsh turn 1/); - } finally { - process.env.HOME = home; - server.close(); - } -}); - -function reuseServer(requests, strategy = ['Reuse this.', 'Then the next step.', 'Then the next step.', 'Then the next step.'], assetId = 'sha256:kept') { - return createServer((request, response) => { - let body = ''; - request.on('data', (chunk) => { body += chunk; }); - request.on('end', () => { - requests.push({ path: request.url, body: JSON.parse(body) }); - response.setHeader('Content-Type', 'application/json'); - response.end(JSON.stringify( - request.url === '/asset/search' - ? { results: [{ asset_type: 'Gene', asset_id: assetId, has_strategy: true, similarity: 0.9 }] } - : request.url === '/asset/fetch' - ? { assets: [{ asset_id: assetId, strategy }] } - : { ok: true }, - )); - }); - }); -} - -test('an injected asset is reported from disk even when the process that injected it is gone', async () => { - const projectDir = gitDirectory('evolver-durable-'); - const requests = []; - const server = reuseServer(requests); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-durable-home-')); - - try { - const config = Config({ projectDir, proxyPort: server.address().port }); - const first = fakeContext(); - apply(first.ctx, config); - const { agent } = fakeAgent({ sessionId: 'session-durable', cwd: projectDir }); - assert.match((await primedBy(first.listeners, agent)).at(-1).content[0].text, /sha256:kept/); - - const restarted = fakeContext(); - apply(restarted.ctx, config); - restarted.listeners.get('session/event')( - { id: 'session-durable', header: { cwd: projectDir } }, - { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }, - ); - await untilRequest(requests, '/asset/reuse-result'); - - const [report] = requests.filter((request) => request.path === '/asset/reuse-result'); - assert.equal(report.body.asset_id, 'sha256:kept'); - assert.equal(report.body.outcome, 'success'); - } finally { - process.env.HOME = home; - server.close(); - } -}); - -test('a prompt that says the answer did not hold revises the verdict once, and only once', async () => { - const projectDir = gitDirectory('evolver-correct-'); - const requests = []; - const server = reuseServer(requests); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-correct-home-')); - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, proxyPort: server.address().port })); - const { agent } = fakeAgent({ sessionId: 'session-correct', cwd: projectDir }); - - await primedBy(listeners, agent, 'add a retry to the uploader', 1, 1); - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); - await untilRequest(requests, '/asset/reuse-result'); - const reuseResults = () => requests.filter((request) => request.path === '/asset/reuse-result'); - assert.equal(reuseResults()[0].body.outcome, 'success'); - - await primedBy(listeners, agent, '还是不行,报一样的错', 2, 1); - const isFailedReport = (request) => request.path === '/asset/reuse-result' && request.body?.outcome === 'failed'; - await untilCount(requests, isFailedReport, 1); - - const failed = requests.filter(isFailedReport); - assert.equal(failed.length, 1, 'the correction is sent exactly once'); - assert.equal(failed[0].body.asset_id, 'sha256:kept'); - assert.notEqual(failed[0].body.task_id, reuseResults()[0].body.task_id, 'sharing the key with the verdict it revises would have the Hub dismiss it as a retry'); - assert.match(failed[0].body.task_id, /:correction$/); - assert.match(failed[0].body.reason, /read as a correction/); - - await primedBy(listeners, agent, '还是不行啊', 3, 1); - await new Promise((resolve) => setTimeout(resolve, 300)); - assert.equal( - requests.filter(isFailedReport).length, - 1, - 'a second complaint does not send a second negative', - ); - } finally { - process.env.HOME = home; - server.close(); - } -}); - -test('an ordinary follow-up prompt leaves the earlier verdict alone', async () => { - const projectDir = gitDirectory('evolver-nocorrect-'); - const requests = []; - const server = reuseServer(requests); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-nocorrect-home-')); - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, proxyPort: server.address().port })); - const { agent } = fakeAgent({ sessionId: 'session-nocorrect', cwd: projectDir }); - - await primedBy(listeners, agent, 'add a retry to the uploader', 1, 1); - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); - await untilRequest(requests, '/asset/reuse-result'); - - await primedBy(listeners, agent, '这样行不行?再帮我加一个功能', 2, 1); - await new Promise((resolve) => setTimeout(resolve, 150)); - - const outcomes = requests.filter((request) => request.path === '/asset/reuse-result').map((request) => request.body.outcome); - assert.deepEqual(outcomes, ['success']); - } finally { - process.env.HOME = home; - server.close(); - } -}); - -test('a verdict the Hub has no route for still reaches the local ledger', async () => { - const projectDir = gitDirectory('evolver-localledger-'); - const requests = []; - const server = createServer((request, response) => { - let body = ''; - request.on('data', (chunk) => { body += chunk; }); - request.on('end', () => { - requests.push({ path: request.url, body: JSON.parse(body) }); - response.setHeader('Content-Type', 'application/json'); - response.end(JSON.stringify( - request.url === '/asset/search' - ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:unlanded', has_strategy: true, similarity: 0.9 }] } - : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:unlanded', strategy: ['Try it.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } - : { recorded: false, reason: 'hub 404' }, - )); - }); - }); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-localledger-home-')); - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, proxyPort: server.address().port })); - const { agent } = fakeAgent({ sessionId: 'session-local', cwd: projectDir }); - - await primedBy(listeners, agent, 'add a retry to the uploader', 1, 1); - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); - await untilRequest(requests, '/asset/reuse-result'); - await new Promise((resolve) => { setTimeout(resolve, 300); }); - - const log = join(process.env.HOME, '.evomap', 'evolution', 'root_events.jsonl'); - assert.ok(existsSync(log), 'the root event log is written even though the Hub has no route'); - const events = readFileSync(log, 'utf8').trim().split('\n').map((line) => JSON.parse(line)); - const hits = events.filter((event) => event.type === 'value.reuse_hit'); - assert.equal(hits.length, 1); - assert.equal(hits[0].payload.assetId, 'sha256:unlanded'); - assert.equal(hits[0].payload.cycleId, 'dsh:session-local'); - - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 2, reason: { kind: 'completed' } } }); - await new Promise((resolve) => { setTimeout(resolve, 300); }); - assert.equal( - readFileSync(log, 'utf8').trim().split('\n').filter((line) => line.includes('value.reuse_hit')).length, - 1, - 'a verdict that landed locally is not counted twice', - ); - } finally { - process.env.HOME = home; - server.close(); - } -}); -test("a self-report the Hub refused does not retire the plugin's own", async () => { - const projectDir = gitDirectory('evolver-selfreport-404-'); - const requests = []; - const server = createServer((request, response) => { - let body = ''; - request.on('data', (chunk) => { body += chunk; }); - request.on('end', () => { - requests.push({ path: request.url, body: JSON.parse(body) }); - response.setHeader('Content-Type', 'application/json'); - response.end(JSON.stringify( - request.url === '/asset/search' - ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:refused', has_strategy: true, similarity: 0.9 }] } - : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:refused', strategy: ['Reuse this.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } - : { ok: true }, - )); - }); - }); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-selfreport-404-home-')); - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, proxyPort: server.address().port })); - const { agent } = fakeAgent({ sessionId: 'session-refused', cwd: projectDir }); - await primedBy(listeners, agent); - - listeners.get('tools/result')( - { name: 'evolver_asset_reuse_result', agent, arguments: { asset_id: 'sha256:refused', outcome: 'mismatched' } }, - { isError: false, value: { recorded: false, reason: 'hub 404' } }, - ); - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); - await untilRequest(requests, '/asset/reuse-result'); - - const [report] = requests.filter((request) => request.path === '/asset/reuse-result'); - assert.equal(report.body.asset_id, 'sha256:refused'); - assert.match(report.body.reason, /not confirmed as applied/); - } finally { - process.env.HOME = home; - server.close(); - } -}); - -test('a correction writes the negative half of the signal, not another hit', async () => { - const projectDir = gitDirectory('evolver-negative-'); - const requests = []; - const server = createServer((request, response) => { - let body = ''; - request.on('data', (chunk) => { body += chunk; }); - request.on('end', () => { - requests.push({ path: request.url, body: JSON.parse(body) }); - response.setHeader('Content-Type', 'application/json'); - response.end(JSON.stringify( - request.url === '/asset/search' - ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:wrong', has_strategy: true, similarity: 0.9 }] } - : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:wrong', strategy: ['Did not hold.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } - : { recorded: false, reason: 'hub 404' }, - )); - }); - }); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-negative-home-')); - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, proxyPort: server.address().port })); - const { agent } = fakeAgent({ sessionId: 'session-negative', cwd: projectDir }); - - await primedBy(listeners, agent, 'add a retry to the uploader', 1, 1); - listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); - await untilRequest(requests, '/asset/reuse-result'); - await new Promise((resolve) => { setTimeout(resolve, 300); }); - - await primedBy(listeners, agent, '还是不行,报一样的错', 2, 1); - await new Promise((resolve) => { setTimeout(resolve, 400); }); - - const log = join(process.env.HOME, '.evomap', 'evolution', 'root_events.jsonl'); - const events = readFileSync(log, 'utf8').trim().split('\n').map((line) => JSON.parse(line)); - const negatives = events.filter((event) => event.type === 'value.reuse_outcome'); - assert.equal(negatives.length, 1); - assert.equal(negatives[0].payload.outcome, 'failed'); - assert.equal(negatives[0].payload.assetId, 'sha256:wrong'); - assert.equal(events.filter((event) => event.type === 'value.reuse_hit').length, 1); - } finally { - process.env.HOME = home; - server.close(); - } -}); diff --git a/test/capture-coordinator.test.js b/test/capture-coordinator.test.js deleted file mode 100644 index 7637ab8..0000000 --- a/test/capture-coordinator.test.js +++ /dev/null @@ -1,65 +0,0 @@ -import assert from 'node:assert/strict'; -import { test } from 'node:test'; - -import { createCaptureCoordinator } from '../src/capture-coordinator.js'; - -function deferred() { - let resolve; - const promise = new Promise((done) => { - resolve = done; - }); - return { promise, resolve }; -} - -test('snapshots immediately, commits in session order, and flush waits', async () => { - const first = deferred(); - const order = []; - const coordinator = createCaptureCoordinator({ - prepare: async ({ turn }) => { - order.push(`snapshot:${turn}`); - return { turn }; - }, - commit: async ({ turn }) => { - order.push(`commit-start:${turn}`); - if (turn === 1) await first.promise; - order.push(`commit-end:${turn}`); - }, - }); - - coordinator.schedule({ sessionId: 'session-one', projectDir: '/repo', turn: 1 }); - coordinator.schedule({ sessionId: 'session-one', projectDir: '/repo', turn: 2 }); - await new Promise((resolve) => setImmediate(resolve)); - assert.deepEqual(order, ['snapshot:1', 'snapshot:2', 'commit-start:1']); - - first.resolve(); - await coordinator.flush('session-one', '/repo'); - assert.deepEqual(order, [ - 'snapshot:1', - 'snapshot:2', - 'commit-start:1', - 'commit-end:1', - 'commit-start:2', - 'commit-end:2', - ]); -}); - -test('captures for different sessions may settle independently', async () => { - const blocked = deferred(); - const order = []; - const coordinator = createCaptureCoordinator({ - prepare: async (options) => options, - commit: async ({ sessionId }) => { - if (sessionId === 'one') await blocked.promise; - order.push(sessionId); - }, - }); - - coordinator.schedule({ sessionId: 'one', projectDir: '/repo-a' }); - coordinator.schedule({ sessionId: 'two', projectDir: '/repo-b' }); - await coordinator.flush('two', '/repo-b'); - assert.deepEqual(order, ['two']); - - blocked.resolve(); - await coordinator.flushAll(); - assert.deepEqual(order, ['two', 'one']); -}); diff --git a/test/capture.test.js b/test/capture.test.js deleted file mode 100644 index 1c19ee7..0000000 --- a/test/capture.test.js +++ /dev/null @@ -1,360 +0,0 @@ -import assert from 'node:assert/strict'; -import { execFileSync } from 'node:child_process'; -import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, symlinkSync, utimesSync, writeFileSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { after, beforeEach, test } from 'node:test'; - -import { captureOutcome, collectDiff, forgetCaptures, normalizeHubUrl, parseStat, summarize } from '../src/capture.js'; -import { detectSignalsInDiff } from '../src/signals.js'; -import { captureStatePath, findMemoryGraph } from '../src/workspace.js'; - -const logDir = mkdtempSync(join(tmpdir(), 'evolver-log-')); -process.env.EVOLVER_HOOK_LOG_DIR = logDir; - -function repo() { - const dir = realpathSync(mkdtempSync(join(tmpdir(), 'evolver-repo-'))); - const run = (...args) => execFileSync('git', args, { cwd: dir, stdio: 'ignore' }); - run('init', '--quiet'); - run('config', 'user.email', 'test@example.com'); - run('config', 'user.name', 'Test'); - writeFileSync(join(dir, 'app.js'), 'export const rate = 1;\n'); - run('add', '.'); - run('commit', '--quiet', '-m', 'first'); - currentRepo = dir; - forgetCaptures(dir); - return { dir, run }; -} - -function graphFor(_dir) { - const path = join(mkdtempSync(join(tmpdir(), 'evolver-graph-')), 'graph.jsonl'); - process.env.MEMORY_GRAPH_PATH = path; - return path; -} - -function entries(graphPath) { - try { - return readFileSync(graphPath, 'utf8') - .split('\n') - .filter((line) => line.trim()) - .map((line) => JSON.parse(line)); - } catch { - return []; - } -} - -let currentRepo = null; - -beforeEach(() => { - if (currentRepo) forgetCaptures(currentRepo); -}); - -after(() => { - delete process.env.MEMORY_GRAPH_PATH; - rmSync(logDir, { recursive: true, force: true }); -}); - -test('a clean tree records nothing, however many commits it has', async () => { - const { dir, run } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 2;\n'); - run('add', '.'); - run('commit', '--quiet', '-m', 'second'); - const graph = graphFor(dir); - - assert.equal(await captureOutcome(dir, 'completed'), null); - assert.deepEqual(entries(graph), []); -}); - -test('an untracked new file is this turn\'s work', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'added.js'), 'export const added = true;\n'); - const graph = graphFor(dir); - - assert.ok(await captureOutcome(dir, 'completed')); - const [recorded] = entries(graph); - assert.equal(recorded.outcome.status, 'success'); - assert.match(recorded.outcome.note, /added\.js/); -}); - -test('the same working tree is recorded once, across processes as well as turns', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 3;\n'); - const graph = graphFor(dir); - - assert.ok(await captureOutcome(dir, 'completed')); - assert.equal(await captureOutcome(dir, 'completed'), null); - assert.equal(entries(graph).length, 1); - - writeFileSync(join(dir, 'app.js'), 'export const rate = 4;\n'); - assert.ok(await captureOutcome(dir, 'completed')); - assert.equal(entries(graph).length, 2); -}); - -test('a fresh process does not re-record the tree the last one recorded', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 7;\n'); - const graph = graphFor(dir); - - assert.ok(await captureOutcome(dir, 'completed')); - const fromChild = execFileSync( - process.execPath, - [ - '--input-type=module', - '-e', - `const { captureOutcome } = await import(${JSON.stringify(new URL('../src/capture.js', import.meta.url).href)}); - console.log(JSON.stringify(await captureOutcome(${JSON.stringify(dir)}, 'completed')));`, - ], - { encoding: 'utf8', env: { ...process.env, MEMORY_GRAPH_PATH: graph } }, - ); - - assert.equal(fromChild.trim(), 'null'); - assert.equal(entries(graph).length, 1); -}); - -test('editing a still-untracked file is new work each time', async () => { - const { dir } = repo(); - const scratch = join(dir, 'draft.js'); - writeFileSync(scratch, 'export const draft = 1;\n'); - const graph = graphFor(dir); - - assert.ok(await captureOutcome(dir, 'completed')); - writeFileSync(scratch, 'export const draft = 2;\nexport const more = 3;\n'); - assert.ok(await captureOutcome(dir, 'completed')); - assert.equal(entries(graph).length, 2); -}); - -test('workspace state under a nested workspace root is still ignored', async () => { - const { dir } = repo(); - mkdirSync(join(dir, 'workspace', '.evolver'), { recursive: true }); - writeFileSync(join(dir, 'workspace', '.evolver', 'workspace-id'), 'a'.repeat(32)); - const graph = graphFor(dir); - - assert.equal(await captureOutcome(dir, 'completed'), null); - assert.deepEqual(entries(graph), []); -}); - -test('a repository with no commit still sees unstaged work', async () => { - const dir = realpathSync(mkdtempSync(join(tmpdir(), 'evolver-empty-'))); - const run = (...args) => execFileSync('git', args, { cwd: dir, stdio: 'ignore' }); - run('init', '--quiet'); - run('config', 'user.email', 'test@example.com'); - run('config', 'user.name', 'Test'); - writeFileSync(join(dir, 'staged.js'), 'export const staged = 1;\n'); - run('add', 'staged.js'); - currentRepo = dir; - forgetCaptures(dir); - const graph = graphFor(dir); - - assert.ok(await captureOutcome(dir, 'completed')); - writeFileSync(join(dir, 'staged.js'), 'export const staged = 2;\n'); - assert.ok(await captureOutcome(dir, 'completed')); - assert.equal(entries(graph).length, 2); -}); - -test('a diff that could not be recorded anywhere is not marked as done', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 8;\n'); - process.env.MEMORY_GRAPH_PATH = join(dir, 'no-such-dir', 'nested', 'graph.jsonl'); - mkdirSync(join(dir, 'no-such-dir'), { recursive: true }); - writeFileSync(join(dir, 'no-such-dir', 'nested'), 'not a directory'); - - await captureOutcome(dir, 'completed'); - const graph = graphFor(dir); - assert.ok(await captureOutcome(dir, 'completed')); - assert.equal(entries(graph).length, 1); -}); - -test('local memory is durable before a slow Hub request settles', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 12;\n'); - const graph = graphFor(dir); - const originalFetch = globalThis.fetch; - const originalUrl = process.env.EVOMAP_HUB_URL; - const originalKey = process.env.EVOMAP_API_KEY; - let started; - let release; - const fetchStarted = new Promise((resolve) => { - started = resolve; - }); - const fetchRelease = new Promise((resolve) => { - release = resolve; - }); - globalThis.fetch = async () => { - started(); - await fetchRelease; - return { ok: true }; - }; - process.env.EVOMAP_HUB_URL = 'https://evomap.ai'; - process.env.EVOMAP_API_KEY = 'test-key'; - - try { - const pending = captureOutcome(dir, 'completed'); - await fetchStarted; - assert.equal(entries(graph).length, 1); - release(); - assert.ok(await pending); - } finally { - globalThis.fetch = originalFetch; - if (originalUrl === undefined) delete process.env.EVOMAP_HUB_URL; - else process.env.EVOMAP_HUB_URL = originalUrl; - if (originalKey === undefined) delete process.env.EVOMAP_API_KEY; - else process.env.EVOMAP_API_KEY = originalKey; - } -}); - -test('a stale capture lock is recovered before the wait deadline', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 13;\n'); - const graph = graphFor(dir); - const lockPath = `${captureStatePath(dir)}.lock`; - mkdirSync(join(lockPath, '..'), { recursive: true }); - writeFileSync(lockPath, 'crashed'); - const old = new Date(Date.now() - 1_000); - utimesSync(lockPath, old, old); - - assert.ok(await captureOutcome({ - projectDir: dir, - reasonKind: 'completed', - captureLockStaleMs: 10, - captureLockWaitMs: 200, - })); - assert.equal(entries(graph).length, 1); - assert.match(readFileSync(join(logDir, 'evolution.log'), 'utf8'), /Recovered a stale capture lock/); -}); - -test('an unrecoverable lock expiry is visible in the evolution log', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 14;\n'); - graphFor(dir); - const lockPath = `${captureStatePath(dir)}.lock`; - mkdirSync(join(lockPath, '..'), { recursive: true }); - writeFileSync(lockPath, 'still-live'); - - assert.equal(await captureOutcome({ - projectDir: dir, - reasonKind: 'completed', - captureLockStaleMs: 1_000, - captureLockWaitMs: 20, - }), null); - assert.match(readFileSync(join(logDir, 'evolution.log'), 'utf8'), /capture lock wait expired after 20ms/); - rmSync(lockPath, { force: true }); -}); - -test('two overlapping turn ends record one outcome', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 9;\n'); - const graph = graphFor(dir); - - const receipts = await Promise.all([ - captureOutcome(dir, 'completed'), - captureOutcome(dir, 'completed'), - ]); - - assert.equal(receipts.filter(Boolean).length, 1); - assert.equal(entries(graph).length, 1); -}); - -test('a repository with no commit counts both halves of its summary', () => { - const stats = parseStat(' 1 file changed, 2 insertions(+)\n 1 file changed, 3 insertions(+), 1 deletion(-)'); - assert.deepEqual(stats, { files: 2, insertions: 5, deletions: 1 }); -}); - -test('a failed turn with no diff is still recorded with provenance', async () => { - const { dir } = repo(); - const graph = graphFor(dir); - - assert.ok(await captureOutcome({ - projectDir: dir, - reasonKind: 'error', - sessionId: 'session-failed', - turn: 7, - observedSignals: ['recurring_error'], - })); - const [recorded] = entries(graph); - assert.equal(recorded.outcome.status, 'failed'); - assert.equal(recorded.session_id, 'session-failed'); - assert.equal(recorded.turn, 7); - assert.equal(recorded.turn_reason, 'error'); - assert.equal(recorded.diff_scope, 'working_tree'); - assert.match(recorded.diff_hash, /^[a-f0-9]{64}$/); - assert.ok(recorded.signals.includes('recurring_error')); -}); - -test('the same successful diff is suppressed across turns in one session', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 11;\n'); - const graph = graphFor(dir); - - assert.ok(await captureOutcome({ projectDir: dir, reasonKind: 'completed', sessionId: 'session-one', turn: 1 })); - assert.equal(await captureOutcome({ projectDir: dir, reasonKind: 'completed', sessionId: 'session-one', turn: 2 }), null); - assert.equal(entries(graph).length, 1); -}); - -test('a failed turn is recorded as a failure', async () => { - const { dir } = repo(); - writeFileSync(join(dir, 'app.js'), 'export const rate = 5;\n'); - const graph = graphFor(dir); - - assert.ok(await captureOutcome(dir, 'error')); - const [recorded] = entries(graph); - assert.equal(recorded.outcome.status, 'failed'); - assert.match(recorded.outcome.note, /Turn error/); -}); - -test('a written test is not a failing test', () => { - const diff = ['+++ b/app.test.js', '+test("charges once", () => {', '+ expect(charge(order)).toBe(1);', '+});'].join('\n'); - assert.deepEqual(detectSignalsInDiff(diff), []); - - const reported = ['+// note', '+the deploy failed on ci'].join('\n'); - assert.deepEqual(detectSignalsInDiff(reported), ['deployment_issue', 'log_error']); -}); - -test('a completed turn stays a success even when the diff mentions a recurring error', () => { - const outcome = summarize( - { statText: ' 1 file changed, 1 insertion(+)', body: '+the same error keeps failing here', untracked: [] }, - 'completed', - ); - assert.deepEqual(outcome.signals, ['recurring_error']); - assert.equal(outcome.status, 'success'); -}); - -test('a symlinked memory graph inside the project is never appended to', async () => { - const { dir } = repo(); - const outside = mkdtempSync(join(tmpdir(), 'evolver-outside-')); - const target = join(outside, 'secret.jsonl'); - writeFileSync(target, ''); - const planted = join(dir, 'graph.jsonl'); - symlinkSync(target, planted); - process.env.MEMORY_GRAPH_PATH = planted; - writeFileSync(join(dir, 'app.js'), 'export const rate = 6;\n'); - - await captureOutcome(dir, 'completed'); - assert.equal(readFileSync(target, 'utf8'), ''); -}); - -test('a symlinked project memory directory is rejected', () => { - const { dir } = repo(); - const outside = mkdtempSync(join(tmpdir(), 'evolver-outside-memory-')); - mkdirSync(join(outside, 'evolution'), { recursive: true }); - const target = join(outside, 'evolution', 'memory_graph.jsonl'); - writeFileSync(target, ''); - symlinkSync(outside, join(dir, 'memory')); - delete process.env.MEMORY_GRAPH_PATH; - - assert.notEqual(findMemoryGraph(dir), target); -}); - -test('Hub recording requires HTTPS except on loopback', () => { - assert.equal(normalizeHubUrl('http://evomap.ai'), null); - assert.equal(normalizeHubUrl('https://user:secret@evomap.ai'), null); - assert.equal(normalizeHubUrl('https://evomap.ai').hostname, 'evomap.ai'); - assert.equal(normalizeHubUrl('http://127.0.0.1:4000').hostname, '127.0.0.1'); - assert.equal(normalizeHubUrl('not a url'), null); -}); - -test('a directory outside git reports no repository', async () => { - const dir = mkdtempSync(join(tmpdir(), 'evolver-plain-')); - const diff = await collectDiff(dir); - assert.equal(diff.isRepo, false); - assert.equal(await captureOutcome(dir, 'completed'), null); -}); diff --git a/test/checks.test.js b/test/checks.test.js new file mode 100644 index 0000000..aba6986 --- /dev/null +++ b/test/checks.test.js @@ -0,0 +1,37 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; + +import { checkEvidenceOf, checkIdentityOf, checkKindOf } from '../src/checks.js'; + +test('every check command maps onto the four kinds the Hub accepts', () => { + assert.equal(checkKindOf('npx -p typescript tsc -p .'), 'typecheck'); + assert.equal(checkKindOf('npm run lint'), 'lint'); + assert.equal(checkKindOf('npm run validate'), 'lint'); + assert.equal(checkKindOf('node --test test/*.test.js'), 'test'); + assert.equal(checkKindOf('cargo build --release'), 'build'); + assert.equal(checkKindOf('make'), 'build'); +}); + +test('the last check decides, and a failure of that same command earlier marks a turnaround', () => { + assert.equal(checkEvidenceOf([]), undefined); + assert.deepEqual(checkEvidenceOf([{ command: 'npm test', failed: true }, { command: 'npm test', failed: false }]), { kind: 'test', passed: true, fail_then_pass: true }); + assert.deepEqual(checkEvidenceOf([{ command: 'npm run lint', failed: true }, { command: 'npm test', failed: false }]), { kind: 'test', passed: true, fail_then_pass: false }); + assert.deepEqual(checkEvidenceOf([{ command: 'npm test', failed: false }, { command: 'npm test', failed: true }]), { kind: 'test', passed: false, fail_then_pass: false }); +}); + +test('the same check is recognised when only its output trimming differs', () => { + const checks = [ + { command: 'cd /tmp/lru-e2e && node --test 2>&1 | tail -12', failed: true }, + { command: 'cd /tmp/lru-e2e && node --test 2>&1 | tail -9', failed: false }, + ]; + assert.deepEqual(checkEvidenceOf(checks), { kind: 'test', passed: true, fail_then_pass: true }); + assert.equal(checkIdentityOf('npm test 2>&1 | grep -v "^ok" | head -40'), 'npm test'); + assert.equal(checkIdentityOf('npm run build > build.log 2>&1 && npm test | tail -3'), 'npm run build && npm test'); +}); + +test('different checks stay different, and a logical or is not an output pipe', () => { + assert.notEqual(checkIdentityOf('npm test | tail -5'), checkIdentityOf('npm run lint | tail -5')); + assert.notEqual(checkIdentityOf('cd a && npm test'), checkIdentityOf('cd b && npm test')); + assert.equal(checkIdentityOf('npm test || true'), 'npm test || true'); + assert.equal(checkEvidenceOf([{ command: 'npm run lint | tail -3', failed: true }, { command: 'npm test | tail -3', failed: false }]).fail_then_pass, false); +}); diff --git a/test/diagnostics.test.js b/test/diagnostics.test.js index bc6d58d..7b94331 100644 --- a/test/diagnostics.test.js +++ b/test/diagnostics.test.js @@ -4,7 +4,7 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { test } from 'node:test'; -import { createRecallLog, pluginDiagnostics } from '../src/diagnostics.js'; +import { createRecallLog, createReportLog, pluginDiagnostics } from '../src/diagnostics.js'; import { hubGene } from '../src/prime.js'; import { evolverTools } from '../src/tools.js'; @@ -75,6 +75,23 @@ test('the recall log keeps the last outcome with its timing and counts every sta assert.equal(last.at, new Date(2_500).toISOString()); }); +test('the report log keeps the Hub verdict of delivered reports and why rejected ones were refused', () => { + const log = createReportLog(() => 3_000); + log.record({ verdict: 'sent', taskId: 'dsh:s:1', assetId: 'sha256:a', receipt: { eligible: true, counted: true, weight: 0.6 } }); + log.record({ verdict: 'sent', taskId: 'dsh:s:1:correction', assetId: 'sha256:a', receipt: { eligible: false, reason: 'daily_cap' } }); + log.record({ verdict: 'rejected', taskId: 'dsh:s:2', assetId: 'sha256:b', reason: 'invalid_evidence', status: 400 }); + const at = new Date(3_000).toISOString(); + assert.deepEqual(log.snapshot(), { + delivered: [ + { at, task_id: 'dsh:s:1', asset_id: 'sha256:a', eligible: true, counted: true, weight: 0.6 }, + { at, task_id: 'dsh:s:1:correction', asset_id: 'sha256:a', eligible: false, reason: 'daily_cap' }, + ], + rejected: [{ at, task_id: 'dsh:s:2', asset_id: 'sha256:b', reason: 'invalid_evidence', status: 400 }], + }); + for (let index = 0; index < 12; index += 1) log.record({ verdict: 'rejected', taskId: `dsh:s:${index}`, assetId: 'sha256:c', reason: 'x' }); + assert.equal(log.snapshot().rejected.length, 10); +}); + test('evolver_status reports an unreachable Proxy instead of failing, and still shows the plugin view', async () => { const [status] = evolverTools(async () => ({ ok: false, error: 'Proxy connection failed: ECONNREFUSED.' }), { diagnostics: async () => ({ plugin: { version: '9.9.9' } }), @@ -97,8 +114,9 @@ test('diagnostics flag a running Proxy older than the minimum even when PATH has })); const report = await pluginDiagnostics({ config: { assetPrimeWaitMs: 5_000 }, - engineVersion: Promise.resolve('2.0.39'), + engineVersion: async () => '2.0.39', recallLog: createRecallLog(), + reportLog: createReportLog(), }); assert.equal(report.plugin.name, '@evomap/dsh-evolver'); assert.equal(report.proxy_endpoint.url, 'http://127.0.0.1:19999'); @@ -115,6 +133,7 @@ test('diagnostics flag a running Proxy older than the minimum even when PATH has assert.equal(report.recall.wait_ms, 5_000); assert.equal(report.recall.max_strategy_chars, 4_000); assert.equal(report.runtime.platform, process.platform); + assert.deepEqual(report.reuse_reports, { delivered: [], rejected: [] }); } finally { process.env.HOME = home; } diff --git a/test/engine-setup.test.js b/test/engine-setup.test.js new file mode 100644 index 0000000..96b09f2 --- /dev/null +++ b/test/engine-setup.test.js @@ -0,0 +1,65 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; + +import { prepareEngine, setupNoticeText } from '../src/engine-setup.js'; + +function machine({ versions, reachable = false, failing = [] }) { + const calls = []; + const left = [...versions]; + return { + calls, + probeVersion: async () => left.shift() ?? null, + isProxyReachable: async () => reachable, + run: async (command, args) => { + const line = [command, ...args].join(' '); + calls.push(line); + return failing.some((prefix) => line.startsWith(prefix)) ? { ok: false, error: `${command} failed` } : { ok: true, error: null }; + }, + }; +} + +test('a current Evolver with its Proxy serving needs nothing', async () => { + const host = machine({ versions: ['2.0.39'], reachable: true }); + assert.deepEqual(await prepareEngine(host), { status: 'ready', version: '2.0.39' }); + assert.deepEqual(host.calls, []); + assert.equal(setupNoticeText({ status: 'ready' }), null); +}); + +test('a current Evolver whose Proxy is down is started', async () => { + const host = machine({ versions: ['2.0.42'] }); + assert.deepEqual(await prepareEngine(host), { status: 'started', version: '2.0.42' }); + assert.deepEqual(host.calls, ['evolver lifecycle start']); +}); + +test('a start that fails is reported', async () => { + const host = machine({ versions: ['2.0.42'], failing: ['evolver lifecycle'] }); + const outcome = await prepareEngine(host); + assert.deepEqual(outcome, { status: 'start_failed', version: '2.0.42', error: 'evolver failed' }); + assert.match(setupNoticeText(outcome), /`evolver lifecycle start` failed \(evolver failed\)/); +}); + +test('an old Evolver is upgraded and restarted; a missing one is installed and started', async () => { + const old = machine({ versions: ['2.0.38', '2.0.42'], reachable: true }); + assert.deepEqual(await prepareEngine(old), { status: 'upgraded', previous: '2.0.38', version: '2.0.42' }); + assert.deepEqual(old.calls, ['npm install -g @evomap/evolver@latest', 'evolver lifecycle restart']); + + const missing = machine({ versions: [null, '2.0.42'] }); + assert.deepEqual(await prepareEngine(missing), { status: 'installed', previous: null, version: '2.0.42' }); + assert.deepEqual(missing.calls, ['npm install -g @evomap/evolver@latest', 'evolver lifecycle start']); +}); + +test('an install that fails, or leaves an old evolver first on PATH, is reported with the manual steps', async () => { + const denied = await prepareEngine(machine({ versions: [null], failing: ['npm'] })); + assert.deepEqual(denied, { status: 'install_failed', previous: null, version: null, error: 'npm failed' }); + assert.match(setupNoticeText(denied), /Evolver is not installed and installing the latest release failed \(npm failed\)\. Run `npm install -g @evomap\/evolver@latest`, then `evolver lifecycle start`\./); + + const shadowed = await prepareEngine(machine({ versions: ['2.0.30', '2.0.30'] })); + assert.equal(shadowed.status, 'install_failed'); + assert.equal(shadowed.error, 'npm installed 2.0.30'); +}); + +test('an upgrade whose Proxy did not restart says so', async () => { + const outcome = await prepareEngine(machine({ versions: ['2.0.38', '2.0.42'], failing: ['evolver lifecycle'] })); + assert.equal(outcome.startError, 'evolver failed'); + assert.match(setupNoticeText(outcome), /upgraded it to 2\.0\.42, but its Proxy did not start \(evolver failed\); run `evolver lifecycle start`\./); +}); diff --git a/test/engine-version.test.js b/test/engine-version.test.js index 27e5a24..e8e720c 100644 --- a/test/engine-version.test.js +++ b/test/engine-version.test.js @@ -4,7 +4,7 @@ import { tmpdir } from 'node:os'; import { delimiter, join } from 'node:path'; import { afterEach, test } from 'node:test'; -import { MIN_EVOLVER_VERSION, installedEvolverVersion, isOlderThan, upgradeNoticeText } from '../src/engine-version.js'; +import { installedEvolverVersion, isOlderThan } from '../src/engine-version.js'; const previousPath = process.env.PATH; afterEach(() => { @@ -28,13 +28,6 @@ test('compares release numbers numerically, not as text', () => { assert.equal(isOlderThan('not a version', '2.0.39'), false); }); -test('asks for an upgrade only below the minimum', () => { - assert.match(upgradeNoticeText('2.0.38'), /2\.0\.38.*2\.0\.39.*npm install -g @evomap\/evolver@latest/s); - assert.equal(upgradeNoticeText(MIN_EVOLVER_VERSION), null); - assert.equal(upgradeNoticeText('2.1.0'), null); - assert.equal(upgradeNoticeText(null), null); -}); - test('reads the version the installed CLI prints', { skip: process.platform === 'win32' }, async () => { evolverOnPath('echo "evolver v2.0.38"'); assert.equal(await installedEvolverVersion(), '2.0.38'); diff --git a/test/feedback-label.test.js b/test/feedback-label.test.js new file mode 100644 index 0000000..5855ab1 --- /dev/null +++ b/test/feedback-label.test.js @@ -0,0 +1,20 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; + +import { feedbackLabelOf } from '../src/feedback-label.js'; + +test('a prompt saying the last answer did not hold labels it a correction', () => { + for (const text of ['still failing', "that didn't work", '还是报错', '不对,重来']) assert.equal(feedbackLabelOf(text), 'correction', text); +}); + +test('thanks and confirmations label it praise', () => { + for (const text of ['thanks, works now', 'Perfect', '好了,谢谢', '可以了']) assert.equal(feedbackLabelOf(text), 'praise', text); +}); + +test('a correction wins over politeness', () => { + assert.equal(feedbackLabelOf("thanks, but it still doesn't work"), 'correction'); +}); + +test('a new request carries no label', () => { + for (const text of ['now add a logout button', '这个行不行?', '', undefined]) assert.equal(feedbackLabelOf(text), null, String(text)); +}); diff --git a/test/fixtures/fair-play-session.json b/test/fixtures/fair-play-session.json new file mode 100644 index 0000000..23d4f1b --- /dev/null +++ b/test/fixtures/fair-play-session.json @@ -0,0 +1,494 @@ +{ + "source": "dsh session 793c01b3, turns 1-2, with the home directory renamed and non-check command output shortened", + "turns": [ + { + "turn": 1, + "strategyText": "[Evolution Memory] Pseudo-random distribution with configurable pity timers and weighted loot table (EvoMap network):\n1. Implement PRD algorithm where actual drop probability increases with each failed roll\n2. Add configurable pity timers to guarantee upper bounds on bad luck streaks\n3. Use weighted loot tables with dynamic adjustment based on player segments\n4. Include transparency features to show players bad luck protection progress\n\nThese are the complete steps already recalled from the network, so there is nothing more to search or fetch for this task.\nApply them where they fit, then report the outcome with evolver_asset_reuse_result for sha256:872ad8edfa0d86d74140e386528acc4aee7812c946b9e3200c4464468586d0ea.", + "prompt": "fair play", + "tools": [ + { + "name": "bash", + "arguments": { + "command": "pwd && ls -la /home/dev | head -50", + "description": "Show working directory and its contents" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "(output shortened)" + } + ] + } + }, + { + "name": "bash", + "arguments": { + "command": "ls -la /home/dev", + "description": "List home directory contents" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "(output shortened)" + } + ] + } + }, + { + "name": "bash", + "arguments": { + "command": "ls -la /home/dev/workspace /home/dev/Project 2>/dev/null", + "description": "Inspect workspace and Project folders" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "(output shortened)" + } + ] + } + }, + { + "name": "bash", + "arguments": { + "command": "cat /home/dev/package.json 2>/dev/null; node --version 2>/dev/null; python3 --version 2>/dev/null", + "description": "Check package.json and runtimes" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "(output shortened)" + } + ] + } + }, + { + "name": "todo_write", + "arguments": { + "todos": [ + { + "content": "Scaffold fair-play project (package.json, dirs)", + "status": "in_progress" + }, + { + "content": "Implement PRD engine with pity caps + periodic pity timers (src/fair-play.js)", + "status": "pending" + }, + { + "content": "Implement weighted loot table with segment-based dynamic adjustment (src/loot-table.js)", + "status": "pending" + }, + { + "content": "Add transparency/progress reporting + state persistence", + "status": "pending" + }, + { + "content": "Write node:test tests covering PRD, pity guarantee, tables, segments, timers, persistence", + "status": "pending" + }, + { + "content": "Write simulation + fairness report script and run it", + "status": "pending" + }, + { + "content": "Write README, run all tests, report evolver_asset_reuse_result", + "status": "pending" + } + ] + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "Updated todo list: 6 pending, 1 in progress, 0 completed." + } + ] + } + }, + { + "name": "write", + "arguments": { + "file_path": "/home/dev/fair-play/package.json", + "content": "{\n \"name\": \"fair-play\",\n \"version\": \"1.0.0\",\n \"description\": \"Fair-play drop/loot engine: pseudo-random distribution (PRD) with pity caps, configurable pity timers, segment-adjusted weighted loot tables, and transparency reporting.\",\n \"type\": \"module\",\n \"main\": \"src/index.js\",\n \"exports\": {\n \".\": \"./src/index.js\"\n },\n \"scripts\": {\n \"test\": \"node --test test/\",\n \"simulate\": \"node simulate.js\"\n },\n \"engines\": {\n \"node\": \">=18\"\n },\n \"license\": \"MIT\"\n}\n" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "/home/dev/fair-play/package.json\nfile\n\nCreated file\n" + } + ] + } + }, + { + "name": "write", + "arguments": { + "content": "/**\n * Fair-play drop engine.\n *\n * Implements the four pillars of \"fair play\" loot design:\n * 1. PRD (pseudo-random distribution): the effective drop probability grows\n * with every consecutive failure, so bad-luck streaks are front-loaded\n * away and the long-run average matches the configured base rate.\n * 2. Configurable pity caps: a hard guarantee that bounds the worst-case\n * streak (roll N is always a drop), plus periodic pity timers that\n * reset progress on a calendar window (daily/weekly/monthly events).\n * 3. Weighted loot tables with segment-based dynamic adjustment\n * (see src/loot-table.js).\n * 4. Transparency: players can always read their current streak, the\n * effective chance of the next roll, and how many rolls remain until\n * the guaranteed drop.\n *\n * Zero runtime dependencies; the RNG and the clock are injectable so tests\n * and simulations are fully deterministic.\n */\n\nconst DAY_MS = 86_400_000;\nconst WEEK_MS = 7 * DAY_MS;\n\n/** @typedef {{baseRate?: number, increment?: number, pityCap?: number,\n * periodicReset?: 'none'|'daily'|'weekly'|'monthly', resetHourUtc?: number,\n * rng?: () => number, now?: () => number, historySize?: number}} EngineConfig */\n\nfunction assertPositiveInt(name, value) {\n if (!Number.isInteger(value) || value < 1) {\n throw new RangeError(`${name} must be a positive integer, got ${value}`);\n }\n}\n\n/** Bucket a timestamp into the current pity window id. */\nfunction windowId(periodicReset, resetHourUtc, nowMs) {\n if (periodicReset === 'none') return 'always';\n const shifted = new Date(nowMs + resetHourUtc * 3_600_000);\n if (periodicReset === 'daily') {\n return shifted.toISOString().slice(0, 10); // YYYY-MM-DD (UTC-shifted)\n }\n if (periodicReset === 'monthly') {\n return shifted.toISOString().slice(0, 7); // YYYY-MM\n }\n if (periodicReset === 'weekly') {\n // ISO week number of the shifted date.\n const d = new Date(Date.UTC(shifted.getUTCFullYear(), shifted.getUTCMonth(), shifted.getUTCDate()));\n const dayNum = (d.getUTCDay() + 6) % 7; // Mon = 0\n const thursday = new Date(d);\n thursday.setUTCDate(d.getUTCDate() - dayNum + 3);\n const isoYear = thursday.getUTCFullYear();\n const firstThursday = new Date(Date.UTC(isoYear, 0, 4));\n const firstDayNum = (firstThursday.getUTCDay() + 6) % 7;\n firstThursday.setUTCDate(firstThursday.getUTCDate() - firstDayNum + 3);\n const week = 1 + Math.round((thursday - firstThursday) / WEEK_MS);\n return `${isoYear}-W${String(week).padStart(2, '0')}`;\n }\n throw new RangeError(`Unsupported periodicReset: ${periodicReset}`);\n}\n\nexport class FairPlayEngine {\n /**\n * @param {EngineConfig} [config]\n */\n constructor(config = {}) {\n this.config = {\n baseRate: 0.02, // per-roll probability at streak 0\n increment: 0.005, // PRD ramp: added per consecutive failure\n pityCap: 90, // hard cap: the 90th consecutive failure drops for sure\n periodicReset: 'none', // 'none' | 'daily' | 'weekly' | 'monthly'\n resetHourUtc: 0, // hour (UTC) at which a new pity window starts\n rng: Math.random,\n now: () => Date.now(),\n historySize: 50,\n ...config,\n };\n const { baseRate, increment, pityCap, periodicReset, rng, now } = this.config;\n if (!(baseRate > 0 && baseRate < 1)) {\n throw new RangeError(`baseRate must be in (0, 1), got ${baseRate}`);\n }\n if (!(increment >= 0 && increment < 1)) {\n throw new RangeError(`increment must be in [0, 1), got ${increment}`);\n }\n assertPositiveInt('pityCap', pityCap);\n if (typeof rng !== 'function' || typeof now !== 'function') {\n throw new TypeError('rng and now must be functions');\n }\n const w = this.currentWindowId();\n this.state = {\n streak: 0, // consecutive failures since the last drop\n windowId: w, // pity window the streak belongs to\n lastDropAt: null, // epoch ms of the last drop (null = no drop yet)\n totalRolls: 0,\n totalDrops: 0,\n resetsFromWindow: 0, // how often a new window wiped a streak (transparency)\n history: [], // recent drops: { roll, at, streakBefore }\n };\n }\n\n /** @returns {string} the current pity-window id. */\n currentWindowId() {\n return windowId(this.config.periodicReset, this.config.resetHourUtc, this.config.now());\n }\n\n /**\n * Effective chance of a drop for a given number of consecutive failures,\n * before the pity guarantee kicks in. Monotonically non-decreasing in s.\n * @param {number} streak consecutive failures so far\n * @returns {number} probability in [0, 1]\n */\n effectiveChanceAt(streak) {\n if (streak >= this.config.pityCap - 1) return 1;\n return Math.min(this.config.baseRate + this.config.increment * streak, 1);\n }\n\n /**\n * Expected long-run drop rate (drops per roll) for the configured PRD + pity,\n * derived analytically from the renewal process: E[rolls per drop] =\n * sum over n of P(T >= n). Used to validate simulations and to show players\n * the \"true\" rate the PRD is calibrated to.\n */\n theoreticalRate() {\n const { pityCap } = this.config;\n let survival = 1; // P(no drop in the first 0 rolls)\n let expectation = 0;\n for (let n = 1; n <= pityCap; n++) {\n expectation += survival; // P(T >= n)\n const p = this.effectiveChanceAt(n - 1);\n survival *= 1 - p;\n if (survival === 0) break;\n }\n return 1 / expectation;\n }\n\n /**\n * Roll once. Returns true when the drop happens.\n * @returns {boolean}\n */\n roll() {\n const { periodicReset, pityCap, rng, now } = this.config;\n const s = this.state.streak;\n\n // Pity timer: if the streak belongs to a previous calendar window, the\n // window closed without a drop — progress expires (documented behavior).\n if (periodicReset !== 'none') {\n const w = this.currentWindowId();\n if (w !== this.state.windowId) {\n this.state.streak = 0;\n this.state.windowId = w;\n this.state.resetsFromWindow += 1;\n }\n }\n\n const nextStreak = this.state.streak;\n let dropped = false;\n if (nextStreak >= pityCap - 1) {\n dropped = true; // hard pity guarantee\n } else {\n dropped = rng() < this.effectiveChanceAt(nextStreak);\n }\n\n this.state.totalRolls += 1;\n if (dropped) {\n const at = now();\n this.state.history.push({\n roll: this.state.totalRolls,\n at,\n streakBefore: nextStreak,\n });\n if (this.state.history.length > this.config.historySize) {\n this.state.history.shift();\n }\n this.state.streak = 0;\n this.state.totalDrops += 1;\n this.state.lastDropAt = at;\n } else {\n this.state.streak = nextStreak + 1;\n }\n return dropped;\n }\n\n /**\n * Transparency snapshot for UI / logs. Shows players exactly where their\n * bad-luck protection stands.\n */\n report() {\n const s = this.state.streak;\n const effectiveChance = this.effectiveChanceAt(s);\n return {\n streak: s,\n effectiveChance,\n guaranteedNext: effectiveChance === 1,\n rollsToGuarantee: Math.max(0, this.config.pityCap - s),\n pityCap: this.config.pityCap,\n pityWindow: this.config.periodicReset,\n windowId: this.state.windowId,\n resetsFromWindow: this.state.resetsFromWindow,\n totalRolls: this.state.totalRolls,\n totalDrops: this.state.totalDrops,\n observedRate: this.state.totalRolls\n ? this.state.totalDrops / this.state.totalRolls\n : null,\n theoreticalRate: this.theoreticalRate(),\n lastDropAt: this.state.lastDropAt,\n recentDrops: this.state.history.slice(-5),\n };\n }\n\n /** Serializable state (for persistence across sessions/reboots). */\n toJSON() {\n return {\n config: { ...this.config, rng: undefined, now: undefined },\n state: this.state,\n };\n }\n\n /**\n * Restore state from toJSON(). The stored config (minus the injectables) is\n * reused unless overrides are passed.\n * @param {object} json\n * @param {EngineConfig} [overrides]\n */\n static fromJSON(json, overrides = {}) {\n const engine = new FairPlayEngine({ ...json.config, ...overrides });\n const w = engine.currentWindowId();\n engine.state = {\n ...json.state,\n windowId: w, // always re-anchor to the current window on restore\n history: Array.isArray(json.state?.history) ? json.state.history : [],\n };\n return engine;\n }\n}\n", + "file_path": "/home/dev/fair-play/src/fair-play.js" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "/home/dev/fair-play/src/fair-play.js\nfile\n\nCreated file\n" + } + ] + } + }, + { + "name": "write", + "arguments": { + "content": "/**\n * Weighted loot table with segment-based dynamic adjustment.\n *\n * - Static weights: each entry has a base weight; the relative share of each\n * entry is weight / sum(weights) (entries filtered by segment eligibility).\n * - Segment adjustment: player segments (e.g. \"new\", \"returning\", \"veteran\")\n * can multiply entry weights. Multipliers are bounded to keep the economy\n * stable, and every adjustment is fully visible to players via\n * `adjustedWeights()` — no hidden skew.\n *\n * Fair-play guardrails enforced here:\n * * segment multipliers are clamped to a configurable range (default 0.5x–2x);\n * * the adjusted table always re-normalizes, so all rows sum to 1;\n * * per-segment expected rates are computable and exportable, so game-design\n * can audit that no segment is exploited.\n */\n\n/** @typedef {{id: string, weight: number, label?: string,\n * minSegment?: string, maxSegment?: string}} TableEntry */\n\nconst DEFAULT_MULTIPLIER_BOUNDS = { min: 0.5, max: 2 };\n\nfunction normalizeEntry(entry) {\n if (!entry || typeof entry.id !== 'string' || entry.id.length === 0) {\n throw new TypeError('each entry needs a non-empty string id');\n }\n if (!(entry.weight > 0) || !Number.isFinite(entry.weight)) {\n throw new RangeError(`entry \"${entry.id}\" weight must be a positive finite number`);\n }\n return { label: entry.id, ...entry };\n}\n\nexport class WeightedLootTable {\n /**\n * @param {object} params\n * @param {TableEntry[]} params.entries\n * @param {Record>} [params.segmentMultipliers]\n * e.g. { new: { sword: 1.5 } } — extra multiplier for entry \"sword\" for\n * segment \"new\". Missing entries default to 1.\n * @param {{min: number, max: number}} [params.multiplierBounds]\n */\n constructor({ entries, segmentMultipliers = {}, multiplierBounds = DEFAULT_MULTIPLIER_BOUNDS }) {\n if (!Array.isArray(entries) || entries.length === 0) {\n throw new RangeError('entries must be a non-empty array');\n }\n this.entries = entries.map(normalizeEntry);\n this.multiplierBounds = multiplierBounds;\n this.segmentMultipliers = segmentMultipliers;\n }\n\n /**\n * Convenience constructor from a plain { id: weight } map.\n * @param {Record} weights\n */\n static fromWeights(weights) {\n return new WeightedLootTable({\n entries: Object.entries(weights).map(([id, weight]) => ({ id, weight })),\n });\n }\n\n /**\n * Effective weights for a segment after eligibility filter + multipliers.\n * @param {string} [segment='default']\n * @returns {Array<{id: string, label: string, weight: number}>}\n */\n effectiveEntries(segment = 'default') {\n const bounds = this.multiplierBounds;\n const seg = this.segmentMultipliers[segment] ?? {};\n const out = [];\n for (const e of this.entries) {\n if (e.minSegment && segment < e.minSegment) continue;\n if (e.maxSegment && segment > e.maxSegment) continue;\n let weight = e.weight;\n const mult = seg[e.id];\n if (mult !== undefined) {\n if (!Number.isFinite(mult) || mult <= 0) {\n throw new RangeError(`segment multiplier for \"${e.id}\" must be a positive finite number`);\n }\n weight *= Math.min(Math.max(mult, bounds.min), bounds.max);\n }\n out.push({ id: e.id, label: e.label, weight });\n }\n return out;\n }\n\n /**\n * Chances after adjustment for a segment (sums to 1). This is the\n * transparency view: what the player is actually rolling against.\n * @param {string} [segment]\n * @returns {Array<{id: string, label: string, weight: number, chance: number}>}\n */\n adjustedWeights(segment = 'default') {\n const effective = this.effectiveEntries(segment);\n const total = effective.reduce((sum, e) => sum + e.weight, 0);\n return effective.map((e) => ({ ...e, chance: e.weight / total }));\n }\n\n /**\n * Pick one entry by weighted random draw.\n * @param {string} [segment]\n * @param {() => number} [rng]\n * @returns {{id: string, label: string, chance: number}}\n */\n roll(segment = 'default', rng = Math.random) {\n const rows = this.adjustedWeights(segment);\n const u = rng() * rows.reduce((sum, r) => sum + r.weight, 0);\n let acc = 0;\n for (const r of rows) {\n acc += r.weight;\n if (u < acc) return r;\n }\n return rows[rows.length - 1]; // float-safety fallback\n }\n}\n", + "file_path": "/home/dev/fair-play/src/loot-table.js" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "/home/dev/fair-play/src/loot-table.js\nfile\n\nCreated file\n" + } + ] + } + }, + { + "name": "write", + "arguments": { + "content": "/**\n * fair-play — drop/loot engine with PRD, pity caps, pity timers,\n * segment-adjusted weighted tables, and transparency reporting.\n */\n\nexport { FairPlayEngine } from './fair-play.js';\nexport { WeightedLootTable } from './loot-table.js';\n", + "file_path": "/home/dev/fair-play/src/index.js" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "/home/dev/fair-play/src/index.js\nfile\n\nCreated file\n" + } + ] + } + }, + { + "name": "write", + "arguments": { + "content": "import { test } from 'node:test';\nimport assert from 'node:assert/strict';\nimport { FairPlayEngine, WeightedLootTable } from '../src/index.js';\n\n/** RNG stub that always fails (returns ~1), for forcing bad luck. */\nconst alwaysFail = () => 0.9999999;\n/** RNG stub that always succeeds (returns ~0). */\nconst alwaysWin = () => 0.0000001;\n/** Deterministic LCG for reproducible distributions. */\nfunction lcg(seed) {\n let s = seed >>> 0;\n return () => {\n s = (s * 1664525 + 1013904223) >>> 0;\n return s / 4294967296;\n };\n}\n\nfunction makeClock() {\n let t = 1_700_000_000_000;\n return {\n now: () => t,\n advance: (ms) => { t += ms; },\n };\n}\n\ntest('PRD: effective chance grows monotonically with consecutive failures', () => {\n const e = new FairPlayEngine({ baseRate: 0.02, increment: 0.01, pityCap: 90, rng: alwaysFail });\n let prev = 0;\n for (let s = 0; s < 80; s++) {\n const p = e.effectiveChanceAt(s);\n assert.ok(p >= prev, `chance must not decrease at streak ${s}`);\n prev = p;\n }\n assert.equal(e.effectiveChanceAt(0), 0.02);\n assert.equal(e.effectiveChanceAt(1), 0.03);\n});\n\ntest('pity guarantee: the cap-th consecutive failure is a guaranteed drop', () => {\n const e = new FairPlayEngine({ baseRate: 0.01, increment: 0.001, pityCap: 50, rng: alwaysFail });\n let drops = 0;\n for (let roll = 1; roll <= 50; roll++) {\n const dropped = e.roll();\n if (roll < 50) {\n assert.equal(dropped, false, `roll ${roll} should fail before the cap`);\n } else {\n assert.equal(dropped, true, `roll 50 must be guaranteed`);\n drops++;\n }\n }\n assert.equal(drops, 1);\n // Streak resets after the pity drop.\n assert.equal(e.report().streak, 0);\n});\n\ntest('streak resets on a normal drop, totals accumulate', () => {\n const e = new FairPlayEngine({ baseRate: 0.5, increment: 0, pityCap: 10, rng: alwaysWin });\n assert.equal(e.roll(), true);\n assert.equal(e.roll(), true);\n const r = e.report();\n assert.equal(r.totalRolls, 2);\n assert.equal(r.totalDrops, 2);\n assert.equal(r.streak, 0);\n});\n\ntest('worst streak never exceeds pityCap - 1 across a long adversarial run', () => {\n const e = new FairPlayEngine({ baseRate: 0.005, increment: 0.0005, pityCap: 40, rng: lcg(7) });\n let maxStreak = 0;\n for (let i = 0; i < 200_000; i++) {\n e.roll();\n maxStreak = Math.max(maxStreak, e.report().streak);\n }\n assert.ok(maxStreak <= 39, `max streak ${maxStreak} must not exceed pityCap - 1 = 39`);\n});\n\ntest('long-run observed rate converges to the analytic theoretical rate', () => {\n const e = new FairPlayEngine({ baseRate: 0.02, increment: 0.005, pityCap: 90, rng: lcg(42) });\n const rolls = 500_000;\n for (let i = 0; i < rolls; i++) e.roll();\n const r = e.report();\n assert.ok(r.theoreticalRate > 0.02, 'PRD must raise the true rate above the base rate');\n assert.ok(\n Math.abs(r.observedRate - r.theoreticalRate) < 0.0015,\n `observed ${r.observedRate.toFixed(5)} vs theoretical ${r.theoreticalRate.toFixed(5)}`\n );\n});\n\ntest('transparency report exposes streak, chance, and rolls-to-guarantee', () => {\n const e = new FairPlayEngine({ baseRate: 0.05, increment: 0.02, pityCap: 20, rng: alwaysFail });\n e.roll(); e.roll();\n const r = e.report();\n assert.equal(r.streak, 2);\n assert.equal(r.effectiveChance, 0.05 + 0.02 * 2);\n assert.equal(r.rollsToGuarantee, 20 - 2);\n assert.equal(r.guaranteedNext, false);\n assert.ok(Number.isFinite(r.theoreticalRate));\n});\n\ntest('pity timer (daily): streak expires when the calendar window flips', () => {\n const clock = makeClock();\n const e = new FairPlayEngine({\n baseRate: 0.02, increment: 0.01, pityCap: 90,\n periodicReset: 'daily', resetHourUtc: 0, rng: alwaysFail, now: clock.now,\n });\n // 2023-11-14T21:33:20Z — one day later crosses into 2023-11-15 UTC.\n for (let i = 0; i < 5; i++) e.roll();\n assert.equal(e.report().streak, 5);\n clock.advance(86_400_000);\n e.roll(); // triggers the window check\n const r = e.report();\n assert.equal(r.streak, 1, 'streak must reset when the window flips');\n assert.equal(r.resetsFromWindow, 1);\n assert.equal(r.totalRolls, 6, 'totals must survive the window flip');\n});\n\ntest('weekly window id is ISO-week based and stable across days in the same week', () => {\n const e = new FairPlayEngine({ baseRate: 0.1, increment: 0, pityCap: 5, periodicReset: 'weekly' });\n // 2024-01-01 is a Monday → ISO week 2024-W01; +3 days stays in W01.\n const c1 = { ...e, state: { ...e.state } };\n assert.equal(c1.currentWindowId(), '2024-W01');\n const clock = makeClock();\n const tuesday = clock.now() + 86_400_000;\n assert.equal(FairPlayEngine.prototype.currentWindowId.call({ config: { periodicReset: 'weekly', resetHourUtc: 0, now: () => tuesday } }), '2024-W01');\n});\n\ntest('pity timer (monthly) resets across months but not within a month', () => {\n const clock = makeClock();\n const e = new FairPlayEngine({\n baseRate: 0.05, increment: 0, pityCap: 50,\n periodicReset: 'monthly', rng: alwaysFail, now: clock.now,\n });\n e.roll();\n clock.advance(30 * 86_400_000); // +30d stays within November 2023\n e.roll();\n assert.equal(e.report().streak, 2, 'no reset within the same month');\n clock.advance(20 * 86_400_000); // now December\n e.roll();\n assert.equal(e.report().streak, 1, 'reset after month flip');\n});\n\ntest('persistence: toJSON/fromJSON round-trips streak and totals, re-anchors window', () => {\n const clock = makeClock();\n const e = new FairPlayEngine({ baseRate: 0.02, increment: 0.01, pityCap: 90, rng: alwaysFail, now: clock.now });\n e.roll(); e.roll(); e.roll();\n const json = e.toJSON();\n const restored = FairPlayEngine.fromJSON(json, { rng: alwaysFail, now: clock.now });\n const r = restored.report();\n assert.equal(r.streak, 3);\n assert.equal(r.totalRolls, 3);\n assert.equal(r.windowId, e.report().windowId);\n});\n\n// ---------------- Weighted loot table ----------------\n\ntest('loot table: static weights normalize to 1 and respect relative shares', () => {\n const t = WeightedLootTable.fromWeights({ common: 70, rare: 25, epic: 4, legendary: 1 });\n const rows = t.adjustedWeights();\n const total = rows.reduce((s, r) => s + r.chance, 0);\n assert.ok(Math.abs(total - 1) < 1e-12);\n const legendary = rows.find((r) => r.id === 'legendary');\n assert.equal(legendary.chance, 0.01);\n});\n\ntest('loot table: weighted draws match expected shares within tolerance', () => {\n const t = WeightedLootTable.fromWeights({ a: 60, b: 30, c: 10 });\n const counts = { a: 0, b: 0, c: 0 };\n const rng = lcg(1234);\n const N = 60_000;\n for (let i = 0; i < N; i++) counts[t.roll('default', rng).id]++;\n assert.ok(Math.abs(counts.a / N - 0.6) < 0.01, `a=${counts.a / N}`);\n assert.ok(Math.abs(counts.b / N - 0.3) < 0.01, `b=${counts.b / N}`);\n assert.ok(Math.abs(counts.c / N - 0.1) < 0.01, `c=${counts.c / N}`);\n});\n\ntest('segment adjustment: multipliers shift shares, rows still sum to 1, bounds enforced', () => {\n const t = new WeightedLootTable({\n entries: [{ id: 'common', weight: 100 }, { id: 'rare', weight: 10 }, { id: 'epic', weight: 1 }],\n segmentMultipliers: { new: { epic: 50, rare: 5 } }, // way out of bounds on purpose\n multiplierBounds: { min: 0.5, max: 2 },\n });\n const base = t.adjustedWeights('default');\n const boosted = t.adjustedWeights('new');\n const epicBase = base.find((r) => r.id === 'epic').chance;\n const epicNew = boosted.find((r) => r.id === 'epic').chance;\n // multiplier 50 clamps to max 2 → epic chance at most doubles the base chance\n assert.ok(epicNew <= epicBase * 2.0001, `epicNew=${epicNew} epicBase=${epicBase}`);\n assert.ok(epicNew > epicBase, 'boosted epic must still be more likely than default');\n assert.ok(Math.abs(boosted.reduce((s, r) => s + r.chance, 0) - 1) < 1e-12);\n});\n\ntest('segment adjustment: eligibility ranges filter entries per segment', () => {\n const t = new WeightedLootTable({\n entries: [\n { id: 'starter-potion', weight: 1, maxSegment: 'new' },\n { id: 'endgame-gem', weight: 1, minSegment: 'veteran' },\n { id: 'common-drop', weight: 1 },\n ],\n });\n const ids = (seg) => t.adjustedWeights(seg).map((r) => r.id);\n assert.deepEqual(ids('new'), ['starter-potion', 'common-drop']);\n assert.deepEqual(ids('regular'), ['common-drop']);\n assert.deepEqual(ids('veteran'), ['endgame-gem', 'common-drop']);\n});\n\ntest('segment adjustment: draws follow the adjusted distribution', () => {\n const t = new WeightedLootTable({\n entries: [{ id: 'a', weight: 1 }, { id: 'b', weight: 1 }],\n segmentMultipliers: { vip: { a: 3 } },\n });\n const counts = { a: 0, b: 0 };\n const rng = lcg(99);\n const N = 20_000;\n for (let i = 0; i < N; i++) counts[t.roll('vip', rng).id]++;\n assert.ok(Math.abs(counts.a / N - 0.75) < 0.02, `a=${counts.a / N}`);\n});\n\ntest('validation: bad entries and multipliers throw', () => {\n assert.throws(() => WeightedLootTable.fromWeights({ x: 0 }), /positive finite/);\n assert.throws(() => new WeightedLootTable({ entries: [{ id: '', weight: 1 }] }), /non-empty string id/);\n const t = WeightedLootTable.fromWeights({ a: 1, b: 1 });\n assert.throws(() => t.adjustedWeights('bad'), /must be a positive finite number/);\n});\n\ntest('engine validation: invalid config throws', () => {\n assert.throws(() => new FairPlayEngine({ baseRate: 0 }), /baseRate/);\n assert.throws(() => new FairPlayEngine({ baseRate: 1 }), /baseRate/);\n assert.throws(() => new FairPlayEngine({ baseRate: 0.1, increment: 2 }), /increment/);\n assert.throws(() => new FairPlayEngine({ baseRate: 0.1, pityCap: 0 }), /pityCap/);\n});\n", + "file_path": "/home/dev/fair-play/test/fair-play.test.js" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "/home/dev/fair-play/test/fair-play.test.js\nfile\n\nCreated file\n" + } + ] + } + }, + { + "name": "edit", + "arguments": { + "file_path": "/home/dev/fair-play/test/fair-play.test.js", + "old_string": "test('weekly window id is ISO-week based and stable across days in the same week', () => {\n const e = new FairPlayEngine({ baseRate: 0.1, increment: 0, pityCap: 5, periodicReset: 'weekly' });\n // 2024-01-01 is a Monday → ISO week 2024-W01; +3 days stays in W01.\n const c1 = { ...e, state: { ...e.state } };\n assert.equal(c1.currentWindowId(), '2024-W01');\n const clock = makeClock();\n const tuesday = clock.now() + 86_400_000;\n assert.equal(FairPlayEngine.prototype.currentWindowId.call({ config: { periodicReset: 'weekly', resetHourUtc: 0, now: () => tuesday } }), '2024-W01');\n});", + "new_string": "test('weekly window id is ISO-week based and stable across days in the same week', () => {\n // 2024-01-01 is a Monday → ISO week 2024-W01; +3 days stays in W01.\n const mk = (t) => new FairPlayEngine({ baseRate: 0.1, increment: 0, pityCap: 5, periodicReset: 'weekly', now: () => t });\n const jan1 = Date.UTC(2024, 0, 1);\n const jan4 = Date.UTC(2024, 0, 4);\n assert.equal(mk(jan1).currentWindowId(), '2024-W01');\n assert.equal(mk(jan4).currentWindowId(), '2024-W01');\n});" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "The file /home/dev/fair-play/test/fair-play.test.js has been updated successfully." + } + ] + } + }, + { + "name": "edit", + "arguments": { + "file_path": "/home/dev/fair-play/test/fair-play.test.js", + "new_string": "test('pity timer (monthly) resets across months but not within a month', () => {\n const clock = makeClock(); // 2023-11-14T21:33:20Z\n const e = new FairPlayEngine({\n baseRate: 0.05, increment: 0, pityCap: 50,\n periodicReset: 'monthly', rng: alwaysFail, now: clock.now,\n });\n e.roll();\n clock.advance(10 * 86_400_000); // 2023-11-24: still November\n e.roll();\n assert.equal(e.report().streak, 2, 'no reset within the same month');\n clock.advance(20 * 86_400_000); // 2023-12-14: December\n e.roll();\n assert.equal(e.report().streak, 1, 'reset after month flip');\n});", + "old_string": "test('pity timer (monthly) resets across months but not within a month', () => {\n const clock = makeClock();\n const e = new FairPlayEngine({\n baseRate: 0.05, increment: 0, pityCap: 50,\n periodicReset: 'monthly', rng: alwaysFail, now: clock.now,\n });\n e.roll();\n clock.advance(30 * 86_400_000); // +30d stays within November 2023\n e.roll();\n assert.equal(e.report().streak, 2, 'no reset within the same month');\n clock.advance(20 * 86_400_000); // now December\n e.roll();\n assert.equal(e.report().streak, 1, 'reset after month flip');\n});" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "The file /home/dev/fair-play/test/fair-play.test.js has been updated successfully." + } + ] + } + }, + { + "name": "edit", + "arguments": { + "file_path": "/home/dev/fair-play/test/fair-play.test.js", + "new_string": "test('validation: bad entries and multipliers throw', () => {\n assert.throws(() => WeightedLootTable.fromWeights({ x: 0 }), /positive finite/);\n assert.throws(() => new WeightedLootTable({ entries: [{ id: '', weight: 1 }] }), /non-empty string id/);\n const t = new WeightedLootTable({\n entries: [{ id: 'a', weight: 1 }, { id: 'b', weight: 1 }],\n segmentMultipliers: { bad: { a: -3 } },\n });\n assert.throws(() => t.adjustedWeights('bad'), /positive finite number/);\n});", + "old_string": "test('validation: bad entries and multipliers throw', () => {\n assert.throws(() => WeightedLootTable.fromWeights({ x: 0 }), /positive finite/);\n assert.throws(() => new WeightedLootTable({ entries: [{ id: '', weight: 1 }] }), /non-empty string id/);\n const t = WeightedLootTable.fromWeights({ a: 1, b: 1 });\n assert.throws(() => t.adjustedWeights('bad'), /must be a positive finite number/);\n});" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "The file /home/dev/fair-play/test/fair-play.test.js has been updated successfully." + } + ] + } + }, + { + "name": "write", + "arguments": { + "content": "/**\n * Fairness simulation: runs many synthetic players through the engine and\n * prints a report that verifies the four fair-play pillars empirically.\n *\n * node simulate.js\n */\n\nimport { FairPlayEngine, WeightedLootTable } from './src/index.js';\n\nfunction lcg(seed) {\n let s = seed >>> 0;\n return () => {\n s = (s * 1664525 + 1013904223) >>> 0;\n return s / 4294967296;\n };\n}\n\nconst PLAYERS = 50_000;\nconst ROLLS_PER_PLAYER = 200;\nconst CONFIG = { baseRate: 0.02, increment: 0.005, pityCap: 90 };\nconst TABLE = new WeightedLootTable({\n entries: [\n { id: 'common', weight: 68, label: 'Common' },\n { id: 'rare', weight: 22, label: 'Rare' },\n { id: 'epic', weight: 8, label: 'Epic' },\n { id: 'legendary', weight: 2, label: 'Legendary' },\n ],\n segmentMultipliers: {\n new: { epic: 1.5, legendary: 1.5 }, // onboarding boost for newcomers\n },\n});\n\nconst players = Array.from({ length: PLAYERS }, (_, i) => {\n const engine = new FairPlayEngine({ ...CONFIG, rng: lcg(i + 1) });\n const drops = { common: 0, rare: 0, epic: 0, legendary: 0 };\n let maxStreak = 0;\n for (let r = 0; r < ROLLS_PER_PLAYER; r++) {\n if (engine.roll()) {\n drops[TABLE.roll('default', engine.config.rng).id]++;\n }\n maxStreak = Math.max(maxStreak, engine.report().streak);\n }\n const rep = engine.report();\n return {\n drops: Object.values(drops).reduce((a, b) => a + b, 0),\n byTier: drops,\n maxStreak,\n observedRate: rep.observedRate,\n };\n});\n\nconst totalRolls = PLAYERS * ROLLS_PER_PLAYER;\nconst totalDrops = players.reduce((s, p) => s + p.drops, 0);\nconst observed = totalDrops / totalRolls;\nconst engine0 = new FairPlayEngine(CONFIG);\nconst theoretical = engine0.theoreticalRate();\n\nconst maxStreakGlobal = Math.max(...players.map((p) => p.maxStreak));\nconst worstPlayer = players.reduce((a, b) => (b.drops < a.drops ? b : a));\n\n// Gini coefficient over drops-per-player — 0 = perfectly fair.\nconst sorted = players.map((p) => p.drops).sort((a, b) => a - b);\nconst n = sorted.length;\nlet giniNum = 0;\nfor (let i = 0; i < n; i++) giniNum += (2 * i - n + 1) * sorted[i];\nconst gini = giniNum / (n * sorted.reduce((s, x) => s + x, 0));\n\nconst fmt = (x) => x.toFixed(4);\nconst line = '─'.repeat(62);\n\nconsole.log(`\\n${line}`);\nconsole.log('FAIR-PLAY DROP ENGINE — SIMULATION REPORT');\nconsole.log(line);\nconsole.log(`players × rolls : ${PLAYERS.toLocaleString()} × ${ROLLS_PER_PLAYER}`);\nconsole.log(`config : base=${CONFIG.baseRate} increment=${CONFIG.increment} pityCap=${CONFIG.pityCap}`);\nconsole.log(`theoretical rate : ${fmt(theoretical)} (analytic renewal expectation)`);\nconsole.log(`observed rate : ${fmt(observed)}`);\nconsole.log(`deviation : ${(observed - theoretical).toFixed(5)}`);\nconsole.log(`${line}`);\nconsole.log('BAD-LUCK BOUNDS');\nconsole.log(`worst streak (global) : ${maxStreakGlobal} (pityCap ${CONFIG.pityCap} ⇒ guaranteed at ${CONFIG.pityCap - 1} failures)`);\nconsole.log(`worst player drops : ${worstPlayer.drops}/${ROLLS_PER_PLAYER} (rate ${fmt(worstPlayer.observedRate)})`);\nconsole.log(`${line}`);\nconsole.log('FAIRNESS (drops per player)');\nconsole.log(`mean drops/player : ${fmt(players.reduce((s, p) => s + p.drops, 0) / PLAYERS)}`);\nconsole.log(`gini coefficient : ${fmt(gini)} (0 = perfectly even distribution)`);\nconsole.log(`${line}`);\nconsole.log('LOOT TABLE (share of dropped items, expected vs observed)');\nconst totalByTier = { common: 0, rare: 0, epic: 0, legendary: 0 };\nfor (const p of players) for (const k of Object.keys(totalByTier)) totalByTier[k] += p.byTier[k];\nfor (const row of TABLE.adjustedWeights('default')) {\n const obs = totalByTier[row.id] / totalDrops;\n console.log(\n ` ${row.id.padEnd(10)} expected ${fmt(row.chance).padStart(6)} observed ${fmt(obs).padStart(6)}`\n );\n}\nconsole.log(line);\nconsole.log('TRANSPARENCY SAMPLE (player-facing progress view)');\nconst sample = new FairPlayEngine({ ...CONFIG, rng: lcg(999) });\nfor (let i = 0; i < 57; i++) sample.roll();\nconst r = sample.report();\nconsole.log(` streak : ${r.streak}`);\nconsole.log(` next-roll chance : ${fmt(r.effectiveChance)}`);\nconsole.log(` rolls-to-guarantee: ${r.rollsToGuarantee}`);\nconsole.log(` guaranteed next : ${r.guaranteedNext}`);\nconsole.log(`${line}\\n`);\n", + "file_path": "/home/dev/fair-play/simulate.js" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "/home/dev/fair-play/simulate.js\nfile\n\nCreated file\n" + } + ] + } + }, + { + "name": "bash", + "arguments": { + "command": "cd /home/dev/fair-play && node --test test/ 2>&1 | tail -30", + "description": "Run the test suite" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "\nError: Cannot find module '/home/dev/fair-play/test'\n at Module._resolveFilename (node:internal/modules/cjs/loader:1475:15)\n at wrapResolveFilename (node:internal/modules/cjs/loader:1048:27)\n at defaultResolveImplForCJSLoading (node:internal/modules/cjs/loader:1072:10)\n at resolveForCJSWithHooks (node:internal/modules/cjs/loader:1093:12)\n at Module._load (node:internal/modules/cjs/loader:1261:25)\n at wrapModuleLoad (node:internal/modules/cjs/loader:255:19)\n at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:154:5)\n at node:internal/main/run_main_module:33:47 {\n code: 'MODULE_NOT_FOUND',\n requireStack: []\n}\n\nNode.js v26.0.0\n✖ test (59.955542ms)\nℹ tests 1\nℹ suites 0\nℹ pass 0\nℹ fail 1\nℹ cancelled 0\nℹ skipped 0\nℹ todo 0\nℹ duration_ms 63.988792\n\n✖ failing tests:\n\ntest at test:1:1\n✖ test (59.955542ms)\n 'test failed'\n" + } + ] + } + }, + { + "name": "bash", + "arguments": { + "command": "cd /home/dev/fair-play && node --test 2>&1 | tail -25", + "description": "Run tests with auto-discovery" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "ℹ fail 1\nℹ cancelled 0\nℹ skipped 0\nℹ todo 0\nℹ duration_ms 142.962542\n\n✖ failing tests:\n\ntest at test/fair-play.test.js:203:1\n✖ segment adjustment: draws follow the adjusted distribution (4.077708ms)\n AssertionError [ERR_ASSERTION]: a=0.6732\n at TestContext. (file:///home/dev/fair-play/test/fair-play.test.js:212:10)\n at Test.runInAsyncScope (node:async_hooks:226:14)\n at Test.run (node:internal/test_runner/test:1201:25)\n at Test.processPendingSubtests (node:internal/test_runner/test:831:18)\n at Test.postRun (node:internal/test_runner/test:1330:19)\n at Test.run (node:internal/test_runner/test:1258:12)\n at async Test.processPendingSubtests (node:internal/test_runner/test:831:7) {\n generatedMessage: false,\n code: 'ERR_ASSERTION',\n actual: false,\n expected: true,\n operator: '==',\n diff: 'simple'\n }\n" + } + ] + } + }, + { + "name": "edit", + "arguments": { + "file_path": "/home/dev/fair-play/test/fair-play.test.js", + "new_string": " const counts = { a: 0, b: 0 };\n const rng = lcg(99);\n const N = 20_000;\n for (let i = 0; i < N; i++) counts[t.roll('vip', rng).id]++;\n // multiplier 3 clamps to the default max of 2 → a = 2/3 of the mass.\n assert.ok(Math.abs(counts.a / N - 2 / 3) < 0.02, `a=${counts.a / N}`);", + "old_string": " const counts = { a: 0, b: 0 };\n const rng = lcg(99);\n const N = 20_000;\n for (let i = 0; i < N; i++) counts[t.roll('vip', rng).id]++;\n assert.ok(Math.abs(counts.a / N - 0.75) < 0.02, `a=${counts.a / N}`);" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "The file /home/dev/fair-play/test/fair-play.test.js has been updated successfully." + } + ] + } + }, + { + "name": "bash", + "arguments": { + "command": "cd /home/dev/fair-play && node --test 2>&1 | tail -12", + "description": "Re-run the test suite" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "✔ segment adjustment: eligibility ranges filter entries per segment (0.603666ms)\n✔ segment adjustment: draws follow the adjusted distribution (3.027917ms)\n✔ validation: bad entries and multipliers throw (0.222958ms)\n✔ engine validation: invalid config throws (0.10975ms)\nℹ tests 17\nℹ suites 0\nℹ pass 17\nℹ fail 0\nℹ cancelled 0\nℹ skipped 0\nℹ todo 0\nℹ duration_ms 129.154042\n" + } + ] + } + }, + { + "name": "bash", + "arguments": { + "command": "cd /home/dev/fair-play && node simulate.js", + "description": "Run fairness simulation report" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "(output shortened)" + } + ] + } + }, + { + "name": "edit", + "arguments": { + "file_path": "/home/dev/fair-play/package.json", + "new_string": " \"test\": \"node --test\",", + "old_string": " \"test\": \"node --test test/\"," + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "The file /home/dev/fair-play/package.json has been updated successfully." + } + ] + } + }, + { + "name": "write", + "arguments": { + "content": "# fair-play\n\nA dependency-free drop/loot engine that makes **bad luck protection** first-class:\npseudo-random distribution (PRD), hard pity caps, configurable pity *timers*,\nsegment-adjusted weighted loot tables, and full player-facing transparency.\n\nBuilt for game backends, gacha / card-draw systems, and any \"random reward\"\nloop where fairness is a product requirement.\n\n## The four fair-play pillars\n\n| Pillar | Implementation |\n|---|---|\n| **1. PRD — chance grows with each miss** | `baseRate + increment × streak`, so streaks are front-loaded away and the long-run average converges to an analytic target (see `theoreticalRate()`). |\n| **2. Pity caps + pity timers** | `pityCap` guarantees the cap-th consecutive failure always drops. `periodicReset: 'daily' | 'weekly' | 'monthly'` gives calendar-bound pity windows (e.g. event banners) with injectable clocks for testing. |\n| **3. Weighted loot tables, segment-adjusted** | `WeightedLootTable` with per-entry weights, segment eligibility ranges, and bounded per-segment multipliers (default 0.5×–2×) for onboarding/returning-player boosts that cannot destabilize the economy. |\n| **4. Transparency** | `engine.report()` exposes streak, next-roll chance, rolls-to-guarantee, observed vs theoretical rate, and recent drops — exactly what a UI \"pity meter\" should show. State is serializable (`toJSON`/`fromJSON`) so progress survives sessions. |\n\n## Quick start\n\n```js\nimport { FairPlayEngine, WeightedLootTable } from 'fair-play';\n\n// 1. The engine: PRD + pity.\nconst engine = new FairPlayEngine({\n baseRate: 0.02, // per-roll chance at streak 0\n increment: 0.005, // added per consecutive failure\n pityCap: 90, // 90th consecutive failure always drops\n periodicReset: 'none', // 'daily' | 'weekly' | 'monthly' for event pity timers\n});\n\n// 2. The loot table: weighted, with a bounded onboarding boost for new players.\nconst table = new WeightedLootTable({\n entries: [\n { id: 'common', weight: 68 },\n { id: 'rare', weight: 22 },\n { id: 'epic', weight: 8 },\n { id: 'legendary', weight: 2 },\n ],\n segmentMultipliers: { new: { epic: 1.5, legendary: 1.5 } },\n});\n\n// 3. Roll.\nif (engine.roll()) {\n const item = table.roll(segment); // weighted pick\n}\n\n// 4. Show the player their protection progress.\nconsole.log(engine.report());\n// { streak: 5, effectiveChance: 0.045, rollsToGuarantee: 85, ... }\n```\n\n## API\n\n### `FairPlayEngine`\n\n| Member | Description |\n|---|---|\n| `new FairPlayEngine(config?)` | `baseRate` (0,1), `increment` [0,1), `pityCap` ≥1, `periodicReset`, `resetHourUtc`, injectable `rng()` and `now()` |\n| `roll()` | one roll; returns `true` on drop; resets streak; enforces pity window expiry |\n| `effectiveChanceAt(streak)` | probability of the next roll for a given miss streak |\n| `theoreticalRate()` | analytic long-run drops-per-roll for the configured PRD+pity |\n| `report()` | transparency snapshot (streak, chance, guarantee countdown, totals, history) |\n| `toJSON()` / `fromJSON()` | persist/restore state across sessions |\n\n### `WeightedLootTable`\n\n| Member | Description |\n|---|---|\n| `new WeightedLootTable({ entries, segmentMultipliers, multiplierBounds })` | entries: `{ id, weight, label?, minSegment?, maxSegment? }` |\n| `static fromWeights({ id: weight })` | convenience constructor |\n| `adjustedWeights(segment)` | normalized chances after eligibility + multipliers (the player-facing table) |\n| `roll(segment, rng?)` | weighted pick |\n\n## Fairness guards built in\n\n- The hard pity cap means **no player can ever exceed `pityCap - 1` misses** — verified empirically in the simulation and by test.\n- Segment multipliers are **clamped** (default 0.5×–2×), so onboarding boosts stay bounded and long-run rates converge for every segment.\n- `theoreticalRate()` lets designers calibrate `baseRate`/`increment` so the *true* drop rate matches the marketing number — the observed rate in `simulate.js` lands within 0.2% of it.\n\n## Run it\n\n```bash\nnpm test # 17 tests: PRD, pity guarantee, timers, tables, persistence\nnode simulate.js # 50k players × 200 rolls: fairness + distribution report\n```\n\n## License\n\nMIT\n", + "file_path": "/home/dev/fair-play/README.md" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "/home/dev/fair-play/README.md\nfile\n\nCreated file\n" + } + ] + } + }, + { + "name": "bash", + "arguments": { + "command": "cd /home/dev/fair-play && npm test 2>&1 | tail -8 && echo \"---\" && find . -type f -not -path './node_modules/*' | sort", + "description": "Final test run and file listing" + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "ℹ tests 17\nℹ suites 0\nℹ pass 17\nℹ fail 0\nℹ cancelled 0\nℹ skipped 0\nℹ todo 0\nℹ duration_ms 121.145875\n---\n./README.md\n./package.json\n./simulate.js\n./src/fair-play.js\n./src/index.js\n./src/loot-table.js\n./test/fair-play.test.js\n" + } + ] + } + }, + { + "name": "evolver_asset_reuse_result", + "arguments": { + "asset_id": "sha256:872ad8edfa0d86d74140e386528acc4aee7812c946b9e3200c4464468586d0ea", + "outcome": "success", + "reason": "Applied the recalled PRD+pity strategy end-to-end: implemented a dependency-free fair-play drop engine (PRD ramp, hard pity caps, daily/weekly/monthly pity timers, segment-adjusted weighted loot tables, transparency reporting) with 17 passing tests and a 50k-player simulation confirming the observed rate matches the analytic expectation within 0.2%.", + "task_id": "fair-play-drop-engine", + "time_saved_seconds": 1200 + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "{\n \"recorded\": true,\n \"reuseStats\": {\n \"total\": 1,\n \"success\": 1,\n \"successRate\": 1\n },\n \"eligible\": true,\n \"counted\": true,\n \"weight\": 0.1\n}" + } + ] + } + }, + { + "name": "todo_write", + "arguments": { + "todos": [ + { + "content": "Scaffold fair-play project (package.json, dirs)", + "status": "completed" + }, + { + "content": "Implement PRD engine with pity caps + periodic pity timers (src/fair-play.js)", + "status": "completed" + }, + { + "content": "Implement weighted loot table with segment-based dynamic adjustment (src/loot-table.js)", + "status": "completed" + }, + { + "content": "Add transparency/progress reporting + state persistence", + "status": "completed" + }, + { + "content": "Write node:test tests covering PRD, pity guarantee, tables, segments, timers, persistence", + "status": "completed" + }, + { + "content": "Write simulation + fairness report script and run it", + "status": "completed" + }, + { + "content": "Write README, run all tests, report evolver_asset_reuse_result", + "status": "completed" + } + ] + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "Updated todo list: 0 pending, 0 in progress, 7 completed." + } + ] + } + } + ], + "answer": "Built **fair-play** — a dependency-free drop/loot engine implementing all four recalled steps. `npm test` → 17/17 pass; simulation → observed rate within 0.2% of the analytic expectation.\n\n## What's in `/home/dev/fair-play/`\n\n| File | Purpose |\n|---|---|\n| [src/fair-play.js]() | PRD engine: chance = `baseRate + increment × streak`, hard `pityCap` guarantee, daily/weekly/monthly **pity timers** (injectable clock), `toJSON`/`fromJSON` persistence |\n| [src/loot-table.js]() | Weighted loot table with segment eligibility + **bounded** segment multipliers (0.5×–2×) and normalized player-facing rates |\n| [src/index.js]() | Public exports |\n| [test/fair-play.test.js]() | 17 tests: monotonic PRD, pity-at-cap guarantee, worst-streak bound, rate convergence, table distribution, segment adjustment, timer resets, persistence round-trip |\n| [simulate.js]() | 50k-player × 200-roll fairness report |\n| [README.md]() | Docs + usage example |\n\n## Verification highlights\n\n```\ntheoretical rate : 0.0673 (analytic renewal expectation)\nobserved rate : 0.0658 (deviation -0.0015)\nworst streak (global) : 66 (pityCap 90 ⇒ guaranteed at 89 misses)\ngini coefficient : 0.0922 (0 = perfectly even drops per player)\nlegendary expected 0.0200 observed 0.0200\n```\n\nThe four pillars, mapped:\n1. **PRD** — effective chance grows per miss (`effectiveChanceAt`), so streaks get front-loaded; long-run average converges to `theoreticalRate()`.\n2. **Pity caps + timers** — hard guarantee bounds any streak at `pityCap − 1`; `periodicReset` gives calendar-bound event pity windows that expire a stale streak (documented behavior, tested).\n3. **Weighted tables + segments** — per-entry weights, eligibility ranges, clamped segment boosts (e.g. onboarding boost for `new` players that can't destabilize the economy).\n4. **Transparency** — `engine.report()` returns streak, next-roll chance, rolls-to-guarantee, observed vs theoretical rate, and recent drops — ready for a UI pity meter.\n\nThe asset reuse was recorded as `success` (estimated ~20 min saved by applying the recalled strategy directly). If \"fair play\" actually referred to an existing project of yours (I found none in the workspace), tell me where and I'll port the engine there.", + "reason": { + "kind": "completed" + } + }, + { + "turn": 2, + "strategyText": "[Evolution Memory] Pipeline-generated simulation capsule for Select and apply structural craft pack (EvoMap network):\n1. Resolution order: enforce explicit scope and safety boundaries first; for the same API, syntax, or config surface under matching applicability, follow the source-grounded canonical contract and its executable exemplar over a generalized procedure step; otherwise follow the generalized procedure.\n2. 1. Open story.md, confirm that genre or sub-genre is recorded, and confirm that a plot structure exists or is underway.\n3. 2. Map each recorded genre to its reference pack: mystery/crime/detective to mystery-fair-play; romance to romance-beats; thriller/action/suspense to thriller; horror to horror; middle-grade/young-adult to mg-ya; science fiction to scifi-pipeline; and serial/episodic/web serial to serial-episodic. Exact anchors: Pick the pack..\n4. 3. For a multi-genre story, load every applicable pack. Identify conflicting requirements, ask the user which genre contract dominates each affected section, and record each decision in story.md notes. Exact anchors: Multi-genre stories: load each applicable pack.\n5. 4. Set planning constraints from the selected packs. For mystery, create continuity/clues and ledger clues and red herrings with planted/payoff metadata; for thriller, log deadlines in continuity/promises and track story-time in plot/timeline.md; for serial, add season-goal to story.md and episode-question to each installment; for MG/YA, add target-words and check age/adult-involvement constraints\n6. 5. During drafting, apply the selected pack's beat concepts and rules alongside chapter- and scene-level workflows. For thrillers, use the Fichtean-curve connection; for thrillers and serials, inspect chapter hook values and runs of resolution endings; for serials, connect book-level canon to series continuity. Exact anchors: Draft against the pack..\n7. 6. Maintain entity conventions: use kebab-case IDs, YAML frontmatter, collection registries, forward plant-to-payoff ordering, and reciprocal clue/suspect/chapter links. Exact anchors: kebab-case ids | YAML frontmatter | Bidirectional links apply.\n8. 7. After genre entities change, refresh indexes and links, then run validation and continuity checks; for mystery also inspect the fair-play clue matrix. Use the direct Story CLI, supported Bun or Node fallback, or equivalent manual checks when no CLI is available. Exact anchors: After adding or changing genre entities.\n9. 8. During developmental revision, execute the selected reference pack's audit checklist and write findings into the revision plan or a continuity audit file. Exact anchors: Run the pack audit in revision..\n10. 9. Record every deliberate departure from a pack in story.md with its reason so later audits preserve rather than automatically repair it. Exact anchors: Record genre decisions..\n11. Canonical syntax contract 1 (applies when: After adding or changing genre entities such as clues, deadlines, season goals, or episode questions, and the Story CLI is available.)\nExecutable exemplar (verbatim):\nstory reindex .\nstory links .\nstory validate .\nstory continuity .\nstory clues . # mystery: fair-play matrix and warnings\n12. Safety boundary: Use these packs for structural genre craft, not line-level genre voice.\n13. Safety boundary: Do not impose a deterministic genre pack on literary fiction.\n14. Safety boundary: For multi-genre conflicts, obtain the user's decision about which genre contract dominates each affected section and record it in story.md notes.\n15. Safety boundary: Do not silently normalize a deliberate genre deviation; preserve the deviation and its reason in story.md.\n16. Safety boundary: Store audit findings in the revision plan or a continuity audit file, never only in chat.\n17. Safety boundary: If no Story CLI invocation is available, perform registry, backlink, and word-count checks manually rather than skipping them.\n\nThese are the complete steps already recalled from the network, so there is nothing more to search or fetch for this task.\nApply them where they fit, then report the outcome with evolver_asset_reuse_result for sha256:ed3b8595561c58c86c0c630e1c77f698a8a5b3e0e5d1aabd6acce1a8fa3a3750.", + "prompt": "fair play是什么意思?", + "tools": [ + { + "name": "evolver_asset_reuse_result", + "arguments": { + "asset_id": "sha256:ed3b8595561c58c86c0c630e1c77f698a8a5b3e0e5d1aabd6acce1a8fa3a3750", + "outcome": "mismatched", + "reason": "The recalled capsule is a story-craft pipeline (story.md genre packs, Story CLI, mystery-fair-play pack selection). The user's actual request was a terminology question (\"what does fair play mean\"), not a story-writing task, so none of the pipeline steps applied; I only used its mystery-fair-play reference to explain the genre meaning.", + "task_id": "fair-play-definition", + "time_saved_seconds": 0 + }, + "result": { + "isError": false, + "content": [ + { + "type": "text", + "text": "{\n \"recorded\": true,\n \"reuseStats\": {\n \"total\": 1,\n \"success\": 0,\n \"successRate\": 0\n },\n \"eligible\": true,\n \"counted\": true,\n \"weight\": 0.1\n}" + } + ] + } + } + ], + "answer": "\"fair play\" 字面意思是**公平竞赛、光明正大**,源于体育(英式绅士运动理念),引申为\"遵守规则、不耍赖、给对手同等机会\"。在不同语境下有几个具体含义:\n\n## 1. 体育 / 竞技\n遵守规则、尊重对手、不以不正当手段取胜。就是我们常说的\"公平竞争\"。\n\n## 2. 游戏掉落 / 抽卡(我们刚建的那个 fair-play 项目取的就是这个意思)\n指**公平掉落机制**——\"坏运气保护\"(bad luck protection):\n- **伪随机分布(PRD)**:连续失败后下次掉落概率逐步提高,把坏运气摊平;\n- **保底(pity)**:连续 N 次不出货,第 N 次必定出货,给运气最差的玩家一个可预期的下限;\n- **透明**:界面上显示\"距离保底还有 X 次\"\"当前出货概率\",玩家看得见自己的进度。\n\n这是现代手游/网游让\"抽卡\"不那么伤人的标准做法——不是让每个玩家拿到一样的东西,而是**保证运气差的人也有兜底**,所以叫 fair play。\n\n## 3. 推理小说(本格派的核心原则)\n**Fair-play mystery / 公平游戏推理**——作者向读者承诺:所有破案线索都已经摆在读者面前,读者在真相揭晓前**有足够信息自行推理出凶手**;作者不得用读者无法获取的隐藏信息作弊。黄金时代推理(诺克斯\"十诫\"、范达因\"二十条\")就是这套公平竞赛规则。这也是刚才注入记忆里 `mystery-fair-play` 题材卡组所指的东西——写推理时要把\"线索布置\"做成可审计的公平游戏。\n\n## 小结\n一句话:**fair play = 规则透明、结果可预期、不欺负参与者**——不管是赛场、抽卡还是推理小说,都是这个内核。\n\n那条关于写作结构卡组的记忆与你的提问不匹配(它是给写小说用的流程),我已按 `mismatched` 回报;如果你其实是想用 mystery-fair-play 卡组写推理小说,告诉我,我可以按那套流程给你搭 story 结构。", + "reason": { + "kind": "completed" + } + } + ] +} \ No newline at end of file diff --git a/test/injected-assets.test.js b/test/injected-assets.test.js new file mode 100644 index 0000000..890223e --- /dev/null +++ b/test/injected-assets.test.js @@ -0,0 +1,28 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; + +import { unreportedOf, withInjected, withReported } from '../src/injected-assets.js'; + +const DAY = 24 * 60 * 60 * 1000; + +test('an injection keeps its turn and strategy until reported', () => { + const ledger = withInjected({}, 'sha256:a', { turn: 2, strategyText: '1. Step' }, 0); + assert.deepEqual(unreportedOf(ledger, 2), [{ assetId: 'sha256:a', strategyText: '1. Step' }]); + assert.deepEqual(unreportedOf(withReported(ledger, ['sha256:a'], 1), 2), []); +}); + +test('an asset is recorded once per session', () => { + const first = withInjected({}, 'sha256:a', { turn: 1, strategyText: 'one' }, 0); + assert.equal(withInjected(first, 'sha256:a', { turn: 3, strategyText: 'two' }, 1)['sha256:a'].turn, 1); +}); + +test('a strategy that lands after its turn ended is reported at the next turn end', () => { + const ledger = withInjected({}, 'sha256:a', { turn: 2, strategyText: '' }, 0); + assert.deepEqual(unreportedOf(ledger, 1), []); + assert.equal(unreportedOf(ledger, 3).length, 1); +}); + +test('entries older than a week are pruned', () => { + const ledger = withInjected({}, 'sha256:old', { turn: 1, strategyText: '' }, 0); + assert.deepEqual(Object.keys(withInjected(ledger, 'sha256:new', { turn: 2, strategyText: '' }, 8 * DAY)), ['sha256:new']); +}); diff --git a/test/prime.test.js b/test/prime.test.js index eb625ad..57c0863 100644 --- a/test/prime.test.js +++ b/test/prime.test.js @@ -32,6 +32,14 @@ test('the prompt text comes from the user, not from dsh\'s own injections', () = assert.equal(text, 'add a retry to the uploader'); }); +test('keys and tokens in the prompt never leave the machine', () => { + const text = promptTextOf([ + { content: [{ type: 'text', text: 'deploy with api_key=sk-live-abcdefghijklmnop1234 and Bearer abcdefghijklmnopqrstuvwxyz' }], source: { kind: 'user' } }, + ]); + assert.doesNotMatch(text, /abcdefghijklmnop1234|abcdefghijklmnopqrstuvwxyz/); + assert.match(text, /^deploy with /); +}); + test('one call recalls by text and injects that asset\'s strategy alone', async () => { const { proxyFetch, calls } = stubProxy({ assets: [ASSET] }); const controller = new AbortController(); diff --git a/test/report-outbox.test.js b/test/report-outbox.test.js new file mode 100644 index 0000000..61e5ff1 --- /dev/null +++ b/test/report-outbox.test.js @@ -0,0 +1,44 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; + +import { OUTBOX_TTL_MS, isSettled, withAttempt, withQueued } from '../src/report-outbox.js'; + +const report = (taskId, assetId = 'sha256:a') => ({ asset_id: assetId, task_id: taskId, outcome: 'success', evidence: { tier: 'L3' } }); + +test('the same report is queued once, keyed by task id and asset', () => { + const once = withQueued({}, [{ report: report('t:1:correction'), sessionId: 's' }], 0); + const twice = withQueued(withAttempt(once, Object.keys(once)[0], { verdict: 'sent' }, 1), [{ report: report('t:1:correction'), sessionId: 's' }], 2); + assert.equal(Object.keys(twice).length, 1); + assert.equal(Object.values(twice)[0].sentAt, 1); + assert.equal(Object.keys(withQueued(twice, [{ report: report('t:1:correction', 'sha256:b'), sessionId: 's' }], 3)).length, 2); +}); + +test('reports older than eight days are dropped', () => { + const queued = withQueued({}, [{ report: report('t:1'), sessionId: 's' }], 0); + assert.deepEqual(withQueued(queued, [], OUTBOX_TTL_MS + 1), {}); +}); + +test('each delivery attempt is counted', () => { + const queued = withQueued({}, [{ report: report('t:1'), sessionId: 's' }], 0); + const key = Object.keys(queued)[0]; + assert.equal(withAttempt(withAttempt(queued, key, { verdict: 'retry' }, 1), key, { verdict: 'retry' }, 2)[key].attempts, 2); + assert.equal(withAttempt(queued, 'missing', { verdict: 'sent' }, 1), queued); +}); + +test('a rejected report is settled with its reason and never marked sent', () => { + const queued = withQueued({}, [{ report: report('t:1'), sessionId: 's' }], 0); + const key = Object.keys(queued)[0]; + const rejected = withAttempt(queued, key, { verdict: 'rejected', reason: 'hub 400', status: 400 }, 5)[key]; + assert.equal(rejected.sentAt, undefined); + assert.equal(rejected.rejectedAt, 5); + assert.deepEqual(rejected.rejection, { reason: 'hub 400', status: 400 }); + assert.equal(isSettled(rejected), true); + assert.equal(isSettled(withAttempt(queued, key, { verdict: 'retry' }, 5)[key]), false); +}); + +test('the local record is noted once it is written', () => { + const queued = withQueued({}, [{ report: report('t:1'), sessionId: 's' }], 0); + const key = Object.keys(queued)[0]; + assert.equal(withAttempt(queued, key, { verdict: 'retry', isLocal: true }, 3)[key].localAt, 3); + assert.equal('localAt' in withAttempt(queued, key, { verdict: 'retry', isLocal: false }, 3)[key], false); +}); diff --git a/test/reporting.test.js b/test/reporting.test.js new file mode 100644 index 0000000..a1dc77f --- /dev/null +++ b/test/reporting.test.js @@ -0,0 +1,352 @@ +import assert from 'node:assert/strict'; +import { existsSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { test } from 'node:test'; + +import { Config, apply } from '../src/index.js'; +import { + fakeAgent, fakeContext, freshHome, primedBy, projectDirectory, startProxy, untilCount, untilInjected, untilRequest, +} from './support/host.js'; + +const SESSION = JSON.parse(readFileSync(new URL('./fixtures/fair-play-session.json', import.meta.url), 'utf8')); +const STEPS = ['Reuse this.', 'Then the next step.', 'Then the next step.', 'Then the next step.']; + +function assetFromInjection(strategyText) { + const lines = strategyText.split('\n'); + return { + asset_id: /for (sha256:[0-9a-f]+)\.$/.exec(lines.at(-1))[1], + short_title: /^\[Evolution Memory\] (.*) \(EvoMap network\):$/.exec(lines[0])[1], + strategy: lines.filter((line) => /^\d+\. /.test(line)).map((line) => line.replace(/^\d+\. /, '')), + }; +} + +function hubAnswering({ assets, reuse = () => ({ recorded: true, eligible: true, counted: true, weight: 0.6 }) }) { + return (path, body) => { + if (path === '/asset/fetch') return { assets: assets(body.text) }; + if (path === '/asset/reuse-result') return reuse(body); + return {}; + }; +} + +const oneAsset = (assetId, strategy = STEPS) => hubAnswering({ assets: () => [{ asset_id: assetId, strategy }] }); +const reportsIn = (requests) => requests.filter((request) => request.path === '/asset/reuse-result').map((request) => request.body); + +async function runTool(listeners, agent, { name, arguments: args, result, sideEffect }) { + return listeners.get('tools/execute')({ name, agent, arguments: args }, async () => { + sideEffect?.(); + return result; + }); +} + +function answer(listeners, agent, turn, text) { + listeners.get('session/event')(agent.session, { + type: 'assistant/message', + data: { turn, step: 1, message: { role: 'assistant', content: [{ type: 'text', text }] } }, + }); +} + +function endTurn(listeners, agent, turn, kind = 'completed') { + listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn, reason: { kind } } }); +} + +const bashResult = (text) => ({ isError: false, content: [{ type: 'text', text }] }); + +async function harness(t, prefix, answerOf) { + t.after(freshHome(`${prefix}-home-`)); + const proxy = await startProxy(answerOf); + t.after(() => proxy.close()); + const projectDir = projectDirectory(`${prefix}-`); + const { ctx, listeners, registered } = fakeContext(); + apply(ctx, Config({ projectDir, proxyPort: proxy.port })); + return { proxy, projectDir, listeners, registered }; +} + +test('a recorded dsh session reports each turn with the evidence it left', async (t) => { + const byPrompt = new Map(SESSION.turns.map((turn) => [turn.prompt, assetFromInjection(turn.strategyText)])); + const { proxy, listeners } = await harness(t, 'evolver-replay', hubAnswering({ assets: (text) => [byPrompt.get(text)] })); + const { agent, injected } = fakeAgent({ sessionId: 'session-fair-play' }); + + for (const turn of SESSION.turns) { + const [strategy] = await primedBy(listeners, agent, turn.prompt, turn.turn); + assert.equal(strategy.content[0].text.split('\n')[1], turn.strategyText.split('\n')[1]); + for (const tool of turn.tools) await runTool(listeners, agent, tool); + answer(listeners, agent, turn.turn, turn.answer); + endTurn(listeners, agent, turn.turn, turn.reason.kind); + await untilCount(proxy.requests, (request) => request.path === '/asset/reuse-result', turn.turn); + } + + const [built, explained] = reportsIn(proxy.requests); + assert.deepEqual(built, { + asset_id: 'sha256:872ad8edfa0d86d74140e386528acc4aee7812c946b9e3200c4464468586d0ea', + outcome: 'success', + task_id: 'dsh:session-fair-play:1', + reason: 'Injected by Evolver into dsh turn 1; applied: used; last test passed.', + evidence: { tier: 'L1', applied: 'used', check: { kind: 'test', passed: true, fail_then_pass: false }, aborted: false, stage: 'immediate' }, + }); + assert.deepEqual(explained, { + asset_id: 'sha256:ed3b8595561c58c86c0c630e1c77f698a8a5b3e0e5d1aabd6acce1a8fa3a3750', + outcome: 'success', + task_id: 'dsh:session-fair-play:2', + reason: 'Injected by Evolver into dsh turn 2; applied: unknown; no check ran.', + evidence: { tier: 'none', applied: 'partial', aborted: false, stage: 'immediate' }, + }); + assert.deepEqual( + injected.map((message) => message.content[0].text), + ['[Evolution Signal] Detected: [user_feature_request] in /home/dev/fair-play/README.md.'], + ); + + await primedBy(listeners, agent, '不对,我问的是推理小说里的意思', 3); + await untilCount(proxy.requests, (request) => request.path === '/asset/reuse-result', 3); + assert.deepEqual(reportsIn(proxy.requests)[2], { + asset_id: 'sha256:ed3b8595561c58c86c0c630e1c77f698a8a5b3e0e5d1aabd6acce1a8fa3a3750', + outcome: 'failed', + task_id: 'dsh:session-fair-play:2:correction', + reason: "The person's next prompt read as correction of the turn that reused this strategy.", + evidence: { tier: 'L3', applied: 'partial', stage: 'correction' }, + }); +}); + +test('the same check failing and then passing behind a different tail is a turnaround', async (t) => { + const { proxy, listeners } = await harness(t, 'evolver-turnaround', oneAsset('sha256:lru', [ + 'Use a Map for O(1) key lookups', 'Move the accessed node to the head on get', 'Evict the tail node when capacity is exceeded', 'Track hits for a stats method', + ])); + const { agent } = fakeAgent({ sessionId: 'session-turnaround' }); + await primedBy(listeners, agent, 'write an LRU cache with stats', 1); + await runTool(listeners, agent, { name: 'write', arguments: { file_path: '/w/lru.js', content: 'const map = new Map(); // evict tail node, move node to head, stats hits' } }); + await runTool(listeners, agent, { name: 'bash', arguments: { command: 'cd /w && node --test 2>&1 | tail -25' }, result: bashResult('ℹ pass 3\nℹ fail 1\n✖ evicts the tail\n AssertionError: expected 2 to equal 1') }); + await runTool(listeners, agent, { name: 'bash', arguments: { command: 'cd /w && node --test 2>&1 | tail -12' }, result: bashResult('ℹ tests 4\nℹ pass 4\nℹ fail 0') }); + endTurn(listeners, agent, 1); + await untilRequest(proxy.requests, '/asset/reuse-result'); + + const [report] = reportsIn(proxy.requests); + assert.deepEqual(report.evidence.check, { kind: 'test', passed: true, fail_then_pass: true }); + assert.equal(report.evidence.tier, 'L1'); + assert.match(report.reason, /last test passed after failing earlier/); +}); + +test('an interrupted turn whose last check failed reports the failure and the abort', async (t) => { + const { proxy, listeners } = await harness(t, 'evolver-aborted', oneAsset('sha256:aborted')); + const { agent } = fakeAgent({ sessionId: 'session-aborted' }); + await primedBy(listeners, agent, 'add a retry to the uploader', 1); + await runTool(listeners, agent, { name: 'bash', arguments: { command: 'npm test' }, result: { isError: true, content: [{ type: 'text', text: 'exit 1' }] } }); + endTurn(listeners, agent, 1, 'aborted'); + await untilRequest(proxy.requests, '/asset/reuse-result'); + + const [report] = reportsIn(proxy.requests); + assert.equal(report.outcome, 'failed'); + assert.deepEqual(report.evidence, { + tier: 'L1', applied: 'unused', check: { kind: 'test', passed: false, fail_then_pass: false }, aborted: true, stage: 'immediate', + }); +}); + +test('a turn that did nothing the strategy names reports it unused', async (t) => { + const { proxy, listeners } = await harness(t, 'evolver-unused', oneAsset('sha256:unused', [ + 'Configure the Redis client with exponential reconnect backoff', 'Wrap the client in a singleton factory module', 'Close the connection in the global teardown hook', 'Assert reconnection inside the integration suite', + ])); + const { agent } = fakeAgent({ sessionId: 'session-unused' }); + await primedBy(listeners, agent, 'make the flaky redis tests stable', 1); + await runTool(listeners, agent, { name: 'write', arguments: { file_path: '/w/greeting.js', content: 'export const greeting = "hello";\n' } }); + await runTool(listeners, agent, { name: 'bash', arguments: { command: 'npm test' }, result: bashResult('ℹ pass 1\nℹ fail 0') }); + endTurn(listeners, agent, 1); + await untilRequest(proxy.requests, '/asset/reuse-result'); + + assert.deepEqual(reportsIn(proxy.requests)[0].evidence, { + tier: 'L1', applied: 'unused', check: { kind: 'test', passed: true, fail_then_pass: false }, aborted: false, stage: 'immediate', + }); +}); + +test('a follow-up prompt labels the turn once: approval, correction, or nothing', async (t) => { + const { proxy, listeners } = await harness(t, 'evolver-follow-up', oneAsset('sha256:kept')); + const { agent } = fakeAgent({ sessionId: 'session-follow-up' }); + const closeTurn = async (turn, prompt) => { + await primedBy(listeners, agent, prompt, turn); + endTurn(listeners, agent, turn); + }; + + await closeTurn(1, 'add a retry to the uploader'); + await untilCount(proxy.requests, (request) => request.path === '/asset/reuse-result', 1); + await closeTurn(2, '好了,谢谢'); + await untilCount(proxy.requests, (request) => request.path === '/asset/reuse-result', 2); + await closeTurn(3, '还是不行,报一样的错'); + await new Promise((resolve) => { setTimeout(resolve, 200); }); + + const reports = reportsIn(proxy.requests); + assert.deepEqual(reports.map((report) => [report.task_id, report.outcome, report.evidence.stage]), [ + ['dsh:session-follow-up:1', 'success', 'immediate'], + ['dsh:session-follow-up:1:correction', 'success', 'correction'], + ], 'turn 2 had no injection of its own, so the complaint after it revises nothing'); +}); + +test('an ordinary next prompt sends no follow-up', async (t) => { + const { proxy, listeners } = await harness(t, 'evolver-no-label', oneAsset('sha256:plain')); + const { agent } = fakeAgent({ sessionId: 'session-no-label' }); + await primedBy(listeners, agent, 'add a retry to the uploader', 1); + endTurn(listeners, agent, 1); + await untilRequest(proxy.requests, '/asset/reuse-result'); + await primedBy(listeners, agent, '这样行不行?再帮我加一个功能', 2); + await new Promise((resolve) => { setTimeout(resolve, 150); }); + assert.equal(reportsIn(proxy.requests).length, 1); +}); + +for (const [verdict, change, outcome] of [ + ['kept', () => {}, 'success'], + ['reverted', (file) => rmSync(file), 'failed'], +]) { + test(`edits the next session finds ${verdict} send a retention follow-up`, async (t) => { + const { proxy, projectDir, listeners } = await harness(t, `evolver-retention-${verdict}`, oneAsset('sha256:retained')); + const file = join(projectDir, 'uploader.js'); + const first = fakeAgent({ sessionId: `session-${verdict}-a`, cwd: projectDir }).agent; + await primedBy(listeners, first, 'add a retry to the uploader', 1); + await runTool(listeners, first, { + name: 'write', + arguments: { file_path: 'uploader.js', content: 'export const retry = 3;\n' }, + result: { isError: false, content: [] }, + sideEffect: () => writeFileSync(file, 'export const retry = 3;\n'), + }); + endTurn(listeners, first, 1); + await untilRequest(proxy.requests, '/asset/reuse-result'); + + change(file); + listeners.get('agent/session-start')({ agent: fakeAgent({ sessionId: 'session-elsewhere', cwd: projectDirectory('evolver-other-') }).agent, source: 'startup' }); + listeners.get('agent/session-start')({ agent: first, source: 'resume' }); + await new Promise((resolve) => { setTimeout(resolve, 100); }); + assert.equal(reportsIn(proxy.requests).length, 1, 'only a different session in the same project judges retention'); + + listeners.get('agent/session-start')({ agent: fakeAgent({ sessionId: `session-${verdict}-b`, cwd: projectDir }).agent, source: 'startup' }); + await untilCount(proxy.requests, (request) => request.path === '/asset/reuse-result', 2); + assert.deepEqual(reportsIn(proxy.requests)[1], { + asset_id: 'sha256:retained', + outcome, + task_id: `dsh:session-${verdict}-a:1:retention`, + reason: `By the next session the files the turn edited were ${verdict}.`, + evidence: { tier: 'L2', applied: 'unused', stage: 'retention' }, + }); + + listeners.get('agent/session-start')({ agent: fakeAgent({ sessionId: `session-${verdict}-c`, cwd: projectDir }).agent, source: 'startup' }); + await new Promise((resolve) => { setTimeout(resolve, 100); }); + assert.equal(reportsIn(proxy.requests).length, 2, 'retention is judged once'); + }); +} + +test('a report the Hub refuses is settled, shown in evolver_status and never resent', async (t) => { + const refuse = () => ({ recorded: false, terminal: true, reason: 'invalid_evidence', status: 400 }); + const { proxy, listeners, registered } = await harness(t, 'evolver-refused', hubAnswering({ assets: () => [{ asset_id: 'sha256:refused', strategy: STEPS }], reuse: refuse })); + const { agent } = fakeAgent({ sessionId: 'session-refused' }); + await primedBy(listeners, agent, 'add a retry to the uploader', 1); + endTurn(listeners, agent, 1); + await untilRequest(proxy.requests, '/asset/reuse-result'); + endTurn(listeners, agent, 2); + await new Promise((resolve) => { setTimeout(resolve, 150); }); + assert.equal(reportsIn(proxy.requests).length, 1); + + const status = await registered.tools.find((tool) => tool.name === 'evolver_status').execute({}, {}); + assert.deepEqual(status.reuse_reports.rejected.map(({ task_id: taskId, reason, status: code }) => ({ taskId, reason, code })), [ + { taskId: 'dsh:session-refused:1', reason: 'invalid_evidence', code: 400 }, + ]); + const outbox = JSON.parse(readFileSync(join(process.env.HOME, '.evolver', 'state', 'reports-outbox.json'), 'utf8')); + assert.deepEqual(Object.values(outbox)[0].rejection, { reason: 'invalid_evidence', status: 400 }); +}); + +test('a report the Hub could not take yet is resent at the next turn end and written locally once', async (t) => { + let answers = 0; + const { proxy, listeners, registered } = await harness(t, 'evolver-retried', hubAnswering({ + assets: () => [{ asset_id: 'sha256:retried', strategy: STEPS }], + reuse: () => (answers++ === 0 ? { recorded: false, reason: 'hub 404' } : { recorded: true, eligible: true, counted: true, weight: 0.6 }), + })); + const { agent } = fakeAgent({ sessionId: 'session-retried' }); + await primedBy(listeners, agent, 'add a retry to the uploader', 1); + await runTool(listeners, agent, { name: 'edit', arguments: { file_path: '/w/uploader.js', new_string: 'Reuse this. Then the next step.' } }); + await runTool(listeners, agent, { name: 'bash', arguments: { command: 'npm test' }, result: bashResult('ℹ pass 2\nℹ fail 0') }); + endTurn(listeners, agent, 1); + await untilRequest(proxy.requests, '/asset/reuse-result'); + await new Promise((resolve) => { setTimeout(resolve, 150); }); + endTurn(listeners, agent, 2); + await untilCount(proxy.requests, (request) => request.path === '/asset/reuse-result', 2); + await new Promise((resolve) => { setTimeout(resolve, 150); }); + + const [first, second] = reportsIn(proxy.requests); + assert.deepEqual(second, first); + const status = await registered.tools.find((tool) => tool.name === 'evolver_status').execute({}, {}); + assert.deepEqual(status.reuse_reports.delivered.map(({ eligible, counted, weight }) => ({ eligible, counted, weight })), [ + { eligible: true, counted: true, weight: 0.6 }, + ]); + + const log = join(process.env.HOME, '.evomap', 'evolution', 'root_events.jsonl'); + const events = readFileSync(log, 'utf8').trim().split('\n').map((line) => JSON.parse(line)); + assert.deepEqual(events.map((event) => [event.type, event.payload.assetId, event.payload.cycleId]), [ + ['value.reuse_hit', 'sha256:retried', 'dsh:session-retried'], + ]); +}); + +test('a report with no evidence never reaches the local ledger', async (t) => { + const { proxy, listeners } = await harness(t, 'evolver-exposure', oneAsset('sha256:exposure')); + const { agent } = fakeAgent({ sessionId: 'session-exposure' }); + await primedBy(listeners, agent, 'add a retry to the uploader', 1); + endTurn(listeners, agent, 1); + await untilRequest(proxy.requests, '/asset/reuse-result'); + await new Promise((resolve) => { setTimeout(resolve, 150); }); + assert.equal(existsSync(join(process.env.HOME, '.evomap', 'evolution', 'root_events.jsonl')), false); +}); + +test("the model's own report carries L5 evidence, passes the Hub verdict through, and leaves the automatic one in place", async (t) => { + const { proxy, listeners, registered } = await harness(t, 'evolver-self-report', hubAnswering({ + assets: () => [{ asset_id: 'sha256:owned', strategy: STEPS }], + reuse: (body) => ({ recorded: true, eligible: true, counted: body.evidence.tier !== 'L5', weight: body.evidence.tier === 'L5' ? 0.1 : 0 }), + })); + const { agent } = fakeAgent({ sessionId: 'session-owned' }); + await primedBy(listeners, agent, 'add a retry to the uploader', 1); + + const tool = registered.tools.find((candidate) => candidate.name === 'evolver_asset_reuse_result'); + const receipt = await tool.execute({ asset_id: 'sha256:owned', outcome: 'mismatched', reason: 'not a fit' }, {}); + assert.deepEqual(receipt, { recorded: true, eligible: true, counted: false, weight: 0.1 }); + endTurn(listeners, agent, 1); + await untilCount(proxy.requests, (request) => request.path === '/asset/reuse-result', 2); + + const [own, automatic] = reportsIn(proxy.requests); + assert.deepEqual(own.evidence, { tier: 'L5', applied: 'unused', stage: 'immediate' }); + assert.equal(automatic.task_id, 'dsh:session-owned:1'); + assert.equal(automatic.evidence.tier, 'none'); +}); + +test('an injected asset is reported from disk after the process that injected it is gone', async (t) => { + const { proxy, projectDir, listeners } = await harness(t, 'evolver-durable', oneAsset('sha256:durable')); + const { agent } = fakeAgent({ sessionId: 'session-durable', cwd: projectDir }); + await primedBy(listeners, agent, 'add a retry to the uploader', 1); + + const restarted = fakeContext(); + apply(restarted.ctx, Config({ projectDir, proxyPort: proxy.port })); + restarted.listeners.get('session/event')(agent.session, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); + await untilRequest(proxy.requests, '/asset/reuse-result'); + assert.equal(reportsIn(proxy.requests)[0].asset_id, 'sha256:durable'); +}); + +test('a strategy that lands after its turn ended is reported with the turn that saw it', async (t) => { + t.after(freshHome('evolver-late-home-')); + const proxy = await startProxy(oneAsset('sha256:late'), { delayMs: (path) => (path === '/asset/fetch' ? 150 : 0) }); + t.after(() => proxy.close()); + const { ctx, listeners } = fakeContext(); + apply(ctx, Config({ projectDir: projectDirectory('evolver-late-'), proxyPort: proxy.port, assetPrimeWaitMs: 20 })); + const { agent, injected } = fakeAgent({ sessionId: 'session-late' }); + + assert.deepEqual(await primedBy(listeners, agent, 'add a retry to the uploader', 1), []); + endTurn(listeners, agent, 1); + await untilInjected(injected, 1); + await new Promise((resolve) => { setTimeout(resolve, 100); }); + assert.deepEqual(reportsIn(proxy.requests), []); + + endTurn(listeners, agent, 2); + await untilRequest(proxy.requests, '/asset/reuse-result'); + assert.equal(reportsIn(proxy.requests)[0].task_id, 'dsh:session-late:2'); +}); + +test('subagent turns and synthesized turn ends report nothing', async (t) => { + const { proxy, listeners } = await harness(t, 'evolver-subagent', oneAsset('sha256:child')); + const child = { id: 'session-child', header: { origin: 'subagent' } }; + const { agent } = fakeAgent({ sessionId: 'session-parent' }); + await primedBy(listeners, { ...agent, session: child }, 'add a retry to the uploader', 1); + await primedBy(listeners, agent, 'add a retry to the uploader', 1); + listeners.get('session/event')(child, { type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }); + endTurn(listeners, agent, 1, 'interrupted'); + await new Promise((resolve) => { setTimeout(resolve, 150); }); + assert.deepEqual(reportsIn(proxy.requests), []); +}); diff --git a/test/retention.test.js b/test/retention.test.js new file mode 100644 index 0000000..a8819c5 --- /dev/null +++ b/test/retention.test.js @@ -0,0 +1,25 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; + +import { ABSENT, retentionOf } from '../src/retention.js'; + +test('files still as the turn left them are kept', () => { + assert.deepEqual(retentionOf([{ before: 'a', after: 'b', now: 'b' }, { before: ABSENT, after: 'c', now: 'c' }]), { class: 'kept', files: 2 }); +}); + +test('files back to how they stood before the turn are reverted', () => { + assert.deepEqual(retentionOf([{ before: 'a', after: 'b', now: 'a' }, { before: ABSENT, after: 'c', now: ABSENT }]), { class: 'reverted', files: 2 }); +}); + +test('files changed again since the turn are rewritten', () => { + assert.equal(retentionOf([{ before: 'a', after: 'b', now: 'z' }]).class, 'rewritten'); +}); + +test('a mix of kept and reverted files counts as rewritten', () => { + assert.equal(retentionOf([{ before: 'a', after: 'b', now: 'b' }, { before: 'c', after: 'd', now: 'c' }]).class, 'rewritten'); +}); + +test('edits that netted out to nothing are not judged', () => { + assert.deepEqual(retentionOf([{ before: 'a', after: 'a', now: 'z' }]), { class: 'no_change', files: 0 }); + assert.deepEqual(retentionOf([]), { class: 'no_change', files: 0 }); +}); diff --git a/test/reuse.test.js b/test/reuse.test.js new file mode 100644 index 0000000..c56e3f2 --- /dev/null +++ b/test/reuse.test.js @@ -0,0 +1,241 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; + +import { withQueued } from '../src/report-outbox.js'; +import { + correctionReportOf, + deliverPending, + deliveryOf, + immediateReportOf, + reportedReuseOf, + retentionReportOf, + selfReportedEvidenceOf, +} from '../src/reuse.js'; + +const ASSET = { assetId: 'sha256:a', strategyText: '' }; +const NOW = 1_800_000_000_000; + +function ledgers({ hub = { ok: true, data: {} }, local = true } = {}) { + const posts = []; + const locals = []; + const settled = []; + return { + posts, + locals, + proxyFetch: async (method, path, body) => { + posts.push({ method, path, body }); + if (hub instanceof Error) throw hub; + return hub; + }, + get rejections() { return settled.filter((settlement) => settlement.verdict === 'rejected'); }, + settled, + onSettled: (settlement) => { settled.push(settlement); }, + recordLocally: async (request) => { + locals.push(request); + if (local instanceof Error) throw local; + return local; + }, + }; +} + +const immediate = (overrides = {}) => immediateReportOf({ asset: ASSET, sessionId: 's1', turn: 4, reasonKind: 'completed', checks: [], applied: 'used', ...overrides }); + +test('a turn that ran no check still reports its injection, as tier none', () => { + assert.deepEqual(immediate(), { + asset_id: 'sha256:a', + outcome: 'success', + task_id: 'dsh:s1:4', + reason: 'Injected by Evolver into dsh turn 4; applied: used; no check ran.', + evidence: { tier: 'none', applied: 'used', aborted: false, stage: 'immediate' }, + }); + assert.equal(immediate({ reasonKind: 'error' }).outcome, 'failed'); +}); + +test('a check the turn ran makes the report L1 and decides its outcome', () => { + const passed = immediate({ checks: [{ command: 'npm test', failed: false }] }); + assert.equal(passed.outcome, 'success'); + assert.deepEqual(passed.evidence.check, { kind: 'test', passed: true, fail_then_pass: false }); + assert.equal(passed.evidence.tier, 'L1'); + + const failed = immediate({ checks: [{ command: 'npx tsc -p .', failed: true }], applied: 'partial' }); + assert.equal(failed.outcome, 'failed'); + assert.deepEqual(failed.evidence.check, { kind: 'typecheck', passed: false, fail_then_pass: false }); + assert.equal(failed.evidence.applied, 'partial'); +}); + +test('the same check failing and then passing is recorded as such', () => { + const report = immediate({ + checks: [{ command: 'npm test', failed: true }, { command: 'npm run lint', failed: false }, { command: 'npm test', failed: false }], + }); + assert.deepEqual(report.evidence.check, { kind: 'test', passed: true, fail_then_pass: true }); + assert.match(report.reason, /last test passed after failing earlier/); +}); + +test('an interrupted turn says so', () => { + const report = immediate({ reasonKind: 'aborted' }); + assert.equal(report.evidence.aborted, true); + assert.equal(report.outcome, 'failed'); +}); + +test('a report reveals no command, prompt or code', () => { + const report = immediate({ checks: [{ command: 'API_KEY=hunter22 npm test -- secret/path.js', failed: false }] }); + assert.doesNotMatch(JSON.stringify(report), /hunter22|secret\/path|npm test/); +}); + +test('a correction or approval follow-up is L3 under the turn task id plus :correction', () => { + const reuse = reportedReuseOf(immediate()); + assert.deepEqual(correctionReportOf(reuse, 'correction'), { + asset_id: 'sha256:a', + outcome: 'failed', + task_id: 'dsh:s1:4:correction', + reason: "The person's next prompt read as correction of the turn that reused this strategy.", + evidence: { tier: 'L3', applied: 'used', stage: 'correction' }, + }); + assert.equal(correctionReportOf(reuse, 'praise').outcome, 'success'); + assert.equal(correctionReportOf(reuse, null), null); +}); + +test('kept or reverted edits are an L2 follow-up under :retention; anything else sends nothing', () => { + const reuse = reportedReuseOf(immediate({ applied: 'partial' })); + const kept = retentionReportOf(reuse, 'kept'); + assert.equal(kept.task_id, 'dsh:s1:4:retention'); + assert.equal(kept.outcome, 'success'); + assert.deepEqual(kept.evidence, { tier: 'L2', applied: 'partial', stage: 'retention' }); + assert.equal(retentionReportOf(reuse, 'reverted').outcome, 'failed'); + assert.equal(retentionReportOf(reuse, 'rewritten'), null); + assert.equal(retentionReportOf(reuse, 'no_change'), null); +}); + +test('queued reports go to the Hub until one is taken, then never again', async () => { + const report = immediate({ checks: [{ command: 'npm test', failed: false }] }); + const down = ledgers({ hub: { ok: false, error: 'down' } }); + const outbox = withQueued({}, [{ report, sessionId: 's1' }], NOW); + const retried = await deliverPending(down, outbox, NOW); + assert.equal(down.posts[0].path, '/asset/reuse-result'); + assert.deepEqual(down.posts[0].body, report); + assert.equal(Object.values(retried)[0].sentAt, undefined); + + const up = ledgers(); + const sent = await deliverPending(up, withQueued(retried, [{ report, sessionId: 's1' }], NOW + 1), NOW + 1); + assert.equal(up.posts.length, 1); + assert.equal(Object.values(sent)[0].sentAt, NOW + 1); + + const after = ledgers(); + await deliverPending(after, sent, NOW + 2); + assert.equal(after.posts.length, 0); +}); + +test('a Proxy that answers without recording does not count as delivered', async () => { + const sinks = ledgers({ hub: { ok: true, data: { recorded: false } } }); + const outbox = await deliverPending(sinks, withQueued({}, [{ report: immediate(), sessionId: 's1' }], NOW), NOW); + assert.equal(Object.values(outbox)[0].sentAt, undefined); + assert.equal(Object.values(outbox)[0].attempts, 1); +}); + +test('evidence-bearing reports are recorded locally once; tier none never is', async () => { + const sinks = ledgers({ hub: new Error('down') }); + const verified = immediate({ checks: [{ command: 'npm test', failed: true }] }); + const outbox = withQueued({}, [{ report: verified, sessionId: 's1' }, { report: immediate({ turn: 5 }), sessionId: 's1' }], NOW); + await deliverPending(sinks, await deliverPending(sinks, outbox, NOW), NOW); + assert.deepEqual(sinks.locals, [{ assetId: 'sha256:a', outcome: 'failed', sessionId: 's1' }]); + assert.equal(sinks.posts.length, 4); +}); + +test('a report the Hub already holds counts as delivered and is not retried', async () => { + const sinks = ledgers({ hub: { ok: true, data: { recorded: false, reason: 'duplicate_report' } } }); + const outbox = await deliverPending(sinks, withQueued({}, [{ report: immediate(), sessionId: 's1' }], NOW), NOW); + assert.equal(Object.values(outbox)[0].sentAt, NOW); +}); + +for (const reason of ['daily_cap', 'same_owner', 'no_delivery_proof']) { + test(`a report the Hub judged ineligible (${reason}) is delivered and not retried`, async () => { + const sinks = ledgers({ hub: { ok: true, data: { recorded: true, eligible: false, reason } } }); + const outbox = await deliverPending(sinks, withQueued({}, [{ report: immediate(), sessionId: 's1' }], NOW), NOW); + assert.equal(Object.values(outbox)[0].sentAt, NOW); + await deliverPending(sinks, outbox, NOW + 1); + assert.equal(sinks.posts.length, 1); + }); +} + +const queuedOne = (report) => withQueued({}, [{ report, sessionId: 's1' }], NOW); +const verified = () => immediate({ checks: [{ command: 'npm test', failed: false }] }); + +for (const data of [ + { recorded: false, terminal: true, reason: 'evidence.tier: Invalid input', status: 400 }, + { recorded: false, reason: 'hub 400' }, + { recorded: false, reason: 'hub 422: unprocessable' }, +]) { + test(`a report the Hub refuses (${data.reason}) is settled, logged and never written locally`, async () => { + const sinks = ledgers({ hub: { ok: true, data } }); + const outbox = await deliverPending(sinks, queuedOne(verified()), NOW); + const [entry] = Object.values(outbox); + assert.equal(entry.rejectedAt, NOW); + assert.equal(entry.sentAt, undefined); + assert.equal(entry.rejection.reason, data.reason); + assert.deepEqual(sinks.locals, []); + assert.deepEqual(sinks.rejections.map(({ taskId, assetId, reason }) => ({ taskId, assetId, reason })), [ + { taskId: 'dsh:s1:4', assetId: 'sha256:a', reason: data.reason }, + ]); + + await deliverPending(sinks, outbox, NOW + 1); + assert.equal(sinks.posts.length, 1); + assert.equal(sinks.rejections.length, 1); + }); +} + +for (const hub of [ + { ok: true, data: { recorded: false, reason: 'hub 404' } }, + { ok: true, data: { recorded: false, reason: 'hub 500' } }, + { ok: true, data: { recorded: false, reason: 'hub 503' } }, + { ok: true, data: { recorded: false, reason: 'hub 429', status: 429 } }, + { ok: true, data: { recorded: false, reason: 'hub 401' } }, + { ok: false, error: 'timed out' }, + new Error('ECONNREFUSED'), +]) { + test(`a failure that may pass later (${hub.data?.reason ?? hub.error ?? hub.message}) is retried`, async () => { + const sinks = ledgers({ hub }); + const outbox = await deliverPending(sinks, queuedOne(verified()), NOW); + assert.equal(Object.values(outbox)[0].rejectedAt, undefined); + assert.equal(Object.values(outbox)[0].sentAt, undefined); + await deliverPending(sinks, outbox, NOW + 1); + assert.equal(sinks.posts.length, 2); + assert.deepEqual(sinks.rejections, []); + }); +} + +test('the status a newer Proxy passes through is kept with the rejection', () => { + assert.deepEqual(deliveryOf({ ok: true, data: { recorded: false, terminal: true, reason: 'bad evidence', status: 400 } }), { + verdict: 'rejected', reason: 'bad evidence', status: 400, + }); + assert.deepEqual(deliveryOf({ ok: true, data: { recorded: true, eligible: false, reason: 'daily_cap' } }), { + verdict: 'sent', receipt: { eligible: false, reason: 'daily_cap' }, + }); +}); + +test('a reuse that cannot be judged is exposure only, without the unused penalty', () => { + const report = immediate({ applied: 'unknown' }); + assert.deepEqual(report.evidence, { tier: 'none', applied: 'partial', aborted: false, stage: 'immediate' }); + assert.match(report.reason, /applied: unknown/); + assert.equal(immediate({ applied: 'unknown', checks: [{ command: 'npm test', failed: false }] }).evidence.tier, 'none'); +}); + +test('the Hub verdict on a delivered report is kept with it and passed on', async () => { + const sinks = ledgers({ hub: { ok: true, data: { recorded: true, eligible: true, counted: true, weight: 0.6, asset_id: 'sha256:a', reuse_stats: {} } } }); + const outbox = await deliverPending(sinks, queuedOne(verified()), NOW); + assert.deepEqual(Object.values(outbox)[0].receipt, { eligible: true, counted: true, weight: 0.6 }); + assert.deepEqual(sinks.settled, [{ taskId: 'dsh:s1:4', assetId: 'sha256:a', verdict: 'sent', receipt: { eligible: true, counted: true, weight: 0.6 } }]); +}); + +test('every report body holds exactly the fields the Hub contract names', () => { + const reuse = reportedReuseOf(verified()); + for (const report of [verified(), correctionReportOf(reuse, 'correction'), retentionReportOf(reuse, 'kept')]) { + assert.deepEqual(Object.keys(report).sort(), ['asset_id', 'evidence', 'outcome', 'reason', 'task_id']); + assert.ok(Object.keys(report.evidence).every((key) => ['tier', 'applied', 'check', 'aborted', 'stage'].includes(key))); + } + assert.deepEqual(Object.keys(verified().evidence.check).sort(), ['fail_then_pass', 'kind', 'passed']); +}); + +test("the model's own report is L5, and a mismatch counts as unused", () => { + assert.deepEqual(selfReportedEvidenceOf('success'), { tier: 'L5', applied: 'used', stage: 'immediate' }); + assert.equal(selfReportedEvidenceOf('mismatched').applied, 'unused'); +}); diff --git a/test/session-key.test.js b/test/session-key.test.js index 386c7e2..6816c18 100644 --- a/test/session-key.test.js +++ b/test/session-key.test.js @@ -9,6 +9,6 @@ test('agents and sessions resolve through one session identity seam', () => { assert.equal(sessionKeyOf(agent), 'session-1'); assert.equal(sessionKeyOf(session), 'session-1'); - assert.equal(sessionKeyOf('session-1'), 'session-1'); - assert.equal(sessionKeyOf({}, '/workspace'), 'workspace:/workspace'); + assert.equal(sessionKeyOf({}), null); + assert.equal(sessionKeyOf(undefined), null); }); diff --git a/test/support/host.js b/test/support/host.js new file mode 100644 index 0000000..f1a9c0c --- /dev/null +++ b/test/support/host.js @@ -0,0 +1,143 @@ +import assert from 'node:assert/strict'; +import { chmodSync, mkdtempSync, readFileSync, realpathSync, writeFileSync } from 'node:fs'; +import { createServer } from 'node:http'; +import { tmpdir } from 'node:os'; +import { delimiter, join } from 'node:path'; +import { after } from 'node:test'; + +export const missingClaimFile = join(mkdtempSync(join(tmpdir(), 'evolver-claim-')), 'missing'); +process.env.EVOLVER_CLAIM_URL_PATH = missingClaimFile; +export const sharedStateDir = mkdtempSync(join(tmpdir(), 'evolver-notice-state-')); +process.env.EVOLVER_SESSION_STATE_DIR = sharedStateDir; +const originalPath = process.env.PATH; + +export function evolverOnPath(version, { npmInstalls } = {}) { + const dir = mkdtempSync(join(tmpdir(), 'evolver-bin-')); + const log = join(dir, 'calls.log'); + const versionFile = join(dir, 'version'); + writeFileSync(log, ''); + writeFileSync(versionFile, version ?? ''); + writeFileSync(join(dir, 'evolver'), [ + '#!/bin/sh', + `echo "evolver $*" >> "${log}"`, + `v=$(cat "${versionFile}")`, + '[ -n "$v" ] || exit 127', + '[ "$1" = "--version" ] && echo "$v"', + 'exit 0', + ].join('\n')); + writeFileSync(join(dir, 'npm'), [ + '#!/bin/sh', + `echo "npm $*" >> "${log}"`, + npmInstalls ? `echo ${npmInstalls} > "${versionFile}"` : 'echo "EACCES: permission denied" >&2; exit 1', + ].join('\n')); + chmodSync(join(dir, 'evolver'), 0o755); + chmodSync(join(dir, 'npm'), 0o755); + process.env.PATH = `${dir}${delimiter}${originalPath}`; + return () => readFileSync(log, 'utf8').trim().split('\n').filter(Boolean); +} + +evolverOnPath('2.0.39'); +after(() => { + delete process.env.EVOLVER_CLAIM_URL_PATH; + delete process.env.EVOLVER_SESSION_STATE_DIR; + process.env.PATH = originalPath; +}); + +export function fakeContext() { + const registered = { tools: [], skills: [], commands: [] }; + const listeners = new Map(); + const ctx = { + tools: { register: (definition) => registered.tools.push(definition) }, + skills: { registerProvider: (create) => registered.skills.push(create({})) }, + commands: { register: (definition) => registered.commands.push(definition) }, + on: (event, listener) => { + listeners.set(event, listener); + return () => listeners.delete(event); + }, + inject: (_deps, callback) => callback(ctx), + }; + return { ctx, registered, listeners }; +} + +export function fakeAgent({ id = 'agent-test', sessionId = id, cwd } = {}) { + const injected = []; + const session = { id: sessionId, header: cwd ? { cwd } : {} }; + return { + agent: { id, session, inject: (message) => injected.push(message), followup: () => {} }, + injected, + }; +} + +export function userMessage(text) { + return { content: [{ type: 'text', text }], source: { kind: 'user' } }; +} + +export async function primedBy(listeners, agent, prompt = 'add a retry to the uploader', turn = 1, step = 1) { + const claimed = [userMessage(prompt)]; + const decision = await listeners.get('agent/pre-step')( + { agent, messages: claimed, turn, step, signal: new AbortController().signal }, + async () => ({ kind: 'enter', messages: claimed }), + ); + assert.equal(decision.messages[0], claimed[0]); + return decision.messages.slice(1); +} + +// A watchdog against a hang, not an assertion: each of these returns the moment +// its condition holds, so the budget only has to outlast the slowest runner. Two +// seconds did not -- the correction case takes ~2.0s on an idle laptop, and the +// dsh 0.1.5-rc.3 matrix leg failed on it while every other leg passed. +export async function untilCount(items, predicate, count, deadlineMs = 15_000) { + const start = Date.now(); + while (items.filter(predicate).length < count) { + if (Date.now() - start > deadlineMs) throw new Error(`only ${items.filter(predicate).length} of ${count} expected entries arrived`); + await new Promise((resolve) => { setTimeout(resolve, 10); }); + } + return items.filter(predicate); +} + +export async function untilRequest(requests, path, deadlineMs = 15_000) { + const start = Date.now(); + while (!requests.some((request) => request.path === path)) { + if (Date.now() - start > deadlineMs) throw new Error(`no ${path} request arrived`); + await new Promise((resolve) => { setTimeout(resolve, 10); }); + } + return requests; +} + +export async function untilInjected(injected, count, deadlineMs = 15_000) { + const start = Date.now(); + while (injected.length < count) { + if (Date.now() - start > deadlineMs) throw new Error(`only ${injected.length} of ${count} messages were injected`); + await new Promise((resolve) => { setTimeout(resolve, 10); }); + } + return injected; +} + +export function projectDirectory(prefix = 'evolver-') { + return realpathSync(mkdtempSync(join(tmpdir(), prefix))); +} + + +export async function startProxy(answer, { delayMs = () => 0 } = {}) { + const requests = []; + const server = createServer((request, response) => { + let body = ''; + request.on('data', (chunk) => { body += chunk; }); + request.on('end', () => { + const parsed = body ? JSON.parse(body) : undefined; + requests.push({ path: request.url, body: parsed }); + setTimeout(() => { + response.setHeader('Content-Type', 'application/json'); + response.end(JSON.stringify(answer(request.url, parsed, requests))); + }, delayMs(request.url)); + }); + }); + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); + return { requests, port: server.address().port, close: () => server.close() }; +} + +export function freshHome(prefix) { + const home = process.env.HOME; + process.env.HOME = mkdtempSync(join(tmpdir(), prefix)); + return () => { process.env.HOME = home; }; +} diff --git a/test/turn-outcomes.test.js b/test/turn-outcomes.test.js new file mode 100644 index 0000000..63ff0c3 --- /dev/null +++ b/test/turn-outcomes.test.js @@ -0,0 +1,10 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; + +import { outcomeOfReason } from '../src/turn-outcomes.js'; + +test('every live turn ending has an outcome, but a synthesized one does not', () => { + assert.equal(outcomeOfReason('completed'), 'success'); + for (const kind of ['error', 'aborted', 'blocked', 'max-tokens']) assert.equal(outcomeOfReason(kind), 'failed', kind); + assert.equal(outcomeOfReason('interrupted'), null); +}); diff --git a/test/verification.test.js b/test/verification.test.js new file mode 100644 index 0000000..658432b --- /dev/null +++ b/test/verification.test.js @@ -0,0 +1,40 @@ +import assert from 'node:assert/strict'; +import { test } from 'node:test'; + +import { checkCommandOf, isVerificationCommand, verificationFailed } from '../src/verification.js'; + +const output = (text, isError = false) => ({ isError, content: [{ type: 'text', text }] }); + +test('test, build, lint and type-check runs are verifications; other commands are not', () => { + for (const command of ['npm test', 'pnpm run build', 'yarn lint', 'node --test test/*.test.js', 'npx vitest run', 'go test ./...', + 'cargo clippy', 'npx -p typescript tsc -p .', 'make', 'cd app && npm run typecheck']) { + assert.equal(isVerificationCommand(command), true, command); + } + for (const command of ['ls -la', 'git status', 'cat package.json', 'npm install', 'echo test', 'node simulate.js']) { + assert.equal(isVerificationCommand(command), false, command); + } +}); + +test('an error result fails the verification', () => { + assert.equal(verificationFailed(output('', true)), true); +}); + +test('failure output fails it even when bash reported success', () => { + assert.equal(verificationFailed(output('AssertionError: expected 1 to equal 2\n1 test failed')), true); + assert.equal(verificationFailed(output('TypeError: cannot read properties of undefined')), true); + assert.equal(verificationFailed(output("ℹ pass 0\nℹ fail 1\n\n✖ failing tests:\n\ntest at test:1:1\n✖ test (59.9ms)\n 'test failed'\n")), true); +}); + +test('a passing summary that counts zero failures passes', () => { + assert.equal(verificationFailed(output('ℹ tests 78\nℹ pass 78\nℹ fail 0')), false); + assert.equal(verificationFailed(output('Tests: 0 failed, 12 passed')), false); + assert.equal(verificationFailed(output('failures: 0\nerrors: 0')), false); + assert.equal(verificationFailed(output('✔ Validation passed')), false); +}); + +test('only a check run through bash counts', () => { + assert.equal(checkCommandOf({ name: 'bash', arguments: { command: 'npm test' } }), 'npm test'); + assert.equal(checkCommandOf({ name: 'bash', arguments: { command: 'git status' } }), null); + assert.equal(checkCommandOf({ name: 'read', arguments: { command: 'npm test' } }), null); + assert.equal(checkCommandOf({ name: 'bash', arguments: {} }), null); +}); diff --git a/test/workspace-id-location.test.js b/test/workspace-id-location.test.js deleted file mode 100644 index 25aaa8a..0000000 --- a/test/workspace-id-location.test.js +++ /dev/null @@ -1,79 +0,0 @@ -import assert from 'node:assert/strict'; -import { test } from 'node:test'; -import { execFileSync } from 'node:child_process'; -import { existsSync, mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { realpathSync } from 'node:fs'; - -import { resolveWorkspaceId, workspaceIdPath } from '../src/workspace.js'; - -function gitRepo() { - const directory = realpathSync(mkdtempSync(join(tmpdir(), 'evolver-wsid-'))); - execFileSync('git', ['init', '--quiet'], { cwd: directory }); - return directory; -} - -function withHome(home, run) { - const previousHome = process.env.HOME; - const previousId = process.env.EVOLVER_WORKSPACE_ID; - delete process.env.EVOLVER_WORKSPACE_ID; - process.env.HOME = home; - try { - return run(); - } finally { - if (previousHome === undefined) delete process.env.HOME; - else process.env.HOME = previousHome; - if (previousId !== undefined) process.env.EVOLVER_WORKSPACE_ID = previousId; - } -} - -test('a new workspace id is minted under the home state directory, not in the repository', () => { - const repo = gitRepo(); - const home = realpathSync(mkdtempSync(join(tmpdir(), 'evolver-home-'))); - - // workspaceIdPath reads the home directory too, so it has to be resolved under the same HOME. - const { id, stored } = withHome(home, () => ({ - id: resolveWorkspaceId(repo), - stored: workspaceIdPath(repo), - })); - - assert.match(id, /^[0-9a-f]{32}$/); - assert.equal(existsSync(join(repo, '.evolver', 'workspace-id')), false, 'the repository stays clean'); - assert.equal(readFileSync(stored, 'utf8'), id); -}); - -test('an id minted by an earlier version is adopted, not replaced', () => { - const repo = gitRepo(); - const home = realpathSync(mkdtempSync(join(tmpdir(), 'evolver-home-'))); - const legacyDir = join(repo, '.evolver'); - mkdirSync(legacyDir, { recursive: true, mode: 0o700 }); - writeFileSync(join(legacyDir, 'workspace-id'), 'a'.repeat(32), { mode: 0o600 }); - - const { id, stored } = withHome(home, () => ({ - id: resolveWorkspaceId(repo), - stored: workspaceIdPath(repo), - })); - - // Minting a fresh id here would silently orphan every memory row this workspace already wrote. - assert.equal(id, 'a'.repeat(32)); - assert.equal(readFileSync(stored, 'utf8'), id, 'the legacy id is copied forward'); -}); - -test('the same workspace resolves to the same id across calls', () => { - const repo = gitRepo(); - const home = realpathSync(mkdtempSync(join(tmpdir(), 'evolver-home-'))); - - const first = withHome(home, () => resolveWorkspaceId(repo)); - const second = withHome(home, () => resolveWorkspaceId(repo)); - - assert.equal(first, second); -}); - -test('two workspaces never share an id', () => { - const home = realpathSync(mkdtempSync(join(tmpdir(), 'evolver-home-'))); - const first = withHome(home, () => resolveWorkspaceId(gitRepo())); - const second = withHome(home, () => resolveWorkspaceId(gitRepo())); - - assert.notEqual(first, second); -});