diff --git a/README.md b/README.md index 0066d40..b976627 100644 --- a/README.md +++ b/README.md @@ -20,9 +20,9 @@ Powered by the [Genome Evolution Protocol](https://evomap.ai) and | Seam | dsh event | Behaviour | | --- | --- | --- | -| Asset priming | `agent/pre-step`, once per turn | Searches the Proxy with that turn's own prompt — only the text the person typed — fetches the two leading matches that clear the similarity line and have a strategy, in one call, and injects the strategy of the first that actually came back, announced by the asset's short title rather than its hash, falling back to its description when the Hub's title is a truncated fragment. The hash appears once, on the line that asks for the outcome, because that call needs it. Ids the Hub declined to deliver are remembered for a day and skipped, so a slot is not spent twice on one. A lookup slower than the wait budget is injected into the next step instead of holding the current one. Assets already injected in this session are skipped. | +| Asset priming | `agent/pre-step`, once per turn | Recalls from the Proxy in a single `/asset/fetch` call carrying that turn's own prompt — only the text the person typed — and injects the strategy of the highest-scoring asset that came back carrying between four and eight steps — recall bypasses the Hub's own thin-gene filter, so both bounds are applied here — announced by the asset's short title rather than its hash, falling back to its description when the Hub's title is a truncated fragment. The hash appears once, on the line that asks for the outcome, because that call needs it. A lookup slower than the wait budget is injected into the next step instead of holding the current one. Assets already injected in this session are skipped. | | Signal detection | successful `tools/result` for `write`, `edit`, or `str_replace_editor` | Tags the turn with improvement signals and injects one bounded notice per file/signal set. | -| Reuse feedback | `session/event` → `turn/end` | Reports each strategy injected during that turn to the Proxy with the turn's outcome, marked as automatic and unconfirmed. Injected asset ids are written to `~/.evolver/state/` as they are injected and read back at report time, so a report survives the process dying in between. Reports are keyed on the asset id alone, and each verdict is written twice: to the Proxy, and as a `value.reuse_hit` / `value.reuse_outcome` root event in `~/.evomap/evolution/root_events.jsonl`. The local write is the one that closes the loop — the actuator that re-orders future candidates reads root events, not the Hub. An asset the model reported itself with `evolver_asset_reuse_result` is left alone. | +| Reuse feedback | `session/event` → `turn/end` | Reports each strategy injected during that turn to the Proxy with the turn's outcome, marked as automatic and unconfirmed. Injected asset ids are written to `~/.evolver/state/` as they are injected and read back at report time, so a report survives the process dying in between. Each report names the asset it is about and carries a per-turn id the Hub hashes with it to recognise a retry — a report without one is rejected outright — and a later correction carries its own, so it is not dismissed as that retry. Each verdict is written twice: to the Proxy, and as a `value.reuse_hit` / `value.reuse_outcome` root event in `~/.evomap/evolution/root_events.jsonl`. The local write is the one that closes the loop — the actuator that re-orders future candidates reads root events, not the Hub. An asset the model reported itself with `evolver_asset_reuse_result` is left alone. | | Verdict correction | `agent/pre-step` | When the prompt that opens a later turn in the same session plainly says the last answer did not hold, the verdicts already sent for that session are revised to `failed` — each asset at most once, since the Hub counts every report it receives. | | Capture | `session/event` → `turn/end` | Collects staged, unstaged, and untracked work asynchronously; records the real turn outcome once; drains at `session/flush`. | | Tools | `ctx.tools.register` | Connects directly to the local Evolver Proxy for status, search, fetch, reuse feedback, distillation, publication, and mailbox polling. | @@ -139,9 +139,9 @@ The plugin exports a validated Schemastery `Config`; invalid values fail at load | `proxyTimeoutMs` | `8000` | Native Proxy tool deadline. | | `hubTimeoutMs` | `8000` | Direct outcome-recording deadline. | | `assetPrimeEnabled` | `true` | Look one reusable strategy up per turn and inject it behind that turn's prompt. | -| `assetPrimeMinSimilarity` | `0.3` | Lowest search similarity worth injecting; among the hits that clear it, the highest-scoring one is used. The score swings with phrasing, so the floor is low: measured on a live Hub, one React question scored 0.88 and 0.40 depending on wording, while boilerplate and off-topic hits sat at 0.19–0.22. A Proxy that reports no score is not filtered. | -| `assetPrimeWaitMs` | `6000` | How long a step may wait for that lookup, so a strategy lands behind the prompt it was selected for rather than after the model has already chosen what to do. The budget has to cover two serial Proxy round trips: measured against a live Hub over six prompts, the lookup took 4.8s–9.4s with a 5.1s median, so `4000` never once won and `6000` clears the median and most of the spread. Past it the step proceeds and the strategy, when it lands, is injected into the next step. | -| `assetPrimeTimeoutMs` | `8000` | Deadline for each Proxy call in that lookup. A cold search plus fetch runs into seconds, so this is generous; `assetPrimeWaitMs` is what protects the response. | +| `assetPrimeMinSimilarity` | `0.3` | Lowest similarity worth injecting; among the recalled assets that clear it, the highest-scoring one with a strategy is used. The score swings with phrasing, so the floor is low: measured on a live Hub, one React question scored 0.88 and 0.40 depending on wording, while boilerplate and off-topic hits sat at 0.19–0.22. A Proxy that reports no score is not filtered. | +| `assetPrimeWaitMs` | `6000` | How long a step may wait for that lookup, so a strategy lands behind the prompt it was selected for rather than after the model has already chosen what to do. The budget was sized for the two serial round trips recall replaced, where the lookup took 4.8s–9.4s with a 5.1s median over six prompts against a live Hub; one call should sit well inside it, and the default stays until the single-call cost has been measured the same way. Past it the step proceeds and the strategy, when it lands, is injected into the next step. | +| `assetPrimeTimeoutMs` | `8000` | Deadline for the recall call. A cold Hub query runs into seconds, so this is generous; `assetPrimeWaitMs` is what protects the response. | | `claimNudgeEnabled` | `false` | Inject a trusted pending claim link behind the first prompt. `/evolver-status` remains available when off. | | `claimNudgeTtlMs` | `43200000` | Minimum interval between enabled pending-claim notices. | | `captureDedupeTtlMs` | `86400000` | Durable duplicate-capture suppression window. | @@ -159,7 +159,7 @@ Environment overrides: `MEMORY_GRAPH_PATH`, `EVOLVER_WORKSPACE_ID`, - Node.js 22.13 or newer. - Git for turn capture. - dsh `0.1.5-rc.2` or `0.1.6-alpha.1`. -- Optional network tools: a local Proxy from `@evomap/evolver`. +- Optional network tools: a local Proxy from `@evomap/evolver` 2.0.39 or newer, which is the first release whose `/asset/fetch` recalls by text. In a non-git directory the plugin emits one notice per directory, does not create workspace state, and records nothing. Network strategies are still injected there — they do not diff --git a/scripts/bench-prime.mjs b/scripts/bench-prime.mjs new file mode 100755 index 0000000..1f9dfbf --- /dev/null +++ b/scripts/bench-prime.mjs @@ -0,0 +1,88 @@ +#!/usr/bin/env node +// Measures what priming actually costs and delivers against a live Proxy. +// The pass/fail question it answers is not "is the gene good" -- that needs the +// reuse verdicts -- but "does a strategy land behind the prompt, and what does +// carrying it cost", which is what the wait budget and the context bill turn on. +import { readFileSync } from 'node:fs'; +import { homedir } from 'node:os'; +import { join } from 'node:path'; + +import { hubGene } from '../src/prime.js'; + +const PROMPTS = [ + '帮我写一篇小红书的种草笔记', + 'add a retry with backoff to the uploader', + '优化这个服务的缓存命中率', + '写个脚本批量重命名文件', + '这个 SQL 查询太慢了帮我优化', + 'the CI job times out on the coverage shard, find out why', + '把这个 React 组件的重渲染次数降下来', + 'design a rate limiter for a public HTTP API', + '这段 Python 有内存泄漏,帮我定位', + 'migrate this Express service to Fastify', + '给这个仓库加一个 GitHub Actions 的 lint 工作流', + 'explain why this goroutine deadlocks', +]; + +const WAIT_BUDGET_MS = Number(process.env.EVOLVER_BENCH_WAIT_MS ?? 6_000); + +function proxySettings() { + const path = join(homedir(), '.evolver', 'settings.json'); + const { proxy } = JSON.parse(readFileSync(path, 'utf8')); + if (!proxy?.url || !proxy?.token) throw new Error(`no proxy in ${path}; start one with \`evolver proxy\``); + return proxy; +} + +function proxyFetchVia({ url, token }) { + return async (method, path, body, signal) => { + const response = await fetch(url + path, { + method, + headers: { 'content-type': 'application/json', authorization: `Bearer ${token}` }, + body: JSON.stringify(body), + ...(signal ? { signal } : {}), + }); + return { ok: response.ok, data: await response.json() }; + }; +} + +function quantile(sorted, fraction) { + if (sorted.length === 0) return 0; + return sorted[Math.min(sorted.length - 1, Math.floor(sorted.length * fraction))]; +} + +function report(runs) { + const hits = runs.filter((run) => run.injected); + const latencies = runs.map((run) => run.ms).sort((left, right) => left - right); + const sizes = hits.map((run) => run.chars).sort((left, right) => left - right); + const inline = runs.filter((run) => run.ms <= WAIT_BUDGET_MS); + + console.log(`prompts ${runs.length}`); + console.log(`injected ${hits.length}/${runs.length}`); + console.log(`inline (<=${WAIT_BUDGET_MS}ms) ${inline.length}/${runs.length} — past the budget the strategy lands a step late`); + console.log(`latency ms p50 ${quantile(latencies, 0.5)} p90 ${quantile(latencies, 0.9)} max ${latencies.at(-1)}`); + console.log(`injected chars p50 ${quantile(sizes, 0.5)} p90 ${quantile(sizes, 0.9)} max ${sizes.at(-1) ?? 0}`); + console.log(`total chars ${sizes.reduce((sum, n) => sum + n, 0)}`); + console.log(''); + for (const run of runs) { + const late = run.ms > WAIT_BUDGET_MS ? ' LATE' : ''; + const what = run.injected ? `${String(run.chars).padStart(5)}ch ${String(run.steps).padStart(2)}步 ${run.header}` : ' — 无注入'; + console.log(`${String(run.ms).padStart(5)}ms${late.padEnd(5)} ${what}`); + } +} + +const proxyFetch = proxyFetchVia(proxySettings()); +const runs = []; +for (const prompt of PROMPTS) { + const started = Date.now(); + const { ids, text } = await hubGene(proxyFetch, prompt).catch(() => ({ ids: [], text: '' })); + const lines = text ? text.split('\n') : []; + runs.push({ + prompt, + ms: Date.now() - started, + injected: ids.length > 0, + chars: text.length, + steps: lines.filter((line) => /^\d+\. /.test(line)).length, + header: (lines[0] ?? '').replace('[Evolution Memory] ', '').slice(0, 56), + }); +} +report(runs); diff --git a/src/index.js b/src/index.js index e31644a..e6ac236 100644 --- a/src/index.js +++ b/src/index.js @@ -10,7 +10,6 @@ import { Config } from './config.js'; import { EDIT_TOOL_NAMES, editedContent, editedPath } from './edited-content.js'; import { noticeDue, pendingClaimUrl } from './onboarding.js'; import { hubGene, promptTextOf } from './prime.js'; -import { markUndeliverable, undeliverableIds } from './undeliverable.js'; import { createProxyClient } from './proxy.js'; import { looksLikeCorrection } from './dissatisfaction.js'; import { correctableAssets, forgetSession, markCorrected, markReported, rememberInjected, unreportedAssets } from './injected-assets.js'; @@ -133,8 +132,6 @@ function primeSteps(ctx, fallbackDir, config, primeFetch, tracker) { const search = hubGene(primeFetch, promptTextOf(claimed), { signal, listedIds: listed, - skipIds: undeliverableIds(), - onMissing: markUndeliverable, minSimilarity: config.assetPrimeMinSimilarity, }); const inline = await Promise.race([search, afterWait(config.assetPrimeWaitMs ?? DEFAULT_PRIME_WAIT_MS)]); diff --git a/src/prime.js b/src/prime.js index f699d85..de492c7 100644 --- a/src/prime.js +++ b/src/prime.js @@ -1,20 +1,29 @@ // SPDX-License-Identifier: MIT // Copyright (c) 2026 EvoMap -const SEARCH_LIMIT = 5; -// One call carries the whole candidate list — the fetch endpoint is rate -// limited and concurrency-guarded per sender node, so asking n times costs n -// slots where asking once costs one. What that call may contain is bounded by -// the wait budget instead: measured against a live Hub, two ids came back in a -// 1629ms median and three in 5011ms, against a search that costs ~2s on its -// own. Three overran the budget and the strategy landed after the model had -// already answered, which is worth nothing; two lands behind the prompt. -const FETCH_LIMIT = 2; +// One round trip, not two: since evolver 2.0.39 `/asset/fetch` recalls by text +// when it is given no ids, so selecting a candidate and materialising it are the +// same call. The pair it replaces had to buy its second hop out of the wait +// budget, and that budget only ever afforded two candidates; a recall returns +// whole assets, so this limit is about how many the Hub should rank, not about +// what the step can afford to wait for. +const RECALL_LIMIT = 5; const MIN_PROMPT_CHARS = 8; const PROMPT_MAX_CHARS = 400; const STEP_MAX_CHARS = 400; const TITLE_MAX_CHARS = 80; const DEFAULT_MIN_SIMILARITY = 0.3; +// Both bounds live here because recall does not pass through the Hub's search +// tuning: `thinGeneClause`, which drops genes of three steps or fewer, is +// imported only by assetSearchService, so the route that selects by ids and the +// route that recalls by text disagree about what is worth returning. Measured +// against a live Hub over five prompts, recall handed back three-step genes and +// a two-step one. The upper bound is the same judgement in the other direction: +// the same sample carried a 21-step pipeline capsule that injected 5047 +// characters where the median injection is 605, and it is a generated +// transcript rather than an approach anything can reuse. +const MIN_STRATEGY_STEPS = 4; +const MAX_STRATEGY_STEPS = 8; const UNSCORED = -1; const EMPTY_MATCH = { ids: [], text: '' }; @@ -40,14 +49,9 @@ export function promptTextOf(messages) { return parts.join('\n').trim().slice(0, PROMPT_MAX_CHARS); } -function searchHits(data) { - const found = [data?.results, data?.assets, data?.payload?.results].find(Array.isArray) ?? []; - return found.filter((hit) => hit && typeof hit.asset_id === 'string' && hit.asset_id); -} - -function fetchedById(data) { +function recalledAssets(data) { const found = [data?.assets, data?.results, data?.payload?.results].find(Array.isArray) ?? []; - return new Map(found.filter((asset) => asset?.asset_id).map((asset) => [asset.asset_id, asset])); + return found.filter((asset) => asset && typeof asset.asset_id === 'string' && asset.asset_id); } const CJK_SCRIPT = /[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}\p{Script=Hangul}]/u; @@ -66,33 +70,30 @@ function looksLikeAName(text) { return /\s/.test(text); } -function readableNameOf(hit, asset) { - const title = trimmedText(hit?.short_title) || trimmedText(asset?.short_title); +function readableNameOf(asset) { + const title = trimmedText(asset?.short_title); if (looksLikeAName(title)) return title.slice(0, TITLE_MAX_CHARS); - const described = trimmedText(hit?.nl_summary) || trimmedText(asset?.summary); + const described = trimmedText(asset?.nl_summary) || trimmedText(asset?.summary); if (described) return described.slice(0, TITLE_MAX_CHARS); - return title || hit?.asset_type || 'Gene'; + return title || asset?.asset_type || asset?.type || 'Gene'; } -function similarityOf(hit) { - return typeof hit.similarity === 'number' ? hit.similarity : UNSCORED; +function similarityOf(asset) { + return typeof asset.similarity === 'number' ? asset.similarity : UNSCORED; } -// Two things disqualify a hit before it costs a fetch, and the search result -// reports both: no strategy to reuse, and a similarity that says the Hub -// matched a topic rather than this task. The floor is low because the score -// swings with phrasing — the same React question scored 0.88 asked one way and -// 0.40 asked another, while boilerplate and off-topic hits sit at 0.19–0.22. -// They rank by score rather than by the order the Hub listed them in, since -// that order is the Hub's own ranking and weighs more than this prompt. -function rankedCandidates(hits, listedIds, skipIds, minSimilarity) { - return hits - .filter((hit) => !listedIds.has(hit.asset_id)) - .filter((hit) => !skipIds.has(hit.asset_id)) - .filter((hit) => hit.has_strategy !== false) - .filter((hit) => typeof hit.similarity !== 'number' || hit.similarity >= minSimilarity) - .sort((left, right) => similarityOf(right) - similarityOf(left)) - .slice(0, FETCH_LIMIT); +// Recall hands back whole assets, so having a strategy is read off the asset +// itself rather than trusted from a search flag. What is left to judge is the +// score, and it is only advisory: it swings with phrasing — the same React +// question scored 0.88 asked one way and 0.40 asked another, while boilerplate +// and off-topic hits sit at 0.19–0.22 — and a Proxy that reports none at all +// must not be filtered down to nothing. They rank by score rather than by the +// order the Hub listed them in, since that order weighs more than this prompt. +function rankedCandidates(assets, listedIds, minSimilarity) { + return assets + .filter((asset) => !listedIds.has(asset.asset_id)) + .filter((asset) => typeof asset.similarity !== 'number' || asset.similarity >= minSimilarity) + .sort((left, right) => similarityOf(right) - similarityOf(left)); } async function proxyJson(proxyFetch, path, body, signal) { @@ -108,39 +109,20 @@ async function proxyJson(proxyFetch, path, body, signal) { // slow Hub, or a malformed body must cost the turn nothing but the deadline. // One asset's strategy is injected and nothing else — a summary only tells the // model that something exists, while the steps are what it can actually reuse. -export async function hubGene(proxyFetch, text, { signal, listedIds = new Set(), skipIds = new Set(), onMissing, minSimilarity = DEFAULT_MIN_SIMILARITY } = {}) { +export async function hubGene(proxyFetch, text, { signal, listedIds = new Set(), minSimilarity = DEFAULT_MIN_SIMILARITY } = {}) { if (text.length < MIN_PROMPT_CHARS) return EMPTY_MATCH; - const found = await proxyJson(proxyFetch, '/asset/search', { text, limit: SEARCH_LIMIT }, signal); - if (!found) return EMPTY_MATCH; - - const candidates = rankedCandidates(searchHits(found), listedIds, skipIds, minSimilarity); - if (candidates.length === 0) return EMPTY_MATCH; - - // The closest hit is often one this node cannot materialise — the Hub returns - // it under `missing` — so the leading candidates are fetched together in the - // one round trip the endpoint already allows, and the best one that actually - // came back is used. The Hub resolves them one by one, so the list is short. - const fetched = await proxyJson( - proxyFetch, - '/asset/fetch', - { asset_ids: candidates.map((hit) => hit.asset_id) }, - signal, - ); - if (!fetched) return EMPTY_MATCH; + const recalled = await proxyJson(proxyFetch, '/asset/fetch', { text, limit: RECALL_LIMIT }, signal); + if (!recalled) return EMPTY_MATCH; - const byId = fetchedById(fetched); - const undelivered = candidates.filter((hit) => !byId.has(hit.asset_id)).map((hit) => hit.asset_id); - if (undelivered.length > 0) onMissing?.(undelivered); - for (const candidate of candidates) { - const steps = strategySteps(byId.get(candidate.asset_id)); - if (steps.length === 0) continue; + for (const candidate of rankedCandidates(recalledAssets(recalled), listedIds, minSimilarity)) { + const steps = strategySteps(candidate); + if (steps.length < MIN_STRATEGY_STEPS || steps.length > MAX_STRATEGY_STEPS) continue; - const source = found.degraded === true ? 'local cache, the Hub was unavailable' : 'EvoMap network'; return { ids: [candidate.asset_id], text: [ - `[Evolution Memory] ${readableNameOf(candidate, byId.get(candidate.asset_id))} (${source}):`, + `[Evolution Memory] ${readableNameOf(candidate)} (EvoMap network):`, ...steps.map((step, index) => `${index + 1}. ${step}`), '', `Apply it where it fits, then report the outcome with evolver_asset_reuse_result for ${candidate.asset_id}.`, diff --git a/src/reuse.js b/src/reuse.js index 5b61923..53966af 100644 --- a/src/reuse.js +++ b/src/reuse.js @@ -17,31 +17,36 @@ function correctionReason(turn) { + 'in the same session read as a correction, so the earlier reuse did not hold.'; } -// The asset id is the whole address. `summarizeReuseOutcomes` aggregates on it -// alone, and `deriveReuseEntries` anchors an accountable entry on assetId plus -// the cycle id the Proxy mints — a task id is neither, so sending one only -// decorates a report that is already keyed correctly. +// The asset id is what the report is about — `summarizeReuseOutcomes` +// aggregates on it alone. The task id is something else: the Hub hashes it with +// the asset and the reporting node into the event id that makes a retry +// idempotent, and its schema requires one, so a report without it is rejected +// with 400 and reaches no Hub ledger at all. One id per turn per session means +// re-sending the same verdict is recognised, while a later correction carries +// its own and is not mistaken for that retry. // `ok` only says the Proxy answered. The Proxy forwards to the Hub and reports -// the real fate in the body, and today that body reads `{recorded:false, -// reason:"hub 404"}` because no reuse-result route exists on the Hub. Treating -// the transport as the outcome would mark an asset reported that no ledger ever -// saw, and the on-disk record would make that mistake permanent. -async function postToHub(proxyFetch, { assetId, outcome, reason, signal }) { +// the real fate in the body. Treating the transport as the outcome would mark +// an asset reported that no ledger ever saw, and the on-disk record would make +// that mistake permanent. +async function postToHub(proxyFetch, { assetId, outcome, reason, taskId, signal }) { try { - const result = await proxyFetch('POST', '/asset/reuse-result', { asset_id: assetId, outcome, reason }, signal); + const result = await proxyFetch('POST', '/asset/reuse-result', { asset_id: assetId, outcome, reason, task_id: taskId }, signal); return result?.ok === true && result.data?.recorded !== false; } catch { return false; } } -// The local root_event is the half that pays off today. The actuator that -// re-orders candidates reads root_events, not the Hub, and the Hub has no -// reuse-result route to read from anyway. A verdict counts as delivered when -// either ledger took it, so one of them being down does not keep the asset -// pending forever. -async function postOutcome(proxyFetch, { assetId, outcome, reason, sessionId, signal }) { - const hub = await postToHub(proxyFetch, { assetId, outcome, reason, signal }); +function reportId(sessionId, turn) { + return `dsh:${sessionId || 'unknown'}:${turn ?? 'unknown'}`; +} + +// The local root_event is the half that pays off today: the actuator that +// re-orders candidates reads root_events, not the Hub. A verdict counts as +// delivered when either ledger took it, so one of them being down does not +// keep the asset pending forever. +async function postOutcome(proxyFetch, { assetId, outcome, reason, taskId, sessionId, signal }) { + const hub = await postToHub(proxyFetch, { assetId, outcome, reason, taskId, signal }); const local = await recordReuseLocally({ assetId, outcome, sessionId }); return hub || local; } @@ -52,7 +57,7 @@ export async function reportInjectedReuse(proxyFetch, options) { const reported = []; for (const assetId of assetIds) { - if (!await postOutcome(proxyFetch, { assetId, outcome: outcome.status, reason: automaticReason(turn, reasonKind), sessionId, signal })) continue; + if (!await postOutcome(proxyFetch, { assetId, outcome: outcome.status, reason: automaticReason(turn, reasonKind), taskId: reportId(sessionId, turn), sessionId, signal })) continue; reported.push(assetId); onReported?.(assetId, outcome.status); } @@ -63,7 +68,7 @@ export async function reportReuseCorrection(proxyFetch, options) { const { assets, sessionId, signal, onCorrected } = options; const corrected = []; for (const { assetId, turn } of assets) { - if (!await postOutcome(proxyFetch, { assetId, outcome: 'failed', reason: correctionReason(turn), sessionId, signal })) continue; + if (!await postOutcome(proxyFetch, { assetId, outcome: 'failed', reason: correctionReason(turn), taskId: `${reportId(sessionId, turn)}:correction`, sessionId, signal })) continue; corrected.push(assetId); onCorrected?.(assetId); } diff --git a/src/tools.js b/src/tools.js index e71b86f..4860e85 100644 --- a/src/tools.js +++ b/src/tools.js @@ -59,7 +59,13 @@ const assetOutput = { const { assets, missing } = fetchedAssets(value); const parts = assets.map(renderAsset); if (missing.length > 0) { - parts.push(`Not retrievable: ${missing.join(', ')}. They may be unpublished or not visible to this node.`); + // Why the Proxy did not hand these over does not reach here: the adapter + // knows whether the hub lacked the asset, revoked it, or returned one + // whose body no longer hashes to its id, and collapses all of it to a + // bare id. Naming a cause here would be a guess, and the guess this line + // used to make -- unpublished, or invisible to the node -- sent people to + // check permissions on assets that were promoted and plainly visible. + parts.push(`Not retrievable: ${missing.join(', ')}. The Proxy did not report why.`); } if (parts.length === 0) parts.push('No assets returned.'); parts.push('After applying an asset and validating the result, call evolver_asset_reuse_result.'); diff --git a/src/undeliverable.js b/src/undeliverable.js deleted file mode 100644 index 55ef8f5..0000000 --- a/src/undeliverable.js +++ /dev/null @@ -1,64 +0,0 @@ -// SPDX-License-Identifier: MIT -// Copyright (c) 2026 EvoMap - -import crypto from 'node:crypto'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; - -// An asset the Hub lists but will not deliver is stable, not flaky: across a -// day, sixteen ids that came back under `missing` stayed missing on every -// retry, and twenty-four that resolved kept resolving. Nothing in the search -// row predicts which is which — `payload_ready` is false on both — so the only -// way to stop paying a fetch slot for them is to remember. -// -// The cause is not established. `fetchAssetById` collapses "the Hub does not -// have this" and "the client refused the delivery" into the same null, so a -// refusal that some future release fixes would look identical from here. The -// window is therefore a reprieve, not a ban: an id is retried once a day. -const RETRY_AFTER_MS = 24 * 60 * 60 * 1000; - -function storePath() { - return path.join(os.homedir(), '.evolver', 'state', 'undeliverable.json'); -} - -function readStore() { - try { - const parsed = JSON.parse(fs.readFileSync(storePath(), 'utf8')); - return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? parsed : {}; - } catch { - return {}; - } -} - -export function undeliverableIds(now = Date.now()) { - const fresh = new Set(); - for (const [assetId, at] of Object.entries(readStore())) { - if (Number.isFinite(Number(at)) && now - Number(at) < RETRY_AFTER_MS) fresh.add(assetId); - } - return fresh; -} - -export function markUndeliverable(assetIds, now = Date.now()) { - if (!Array.isArray(assetIds) || assetIds.length === 0) return; - const store = {}; - for (const [assetId, at] of Object.entries(readStore())) { - if (Number.isFinite(Number(at)) && now - Number(at) < RETRY_AFTER_MS) store[assetId] = Number(at); - } - for (const assetId of assetIds) if (typeof assetId === 'string' && assetId) store[assetId] = now; - - const target = storePath(); - try { - fs.mkdirSync(path.dirname(target), { recursive: true, mode: 0o700 }); - const temporary = `${target}.${process.pid}.${crypto.randomBytes(6).toString('hex')}.tmp`; - fs.writeFileSync(temporary, JSON.stringify(store), { mode: 0o600, flag: 'wx' }); - try { - fs.renameSync(temporary, target); - } catch (error) { - if (!['EEXIST', 'EPERM'].includes(error?.code)) throw error; - fs.rmSync(target, { force: true }); - fs.renameSync(temporary, target); - } - } catch { - } -} diff --git a/test/apply.test.js b/test/apply.test.js index ff5d0ea..84a461f 100644 --- a/test/apply.test.js +++ b/test/apply.test.js @@ -57,7 +57,11 @@ async function primedBy(listeners, agent, prompt = 'add a retry to the uploader' return decision.messages.slice(1); } -async function untilCount(items, predicate, count, deadlineMs = 2_000) { +// A watchdog against a hang, not an assertion: each of these returns the moment +// its condition holds, so the budget only has to outlast the slowest runner. Two +// seconds did not -- the correction case takes ~2.0s on an idle laptop, and the +// dsh 0.1.5-rc.3 matrix leg failed on it while every other leg passed. +async function untilCount(items, predicate, count, deadlineMs = 15_000) { const start = Date.now(); while (items.filter(predicate).length < count) { if (Date.now() - start > deadlineMs) throw new Error(`only ${items.filter(predicate).length} of ${count} expected entries arrived`); @@ -66,7 +70,7 @@ async function untilCount(items, predicate, count, deadlineMs = 2_000) { return items.filter(predicate); } -async function untilRequest(requests, path, deadlineMs = 2_000) { +async function untilRequest(requests, path, deadlineMs = 15_000) { const start = Date.now(); while (!requests.some((request) => request.path === path)) { if (Date.now() - start > deadlineMs) throw new Error(`no ${path} request arrived`); @@ -75,7 +79,7 @@ async function untilRequest(requests, path, deadlineMs = 2_000) { return requests; } -async function untilInjected(injected, count, deadlineMs = 2_000) { +async function untilInjected(injected, count, deadlineMs = 15_000) { const start = Date.now(); while (injected.length < count) { if (Date.now() - start > deadlineMs) throw new Error(`only ${injected.length} of ${count} messages were injected`); @@ -178,22 +182,14 @@ test('every turn looks the Hub up with its own prompt, without repeating assets' let body = ''; request.on('data', (chunk) => { body += chunk; }); request.on('end', () => { - const parsed = JSON.parse(body); - requests.push({ path: request.url, body: parsed }); + requests.push({ path: request.url, body: JSON.parse(body) }); response.setHeader('Content-Type', 'application/json'); - response.end(JSON.stringify(request.url === '/asset/search' - ? { - results: [ - { asset_type: 'Gene', asset_id: 'sha256:abc', has_strategy: true }, - { asset_type: 'Gene', asset_id: 'sha256:def', has_strategy: true }, - ], - } - : { - assets: parsed.asset_ids.map((id) => ({ - asset_id: id, - strategy: [id === 'sha256:abc' ? 'Retry with backoff.' : 'Chunk the download.'], - })), - })); + response.end(JSON.stringify({ + assets: [ + { asset_type: 'Gene', asset_id: 'sha256:abc', strategy: ['Retry with backoff.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }, + { asset_type: 'Gene', asset_id: 'sha256:def', strategy: ['Chunk the download.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }, + ], + })); }); }); await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); @@ -210,10 +206,10 @@ test('every turn looks the Hub up with its own prompt, without repeating assets' assert.deepEqual( requests.map((request) => request.path), - ['/asset/search', '/asset/fetch', '/asset/search', '/asset/fetch'], + ['/asset/fetch', '/asset/fetch'], ); assert.equal(requests[0].body.text, 'add a retry to the uploader'); - assert.equal(requests[2].body.text, 'now make the download resumable'); + assert.equal(requests[1].body.text, 'now make the download resumable'); assert.match(first.at(-1).content[0].text, /evolver_asset_reuse_result for sha256:abc\./); assert.match(first.at(-1).content[0].text, /^1\. Retry with backoff\.$/m); assert.deepEqual(sameTurn, []); @@ -256,7 +252,7 @@ test('a search slower than the wait budget injects itself instead of holding the request.on('end', () => { const body = request.url === '/asset/search' ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:slow', has_strategy: true }] } - : { assets: [{ asset_id: 'sha256:slow', strategy: ['Arrived late.'] }] }; + : { assets: [{ asset_id: 'sha256:slow', strategy: ['Arrived late.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] }; setTimeout(() => { response.setHeader('Content-Type', 'application/json'); response.end(JSON.stringify(body)); @@ -365,7 +361,7 @@ test('an injected strategy is reported back when the turn ends', async () => { request.url === '/asset/search' ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:used', has_strategy: true, similarity: 0.9 }] } : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:used', strategy: ['Reuse this.'] }] } + ? { assets: [{ asset_id: 'sha256:used', strategy: ['Reuse this.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } : { ok: true }, )); }); @@ -387,7 +383,7 @@ test('an injected strategy is reported back when the turn ends', async () => { const [report] = requests.filter((request) => request.path === '/asset/reuse-result'); assert.equal(report.body.asset_id, 'sha256:used'); assert.equal(report.body.outcome, 'success'); - assert.equal(report.body.task_id, undefined, 'the asset id is the whole address; a task id is not an aggregation key'); + assert.equal(report.body.task_id, 'dsh:session-reuse:1', 'the Hub rejects a report with no dedupe key, and hashes this one with the asset to recognise a retry'); assert.match(report.body.reason, /not confirmed as applied/); } finally { process.env.HOME = home; @@ -407,7 +403,7 @@ test('a strategy the model reported itself is not reported again', async () => { response.end(JSON.stringify( request.url === '/asset/search' ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:owned', has_strategy: true, similarity: 0.9 }] } - : { assets: [{ asset_id: 'sha256:owned', strategy: ['Reuse this.'] }] }, + : { assets: [{ asset_id: 'sha256:owned', strategy: ['Reuse this.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] }, )); }); }); @@ -452,7 +448,7 @@ test('a fetched asset renders as reusable prose, not the raw envelope', async () type: 'Gene', asset_id: 'sha256:abc', summary: 'Fix the pool.', - strategy: ['Step one.', 'Step two.'], + strategy: ['Step one.', 'Step two.', 'Then the next step.', 'Then the next step.'], validation: ['npm test'], signals_match: ['log_error', 'perf_bottleneck'], source_node_id: 'node_x', @@ -484,7 +480,7 @@ test('a strategy that arrived after its own turn ended is still reported, under const payload = request.url === '/asset/search' ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:late', has_strategy: true, similarity: 0.9 }] } : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:late', strategy: ['Arrived after the turn.'] }] } + ? { assets: [{ asset_id: 'sha256:late', strategy: ['Arrived after the turn.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } : { ok: true }; const delay = request.url === '/asset/reuse-result' ? 0 : 150; setTimeout(() => { @@ -514,7 +510,7 @@ test('a strategy that arrived after its own turn ended is still reported, under const [report] = requests.filter((request) => request.path === '/asset/reuse-result'); assert.equal(report.body.asset_id, 'sha256:late'); - assert.equal(report.body.task_id, undefined); + assert.equal(report.body.task_id, 'dsh:session-late:1', 'a late strategy is reported under the turn it was selected for, not the one that ended'); assert.match(report.body.reason, /into dsh turn 1/); } finally { process.env.HOME = home; @@ -522,7 +518,7 @@ test('a strategy that arrived after its own turn ended is still reported, under } }); -function reuseServer(requests, strategy = ['Reuse this.'], assetId = 'sha256:kept') { +function reuseServer(requests, strategy = ['Reuse this.', 'Then the next step.', 'Then the next step.', 'Then the next step.'], assetId = 'sha256:kept') { return createServer((request, response) => { let body = ''; request.on('data', (chunk) => { body += chunk; }); @@ -598,6 +594,8 @@ test('a prompt that says the answer did not hold revises the verdict once, and o const failed = requests.filter(isFailedReport); assert.equal(failed.length, 1, 'the correction is sent exactly once'); assert.equal(failed[0].body.asset_id, 'sha256:kept'); + assert.notEqual(failed[0].body.task_id, reuseResults()[0].body.task_id, 'sharing the key with the verdict it revises would have the Hub dismiss it as a retry'); + assert.match(failed[0].body.task_id, /:correction$/); assert.match(failed[0].body.reason, /read as a correction/); await primedBy(listeners, agent, '还是不行啊', 3, 1); @@ -654,7 +652,7 @@ test('a verdict the Hub has no route for still reaches the local ledger', async request.url === '/asset/search' ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:unlanded', has_strategy: true, similarity: 0.9 }] } : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:unlanded', strategy: ['Try it.'] }] } + ? { assets: [{ asset_id: 'sha256:unlanded', strategy: ['Try it.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } : { recorded: false, reason: 'hub 404' }, )); }); @@ -693,51 +691,6 @@ test('a verdict the Hub has no route for still reaches the local ledger', async server.close(); } }); -test('a slot spent on an asset the Hub would not deliver is not spent on it again', async () => { - const projectDir = gitDirectory('evolver-undeliverable-'); - const requests = []; - const server = createServer((request, response) => { - let body = ''; - request.on('data', (chunk) => { body += chunk; }); - request.on('end', () => { - requests.push({ path: request.url, body: JSON.parse(body) }); - response.setHeader('Content-Type', 'application/json'); - response.end(JSON.stringify( - request.url === '/asset/search' - ? { - results: [ - { asset_type: 'Gene', asset_id: 'sha256:ghost', has_strategy: true, similarity: 0.99 }, - { asset_type: 'Gene', asset_id: 'sha256:real', has_strategy: true, similarity: 0.5 }, - ], - } - : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:real', strategy: ['Delivered.'] }], missing: ['sha256:ghost'] } - : { ok: true }, - )); - }); - }); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), 'evolver-undeliverable-home-')); - - try { - const { ctx, listeners } = fakeContext(); - apply(ctx, Config({ projectDir, proxyPort: server.address().port })); - - const first = fakeAgent({ id: 'a', sessionId: 'session-ghost-1', cwd: projectDir }).agent; - assert.match((await primedBy(listeners, first)).at(-1).content[0].text, /sha256:real/); - const fetches = () => requests.filter((request) => request.path === '/asset/fetch'); - assert.deepEqual(fetches()[0].body.asset_ids, ['sha256:ghost', 'sha256:real']); - - const second = fakeAgent({ id: 'b', sessionId: 'session-ghost-2', cwd: projectDir }).agent; - assert.match((await primedBy(listeners, second)).at(-1).content[0].text, /sha256:real/); - assert.deepEqual(fetches()[1].body.asset_ids, ['sha256:real'], 'the undeliverable id is remembered across sessions'); - } finally { - process.env.HOME = home; - server.close(); - } -}); - test("a self-report the Hub refused does not retire the plugin's own", async () => { const projectDir = gitDirectory('evolver-selfreport-404-'); const requests = []; @@ -751,7 +704,7 @@ test("a self-report the Hub refused does not retire the plugin's own", async () request.url === '/asset/search' ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:refused', has_strategy: true, similarity: 0.9 }] } : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:refused', strategy: ['Reuse this.'] }] } + ? { assets: [{ asset_id: 'sha256:refused', strategy: ['Reuse this.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } : { ok: true }, )); }); @@ -795,7 +748,7 @@ test('a correction writes the negative half of the signal, not another hit', asy request.url === '/asset/search' ? { results: [{ asset_type: 'Gene', asset_id: 'sha256:wrong', has_strategy: true, similarity: 0.9 }] } : request.url === '/asset/fetch' - ? { assets: [{ asset_id: 'sha256:wrong', strategy: ['Did not hold.'] }] } + ? { assets: [{ asset_id: 'sha256:wrong', strategy: ['Did not hold.', 'Then the next step.', 'Then the next step.', 'Then the next step.'] }] } : { recorded: false, reason: 'hub 404' }, )); }); diff --git a/test/prime.test.js b/test/prime.test.js index fdcd957..8b79e6b 100644 --- a/test/prime.test.js +++ b/test/prime.test.js @@ -3,22 +3,21 @@ import { test } from 'node:test'; import { hubGene, promptTextOf } from '../src/prime.js'; -function stubProxy({ search, fetch: fetched }) { +function stubProxy(recalled) { const calls = []; const proxyFetch = async (method, path, body, signal) => { calls.push({ method, path, body, signal }); - const data = path === '/asset/search' ? search : fetched; - if (data instanceof Error) throw data; - return data === undefined ? { ok: false, error: 'nope' } : { ok: true, data }; + if (recalled instanceof Error) throw recalled; + return recalled === undefined ? { ok: false, error: 'nope' } : { ok: true, data: recalled }; }; return { proxyFetch, calls }; } -const HIT = { asset_id: 'sha256:abc', asset_type: 'Gene', has_strategy: true }; const ASSET = { asset_id: 'sha256:abc', + asset_type: 'Gene', summary: 'Retry the upload with backoff.', - strategy: [' Measure the failure rate first. ', 'Add jittered backoff.', ''], + strategy: [' Measure the failure rate first. ', 'Add jittered backoff.', 'Cap the retries.', 'Alert on the cap.', ''], validation: ['npm test'], }; @@ -33,230 +32,130 @@ test('the prompt text comes from the user, not from dsh\'s own injections', () = assert.equal(text, 'add a retry to the uploader'); }); -test('one asset is fetched and injected as its strategy alone', async () => { - const { proxyFetch, calls } = stubProxy({ - search: { results: [HIT, { asset_id: 'sha256:second', has_strategy: true }] }, - fetch: { assets: [ASSET] }, - }); +test('one call recalls by text and injects that asset\'s strategy alone', async () => { + const { proxyFetch, calls } = stubProxy({ assets: [ASSET] }); const controller = new AbortController(); const { ids, text } = await hubGene(proxyFetch, 'add a retry to the uploader', { signal: controller.signal }); - assert.deepEqual(calls.map((call) => call.path), ['/asset/search', '/asset/fetch']); - assert.deepEqual(calls[1].body, { asset_ids: ['sha256:abc', 'sha256:second'] }); + assert.deepEqual(calls.map((call) => call.path), ['/asset/fetch']); + assert.deepEqual(calls[0].body, { text: 'add a retry to the uploader', limit: 5 }); + assert.ok(!('asset_ids' in calls[0].body), 'ids would turn the recall back into a lookup'); assert.equal(calls[0].signal, controller.signal); assert.deepEqual(ids, ['sha256:abc']); assert.match(text, /\[Evolution Memory\] Retry the upload with backoff\. \(EvoMap network\)/); assert.match(text, /evolver_asset_reuse_result for sha256:abc\./); assert.match(text, /^1\. Measure the failure rate first\.$/m); - assert.match(text, /^2\. Add jittered backoff\.$/m); - assert.doesNotMatch(text, /npm test|sha256:second/); - assert.match(text, /evolver_asset_reuse_result/); + assert.match(text, /^4\. Alert on the cap\.$/m); + assert.doesNotMatch(text, /npm test/); }); test('a bounded prompt reaches the Hub, and a trivial one never does', async () => { - const long = stubProxy({ search: { results: [] } }); + const long = stubProxy({ assets: [] }); await hubGene(long.proxyFetch, 'x'.repeat(5_000)); assert.ok(long.calls[0].body.text.length <= 5_000); assert.equal(long.calls[0].body.limit, 5); - const trivial = stubProxy({ search: { results: [HIT] }, fetch: { assets: [ASSET] } }); + const trivial = stubProxy({ assets: [ASSET] }); assert.deepEqual(await hubGene(trivial.proxyFetch, 'hi'), { ids: [], text: '' }); assert.equal(trivial.calls.length, 0); }); -test('hits without a strategy are skipped before they cost a fetch', async () => { - const { proxyFetch, calls } = stubProxy({ - search: { results: [{ asset_id: 'sha256:thin', has_strategy: false }] }, - fetch: { assets: [ASSET] }, - }); - - assert.deepEqual(await hubGene(proxyFetch, 'add a retry to the uploader'), { ids: [], text: '' }); - assert.deepEqual(calls.map((call) => call.path), ['/asset/search']); -}); - test('an asset already injected this session is passed over', async () => { - const { proxyFetch, calls } = stubProxy({ - search: { results: [HIT, { asset_id: 'sha256:next', asset_type: 'Capsule', has_strategy: true }] }, - fetch: { results: [{ asset_id: 'sha256:next', payload: { strategy: 'Drain the queue first.' } }] }, + const { proxyFetch } = stubProxy({ + assets: [ASSET, { asset_id: 'sha256:next', asset_type: 'Capsule', payload: { strategy: ['Drain the queue first.', 'Replay the dead letters.', 'Re-enable the consumer.', 'Watch the depth.'] } }], }); const { ids, text } = await hubGene(proxyFetch, 'add a retry to the uploader', { listedIds: new Set(['sha256:abc']), }); - assert.deepEqual(calls[1].body, { asset_ids: ['sha256:next'] }); - assert.ok(!JSON.stringify(calls[1].body).includes('sha256:abc')); assert.deepEqual(ids, ['sha256:next']); + assert.doesNotMatch(text, /sha256:abc|Measure the failure rate/); assert.match(text, /\[Evolution Memory\] Capsule \(EvoMap network\)/); assert.match(text, /evolver_asset_reuse_result for sha256:next\./); assert.match(text, /^1\. Drain the queue first\.$/m); }); -test('a cached answer says the Hub was unavailable', async () => { - const { proxyFetch } = stubProxy({ search: { degraded: true, results: [HIT] }, fetch: { assets: [ASSET] } }); - const { text } = await hubGene(proxyFetch, 'add a retry to the uploader'); - - assert.match(text, /local cache, the Hub was unavailable/); -}); - -test('a failing Proxy, an empty result, and a strategy-less asset inject nothing', async () => { - const down = stubProxy({ search: undefined }); +test('a failing Proxy, an empty recall, and a strategy-less asset inject nothing', async () => { + const down = stubProxy(undefined); assert.deepEqual(await hubGene(down.proxyFetch, 'add a retry to the uploader'), { ids: [], text: '' }); - const throwing = stubProxy({ search: new Error('socket hang up') }); + const throwing = stubProxy(new Error('socket hang up')); assert.deepEqual(await hubGene(throwing.proxyFetch, 'add a retry to the uploader'), { ids: [], text: '' }); - const empty = stubProxy({ search: { results: [] } }); + const empty = stubProxy({ assets: [] }); assert.deepEqual(await hubGene(empty.proxyFetch, 'add a retry to the uploader'), { ids: [], text: '' }); - const hollow = stubProxy({ search: { results: [HIT] }, fetch: { assets: [{ asset_id: 'sha256:abc', summary: 'only prose' }] } }); + const hollow = stubProxy({ assets: [{ asset_id: 'sha256:abc', summary: 'only prose' }] }); assert.deepEqual(await hubGene(hollow.proxyFetch, 'add a retry to the uploader'), { ids: [], text: '' }); - - const missed = stubProxy({ search: { results: [HIT] }, fetch: undefined }); - assert.deepEqual(await hubGene(missed.proxyFetch, 'add a retry to the uploader'), { ids: [], text: '' }); }); -test('a topical but low-similarity hit is dropped before it costs a fetch', async () => { +test('an asset with no strategy gives way to the next one in the same answer', async () => { const { proxyFetch, calls } = stubProxy({ - search: { results: [{ asset_id: 'sha256:loose', has_strategy: true, similarity: 0.22, short_title: '小红书内容创作工作流' }] }, - fetch: { assets: [ASSET] }, + assets: [ + { asset_id: 'sha256:prose', asset_type: 'Gene', similarity: 0.98, summary: 'Nothing to reuse here.' }, + { asset_id: 'sha256:usable', asset_type: 'Gene', similarity: 0.71, strategy: ['Use the one that carries steps.', 'Then the next.', 'Then the next.', 'Then the last.'] }, + ], + }); + + const { ids, text } = await hubGene(proxyFetch, 'add a retry to the uploader'); + assert.equal(calls.length, 1, 'the fallback costs no second round trip'); + assert.deepEqual(ids, ['sha256:usable']); + assert.match(text, /^1\. Use the one that carries steps\.$/m); + assert.doesNotMatch(text, /sha256:prose/); +}); + +test('a topical but low-similarity asset is dropped', async () => { + const { proxyFetch } = stubProxy({ + assets: [{ asset_id: 'sha256:loose', similarity: 0.22, short_title: '小红书内容创作工作流', strategy: ['先定选题。', '再写标题。', '再写正文。', '最后配图。'] }], }); assert.deepEqual(await hubGene(proxyFetch, 'add a retry to the uploader'), { ids: [], text: '' }); - assert.deepEqual(calls.map((call) => call.path), ['/asset/search']); }); test('the similarity line is configurable, and a Proxy that omits the score still answers', async () => { - const scored = stubProxy({ - search: { results: [{ ...HIT, similarity: 0.35 }] }, - fetch: { assets: [ASSET] }, - }); + const scored = stubProxy({ assets: [{ ...ASSET, similarity: 0.35 }] }); const { ids } = await hubGene(scored.proxyFetch, 'add a retry to the uploader', { minSimilarity: 0.3 }); assert.deepEqual(ids, ['sha256:abc']); - const unscored = stubProxy({ search: { results: [HIT] }, fetch: { assets: [ASSET] } }); + const unscored = stubProxy({ assets: [ASSET] }); assert.deepEqual((await hubGene(unscored.proxyFetch, 'add a retry to the uploader')).ids, ['sha256:abc']); }); -test('the best-scoring hit wins, whatever order the Hub listed them in', async () => { - const { proxyFetch, calls } = stubProxy({ - search: { - results: [ - { asset_id: 'sha256:middling', asset_type: 'Gene', has_strategy: true, similarity: 0.62 }, - { asset_id: 'sha256:closest', asset_type: 'Gene', has_strategy: true, similarity: 0.94 }, - { asset_id: 'sha256:weak', asset_type: 'Gene', has_strategy: true, similarity: 0.51 }, - ], - }, - fetch: { assets: [{ asset_id: 'sha256:closest', strategy: ['Take the closest match.'] }] }, - }); - - const { ids } = await hubGene(proxyFetch, 'add a retry to the uploader'); - assert.deepEqual(calls[1].body.asset_ids[0], 'sha256:closest'); - assert.deepEqual(ids, ['sha256:closest']); -}); - -test('the closest hit the node cannot materialise gives way to the next one', async () => { - const { proxyFetch, calls } = stubProxy({ - search: { - results: [ - { asset_id: 'sha256:unreachable', asset_type: 'Gene', has_strategy: true, similarity: 0.98 }, - { asset_id: 'sha256:present', asset_type: 'Gene', has_strategy: true, similarity: 0.71 }, - ], - }, - fetch: { - assets: [{ asset_id: 'sha256:present', strategy: ['Use the one that came back.'] }], - missing: ['sha256:unreachable'], - }, +test('the best-scoring asset wins, whatever order the Hub listed them in', async () => { + const { proxyFetch } = stubProxy({ + assets: [ + { asset_id: 'sha256:middling', asset_type: 'Gene', similarity: 0.62, strategy: ['Middling.', 'Middling.', 'Middling.', 'Middling.'] }, + { asset_id: 'sha256:closest', asset_type: 'Gene', similarity: 0.94, strategy: ['Take the closest match.', 'Then the next.', 'Then the next.', 'Then the last.'] }, + { asset_id: 'sha256:weak', asset_type: 'Gene', similarity: 0.51, strategy: ['Weak.', 'Weak.', 'Weak.', 'Weak.'] }, + ], }); const { ids, text } = await hubGene(proxyFetch, 'add a retry to the uploader'); - assert.deepEqual(calls[1].body, { asset_ids: ['sha256:unreachable', 'sha256:present'] }); - assert.deepEqual(ids, ['sha256:present']); - assert.match(text, /^1\. Use the one that came back\.$/m); - assert.doesNotMatch(text, /sha256:unreachable/); + assert.deepEqual(ids, ['sha256:closest']); + assert.match(text, /^1\. Take the closest match\.$/m); }); -test('an unscored hit is still usable, but never outranks a scored one', async () => { - const { proxyFetch, calls } = stubProxy({ - search: { - results: [ - { asset_id: 'sha256:unscored', asset_type: 'Gene', has_strategy: true }, - { asset_id: 'sha256:scored', asset_type: 'Gene', has_strategy: true, similarity: 0.55 }, - ], - }, - fetch: { assets: [{ asset_id: 'sha256:scored', strategy: ['Prefer the measured match.'] }] }, +test('an unscored asset is still usable, but never outranks a scored one', async () => { + const { proxyFetch } = stubProxy({ + assets: [ + { asset_id: 'sha256:unscored', asset_type: 'Gene', strategy: ['Unmeasured.', 'Unmeasured.', 'Unmeasured.', 'Unmeasured.'] }, + { asset_id: 'sha256:scored', asset_type: 'Gene', similarity: 0.55, strategy: ['Prefer the measured match.', 'Then the next.', 'Then the next.', 'Then the last.'] }, + ], }); assert.deepEqual((await hubGene(proxyFetch, 'add a retry to the uploader')).ids, ['sha256:scored']); - assert.deepEqual(calls[1].body.asset_ids, ['sha256:scored', 'sha256:unscored']); -}); - -test('a better-scored hit is taken over a closer one that has no strategy', async () => { - const { proxyFetch, calls } = stubProxy({ - search: { - results: [ - { asset_id: 'sha256:thin', has_strategy: false, similarity: 0.99 }, - { asset_id: 'sha256:usable', asset_type: 'Gene', has_strategy: true, similarity: 0.8 }, - { asset_id: 'sha256:lesser', asset_type: 'Gene', has_strategy: true, similarity: 0.6 }, - ], - }, - fetch: { assets: [{ asset_id: 'sha256:usable', strategy: ['Drain the queue first.'] }] }, - }); - - const { ids } = await hubGene(proxyFetch, 'add a retry to the uploader'); - assert.deepEqual(calls[1].body, { asset_ids: ['sha256:usable', 'sha256:lesser'] }); - assert.deepEqual(ids, ['sha256:usable']); -}); - -test('one call carries at most two ids, because a third overruns the wait budget', async () => { - const { proxyFetch, calls } = stubProxy({ - search: { - results: [ - { asset_id: 'sha256:a', asset_type: 'Gene', has_strategy: true, similarity: 0.9 }, - { asset_id: 'sha256:b', asset_type: 'Gene', has_strategy: true, similarity: 0.8 }, - { asset_id: 'sha256:c', asset_type: 'Gene', has_strategy: true, similarity: 0.7 }, - { asset_id: 'sha256:d', asset_type: 'Gene', has_strategy: true, similarity: 0.6 }, - ], - }, - fetch: { assets: [{ asset_id: 'sha256:a', strategy: ['Two is the budget.'] }] }, - }); - - await hubGene(proxyFetch, 'add a retry to the uploader'); - assert.equal(calls.filter((call) => call.path === '/asset/fetch').length, 1, 'still one call, not one per id'); - assert.deepEqual(calls[1].body.asset_ids, ['sha256:a', 'sha256:b']); -}); - -test('an id the Hub would not deliver is reported, and skipped when asked to skip it', async () => { - const undelivered = []; - const { proxyFetch, calls } = stubProxy({ - search: { - results: [ - { asset_id: 'sha256:ghost', asset_type: 'Gene', has_strategy: true, similarity: 0.99 }, - { asset_id: 'sha256:real', asset_type: 'Gene', has_strategy: true, similarity: 0.5 }, - ], - }, - fetch: { assets: [{ asset_id: 'sha256:real', strategy: ['Delivered.'] }], missing: ['sha256:ghost'] }, - }); - - const first = await hubGene(proxyFetch, 'add a retry to the uploader', { onMissing: (ids) => undelivered.push(...ids) }); - assert.deepEqual(undelivered, ['sha256:ghost']); - assert.deepEqual(first.ids, ['sha256:real']); - - const second = await hubGene(proxyFetch, 'add a retry to the uploader', { skipIds: new Set(undelivered) }); - assert.deepEqual(calls.at(-1).body.asset_ids, ['sha256:real'], 'the wasted slot is not spent again'); - assert.deepEqual(second.ids, ['sha256:real']); }); test('a named gene is announced by its name, and its hash only where it gets used', async () => { const { proxyFetch } = stubProxy({ - search: { - results: [{ - asset_id: 'sha256:named', asset_type: 'Gene', has_strategy: true, similarity: 0.9, - short_title: 'Chinese Social Media Writing Template', - nl_summary: 'A long explanation that has no business being in front of the model on every turn.'.repeat(3), - }], - }, - fetch: { assets: [{ asset_id: 'sha256:named', summary: 'Also long, also not the header.', strategy: ['Draft it.'] }] }, + assets: [{ + asset_id: 'sha256:named', asset_type: 'Gene', similarity: 0.9, + short_title: 'Chinese Social Media Writing Template', + nl_summary: 'A long explanation that has no business being in front of the model on every turn.'.repeat(3), + summary: 'Also long, also not the header.', + strategy: ['Draft it.', 'Edit it.', 'Check it.', 'Ship it.'], + }], }); const { text } = await hubGene(proxyFetch, 'add a retry to the uploader'); @@ -269,8 +168,7 @@ test('a named gene is announced by its name, and its hash only where it gets use test('an asset with neither a name nor a description falls back to its type', async () => { const { proxyFetch } = stubProxy({ - search: { results: [{ asset_id: 'sha256:plain', asset_type: 'Capsule', has_strategy: true, similarity: 0.9 }] }, - fetch: { assets: [{ asset_id: 'sha256:plain', strategy: ['Do it.'] }] }, + assets: [{ asset_id: 'sha256:plain', asset_type: 'Capsule', similarity: 0.9, strategy: ['Do it.', 'Do it.', 'Do it.', 'Do it.'] }], }); const { text } = await hubGene(proxyFetch, 'add a retry to the uploader'); @@ -279,14 +177,12 @@ test('an asset with neither a name nor a description falls back to its type', as test('a one-word title is a truncation, not a name, so the summary stands in', async () => { const { proxyFetch } = stubProxy({ - search: { - results: [{ - asset_id: 'sha256:fragment', asset_type: 'Gene', has_strategy: true, similarity: 0.9, - short_title: 'Object', - nl_summary: 'A generic object pool for reusing expensive resources such as database connections, so they are not rebuilt per request.', - }], - }, - fetch: { assets: [{ asset_id: 'sha256:fragment', strategy: ['Initialize the pool.'] }] }, + assets: [{ + asset_id: 'sha256:fragment', asset_type: 'Gene', similarity: 0.9, + short_title: 'Object', + nl_summary: 'A generic object pool for reusing expensive resources such as database connections, so they are not rebuilt per request.', + strategy: ['Initialize the pool.', 'Size it.', 'Lend from it.', 'Return to it.'], + }], }); const { text } = await hubGene(proxyFetch, 'reuse database connections'); @@ -299,14 +195,12 @@ test('a one-word title is a truncation, not a name, so the summary stands in', a test('a short CJK title is a name, and is not mistaken for a fragment', async () => { const { proxyFetch } = stubProxy({ - search: { - results: [{ - asset_id: 'sha256:cjk', asset_type: 'Gene', has_strategy: true, similarity: 0.9, - short_title: '智能缓存优化', - nl_summary: '这个基因用于优化缓存命中率。', - }], - }, - fetch: { assets: [{ asset_id: 'sha256:cjk', strategy: ['预热缓存。'] }] }, + assets: [{ + asset_id: 'sha256:cjk', asset_type: 'Gene', similarity: 0.9, + short_title: '智能缓存优化', + nl_summary: '这个基因用于优化缓存命中率。', + strategy: ['预热缓存。', '设置淘汰策略。', '监控命中率。', '定期复查。'], + }], }); const { text } = await hubGene(proxyFetch, '这个服务的缓存命中率太低了,帮我优化一下'); @@ -315,16 +209,47 @@ test('a short CJK title is a name, and is not mistaken for a fragment', async () test('a truncated CJK title gives way to the summary as well', async () => { const { proxyFetch } = stubProxy({ - search: { - results: [{ - asset_id: 'sha256:cut', asset_type: 'Gene', has_strategy: true, similarity: 0.9, - short_title: '自动化小', - nl_summary: '这个基因能自动帮你创作小红书笔记并一键发布。', - }], - }, - fetch: { assets: [{ asset_id: 'sha256:cut', strategy: ['先定选题。'] }] }, + assets: [{ + asset_id: 'sha256:cut', asset_type: 'Gene', similarity: 0.9, + short_title: '自动化小', + nl_summary: '这个基因能自动帮你创作小红书笔记并一键发布。', + strategy: ['先定选题。', '再写标题。', '再写正文。', '最后配图。'], + }], }); const { text } = await hubGene(proxyFetch, '帮我写一篇小红书的种草笔记'); assert.match(text.split('\n')[0], /这个基因能自动帮你创作小红书笔记/); }); + +test('the Hub\'s own envelope shapes are all read', async () => { + const nested = stubProxy({ payload: { results: [ASSET] } }); + assert.deepEqual((await hubGene(nested.proxyFetch, 'add a retry to the uploader')).ids, ['sha256:abc']); + + const results = stubProxy({ results: [ASSET] }); + assert.deepEqual((await hubGene(results.proxyFetch, 'add a retry to the uploader')).ids, ['sha256:abc']); +}); + +test('a strategy too thin or too long to reuse gives way to one that fits', async () => { + const fits = ['Read the plan.', 'Apply it.', 'Check it.', 'Record it.']; + const { proxyFetch, calls } = stubProxy({ + assets: [ + { asset_id: 'sha256:thin', asset_type: 'Gene', similarity: 0.99, strategy: ['Try harder.', 'Ship it.', 'Hope.'] }, + { asset_id: 'sha256:transcript', asset_type: 'Capsule', similarity: 0.95, strategy: Array.from({ length: 21 }, (step, index) => `Pipeline step ${index}.`) }, + { asset_id: 'sha256:fits', asset_type: 'Gene', similarity: 0.5, strategy: fits }, + ], + }); + + const { ids, text } = await hubGene(proxyFetch, 'add a retry to the uploader'); + assert.equal(calls.length, 1, 'both rejections are read off the one answer'); + assert.deepEqual(ids, ['sha256:fits']); + assert.match(text, /^4\. Record it\.$/m); + assert.doesNotMatch(text, /Pipeline step|Try harder/); +}); + +test('the bounds are inclusive at both ends', async () => { + const four = stubProxy({ assets: [{ asset_id: 'sha256:four', asset_type: 'Gene', strategy: ['a.', 'b.', 'c.', 'd.'] }] }); + assert.deepEqual((await hubGene(four.proxyFetch, 'add a retry to the uploader')).ids, ['sha256:four']); + + const eight = stubProxy({ assets: [{ asset_id: 'sha256:eight', asset_type: 'Gene', strategy: Array.from({ length: 8 }, (step, index) => `step ${index}.`) }] }); + assert.deepEqual((await hubGene(eight.proxyFetch, 'add a retry to the uploader')).ids, ['sha256:eight']); +}); diff --git a/test/undeliverable.test.js b/test/undeliverable.test.js deleted file mode 100644 index 3fb4002..0000000 --- a/test/undeliverable.test.js +++ /dev/null @@ -1,57 +0,0 @@ -import assert from 'node:assert/strict'; -import { mkdtempSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { test } from 'node:test'; - -import { markUndeliverable, undeliverableIds } from '../src/undeliverable.js'; - -const DAY_MS = 24 * 60 * 60 * 1000; - -function isolatedHome(prefix) { - const home = process.env.HOME; - process.env.HOME = mkdtempSync(join(tmpdir(), prefix)); - return () => { process.env.HOME = home; }; -} - -test('an undeliverable id is remembered across processes and forgotten after a day', () => { - const restore = isolatedHome('evolver-undeliverable-ttl-'); - try { - const at = Date.UTC(2026, 0, 1); - markUndeliverable(['sha256:ghost', 'sha256:other'], at); - - assert.deepEqual([...undeliverableIds(at + 60_000)].sort(), ['sha256:ghost', 'sha256:other']); - assert.deepEqual([...undeliverableIds(at + DAY_MS - 1)].sort(), ['sha256:ghost', 'sha256:other']); - assert.deepEqual( - [...undeliverableIds(at + DAY_MS)], - [], - 'the window is a reprieve, not a ban: the cause was never established', - ); - } finally { - restore(); - } -}); - -test('a fresh sighting extends only the id that was sighted', () => { - const restore = isolatedHome('evolver-undeliverable-extend-'); - try { - const at = Date.UTC(2026, 0, 1); - markUndeliverable(['sha256:stale', 'sha256:fresh'], at); - markUndeliverable(['sha256:fresh'], at + DAY_MS - 1); - - assert.deepEqual([...undeliverableIds(at + DAY_MS + 1000)], ['sha256:fresh']); - } finally { - restore(); - } -}); - -test('nothing is written for an empty list, and an unknown id is never skipped', () => { - const restore = isolatedHome('evolver-undeliverable-empty-'); - try { - markUndeliverable([]); - markUndeliverable(undefined); - assert.deepEqual([...undeliverableIds()], []); - } finally { - restore(); - } -});