Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ Powered by the [Genome Evolution Protocol](https://evomap.ai) and

| Seam | dsh event | Behaviour |
| --- | --- | --- |
| Asset priming | `agent/pre-step`, once per turn | Searches the Proxy with that turn's own prompt — only the text the person typed — fetches the two leading matches that clear the similarity line and have a strategy, in one call, and injects the strategy of the first that actually came back, announced by the asset's short title rather than its hash, falling back to its description when the Hub's title is a truncated fragment. The hash appears once, on the line that asks for the outcome, because that call needs it. Ids the Hub declined to deliver are remembered for a day and skipped, so a slot is not spent twice on one. A lookup slower than the wait budget is injected into the next step instead of holding the current one. Assets already injected in this session are skipped. |
| Asset priming | `agent/pre-step`, once per turn | Recalls from the Proxy in a single `/asset/fetch` call carrying that turn's own prompt — only the text the person typed — and injects the strategy of the highest-scoring asset that came back carrying between four and eight steps — recall bypasses the Hub's own thin-gene filter, so both bounds are applied here — announced by the asset's short title rather than its hash, falling back to its description when the Hub's title is a truncated fragment. The hash appears once, on the line that asks for the outcome, because that call needs it. A lookup slower than the wait budget is injected into the next step instead of holding the current one. Assets already injected in this session are skipped. |
| Signal detection | successful `tools/result` for `write`, `edit`, or `str_replace_editor` | Tags the turn with improvement signals and injects one bounded notice per file/signal set. |
| Reuse feedback | `session/event` → `turn/end` | Reports each strategy injected during that turn to the Proxy with the turn's outcome, marked as automatic and unconfirmed. Injected asset ids are written to `~/.evolver/state/` as they are injected and read back at report time, so a report survives the process dying in between. Reports are keyed on the asset id alone, and each verdict is written twice: to the Proxy, and as a `value.reuse_hit` / `value.reuse_outcome` root event in `~/.evomap/evolution/root_events.jsonl`. The local write is the one that closes the loop — the actuator that re-orders future candidates reads root events, not the Hub. An asset the model reported itself with `evolver_asset_reuse_result` is left alone. |
| Reuse feedback | `session/event` → `turn/end` | Reports each strategy injected during that turn to the Proxy with the turn's outcome, marked as automatic and unconfirmed. Injected asset ids are written to `~/.evolver/state/` as they are injected and read back at report time, so a report survives the process dying in between. Each report names the asset it is about and carries a per-turn id the Hub hashes with it to recognise a retry — a report without one is rejected outright — and a later correction carries its own, so it is not dismissed as that retry. Each verdict is written twice: to the Proxy, and as a `value.reuse_hit` / `value.reuse_outcome` root event in `~/.evomap/evolution/root_events.jsonl`. The local write is the one that closes the loop — the actuator that re-orders future candidates reads root events, not the Hub. An asset the model reported itself with `evolver_asset_reuse_result` is left alone. |
| Verdict correction | `agent/pre-step` | When the prompt that opens a later turn in the same session plainly says the last answer did not hold, the verdicts already sent for that session are revised to `failed` — each asset at most once, since the Hub counts every report it receives. |
| Capture | `session/event` → `turn/end` | Collects staged, unstaged, and untracked work asynchronously; records the real turn outcome once; drains at `session/flush`. |
| Tools | `ctx.tools.register` | Connects directly to the local Evolver Proxy for status, search, fetch, reuse feedback, distillation, publication, and mailbox polling. |
Expand Down Expand Up @@ -139,9 +139,9 @@ The plugin exports a validated Schemastery `Config`; invalid values fail at load
| `proxyTimeoutMs` | `8000` | Native Proxy tool deadline. |
| `hubTimeoutMs` | `8000` | Direct outcome-recording deadline. |
| `assetPrimeEnabled` | `true` | Look one reusable strategy up per turn and inject it behind that turn's prompt. |
| `assetPrimeMinSimilarity` | `0.3` | Lowest search similarity worth injecting; among the hits that clear it, the highest-scoring one is used. The score swings with phrasing, so the floor is low: measured on a live Hub, one React question scored 0.88 and 0.40 depending on wording, while boilerplate and off-topic hits sat at 0.19–0.22. A Proxy that reports no score is not filtered. |
| `assetPrimeWaitMs` | `6000` | How long a step may wait for that lookup, so a strategy lands behind the prompt it was selected for rather than after the model has already chosen what to do. The budget has to cover two serial Proxy round trips: measured against a live Hub over six prompts, the lookup took 4.8s–9.4s with a 5.1s median, so `4000` never once won and `6000` clears the median and most of the spread. Past it the step proceeds and the strategy, when it lands, is injected into the next step. |
| `assetPrimeTimeoutMs` | `8000` | Deadline for each Proxy call in that lookup. A cold search plus fetch runs into seconds, so this is generous; `assetPrimeWaitMs` is what protects the response. |
| `assetPrimeMinSimilarity` | `0.3` | Lowest similarity worth injecting; among the recalled assets that clear it, the highest-scoring one with a strategy is used. The score swings with phrasing, so the floor is low: measured on a live Hub, one React question scored 0.88 and 0.40 depending on wording, while boilerplate and off-topic hits sat at 0.19–0.22. A Proxy that reports no score is not filtered. |
| `assetPrimeWaitMs` | `6000` | How long a step may wait for that lookup, so a strategy lands behind the prompt it was selected for rather than after the model has already chosen what to do. The budget was sized for the two serial round trips recall replaced, where the lookup took 4.8s–9.4s with a 5.1s median over six prompts against a live Hub; one call should sit well inside it, and the default stays until the single-call cost has been measured the same way. Past it the step proceeds and the strategy, when it lands, is injected into the next step. |
| `assetPrimeTimeoutMs` | `8000` | Deadline for the recall call. A cold Hub query runs into seconds, so this is generous; `assetPrimeWaitMs` is what protects the response. |
| `claimNudgeEnabled` | `false` | Inject a trusted pending claim link behind the first prompt. `/evolver-status` remains available when off. |
| `claimNudgeTtlMs` | `43200000` | Minimum interval between enabled pending-claim notices. |
| `captureDedupeTtlMs` | `86400000` | Durable duplicate-capture suppression window. |
Expand All @@ -159,7 +159,7 @@ Environment overrides: `MEMORY_GRAPH_PATH`, `EVOLVER_WORKSPACE_ID`,
- Node.js 22.13 or newer.
- Git for turn capture.
- dsh `0.1.5-rc.2` or `0.1.6-alpha.1`.
- Optional network tools: a local Proxy from `@evomap/evolver`.
- Optional network tools: a local Proxy from `@evomap/evolver` 2.0.39 or newer, which is the first release whose `/asset/fetch` recalls by text.

In a non-git directory the plugin emits one notice per directory, does not create workspace
state, and records nothing. Network strategies are still injected there — they do not
Expand Down
88 changes: 88 additions & 0 deletions scripts/bench-prime.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
#!/usr/bin/env node
// Measures what priming actually costs and delivers against a live Proxy.
// The pass/fail question it answers is not "is the gene good" -- that needs the
// reuse verdicts -- but "does a strategy land behind the prompt, and what does
// carrying it cost", which is what the wait budget and the context bill turn on.
import { readFileSync } from 'node:fs';
import { homedir } from 'node:os';
import { join } from 'node:path';

import { hubGene } from '../src/prime.js';

const PROMPTS = [
'帮我写一篇小红书的种草笔记',
'add a retry with backoff to the uploader',
'优化这个服务的缓存命中率',
'写个脚本批量重命名文件',
'这个 SQL 查询太慢了帮我优化',
'the CI job times out on the coverage shard, find out why',
'把这个 React 组件的重渲染次数降下来',
'design a rate limiter for a public HTTP API',
'这段 Python 有内存泄漏,帮我定位',
'migrate this Express service to Fastify',
'给这个仓库加一个 GitHub Actions 的 lint 工作流',
'explain why this goroutine deadlocks',
];

const WAIT_BUDGET_MS = Number(process.env.EVOLVER_BENCH_WAIT_MS ?? 6_000);

function proxySettings() {
const path = join(homedir(), '.evolver', 'settings.json');
const { proxy } = JSON.parse(readFileSync(path, 'utf8'));
if (!proxy?.url || !proxy?.token) throw new Error(`no proxy in ${path}; start one with \`evolver proxy\``);
return proxy;
}

function proxyFetchVia({ url, token }) {
return async (method, path, body, signal) => {
const response = await fetch(url + path, {
method,
headers: { 'content-type': 'application/json', authorization: `Bearer ${token}` },
body: JSON.stringify(body),
...(signal ? { signal } : {}),
});
return { ok: response.ok, data: await response.json() };
};
}

function quantile(sorted, fraction) {
if (sorted.length === 0) return 0;
return sorted[Math.min(sorted.length - 1, Math.floor(sorted.length * fraction))];
}

function report(runs) {
const hits = runs.filter((run) => run.injected);
const latencies = runs.map((run) => run.ms).sort((left, right) => left - right);
const sizes = hits.map((run) => run.chars).sort((left, right) => left - right);
const inline = runs.filter((run) => run.ms <= WAIT_BUDGET_MS);

console.log(`prompts ${runs.length}`);
console.log(`injected ${hits.length}/${runs.length}`);
console.log(`inline (<=${WAIT_BUDGET_MS}ms) ${inline.length}/${runs.length} — past the budget the strategy lands a step late`);
console.log(`latency ms p50 ${quantile(latencies, 0.5)} p90 ${quantile(latencies, 0.9)} max ${latencies.at(-1)}`);
console.log(`injected chars p50 ${quantile(sizes, 0.5)} p90 ${quantile(sizes, 0.9)} max ${sizes.at(-1) ?? 0}`);
console.log(`total chars ${sizes.reduce((sum, n) => sum + n, 0)}`);
console.log('');
for (const run of runs) {
const late = run.ms > WAIT_BUDGET_MS ? ' LATE' : '';
const what = run.injected ? `${String(run.chars).padStart(5)}ch ${String(run.steps).padStart(2)}步 ${run.header}` : ' — 无注入';
console.log(`${String(run.ms).padStart(5)}ms${late.padEnd(5)} ${what}`);
}
}

const proxyFetch = proxyFetchVia(proxySettings());
const runs = [];
for (const prompt of PROMPTS) {
const started = Date.now();
const { ids, text } = await hubGene(proxyFetch, prompt).catch(() => ({ ids: [], text: '' }));
const lines = text ? text.split('\n') : [];
runs.push({
prompt,
ms: Date.now() - started,
injected: ids.length > 0,
chars: text.length,
steps: lines.filter((line) => /^\d+\. /.test(line)).length,
header: (lines[0] ?? '').replace('[Evolution Memory] ', '').slice(0, 56),
});
}
report(runs);
3 changes: 0 additions & 3 deletions src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ import { Config } from './config.js';
import { EDIT_TOOL_NAMES, editedContent, editedPath } from './edited-content.js';
import { noticeDue, pendingClaimUrl } from './onboarding.js';
import { hubGene, promptTextOf } from './prime.js';
import { markUndeliverable, undeliverableIds } from './undeliverable.js';
import { createProxyClient } from './proxy.js';
import { looksLikeCorrection } from './dissatisfaction.js';
import { correctableAssets, forgetSession, markCorrected, markReported, rememberInjected, unreportedAssets } from './injected-assets.js';
Expand Down Expand Up @@ -133,8 +132,6 @@ function primeSteps(ctx, fallbackDir, config, primeFetch, tracker) {
const search = hubGene(primeFetch, promptTextOf(claimed), {
signal,
listedIds: listed,
skipIds: undeliverableIds(),
onMissing: markUndeliverable,
minSimilarity: config.assetPrimeMinSimilarity,
});
const inline = await Promise.race([search, afterWait(config.assetPrimeWaitMs ?? DEFAULT_PRIME_WAIT_MS)]);
Expand Down
Loading
Loading