perf(explore): hash only findings emitted in turn context

[hhyykk]

Goal And Delivered Outcome

Explore's bounded turn context hashes every full finding when stride requests source digests, even though it returns only a recent window, a result page and selected-branch evidence. Large graphs therefore serialize and hash findings that never reach the caller.

This change retains the canonical finding lookup and calculates each digest when that finding is emitted. Recent results, paged results and older audited findings still hash the complete original source. The full result-page revision remains unchanged.

• Outcome basis: a focused engineering performance improvement; no issue closure.
• Intended base: main, implemented from 91b675dfca178db68d3527140272ffb263a3b319.
• Measured outcome: on 5,000 synthetic findings with digests enabled, median projection time fell from 44.63 to 16.61 ms when all findings were writeback results, and 26.69 to 2.79 ms when one in eight was a writeback result. Full output parity passed. These are projection-only measurements, not full Turn latency or model-quality results.

Author Declaration

• Written by: model_agent — OpenAI Codex.

Implemented against

• Basis: the engineering optimization request in this PR; preserve the existing Explore bounded-context contract at the baseline revision.

Criterion	Disposition	Owning path	Validation
Avoid hashing findings omitted from the returned view	implemented	loopx/control_plane/capabilities/explore_turn_context.ts	Controlled baseline/head projection benchmark below
Preserve complete-source digests across recent, paged and audited findings	implemented	Same typed projection owner	tests/control_plane_ts/explore_turn_context.test.ts; changed-source readback
Preserve digest-off output and pagination identity	implemented	Same typed projection owner	Full-packet baseline/head parity, including continuation pages and digest-off cases

• Self-check: reviewed all three digest projection sites, the Effect handler registration and the Python Explore caller; applied clean-code-guard and test-guard. No new cache, strategy, provider or configuration surface was introduced.

Scope And Continuation

• Complete within the projection optimization scope. Existing findings and the existing TypeScript owner are reused; the small shared digest helper keeps all three output surfaces aligned.
• The lookup still scans findings and pagination still hashes its full result set. This does not establish bounded history IO or overall long-running performance.
• Runtime-backed verification must be rerun in an environment that permits local sockets before merge. No deployment, live Goal or model experiment was run.

Validation

• Tested revision: e776e6b3a6bc4c8b3e756a561f8a8b6e2c339428 (final source; premerge and Python checks were run on the same source before the commit).
• Run state: finished.
• Input classes: synthetic, public_fixture.

Check kind	Result	Evidence / limitation
static	passed	npm run typecheck:control-plane; diff hygiene; semantic-inventory advisory found no supported new vocabulary carriers
unit	passed	46 tests: Explore context/result writeback, stride policy and replan semantics; Node 24.21.0 with --test-isolation=none
regression_parity	passed	Complete baseline/head packets match for 100 and 5,000 findings, dense/sparse writeback results, digests on/off and continuation pages
manual	passed	31 alternating timing samples after 8 warmups per case; same synthetic inputs and runtime; reproduction below
real_entrypoint	blocked	Attempted Explore CLI/context and long-chain Python tests: 7 passed, 37 failed while the Effect runtime could not start. A direct startup probe reports listen EPERM on loopback; the unchanged baseline also fails the selected digest integration test at runtime startup
static / integration	failed	canary premerge --from-git-diff --git-diff-base origin/main: all 3 direct checks and 7 of 10 selected checks passed. Semantic-vocabulary, auto-research quota-pause and state-aware-wake checks stop at Effect runtime startup. This is not a green premerge result

The unit tests exercise output behavior without mocks. The new cases cover an audited finding outside both displayed graph windows and verify that later source changes update its digest. Full Python CI, socket-backed execution and real workload gains remain unqualified.

Frontend / Visual Evidence

• UI impact: none. Output shape and content are preserved; visual, attention and craft evidence are N/A. Source data: none.

Type of Change

• Refactoring (no functional changes)
• Test update

LoopX Area

• Control plane
• Capability or extension

Technical Direction

• Existing S10 operational-overhead direction; scoped to the Explore read model. No broader milestone closure is claimed.

Shared-authority RFC fixture impact

• N/A: no storage-provider, authority, lease, quota or runtime-routing change.

Boundary Checklist

• Diff and PR contain synthetic/public inputs and aggregate results only; no private state, credentials, raw traces or machine paths.
• No maintainer-owned benchmark runner, scoring or experiment work changed.
• Change stays within the stated projection optimization.
• UI impact is none.
• Commit includes a DCO Signed-off-by trailer.

Reproduce the projection comparison

Use Node 24.21.0 and separate checkouts of baseline 91b675dfca178db68d3527140272ffb263a3b319 and this PR. Save the following as projection-bench.mjs, then run node --no-warnings --experimental-strip-types projection-bench.mjs BASE_CHECKOUT HEAD_CHECKOUT. Paths are arguments; the script uses only synthetic in-memory inputs. Absolute latency depends on the host.

import assert from 'node:assert/strict';
import {performance} from 'node:perf_hooks';
import {pathToFileURL} from 'node:url';
import {resolve} from 'node:path';
const modulePath = 'loopx/control_plane/capabilities/explore_turn_context.ts';
const {projectExploreTurnContext: baseline} = await import(pathToFileURL(resolve(process.argv[2], modulePath)).href);
const {projectExploreTurnContext: candidate} = await import(pathToFileURL(resolve(process.argv[3], modulePath)).href);

const median = samples => samples.slice().sort((a, b) => a - b)[Math.floor(samples.length / 2)];
for (const count of [100, 5000]) {
  for (const writebackEvery of [1, 8]) {
    for (const enabled of [false, true]) {
      const findings = Array.from({length: count}, (_, index) => ({
        finding_id: `finding-${index}`, node_id: `node-${index % 30}`,
        finding: `Scoped synthetic observation ${index}`, status: 'confirmed',
        summary: 'Applicable only to the measured synthetic input family. '.repeat(20),
        evidence_refs: [`validation:case-${index}`],
        tags: index % writebackEvery === 0 ? ['writeback-result'] : [],
      }));
      const request = {goal_id: 'research', agent_id: 'worker', route: ['loopx'],
        graph_enabled: true, harness_gate: {enabled: true}, include_finding_digests: enabled,
        projection: {nodes: [], findings}, plan: {selected_branches: [{todo_id: 'audit-prior-result',
          typed_evidence_audit: {requested_node_refs: [findings.at(-1).node_id], findings: [findings.at(-1)]}}]}};
      assert.deepEqual(candidate(request), baseline(request));
      const page = baseline(request).graph.result_page;
      assert.deepEqual(candidate({...request, result_offset: 3, result_revision: page.revision}),
        baseline({...request, result_offset: 3, result_revision: page.revision}));
      for (let index = 0; index < 8; index++) {baseline(request); candidate(request);}
      const samples = {baseline: [], candidate: []};
      for (let index = 0; index < 31; index++) {
        const order = index % 2 ? [['candidate', candidate], ['baseline', baseline]]
          : [['baseline', baseline], ['candidate', candidate]];
        for (const [name, project] of order) {
          const start = performance.now(); project(request); samples[name].push(performance.now() - start);
        }
      }
      const before = median(samples.baseline), after = median(samples.candidate);
      console.log(JSON.stringify({count, writebackEvery, enabled, baseline_ms: before,
        candidate_ms: after, ratio: after / before, full_packet_parity: true}));
    }
  }
}

[loopx-agent]

Reviewer: model_agent; gpt-6-astra; OpenAI; runtime_reported; reasoning_effort=high

动机

使用 Explore 历史证据做复盘的 Agent，会反复读取有界上下文。此前即使只返回少量结果，也会序列化并计算所有 finding 的摘要；历史越长，无用计算越多。此修改只为实际返回的结果计算摘要，保持证据内容和读取流程一致。本 PR 不承诺减少输入 token、改善研究路线或提高评测分数，也不解决完整历史读取的 IO 成本。

在精确 head e776e6b3a6bc4c8b3e756a561f8a8b6e2c339428 上复核了同一合成投影工作负载：5,000 条 finding、全部为 writeback 时，开启摘要的中位耗时从 15.85ms 降到 7.29ms；每八条一条 writeback 时，从 8.75ms 降到 1.39ms。31 个交替采样、每种情况先预热 8 次。完整输出一致。这是该投影函数的局部成本证据，不能外推为整个 Turn 的提速。

改动思路

调用链仍是原有 CLI/复盘读取适配器 → 严格加载 Goal 范围内的 Explore 证据 → Effect runtime 的 projectExploreTurnContext → 有界结果。权威源、排序、分页、状态和权限 owner 均未变。修改把 finding_id 到摘要的预计算表，替换为 finding_id 到完整原始 finding 的临时映射，真正返回条目时才计算摘要。

这是现有 TypeScript 投影内部的局部优化；保留完整原始行，并复用三个投影位置的一个私有 helper，避免引入持久缓存及失效协议。本 PR 的边界是减少未输出 finding 的摘要计算，保留全部现有输出语义。最强反对理由是完整 result revision 仍需扫描并摘要结果集，实际调用也可能被 IO 主导；但该限制不抵消已复现的局部收益，且新增维护面很小。没有必要为此新增策略、配置或并行状态源。

具体改动

规范依据：loopx/capabilities/explore/README.md#explore-harness-modes，固定于 91b675dfca178db68d3527140272ffb263a3b319。按该章节的现有条款检查：

• Bounded context：近期结果和建议分支仍按原有数量展示，超出窗口的证据继续通过既有详情路径读取，未把“未显示”解释为不存在。
• Revision-bound pagination：完整结果修订号仍覆盖原有集合；第二页、过期修订号的拒绝与重新读取流程不变。
• Read-only modes：off/evidence/planning 的激活及只读边界不变，不获取 claim、lease、quota 或执行权限。

关键代码讲解

1. findingDigest（explore_turn_context.ts:16）根据 ID 找完整源行再哈希。关闭摘要或没有对应源时仍不输出摘要；没有用已裁剪的显示文本代替原始证据，因此适用条件变化仍能使摘要变化。
2. projectExploreTurnContext（同文件 :59）在开启摘要时构造原始行映射，近期 finding 和当前页结果都调用相同 helper。resultRevision 的完整集合计算及拒绝分支保持不变；历史增长时节省的是未返回行的独立摘要计算。
3. branchContext（同文件 :33）给已选分支的审计证据计算摘要。即使该 finding 不在近期或当前页窗口中，也从完整映射取源；拒绝候选保持原有无摘要路径。
4. 新测试覆盖开关、第二页、窗口外的旧审计证据，以及只修改源适用条件后的摘要变化。差异共一个生产文件 +12/-7、一个测试文件 +58，没有协议词汇或状态机变更。

对主干的风险

未发现阻塞问题。主要回归风险是误对裁剪文本取摘要，导致适用范围变化无法被识别；或优化时丢掉窗口之外的已选分支证据。精确 head 的 46 项 TypeScript 测试通过；真实本地 Effect runtime、CLI 和长链测试 44 项通过，同一不可变 baseline 也通过 44 项。另用相同 fixture 比较了 21 个真实读取观察，包括三种模式、摘要开关、第二页、失效修订号、未绑定分页、非法布尔值、源更新和复盘读取，完整规范化 JSON 一致；只规范化临时目录及现有路由字段中该目录的哈希，证据摘要和分页修订号未做归一化。

故意把 helper 改为仅哈希 finding_id 后，真实 Explore 读取测试在“适用条件改变但摘要未变”的断言处失败；恢复精确 head 后该用例通过。这证明验证不仅检查字段存在。作者历史上的 loopback EPERM 没有在本次环境重现；这里补齐了当时缺失的真实运行路径，未声称修复作者环境。

类型检查、diff hygiene、语义词汇 advisory、premerge 的 3 项直接检查及 10 项风险/目录检查均通过，无 manual hold。按 Goal 配置，本轮后续未等待或轮询远端 CI。完整仓库测试及真实模型收益不在本次验证范围。共享文件刚合入的 #6379 涉及后续分页消费者，需要在集成分支上继续检查组合路径。

我的整体评价

APPROVE。该 PR 完成了“减少未使用摘要计算、保持输出和权威源一致”的独立优化目标。long_horizon：局部投影成本得到改善，长期研究行为保持；user_experience：原有 CLI、读取和错误恢复流程保持。保持原有语义词汇与默认行为，无新配置、迁移或权限。

已做相邻重构判断：当前共享私有 helper 足够；加入跨请求缓存会引入失效与并发复杂度，不应扩大本 PR。相同 finding 同时出现在多个输出位置时可能重复计算，但每次输出有界，现有测量仍有明确收益；不以此要求额外框架。保留完整历史 IO、完整 result revision 的线性成本，以及未验证的模型采纳和全程净收益作为明确限制。

English verdict: APPROVE — e776e6b. Lazy full-source digest calculation preserves bounded views, pagination and disabled behavior. Reviewer reproduced local projection gains, 21 real-entry parity observations, 44 baseline/head runtime tests, 46 TS tests, a caught semantic mutant, typecheck and all selected premerge checks. No claim of token savings or benchmark score gains.