Repository navigation
perf(research): avoid repeated full-file source verification - #7250
Conversation
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
…-read-once-20261024 Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
loopx-agent
left a comment
There was a problem hiding this comment.
Reviewer: model_agent | gpt-6.1-sol | OpenAI | runtime_reported | reasoning_effort=max
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
动机
查看已保存研究报告、需要核对原始引用的用户。
同一源文件中有多个引用时,普通报告读取会为每个引用重复读取、哈希并解码整个文件,报告越完整,重复成本越高。
用户打开包含同一大文件多个摘录的既有报告:原实现重复验证整文件;当前实现每次验证按文件只读一次,仍检查每个声明的摘要和字面摘录,返回原有完整报告。
同一物理合成数据、32 MiB 源文件、两个各含 65 个摘录的已注册包,经真实 HTTP 入口交替执行三对:中位耗时由 4.378 秒降为 0.104 秒;独立观察的整文件逻辑读取由 195 次降为 3 次。有效响应完整相同;同大小、同 mtime 的篡改仍被拒绝,源文件缺失仍保留历史结果。
本次完成既有报告读取的有界重复 I/O 修正,不涉及报告质量、生产分位延迟或长期成本收益资格。没有页面布局、新模型执行入口或持久化缓存;查看报告不触发额外分析。
改动思路
现有 TypeScript prepare/qualify 与研究包 owner 已保存源身份、引用和结果;缺口只在 Python 文件 provider 的重复工作。复用原路径/摘要/摘录检查并共享这次读取即可,无需新缓存失效策略、设置、语义 owner 或数据库迁移。
在一次 _verify_sources 调用内按规范化 artifact 路径分组;原 _source_path 继续验证根目录,源字节读取和 SHA 计算每文件一次,文本按需解码一次,再逐项核对全部期望 SHA 与 literal excerpt。调用结束即丢弃分组。
| 固定 HTTP 工作负载 | 基线 | 当前 |
|---|---|---|
| 三对暖缓存请求中位耗时 | 4.378 s | 0.104 s |
| 每次报告请求的整文件逻辑读取 | 195 | 3 |
表中耗时与读取次数分别测量,不能推导物理磁盘吞吐。
已在原 provider 内完成同域的局部简化:共享一次读取与按需解码。更大的报告路径/语言迁移不属于这次重复 I/O 修正,也不应以持久化缓存扩大维护面。
原实现的完整性、权限、默认和重试规则由原 owner 保留;本次不增加设置、人工输入或新的确认轮次。异步观察和历史结果不变成新执行授权。
具体改动
未发现阻塞项。同一物理合成数据、32 MiB 源文件、两个各含 65 个摘录的已注册包,经真实 HTTP 入口交替执行三对:中位耗时由 4.378 秒降为 0.104 秒;独立观察的整文件逻辑读取由 195 次降为 3 次。有效响应完整相同;同大小、同 mtime 的篡改仍被拒绝,源文件缺失仍保留历史结果。
先读不可变原规格 docs/reference/trajectory-assessment.md(28fa39bd8a2f172c8a1cfde5b144dbd249f165a8),再审完整补丁和调用者。下列 criterion 是本次审计映射标签,非伪造原编号。
-
source-integrity-and-root(implemented):Every declared expected SHA, literal excerpt and contained source path remains checked. -
saved-report-no-dispatch(implemented):Reading a retained report preserves history and dispatches no new analysis. -
repeated-file-work(implemented):Multiple excerpts from one file share one complete source observation per verification invocation. -
production-and-long-term-qualification(out_of_scope):Production latency distribution, research quality and complete long-horizon utility require separate evidence.
3 files +95/-8; production +17/-8。无持久化格式、公共参数、状态值或权限迁移。单次调用内每文件观察一次,不宣称跨文件原子快照。多无效文件的首错顺序变化明确披露;单一故障的完整诊断和历史结果保持。 完整讨论、既有实现复用、DCO 与公开边界已核验。
关键代码讲解
_verify_sources(loopx/capabilities/benchmark_toolkit/trajectory_assessment_store.py:70):按文件共享当前调用的读取、哈希与按需解码,仍遍历每条声明;交错多错误的首错顺序可变。
_source_path(loopx/capabilities/benchmark_toolkit/trajectory_assessment_store.py:92):复用原有 source_root 包含约束;分组并未把未授权路径变成合法源。
inspect_pack(loopx/capabilities/benchmark_toolkit/trajectory_assessment_store.py:478):读取既有包/结果并经 availability 重验原源;失效源保留历史结果,读取不启动模型。
read_citation(loopx/capabilities/benchmark_toolkit/trajectory_assessment_store.py:645):引用读取沿同一包检查与原字面范围返回证据;没有新增独立缓存或放宽引用身份。
对主干的风险
51 项针对性测试通过。标准预合并 6 项直接检查与 6 项目录 smoke 全过,但整体仍为 manual_review_required,benchmark_sensitive 人工 hold 保留。真实 HTTP 配对返回相同完整结果;同一现有 HTTP 回归断言在旧 verifier 下失败(5 次读取而期望 3 次),当前通过。
同大小同 mtime 篡改返回 source_changed;文件缺失返回 source_unavailable,二者都保留原历史报告。稍后引用的错误 SHA、字面摘录不匹配和越根路径继续拒绝。交错的多个无效文件可能改变首个报错:A 正常、B 摘录错、A 后续 SHA 错时,基线 excerpt_mismatch,当前 source_changed;二者拒绝,原规格未承诺多错误优先序。
性能数字来自暖缓存、单并发的固定合成报告,宿主全局负载未控制,不代表冷缓存、物理磁盘吞吐或生产 p95。仅一次合成 provider 初始化报告,读取新增调度为零,无 live 模型调用。Pyinstrument 观察旧 store 的 Python 线程,最大自耗时在 verifier;它不证明 Node CPU、原生 CPU 与阻塞时间的分离,native inspector 的 root_cause_proven/benchmark_qualified 仍为 false。
配对只替换不可变原/当前 verifier 函数;其余生产 AST 相同,真实 HTTP、源文件、注册表、TypeScript 资格和保存结果均参与。完整响应与原 source/job/runtime 字节保留,没有只比较 reason code 或为性能移除证据。Profile 最大自耗时落在原 verifier;其采样 inclusive 时长相互重叠,不可相加,也不是 HTTP wall latency。
语义与 CI 对齐
Only an invocation-local grouping of existing normalized artifact strings and source bytes; derived projection, not persisted fact, user intent, cache or configuration. Existing SHA, scope and saved job retain authority.
Every declared source in the current registered pack is checked; the original contained-root selector and every expected digest/excerpt remain mandatory. Read availability never grants explicit model execution. 此为既有入口的默认修正,没有新增 opt-in/default-off 声称。多个无效文件的首错顺序是有意分组带来的可观察差异,原规格未规定该顺序,本评审没有宣称这些响应逐字相同。 CI 按契约未查询、轮询或等待;本地红项、人工 hold 与评审结论分开保留。
我的整体评价
APPROVE,仅对应 7250@cb9b20f96b680519d3b0e98699bf40c1ba970e16。本次有界用户问题的局部交付达到目标,未扩展为父目标完成。
长期工作:improved;使用体验:improved。同一物理合成数据、32 MiB 源文件、两个各含 65 个摘录的已注册包,经真实 HTTP 入口交替执行三对:中位耗时由 4.378 秒降为 0.104 秒;独立观察的整文件逻辑读取由 195 次降为 3 次。有效响应完整相同;同大小、同 mtime 的篡改仍被拒绝,源文件缺失仍保留历史结果。
已检查实际 HTTP 完整报告读回及既有页面消费者;无渲染布局、样式或素材变化,不需新截图。 未验证范围:性能数字来自暖缓存、单并发的固定合成报告,宿主全局负载未控制,不代表冷缓存、物理磁盘吞吐或生产 p95。仅一次合成 provider 初始化报告,读取新增调度为零,无 live 模型调用。Pyinstrument 观察旧 store 的 Python 线程,最大自耗时在 verifier;它不证明 Node CPU、原生 CPU 与阻塞时间的分离,native inspector 的 root_cause_proven/benchmark_qualified 仍为 false。 安装版及未执行的页面端到端旅程不记为通过。
有界重构判断:已在原 provider 内完成同域的局部简化:共享一次读取与按需解码。更大的报告路径/语言迁移不属于这次重复 I/O 修正,也不应以持久化缓存扩大维护面。 本次完成既有报告读取的有界重复 I/O 修正,不涉及报告质量、生产分位延迟或长期成本收益资格。没有页面布局、新模型执行入口或持久化缓存;查看报告不触发额外分析。
这是 author-owned COMMENTED 结论,GitHub 不支持正式自我批准。批准结论、approval-closeout、合并就绪、未通过验证/人工要求与合并权限分别读回;本评审不合并。
English verdict: APPROVE. APPROVE. A real HTTP, same-fixture comparison preserves complete valid and single-failure responses while reducing repeated logical reads from 195 to 3 for the fixed 32 MiB workload. The unchanged HTTP read-count oracle fails with the original verifier and passes at this head. All 51 focused tests and automated premerge checks pass; the benchmark-sensitive manual hold remains. Interleaved invalid files can report a different first error, explicitly disclosed. No live model or production-tail qualification is claimed.
Goal And Delivered Outcome
Opening a research report reread, hashed and decoded an entire source file for every excerpt. Large files with many excerpts therefore repeated gigabytes of work before the same saved result could appear. Verify each declared artifact once within a pack verification, checking every expected digest and literal excerpt against those bytes. Each later verification still reads fresh bytes.
docs/reference/trajectory-assessment.md, at main28fa39bd8a2f172c8a1cfde5b144dbd249f165a8.loopx-project/loopx:main. Builds on the separately reviewed report-generation change feat(research): guide readers from results to reports and evidence #7233, now in main; its author head was left unchanged.Author Declaration
Written by: model_agent — OpenAI GPT-6 family.
Self-check: inspected the diff, source/authority boundaries and applicable optimization rules. The change stays in one existing verifier, adds no persistent cache, setting, transport call or abstraction, and processes one artifact's bytes at a time. No new semantic vocabulary. Public inputs below are fully synthetic; raw profiles and research material remain private.
Performance Evidence
Synthetic workload: two registered packs, each containing 65 excerpts from one approximately 32 MiB file; the real loopback research GET reads the catalog and selected pack. Python 3.11.17 on macOS, serial alternating base/head requests, three pairs, warmed process/file cache. Both arms use the production request handler; the only production difference is the verifier below, with the base function taken from immutable
ad6d2f6d3. No model jobs are created.This measures logical reads and request latency, not physical disk traffic, cold-cache latency, p95, user comprehension or long-horizon benchmark benefit. A separate isolated saved-report integration check also preserved the complete qualified result, history and source bytes; no active solver, evaluator or model job was touched.
Validation
Tested head:
cb9b20f96b680519d3b0e98699bf40c1ba970e16; source change4efc94dd41182044ab55d6c73a39313d9f03eef7, integrated with main28fa39bd8a2f172c8a1cfde5b144dbd249f165a8. The verifier is byte-identical to the measured candidate. Inputs: synthetic/public fixtures and authorized isolated read-only integration evidence.npm ci --ignore-scripts, all six selected canaries and direct/compile checks pass. The merge gate remains held for the missing exact-scope change-quality receipt and independent benchmark-sensitive review. No merge readiness is claimed.Scope And Continuation
Complete within the proposed source-read optimization. Every digest and excerpt remains checked; grouping takes one byte observation per artifact within that verification, with no reuse across later verifications. Artifacts are visited in first-appearance groups, so a pack containing several independently invalid artifacts can report a different first failure; invalid material is still rejected. There is no atomic snapshot guarantee across distinct files, as before.
The existing research frontend consumes the same HTTP schema and result; no layout, model generation, task scoring, budget, scheduler or authority-store change. Installed adoption and maintainer review/merge remain separate. This author will not self-merge or enable auto-merge. The broader 2.0/research acceptance remains open.
Boundary Checklist