Skip to content

perf: collect producer-owned pre-write inventory - #185

Merged
Iviesever merged 1 commit into
mainfrom
codex/v6-producer-prewrite-collection
Sep 13, 2026
Merged

Iviesever merged 1 commit into
mainfrom
codex/v6-producer-prewrite-collection

Conversation

@Iviesever

Copy link
Copy Markdown
Owner

Publish the previously prepared producer-owned pre-write candidate

Continue #164 after #184. This PR publishes the existing candidate; it does not restart the implementation, alter its source to work around a failed check, or claim native acceptance from older PRs.

  • Exact base: eaf3437f8b210b752f0dca8bc98437adf9c8d9e9.
  • Published head: a012b2b7008822f9aa20571fe34ac72caddf1457.
  • Candidate tree: 4f5594da127e9276e65488ee0e5f0be37c2e1920.
  • This is the identical complete source tree from the previously delivered local commit 2c272201c2ef20c55835d48a6248561f6582167f. The published commit has its own message/metadata; the local SHA is not presented as pushed.
  • 18 files, 478 additions / 44 deletions. 433 base files, 434 candidate files; 82 non-main product translation units and the original 79 native-test executable inventory remain unchanged.
  • VERSION remains 5.5.0. No historical tag, asset or formal Release change. Existing CLI/app/platform implementations, benchmark and CI measurement scripts remain unchanged. This is product collection-interface work, not a test-only diff.
  • Temporary perf: title requests the existing independent benchmark comparison, not a performance-improvement claim.

Implemented scope

SourceDiscovery::collect_known_writes and MsvcToolchainLocator::collect_known_writes expose collection-only entry points using the cache destinations owned by the existing producers. Defaults, overrides and disabling rules are shared rather than guessed. Relative destinations require an explicit execution cwd, without reading ambient cwd. Discovery's empty override still means default; toolchain's empty override still disables caching. Automatic toolchain selection retains possible VS bootstrap effects, even before a portable choice is resolved.

MsvcIncrementalTargetCoordinator::collect_prewrite_inventory accepts upstream inventory and collects ordinary-target candidate compile/link outputs and cache replacement namespaces through the actual coordinators and pure recipe builders. Execution and collection share the existing per-source request factory and effective target link policy. Collection does not load caches or use current hits to omit possible writes. Compiler cwd is not silently used as cache-persistence cwd.

Invalid recipes retain original typed errors and source-order slots. Known declarations, later valid sources, LINK candidates and upstream gaps remain visible. VS bootstrap/temp/native-service effects, unenumerated PCH/additional-object producers and final object/default-library LINK routing remain explicitly unresolved.

These APIs do not execute discovery/bootstrap/tools, create directories or marker files, reserve a domain, recover a marker, or provide a complete-write-set/lease authorization predicate. Normal run/CLI/plan/compdb admission policy is unchanged. Full producer closure, missing-root initialization, multi-domain ordering, transactions and recovery remain separate requirements. #172 and historical C1041/#170/#599/cold-tail debt remain open.

Existing local evidence, not new Windows results

The supplied candidate package records completed GCC 14.2/C++23 ASan+UBSan checks: 28 original recipe checks, 44 new producer checks, the complete incremental compile-coordinator executable, and the complete target-coordinator executable including its existing 84 admission checks. Two compiled mutations were rejected: dropping bootstrap gaps and replacing per-source cwd with target cwd. Collection/failure paths preserve tree bytes/mtime and do not create parents; a separate positive control subsequently executes SourceDiscovery to validate its actual cache destination and warm reuse.

Those are prior local Linux/shim results, not Windows/MSVC/NTFS execution on this published head. Initial type/declaration/link-setup failures remain in the original delivery. During publication, the unchanged verification script was rerun successfully over the complete source archives, all retained file hashes, raw local commit, manifest and exact patch replay. No new local compiler run or performance measurement is claimed here.

Required acceptance gates — unchanged and declared before this PR's measurements

  1. Verify exact base/head/native test-merge trees and full source diff. Complete Windows Native Debug and Release, all shards, self-host, exact runtime-only package and applicable documentation/freshness/inspection gates must pass. Confirm the 44 new producer checks actually execute in both configurations, not only a green workflow summary.
  2. Retain original domain 39 / mapping 33 tests, default 70 / private 42 required positive controls, and six real batch / six real target-cache regressions when triggered. Preserve original unsafe-policy failures and recovery results separately. Untriggered historical research must not be described as rerun.
  3. Independent unchanged original observer-OFF 19×4 ABBA, exact base/head Release binaries and AB/BA/AB/BA order. All 72 instrumented complete counters/breakdowns/compile-link-archive hit/miss identities must match. Four external timings-off counter sets are unavailable, not zero.
  4. External no-op paired-median regression worse by BOTH >1 ms AND >10% blocks acceptance pending investigation. Preserve every favorable/adverse sample and original failure. Do not weaken freshness, discard presumed noise, subtract AA, or retry the same head until green. No historical root-cause or zero-overhead claim.

Publication state and decision

The original branch already existed at the base. A normal GitHub tree operation identified a missing documentation blob; the exact unchanged blob was uploaded and the full tree was created with the expected hash. A new commit was created and the existing branch advanced with force=false, then read back at the published head. Partial header-only trees were not committed or presented as complete. Earlier safety refusals and their records are not rewritten as success.

Draft — submitted for this head's validation, not approved for merge. Native CI and ABBA are not claimed complete. No change to main, VERSION, historical releases, full M1b status or write-lease/recovery authority is authorized by publication. Results and the subsequent acceptance/HOLD decision belong in this PR and #164.

Publish the previously prepared and locally verified candidate without changing its source tree. Share early discovery/toolchain cache destination policies and ordinary-target compile/link request construction with explicit collection-only interfaces. Preserve typed recipe errors, source indices, unresolved native effects and caller-supplied execution contexts; do not infer authorization from known outputs or cache hits.

Add 44 producer checks and preserve the original recipe/target tests. No CLI admission, process cancellation, marker recovery or complete write-set claim. VERSION remains 5.5.0; Windows native validation and independent ABBA are required on this published head before acceptance.

@Iviesever Iviesever left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

实质验收决定:ACCEPT,仅限 producer-owned collection;不是 5.6.0 发布许可

本次按维护者明确指示验收已经提交的 #185,不重复提交源码、不创建竞争 PR。已经读取 #164 正文及全部评论、#185 全部 18 个文件的 diff、当前 main/CI、CONTRIBUTING_ZH、cpp/README 目录/依赖契约及 DEVELOPMENT/SELF_HOSTING 契约。此 COMMENT 是使用当前维护者连接执行的证据审查,不冒充另一名独立人工 reviewer 的 APPROVE。

1. 精确对象与审查边界

  • base/main:eaf3437f8b210b752f0dca8bc98437adf9c8d9e9(写入本审查前已重新读取,未移动)。
  • head:a012b2b7008822f9aa20571fe34ac72caddf1457
  • 原 CI test-merge:ba08816932d1f5da41f136f2abc308c9ac0208a0,父提交为上述 base/head。
  • head 与 test-merge 的完整 tree 都是 4f5594da127e9276e65488ee0e5f0be37c2e1920。只接受这一完整树,不以此前未发布的本地 SHA 代替。
  • 18 files / +478 -44;82 个 non-main production TU、原 79 个 native test executable inventory 不变;VERSION=5.5.0

源码审阅未发现阻止本次有限集成的问题:收集入口使用各 producer 的真实缓存路径策略和普通 compile/link recipe;相对路径要求显式 cwd,不从环境 cwd 猜测;保留原始顺序、typed recipe errors、后续有效条目、upstream gaps 以及 bootstrap/temp/native-service/PCH/additional-object/final LINK routing 未闭合项。compile cache persistence cwd 没有被每源 compiler cwd 偷换。目标 LINK 请求共享构造后的 sanitizer/OpenMP/force 语义与既有执行保持一致。

收集接口没有接入正常 CLI admission,没有启动工具/建目录/建 marker/申请 reservation/执行恢复;路径映射不授予 lease,cache hit 不用于删除候选写入。布局 gate 只登记新增测试头,没有放宽目录或职责约束。上述是有限接口/语义验收,不是 complete-write-set、零开销或完整事务声明。

2. 原始 Windows CI、实际执行与自举/包

原 head 的 17 个 PR 工作流均 completed/success,使用其原始 attempt 1;没有要求重新跑 CI 或 benchmark。相关证据不是本会话新建的 Windows 本地运行,而是逐项审查已完成 Actions 的步骤、原始日志和下载产物。允许的过滤/发布跳过没有计入实际执行。

  • Native Debug #751:四个 paired jobs / 八个 subshards 均成功。新增 prewrite_inventory_cases 44 在 job 103693123494 实际执行;原 target_compile_admission_cases 84103693123532;原 write_inventory_cases 28、domain 39、mapping 33 在 103693123491,均有实际通过日志。
  • Native Release #631:对应完整矩阵成功。新增 44 在 103692890255;原 84 在 103692890260;原 28/domain39/mapping33 在 103692890237,不是只看 workflow 总绿灯。
  • 44 项结尾明确保留 collection_tool_dispatch=false; discovery_positive_control=true; lease_authorized=false;28 项保留 complete_write_set=false;39/33 项保留非完整事务/不授权写入边界。
  • Release self-host/package job 103693144511:pinned historical seed→Stage 0→Stage 1→清空状态→Stage 2,版本均 5.5.0;Stage 1/2 smoke 与 installer install/reinstall/uninstall(含路径/包装器)验证通过。
  • 下载包独立复核:内层 ZIP SHA-256 2920b100fe538a36eb38121fc8a6df89723c8a27d2cbc7e84dfa04e74b6cc79d;sidecar 匹配;扁平且精确七文件 mqb.exe, VERSION, install.bat, install.ps1, uninstall.bat, uninstall.ps1, LICENSE
  • 包内 exe SHA-256 f15177cb856613477f0618179fd2d87640ccf125493092b1d110171b68a22205 与已验证 Stage 1 匹配;不声称 Stage 1/2 binary byte-identical。
  • publication job 103694270350 是 skipped,不是已发布。
  • 原 stdout/stderr、负向编译/链接诊断、退出码,以及 C4100/C4996/D9025 等警告保留;没有按“零诊断”交付。

3. 回归阳性对照与不利结果分开结算

Domain #3:Debug/Release 各 39 domain +33 mapping;checks/provenance/source head 对应当前候选,readonly before/after tree 相同。

Default endpoint #34:70/70 完整采集、collector failures=0;unmanaged-normal、direct drain、scheduler drain 各 14/14 必要阳性对照通过。保留两个首次 B 失败

  1. pch-release-A-started-normal:A=0;B0/B1=2/2,LINK/run 未执行;另记 recovery compile=0。
  2. modules-debug-A-started-cancel:A=1223;B0/B1=2/2,LINK/run 未执行;另记 recovery compile=0。

Private service ownership #36:42/42 完整采集;unmanaged-normal 14/14 阳性对照通过。保留三个首次 B 失败

  1. pch-release-A-started-normal:A=0;B0/B1=2/2。
  2. pch-release-A-started-cancel:A=1223;B0/B1=2/2。
  3. modules-release-A-started-normal:A=0;B0/B1=0/2。
    这些失败的 LINK/run 未执行;各 recovery compile=0 是后续结果,绝不覆盖首次失败。

下载后离线检查 default 的 719 个 process/result records、private 的 411 个 records:stdout/stderr 与对应命令记录齐全(aggregate A 使用 case.argv,实际 native commands 位于 A/ 下)。原诊断含 default C1090,private C1090/C2471/C1051。采集完成不等于受管服务清理策略安全;两组全部 observation 仍为 safe_to_integrate_cancellation=falsesafe_to_transfer_write_lease=false

Real batch #12Real target admission #10:各 6/6,not-run=0,Debug/Release complete/cancel/failure+cancel 原始路径、必要 cache/产物保留对照通过;不授予 CLI 集成或 lease transfer。

4. 独立 ABBA #622:原协议、全样本,不是累计发布比较

Performance comparison #622,job 103691911779:独立 clean Release base/head、原 observer-OFF harness、19 scenarios×4 pairs,顺序 AB/BA/AB/BA。下载 JSON 重新计算全部 76 pairs:72 组 instrumented 的完整 counters、counter_breakdown、compile/link/archive hit/miss 向量一致;4 组 external timings-off counters 不可用,未测 hit/miss 保持 -1,不伪造为零。

外部 no-op 四个 delta,按原顺序:-0.8656, +3.9745, -0.1258, -0.0495 ms;百分比分别 -6.8711501%, +34.5689858%, -1.1217720%, -0.4350998%

配对中位差 -0.08765 ms / -0.7784359%,MAD 0.40805 ms;未触发预先规定的 paired median BOTH >1 ms AND >10% 阻塞。+3.9745 ms / +34.5689858% 不利单样本完整保留,不删样本、不扣 AA、不推断为已证明的噪声或零开销。 四样本的 nearest-rank p95 只是最大观测,不冒充总体尾部结论。

全场景配对中位差 / 百分比 / 最大观测 delta(ms):

scenario median ms median % max delta ms
cold +42.454 +2.62 +105.821
no-op +0.002 +0.05 +0.101
single-tu +1.850 +1.95 +27.232
public-header -20.829 -16.43 +10.749
build-run +0.731 +4.82 +1.630
link-only +2.331 +3.66 +19.387
scale-cold -61.057 -1.51 +30.099
scale-no-op -0.469 -1.35 +0.915
scale-no-op-auto -0.101 -0.32 +0.099
scale-no-op-j1 -0.243 -0.56 +3.162
scale-common-header-no-op +0.013 +0.04 +0.395
scale-single-tu +3.248 +2.49 +25.476
discovery-cold -31.540 -1.91 +157.975
discovery-no-op -0.118 -1.42 +0.256
discovery-header +3.995 +3.52 +10.230
modules-cold -52.897 -2.98 +15.474
modules-no-op -0.109 -2.14 +0.097
timings-enabled-no-op -0.098 -2.55 +0.101
timings-disabled-no-op -0.088 -0.78 +3.975

这是 eaf3437→a012b2b 的单 PR 比较,不是 released v5.5.0→完整 5.6.0 候选的累计回归,不能替代后者。

5. 原产物身份与复核方式

以下原 artifact ZIP 均已下载并校验 SHA-256/ZIP CRC,无原数据改动:

  • performance 103141013957951af6de742ac019129c66bbf6c74c9e947a8a4c9e01964e055af2b4766c6eb
  • default 1031428156167337995169227320e7e94d733313d2070dab8af133445a4eb124ff14d7c6ea6
  • private 103143759697461d356946d5fcc6e5e73870e612fc5ed1ffed9416ec2dc9e0cac0a4ab40c31
  • batch 103144904515fa792765b0514598f7e3bd69361d10882295f48e896421e361351ff683305e3
  • target 103128548608e973133cfae9dff09517e79891083e46d3237afeb46316010f1819a864f20e5
  • domain Debug 1031414093583deb99ee6ff7c338a4186a6a0da1ce88d2a88ff8824975da7f6da0899cddd5d
  • domain Release 103138516016c6ddd7daea5c5ec2fc373ed870cd1ee7b6c286fde5781cf8682533dcc934804
  • runtime package outer artifact 10314685188b393ffa97b1d0f36f70f02621f737ff0be3a1d1de6acb42f0a24cdf81bdf8ce0

额外离线审计脚本曾因其自身把 inner iteration 当 outer pair、把 aggregate A 当单工具命令而两次拒绝;原失败脚本及原因已保留,按原 JSON/文件布局修正读取映射后通过。没有重新执行任何 Windows CI/build/benchmark,没有改原样本或验收阈值。

6. 结论与持续阻塞

允许将上述 exact head 作为有限 producer collection 集成。 决定依据是源码边界、当前完整 native gates、必要阳性对照、原始诊断及预定 ABBA 门槛,而非工作流数量或抹掉失败。

不授权发布 5.6.0,不关闭 #164 的 6.0.0 路线。 #172 继续 HOLD。历史 #172/#176 的 C1041、#170 的 +1.6946 ms/+13.7988%、#180 原 #599 的 +2.07175 ms/+16.2700%、common129 cold tails 保留。#170/#180 此前的有限开发集成风险接受不是 stable release 豁免;本次新绿灯不解释或清零旧失败。发布阻塞须有逐项证据处置,必要时修复/限定范围/回退,并在完整候选上验证,不自动豁免。

下一接手主题:合并后冻结 5.6 新功能,以新的 exact main 为 base,先做累计正确性与兼容性回归;随后 released v5.5.0 对完整候选性能及历史风险处置,再文档/打包收口。独立 VERSION/release PR 及最终发布须另行授权。开发期间 VERSION 仍为 5.5.0,历史 tag/资产不动。

@Iviesever
Iviesever marked this pull request as ready for review September 13, 2026 08:34
@Iviesever
Iviesever merged commit b7e322c into main Sep 13, 2026
36 checks passed
@Iviesever
Iviesever deleted the codex/v6-producer-prewrite-collection branch September 13, 2026 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant