test(perf): retain fixed-budget attribution for original foreground no-op crossing - #181
Conversation
Reuse the accepted observer/collector unchanged for eight AA and eight AB pairs. Verify exact original #599 source/tree identities with negative controls, archive source and newly rebuilt binaries, and retain complete or partial observations. Keep this evidence-only PR separate from #180 and independent observer-OFF ABBA. No product, freshness, CLI, VERSION or historical asset changes; no automatic performance clearance.
Fixed study completed once; independent raw replay accepts its evidence, not PR180Actual Foreground Attribution #1/34463275198, attempt1, evidence artifact 10146995073:10,983,823bytes,SHA256 Independent Python replay read all32reports,672final records and672started markers. It checked the fixed interleaved schedule against actual sequential monotonic intervals, four clock-boundary arithmetic, exact A/B executable paths/hashes, original merged output/argv/exit0, raw timing JSON-to-report phase/counter/attribution binding, all304pairs and38statistics rows. 288 instrumented semantic pairs have complete matching counters/breakdowns/compile-link-archive hit/miss identities; 16 external pairs have unavailable counters, not zeros. All favorable/adverse observations remain. Actual Windows contracts: 13 synthetic fixed-source checks plus 32 existing observer checks passed. The32-check artifact10146429644 is12,944bytes,SHA256 Source and physical evidenceRuntime A=b16cf049d0c8443288d48a6f712e22e9df6309ea; runtime B=original5835825a0f5337b6491433f9f402ef9434087d09, not input-fixed56b8f08. Observer39b9a84915ada9ef461320680ed361317dc8d437. Canonical Git trees reconstructed from the archived sources match24346d15/e8819f5d/7aea95c2 exactly. All418existing main files remain unchanged in the420-file evidence candidate; only the2declared files were added. The Windows-generated source ZIPs contain CRLF text:417A/419B/419observer text files differ from canonical LF bytes. The actual Archived measured A exe SHA256 Complete observed external-clock results
All rows here use the additional observed external clock, including18scenarios that also emit separate internal timings. They are not the observer-OFFABBA rows. Eight-pairP95 is the maximum; component medians/overlapping work are not additive elapsed. Timings-off no-op exact paired medians: AA +0.0139ms/+0.1444787034%; AB -0.19565ms/-1.7363737221%. Neither crosses BOTH>1ms AND>10%. Keep the AA same-binary+5.3568ms/+49.0738% tail and AB+1.7768ms/+14.3846% tail. No AA subtraction or presumed-noise deletion. On that worst AB no-op pair(block2,BA), recorded delta splits into +1.1622ms push/pop span and +0.6146ms native-invocation+capture span. This is a decomposition of THIS observed pair, not historical599, and the latter is not child CPU time. It does not identify an OS/antivirus/compiler cause. Original#599 +2.07175ms/+16.2700359633% remains a real unresolved crossing. Interpretation and next concrete handoffAccept the evidence as a completed fixed-budget investigation, not proof of a product speedup or historical root cause. #180 is not approved for merge. Do not add unbounded identical-runtime AA/AB runs or let favorable new medians overwrite599. The next useful product iteration is to independently extract the proven vcvars stdin correction onto accepted main, with its native positive/negative controls and its own ABBA, WITHOUT carrying the held build/run structural refactor. Preserve #180 for later redesign/explicit risk review, not a duplicate observer expansion. Scope note: the workflow's run-attempt guard is in the contract job. It must not be described as a global idempotency guarantee against selective job reruns or another manual dispatch. This actual study is a single attempt1; no rerun was requested. The local offline auditor corrected two assumptions(untimed priming calls and CRLF source archives), not product code or data. Those local rejections are retained in the delivery notes. |
Iviesever
left a comment
Iviesever
left a comment
There was a problem hiding this comment.
Final review: ACCEPT this two-file evidence iteration; DO NOT merge #180
Reviewed exact base b16cf049d0c8443288d48a6f712e22e9df6309ea, head 39b9a84915ada9ef461320680ed361317dc8d437, tree 7aea95c20273ec62dbe5469b482b47515898911e. Native test-merge 8669fd33fb48f2732643548eb25c217ecef22339 has that tree and the exact ordered base/head parents. Canonical source-tree reconstruction and git diff --check pass;418existing files unchanged,2new files/211lines. Original product,CLI,freshness,79native executable inventory,benchmark/collector/comparator,VERSION5.5.0 and history are unchanged.
Actual exact-source validation
- Native C++ #740/34463275143: real Debug build, runtime/parameter/freshness special checks, all four paired-shard jobs and final Native C++ gate passed.
- Native Release #625/34463275087: Stage0/shared library, all four paired Release-shard jobs, self-host and exact runtime-only package validation passed. publish-release skipped, not released. Retained package artifact10147204101 SHA256
5ff94d7d630a8438b68a8fbac3566a644baa4ff3ceac335b83a0c08f8987be05; inner package SHA256ac06c1a5c4ee8e7ebd09d38824a6249de0a311183da7fd2d1eeff44b7bbed594matches its checksum and contains exactly the7runtime/install/license/version files, no compiler/build cache. - Documentation #186/34463275112 passed.
- Foreground Attribution #1/34463275198, attempt1: actual13source and32existing observer contracts, both real MSVC-built runtime binaries and the fixed32reports/672calls completed. Full source/binary/record audit and all38observed rows.
- Existing ownership/default/private/batch/target/ETW/Cross-Stage research workflows did not trigger for these2paths. They are NOT called rerun or silently substituted with these results. Their accepted sources and historical debts remain unchanged.
Separate original observer-OFF19x4ABBA
Performance #604/34463275188, actual compare job102825843405, built the exact PR base and candidate separately and performed the original observer-OFF comparison. Artifact 10146670319,35,029bytes,SHA256 e4729cee012239d4f2652fd9edfa82bbea6c9f397e91b6309d098e9b70015569, CRC passes. Independent replay recomputed all76pairs, originalAB/BA/AB/BA schedule,72complete instrumented counter/breakdown/compile-link-archive hit/miss identities; four external counter sets remain unavailable.
External no-op paired median +0.28065ms/+3.2539847375%, MAD0.35655ms,maximum+0.9885ms,3/4slower: does not cross BOTH original limits (>1ms AND>10%). This is not proof of zero overhead despite unchanged product source.
| Scenario | Paired median Δms | Paired median Δ% | MAD ms | Max Δms | Slower |
|---|---|---|---|---|---|
| cold | +22.223 | +1.82 | 32.889 | +56.619 | 2/4 |
| no-op | +0.012 | +0.36 | 0.060 | +0.084 | 2/4 |
| single-tu | -91.966 | -50.62 | 67.605 | +0.492 | 1/4 |
| public-header | -18.745 | -15.72 | 29.506 | +414.348 | 1/4 |
| build-run | -0.516 | -4.38 | 0.405 | +0.049 | 1/4 |
| link-only | -15.139 | -20.79 | 12.366 | -1.550 | 0/4 |
| target-scale-cold | -22.778 | -0.72 | 65.210 | +45.746 | 2/4 |
| target-scale-no-op | -0.533 | -2.31 | 0.562 | +0.283 | 1/4 |
| target-scale-no-op-auto | -0.456 | -2.11 | 0.571 | +0.562 | 1/4 |
| target-scale-no-op-j1 | -1.304 | -4.45 | 0.834 | +1.289 | 1/4 |
| target-scale-common-header-no-op | +0.307 | +1.28 | 0.764 | +1.230 | 2/4 |
| target-scale-single-tu | +1.488 | +1.56 | 26.730 | +69.886 | 2/4 |
| discovery-cold | -0.397 | -0.03 | 20.909 | +1481.324 | 2/4 |
| discovery-no-op | +0.095 | +1.64 | 0.079 | +0.267 | 4/4 |
| discovery-header | +4.337 | +4.88 | 2.906 | +358.675 | 3/4 |
| modules-cold | -0.115 | -0.01 | 39.275 | +1908.931 | 2/4 |
| modules-no-op | -0.073 | -1.89 | 0.111 | +0.763 | 2/4 |
| timings-enabled-no-op | +0.019 | +0.69 | 0.106 | +0.232 | 2/4 |
| timings-disabled-no-op | +0.281 | +3.25 | 0.357 | +0.989 | 3/4 |
Only the last row uses this original external clock. All adverse/favorable pairs remain, especially Modules cold+1908.931ms/discovery cold+1481.324ms and discovery-no-op4/4slower. No offsetting, AA subtraction, presumed-noise deletion or same-head remeasurement. The comparisonJSON does not archive its own binary pair/OS trace; the separately retained study binaries are not mislabeled as #604 binaries.
Explicit decision and scope
Accept #181 for ordinary merge with expected-head verification, only as tested fixed-source evidence infrastructure and a completed bounded investigation. Required repository rules remain enabled; no bypass. Remove the temporary perf title before marking ready so a ready-for-review event cannot launch another measurement.
Observed fixed-study no-op AA=+0.0139ms/+0.1445%,AB=-0.19565ms/-1.7364%; this new sample set does not cross the threshold. It neither repairs nor explains original#599 +2.07175ms/+16.2700%. AA tails do not certify an OS/antivirus cause, and native+capture is not CPU time. #180 remains Draft/HOLD; its latest native stdin fix and older fixture/cold-input failures remain distinct. No CLI cancellation, lease,writer-quiescence,transaction,daemon,#172 or final-release gate is approved.
The run-attempt check resides in the contract job, not a repository-wide idempotency mechanism against selective reruns/manual redispatch. No such action occurred in this study. Offline verifier assumptions about priming records and Windows CRLF archives were corrected and recorded without changing data or rerunning MSVC. Delivery retains complete study data/build transcript/source ZIPs, both historical ABBA ZIPs, own ABBA/package/contracts and replayers; it does not claim all native job logs are included.
Next product handoff: independently extract the proven vcvars stdin fix on the accepted main, preserving its original native positive/negative controls and adding that PR's own ABBA. Do not include the held build/run refactor or restart an unbounded observer investigation.
|
Precision correction to review5165888227: the exact #604 external no-op MAD is0.3572500000000005ms, not the mistyped0.35655ms in that review's prose. The19-row rounded table's0.357ms is correct. Original four paired deltas are [-2.2207,+0.2873,+0.2740,+0.9885]ms; paired median+0.28065ms,median percentage+3.2539847374786346%,maximum+0.9885ms,3/4slower. The raw ZIP, independent replay JSON, unchanged threshold and acceptance decision are unchanged. This note corrects descriptive precision only and is retained alongside the review. |
Iviesever
changed the title
…din fix Preserve the complete #180 history and original failures. Bring in accepted #181 diagnostic entry points and #182's identical stdin/CLI test file versions without rewriting history. Relative to current main, retain only the nine-file application-private build completion/foreground execution boundary and its completion/module tests. No new cancellation, lease, observer or VERSION change. This combined source needs its own full native/self-host/package and unchanged independent ABBA; historical #599 remains an unresolved risk, not a repaired measurement.
1 participant
Accepted evidence-only iteration; PR180 remains held
#181 is accepted for ordinary merge after its exact-source gates. This is not approval of #180 or a performance fix. Continue #164 without rewriting its roadmap. VERSION stays5.5.0; no historical tags/assets or formal release change.
b16cf049d0c8443288d48a6f712e22e9df6309ea, tree24346d15b7f245396c972dfd2478b2ef0ecaa3a4.39b9a84915ada9ef461320680ed361317dc8d437, tree7aea95c20273ec62dbe5469b482b47515898911e.8669fd33fb48f2732643548eb25c217ecef22339, same tree, exact base/head parents.Premeasurement fixed-budget declaration · Full raw study audit/all38rows · Final review/all19independent ABBA rows · Exact MAD precision correction.
Code and fixed study
The new runner checks original source/tree identity, clean tracked state,VERSION and unchanged measurement/build helpers; archives A/B/observer source and newly rebuilt Release binaries; and invokes the already accepted collector unchanged. It does not weaken #171's original identical-product experiment: this separate study explicitly compares different product compositions.
Runtime A=b16cf049d0c8443288d48a6f712e22e9df6309ea, B=original#599head5835825a0f5337b6491433f9f402ef9434087d09. B is not silently replaced by the input-repaired56b8f08 head. Fixed8AA+8AB,32reports,672invocations including64priming calls; interleaved contrast/order,AA same A path/bytes. Original outputs,failures and unavailable counters are retained. No extra adaptive samples,AA subtraction,newETW,product instrumentation,compiler Job or input-policy edit.
**Actual Foreground Attribution#1/34463275198 attempt1 completed.**13source and32existing observer contracts passed. Independent replay validates32reports/672final+672started records,actual sequential clock order,raw clock arithmetic,original timing-to-report binding,all304pairs/38rows,288complete instrumented semantic pairs and16unavailable external pairs. Seven offline replayer rejection controls are not MSVC measurements.
The source ZIPs contain Windows CRLF text. Their original bytes/ZIP hashes are retained; canonical LF reconstruction, leaving the sole binary-attribute install.bat untouched, matches all3exact Git trees and commit ZIP comments. Raw archive byte trees are separately labeled, not misrepresented as canonical Git trees. New A/B binaries,source,build transcript,allowlisted environment and output records are retained; historical599 binaries/OS traces are not reconstructed.
Actual results — keep clock provenance separate
AA includes a same-binary+5.3568ms tail;AB includes+1.7768ms. On the worst observed AB no-op pair,+1.1622ms is in push/pop and+0.6146ms in native-invocation+capture. This is not CPU/OS attribution or evidence about the missing historical trace. Independent604 retains Modules cold+1908.931ms,discovery cold+1481.324ms and discovery-no-op4/4slower. No adverse/favorable sample is discarded or offset; nearest-rankP95 at4/8pairs is a maximum,not a reliable population tail.
Exact-source gates and artifacts
Native Debug#740/34463275143 actual build,special runtime/parameter/freshness checks,all shards/final gate passed. Native Release#625/34463275087 Stage0,all shards,self-host/exact runtime-only package passed; publish-release skipped. Documentation#186/34463275112 passed. Own Performance#604/34463275188 actually built exactbase/head and ran unchanged19x4ABBA; all76pairs independently recomputed,72instrumented semantic pairs matched,four external counter sets unavailable.
b879b66f1c3f0a51380b2103868fa69f020e10b5cb0c4c951978a19b4fc67f3e56ebfcd65e3b566d1e398694a099aa14bf19c7de18d69fb95fb897fdaa33a67de4729cee012239d4f2652fd9edfa82bbea6c9f397e91b6309d098e9b700155695ff94d7d630a8438b68a8fbac3566a644baa4ff3ceac335b83a0c08f8987be05All listed ZIPs pass SHA/CRC. Inner runtime package SHA256
ac06c1a5c4ee8e7ebd09d38824a6249de0a311183da7fd2d1eeff44b7bbed594matches its checksum and exact7file inventory. Study binaries are not mislabeled as the separate604benchmark binary pair. Delivery is not a complete archive of every native job log. Untriggered ownership/default/private/batch/target/ETW/Cross-Stage workflows are not claimed rerun. No local MSVC/PowerShell/ETW execution is claimed.Limits and next handoff
The contract job rejects repeated run attempts, but that is not a repository-wide idempotency guard against selective job reruns or a fresh manual dispatch. No such rerun occurred. Local replay development corrected untimed-prime and CRLF-archive assumptions without changing product/data; recorded separately. The temporary perf title is removed only after actual604completed, before ready-for-review; resulting skipped checks are not new measurements.
#180 stays Draft/HOLD at56b8f086e45ddecef56f63a04c347bd80bd8ed84. Its current native stdin repair and historical Debug/MT and cold-input84 failures remain distinct. This study does not explain599 or approve build/run architecture,CLI cancellation,transactions,external-writer/lease recovery,#172,daemon or final VERSION/release.
Next concrete product iteration: independently extract the proven vcvars stdin correction onto accepted main, retaining native positive/negative controls and its own independent ABBA, WITHOUT the held build/run structural refactor. Do not restart an unbounded identical-runtime observer investigation. Final merge SHA and route update will be recorded in #164.