feat: map known recipe writes with read-only physical identities - #184
Conversation
Recover the existing five core blobs and complete their single-topic integration. Preserve known-output provenance and unresolved native effects; reuse compiler/linker/archive recipe authorities without tool execution. Map exact held physical parents without creating missing directories or markers; retain per-item failures, reparse/hardlink/case-alias uncertainty and missing-leaf distinctions. No completeness, reservation or lease authority. Add 28 pure recipe checks and 33 fixed native read-only checks, preserving the original 39 domain cases and all existing pipeline/benchmark behavior. VERSION stays 5.5.0; historical runtime debts remain.
Resumed handoff: verify the published candidate, do not overwrite it with the alternate local packageLive reads show this PR already exists at cae3ad2, base e64419c, tree 1aaa474b03b3a21c920b14fa8f4a4ae6edc535e3. The earlier chat statement that the branch was still at base was stale. The attached local b340e52/81a53040 package was independently verified but is a DIFFERENT 17-file candidate, not the published commit. It will not be force-pushed or silently substituted. The complete published source has now been reconstructed from the verified 427-file base and the actual PR diff; all 433 files produce the exact Git tree 1aaa474b.... Net diff is 14files/716additions/2deletions. The five recovered core blobs match both candidates. Published code adds the conservative second read-only case-alias check before classifying a missing exact leaf; no statement that the alternate package's 85 local syntax cases ran against this published source will be made. Dedicated native artifacts10301488844(Debug) and10301069780(Release) were downloaded and CRC/SHA checked against GitHub metadata. Each records33mapping checks plus39unchanged domain checks, headcae3ad2,attempt1. Actual observations distinguish missing parent from missing leaf, retain25source indices and native errors, and resolve case/junction aliases to physical identity. Detailed record/provenance/source and existing regression/ABBA review is underway; all current workflow statuses have completed, but green statuses alone are not merge authorization. This resumption has not edited product source, moved the branch, rerun CI/benchmark, or created a competing PR. VERSION5.5.0, #172HOLD and all write-set/lease/recovery limits remain. The final decision will use this exact published head and its own unmodified artifacts. |
Iviesever
left a comment
Iviesever
left a comment
There was a problem hiding this comment.
Final resumed review — accept the published read-only inventory, not a complete write-set or lease authority
Source/evidence self-review with separate read-only source reconstruction, native-log inspection and raw-artifact replay; not an independent human approval. The existing published PR was reviewed rather than overwritten with the attached alternative local candidate.
Exact source
- Base e64419c, tree ac25db90468660b8945bc18a7c926830c272d840.
- Reviewed head cae3ad2, tree 1aaa474b03b3a21c920b14fa8f4a4ae6edc535e3.
- Native test-merge db04123 has that identical tree and exactly those base/head parents.
- 14 files, 716 additions/2 deletions; 427 base files, 419 byte-identical, 8 modified plus 6 new, 433 candidate files. Complete canonical base/candidate archives and patch replay reproduce both Git trees.
- Production manifest matches 82 non-main translation units; original 79 native-test executable inventory unchanged. Existing compiler/linker/cache/freshness implementation, CLI callers, original benchmark and domain reservation/recovery policy remain unchanged. New public interfaces/backend code mean this is not a test-only diff.
The attached local b340e52/81a53040 package is valid but DIFFERENT: 17 files, including alternative mapping/tests. Its local 85 path tests are not claimed as current-head coverage. The original five recovered blobs match; remaining published changes were reconstructed from the actual PR patch. No competing PR, force push or post-publication source edit was made during this resumption.
API and boundary review
WriteInventory collects additive known destinations and unresolved reasons without executing tools, reading cwd/environment or inventing cache naming rules. MsvcWriteInventory reuses existing compile/Header Unit/PCH/module/scan/link/archive recipes and MsvcLinker side-output authority. Every native recipe retains an unresolved-effects record. Extra native parameters, mutable ProcessSpec, environment, temporary/shared-service writes and producers not yet connected prevent any complete-write-set claim.
WindowsWriteInventory retains original source indices and private read-only parent pins. Missing parents remain errors rather than being created or replaced by the nearest existing ancestor. Only actual physical identities are deduplicated; nested/external parents remain separate. Leaf observations use FILE_OPEN relative to the held directory with metadata access and sharing, not a pathname-based create. Directory leaves, final reparses, multi-hardlink ambiguity and unsafe path syntax remain per-item errors without hiding later entries.
The published candidate includes a conservative case-alias boundary absent from the attached alternative: after exact-case lookup reports missing, a second read-only case-insensitive lookup must also report missing before absence is accepted. An existing or otherwise uncertain alias is not declared absent. In this native batch the uppercase alias directly resolved to the same identity; that does not prove every case-sensitive-directory/race branch was exercised.
This API exposes no reserve/withdraw, complete/ready or lease-safe predicate. Empty mapping errors cannot close the unresolved native effects. A held parent does not pin a future leaf replacement, recursively contain all outputs, serialize multiple domains or authorize cleanup. Non-delete-sharing parent pins can constrain rename while held; read-only does not mean zero OS interaction. Exceptions/OOM, hostile races, recursive write closure and recovery remain outside the demonstrated contract. Existing WindowsWriteDomain marker lifecycle and its 39-case source are preserved.
Current-head execution actually inspected
All 17 original applicable PR workflow runs completed successfully. Key gates:
| Gate | Actual run | Evidence |
|---|---|---|
| Native Debug | #749 / 34705277151 | Actual current build, every paired subshard, runtime/ambient/include/parameter/freshness/side-output checks and final gate |
| Native Release | #630 / 34705277183 | Stage0, all paired subshards, full self-host and exact runtime-only package validation; publish-release skipped |
| Physical domain/inventory | #2 / 34705277166 | Debug and Release each execute 33 mapping checks plus unchanged 39 domain checks |
| Default/private ownership | #33 / 34705277194; #35 / 34705277231 | 70/70 and 42/42 complete; required positive controls pass |
| True-MSVC batch/target | #11 / 34705277170; #9 / 34705277200 | Six fixed cases in each; original errors and target-cache barriers preserved |
| Independent ABBA | #619 / 34705277177 | Both exact checkouts/builds, instrumentation check, 19x4 measurement and upload actually run |
| Documentation/Cross-Stage | #192 / 34705277172; #224 / 34705277212 | Passed |
The other eight inspection/reporting workflow statuses pass; this is not a claim to have independently downloaded every ancillary artifact. Untriggered old File Event/PDB Open studies are not labeled current reruns.
Actual Debug job103585591044 and Release job103585152607 show compile/link/run and PASS for 28 pure inventory checks, the original process lifetime checks, 39 domain checks and 33 mapping checks inside the standard native suite. This is not just a green summary. Release self-host/package job 103585464656 completed all stages. Existing C4996/D9025 and tool warnings remain; no warning-free claim. No same-head native/CI/benchmark retry was requested.
Native physical observations independently replayed
Both dedicated ZIPs were CRC/SHA checked, actual probe executable bytes matched provenance, and every original check label/order, native error and data record was independently examined. Both configurations record attempt1 and image20260907.229.1.
Each read-only study preserves 25 original destination indices, three distinct physical parents and two exact unresolved reasons. Existing case/dot/junction aliases identify the same parent/file; missing leaf differs from missing parent/namespace. Nested and external destinations are not collapsed. Multi-hardlink, reparse, directory-leaf, file-parent and eight unsafe-path entries are refused while subsequent valid COM10.obj/last.obj entries remain visible. A retargeted junction produces a new observation of the external parent without changing the old retained identity. Exact before/after tree-byte/mtime records match, and no missing parents or reservation markers are created by the mapping study.
All 39 original domain assertions and raw contender/death-control files also reproduce. One of two same-domain contenders wins, both distinct-domain contenders win, and an independent child writes after its retained owner dies. Persistent unresolved markers still refuse takeover after both owner and writer exit. The mapper does not introduce recovery authority. These are retained native test records, not a syscall trace or proof of all MSVC writers.
Dedicated artifacts: Debug10301488844 SHA256 17a48bfffb66066b9735289150784de009bd8763c19baa31654adfa34aa314a3; Release10301069780 917897ed9704b41bbd295109523851116d03104b02bf31cadc77247397b35d88. Probe hashes match f831cd83… and 3fc1f258… respectively; these are not the independent ABBA binary pair.
Existing correctness and adverse evidence retained
The unchanged original auditors verify all required positive default70/private42 controls. 150 ownership record expectations match 113 accepted/37 rejected with the exact CRLF collector hash. Fourteen scheduler/direct pairs match 56 sources and 70 argv records after only case-directory normalization; native inputs and helper/refusal outputs match provenance.
Default retains four adverse managed-policy B failures; private retains ten, with original C1090/C1051/C1001 diagnoses and exits. Link/run remain unattempted after failed compilation, and separate recovery0 never replaces the first failure. Across both matrices all 28 A-started managed endings kill the original shared service. Audited 715/397 tool results, 1430/794 diagnostic files, owner identity 4/70 and 12/42, and 913/502 post-case binary inventories. These sparse snapshots and post-case hashes do not prove writer quiescence.
The six batch cases retain 18 original results/36 diagnostics, including both C2338 failure-plus-cancel cases remaining failed. The six actual target cases retain 48 results/96 diagnostics and 96 raw cache copies: stopped EXE/link-cache snapshots remain unchanged while product freshness correctly requires unfinished sources/relinking. Completed sibling caches remain valid. Their injected static assertions are not historical C1041 reproduction. Exact input binaries, 32 target record expectations and all recorded audits reproduce. Existing MSVC pipelines do not yet invoke the new inventory API; these regressions are not complete write-closure validation.
Independent original ABBA: every scenario preserved
Artifact 10301349552, 35,569 bytes, SHA256 dbc356d5ee6c01834a4662b262c3fbcd9537a6803dfe314120c7b60cd2a46cfb. All 76 pairs, original AB/BA/AB/BA order, stored arithmetic, paired medians/MAD/nearest-rank P95 and marginal medians recomputed. All 72 instrumented complete counter/breakdown/compile-link-archive hit-miss vectors agree. Four external counter sets remain unavailable, not zero. Only the last row uses timings-off external elapsed; four-pair P95 is the maximum, not a population-tail estimate.
| Scenario | Median delta ms | Median delta % | MAD ms | Maximum delta ms | Slower |
|---|---|---|---|---|---|
| cold | +57.576 | +3.60 | 155.264 | +3245.874 | 2/4 |
| no-op | +0.371 | +8.29 | 0.339 | +0.719 | 3/4 |
| single-tu | +0.956 | +0.59 | 29.597 | +1504.279 | 2/4 |
| public-header | +12.567 | +11.25 | 14.985 | +174.077 | 2/4 |
| build-run | −1.518 | −8.32 | 2.420 | +1.830 | 1/4 |
| link-only | +2.442 | +4.31 | 8.417 | +44.035 | 2/4 |
| target-scale-cold | +94.885 | +2.28 | 154.753 | +675.981 | 2/4 |
| target-scale-no-op | +2.485 | +7.43 | 2.188 | +5.579 | 3/4 |
| target-scale-no-op-auto | +1.265 | +3.87 | 0.793 | +9.330 | 3/4 |
| target-scale-no-op-j1 | +0.103 | +0.23 | 2.135 | +9.874 | 2/4 |
| target-scale-common-header-no-op | +2.144 | +5.80 | 1.349 | +16.705 | 3/4 |
| target-scale-single-tu | +14.780 | +12.26 | 11.152 | +85.061 | 4/4 |
| discovery-cold | +40.007 | +2.41 | 35.607 | +758.469 | 3/4 |
| discovery-no-op | −0.102 | −0.97 | 0.735 | +0.862 | 2/4 |
| discovery-header | +1.448 | +1.29 | 76.882 | +172.755 | 3/4 |
| modules-cold | +43.169 | +2.49 | 72.411 | +151.762 | 3/4 |
| modules-no-op | −0.170 | −3.10 | 0.356 | +3.627 | 2/4 |
| timings-enabled-no-op | +0.034 | +0.91 | 0.823 | +1.848 | 2/4 |
| timings-disabled-no-op (external) | −1.190 | −8.74 | 1.834 | +8.348 | 1/4 |
Unrounded external medians −1.1895ms/−8.7446615410% do not cross the unchanged BOTH >1ms AND >10% regression gate. Preserve the external +8.3479ms adverse pair, cold +3.245874s tail, single-TU +1.504279s tail and 4/4 slower scale-single-TU. No speedup/zero-overhead, harmless-noise or causal source attribution claim. No sample deletion, AA subtraction or historical #170/#599/C1041/cold-tail debt closure. Benchmark JSON is not an archive of its actual executable pair or OS trace.
Decision and handoff
Accept normal expected-head merge of cae3ad2, subject to unchanged main/protection. The predeclared 28/33/39 checks, full native/selfhost/package, original controls and independent ABBA gates all pass. This accepts the bounded read-only interfaces, not a declaration that every project writer is known or that a build can safely acquire/release leases. VERSION5.5.0, #172 HOLD and M1b's ownership/recovery gates remain.
Next independent small topic: connect the actual early discovery/toolchain cache owners and ordinary-target producers to an explicit pre-write inventory collection boundary, reusing their path authorities. Preserve unresolved native/temporary/provider effects rather than converting them into successful closure. No writer should start before its necessary admission requirements are known. Missing-root initialization, overlapping/multiple domains and safe commit/recovery remain separate steps; do not add a stale-marker deletion shortcut or another observer.
Delivery boundaries
Twelve immutable core ZIPs, complete exact base/head source archives, the published 14-file patch, hashes and a portable offline replay were generated and the replay executed successfully. Native log inspection notes are labeled as notes, not complete CI log archives. The alternate local package was validated but not applied, and its local tests are not substituted for published-head evidence. No local MSVC/PowerShell/ETW or new benchmark execution is claimed. Source reconstruction used the connector diff and verified mounted base after container network failure; exact Git trees prevent guessed replacement content. The offline domain auditor only narrows archive selection to the unchanged domain checks now co-located with inventory checks; none of its acceptance assertions was relaxed. No raw artifact or product source was changed to make a replay pass. Artifact retention is 30 days. Any later title/ready skip or post-merge push CI is not a replacement for completed exact-tree validation.
Iviesever
changed the title
1 participant
Final decision: accept the published read-only inventory
Final exact-source, native evidence, complete ABBA table and acceptance review
Accept normal expected-head merge. This is a bounded public inventory/mapping API, not a complete project write set, CLI lease or recovery authority. #172 and M1b's remaining ownership/recovery gates stay independent.
Exact accepted source
e64419c29b0e1caa86a69eb2b71e0066457c17e9.cae3ad239bcc7f4b09ba091bcf36d3ce22ef6e9e.1aaa474b03b3a21c920b14fa8f4a4ae6edc535e3.db04123afbe29fee3ba14621d500e4412de45fb5, exact base/head parents.The attached local b340e52/81a53040 package is a different unpublished17-file candidate. It was verified but not substituted for this PR. The stale chat handoff did not override the actual published source. No source revision, force push, competing PR or same-head native/benchmark retry was made during resumed acceptance.
Implemented boundary
WriteInventory preserves known stage/path/reason entries and unresolved reasons. Native recipe extraction reuses actual compile/Header Unit/PCH/module/scan/link/archive models and the established linker side-output paths; cache paths come from their owner. No alternate argv parser or cache naming rule is introduced. Every native recipe preserves an unresolved side-effect record, so unknown native parameters/environment/temp/service writes are not erased.
WindowsWriteInventory maps each original entry to its exact held physical parent without creating directories or markers. Missing parent and missing file remain distinct. Only physical parent identities are deduplicated; nested/external roots are not silently collapsed. Directory leaves, final reparses, multi-hardlink ambiguity and unsafe paths retain per-item errors. Subsequent valid entries and all unresolved reasons remain visible.
After an exact-case missing lookup, the published candidate uses a second read-only case-insensitive lookup before accepting absence. An existing/uncertain alias is not treated as a new missing file. Native uppercase-alias tests directly resolve to the same identity in this batch; they are not claimed exhaustive case-sensitive/race coverage.
Private pins expose no reservation or cleanup API, and there is no complete/ready/lease-safe predicate. A held parent does not pin future leaf changes or establish a recursive/multiple-domain write closure. Existing marker abandonment/recovery rules stay unchanged. No existing build/CLI/plan/compdb caller adopts the inventory yet.
Completed current-head gates
Standard native suites also execute the original39 and new33 checks. Dedicated raw records preserve25destination indices,3physical parents and2exact unresolved reasons. Before/after tree bytes/mtime records match; no missing parents or reservation markers are created by mapping. Real retargeted junctions produce the expected new observation without changing the old pin. Original owner-death/residual-child-write and two-process contender records remain valid. These are bounded native assertions/records, not a syscall trace or all-writer proof.
Existing warnings are preserved. Untriggered File Event/PDB Open studies are not labeled rerun. No local MSVC/PowerShell/ETW execution or local85-case coverage against this different published candidate is claimed.
Adverse outcomes and performance
Twelve immutable core ZIPs were SHA256/CRC checked and completely replayed.150ownership expectations match113accepted/37rejected,32target record expectations match,14scheduler/direct pairs match56sources+70argv, and exact native inputs match provenance. Batch18results/36diagnostics and target48results/96diagnostics preserve original C2338 failures and real target-cache freshness barriers.
Default/private adverse managed-policy tests retain 4/10 original B failures,715/397tool results,1430/794diagnostics and separate recoveries. All28A-started managed endings still kill the original service. Owner identities4/70 and12/42 are incomplete; post-case hashes are not failure-time snapshots. New inventory APIs are not used by these existing pipelines, so this does not validate a closed production write set.
External timings-off no-op paired median −1.1895ms/−8.7446615410% does not cross the unchanged BOTH>1ms AND>10% gate. Preserve external+8.3479ms, cold+3245.874ms and single-TU+1504.279ms adverse samples, and4/4slower scale-single-TU. No speedup/zero-overhead, noise deletion, AA subtraction or historical debt closure. Four external counter sets are unavailable,not zero; all other rows are instrumented. Four-pairP95 is only a maximum. The raw comparison is not an archive of its executable pair or OS trace.
Decision and next handoff
The predeclared current correctness, source and performance gates all pass; none was waived. Accept only the bounded read-only interfaces. Historical #170/#599/C1041/cold-tail debt, #172HOLD, full write closure, CLI leases and project commit/recovery remain unresolved.
Next independent small topic: connect actual early discovery/toolchain cache owners and ordinary-target producers to a pre-write inventory boundary using their existing path authorities. Retain unresolved native/temp/provider effects and require missing ownership information before starting writers; do not reinterpret mapping success as authorization. Missing-root initialization, overlapping/multiple domains and safe recovery remain separate. No stale-marker deletion shortcut or new observer.
Delivery contains12raw coreZIPs, exact base/head source archives, the accepted14-file patch, manifests and executed portable offline replay; not every Actions artifact or complete native-log archive. Input probes are not the independent ABBA binary pair. The domain replay helper only selects its unchanged39-case record from an archive now containing two suites; acceptance assertions were not relaxed. Core retention30days. Later title/ready-triggered skips or post-merge push CI are not substituted for completed exact-tree evidence.