Integrate faithful CPU execution: packed kernels and trained FP semantic exit - #27
tugrapaydiner wants to merge 25 commits into
Conversation
… and fresh native builds
…scopes, and exact replay
… and invalid measurements
… Python 3.11 and 3.13
Final-head CI and follow-up audit receiptHead remains New runtime CI artifacts inspectedBoth downloaded archives passed CRC and matched GitHub's artifact SHA256. Their recorded source trees exactly match the local tested tree. Each contains 228 focused passing tests, 11 dependency-free installed-wheel checks, six installed-native runtime arms, 1,344 measurements, and 192 successful within-CI execution replays.
Artifact IDs: Portability limit uncovered rather than hiddenAdditional strict replay of the Intel CI recordings on the local AMD environment fails bitwise matching, even using the unchanged historical checked runtime. For each CI archive, 156/192 execution-output hash comparisons differ across environments, including 26/32 historical checked-trace cases. A complete diagnostic establishes zero disagreements between the six runtimes within the local environment, zero cross-environment decoded-answer differences, and zero validity differences in these fixtures. Both CI environments independently passed their own exact six-arm checks and replays. Therefore the result is bitwise preservation relative to the historical runtime within the same environment, not a promise of cross-hardware/Python-environment bitwise floating-point portability. The strict replay failures and full diagnostic are retained. The verifier was not weakened, and these failed cross-environment replays are not counted as passes. Stronger alternative controlA separate exploratory full-trace comparison expands the same packed artifacts into dense FP32 weights and uses PyTorch linear operations. Across the same 32 cases / 448 calls, its ratio of summed case medians versus blocked tracing is This alternative has different reduction arithmetic and memory tradeoffs and was not silently added to the exact public API. Its script, all raw observations, and outputs are retained separately from the frozen six-arm experiment in the delivery bundle. The new packed kernel does not establish superiority over optimized dense libraries. Useful trained-task performance and an explicit numerical/memory tradeoff study remain the next decision-changing work. |
…ation-guard validation
Delivery receipt — final source, artifacts, and current checksFinal head The full local suite was rerun once more after the final documentation changes, with the exact final tree recorded before testing: 1,396 passed, 16 slow tests deselected, two historical warnings. The final wheel was rebuilt from its source distribution and passed all 11 dependency-free installed-package checks. All 567 source-archive member bytes match the final source tree. At the delivery check, eight final-head workflows have completed successfully: repository audit, public package, native contract, task-data contract, indexed efficiency, cached residuals, SAT workload admission, and integrated runtime. The broader CPU regression workflow ( Final integrated-runtime run Tail disclosureNo case-median regressions occur in the fixed local or either final CI matrix. However, two local cases have worse empirical P95 values against the median-selected comparator, with ratios 1.25074 and 1.00296. Seven samples per arm/case support only descriptive quantiles, not stable production-tail claims. All observations remain retained. The faster-but-non-bitwise dense control and failed cross-environment bitwise replays also remain disclosed and preserved. Delivered archive
Bundle size: 64,585,512 bytes. Implementation is complete on this branch. The pending workflow is reported honestly; no automatic merge, permissions change, or scientific-claim expansion is requested by this receipt. |
|
Continuation checkpoint: all nine workflows on 8c1abb3 are now successful. Main remains unchanged. Trained FP protocol is committed at 3cb7073. The initial local development matrix has 14,336 observations, 512 model/input cases, exact answer/depth/work agreement, and 478 verified outcomes for each neural arm. Prepared/native ratio 0.571750, problem-bootstrap interval [0.567691, 0.575769]; all four strata pass the declared development gate. The existing symbolic solver still solves all 512 outcomes faster. All observations, 20 descriptive per-case P95 regressions, and the executable snapshot are retained; these are development observations, not final-head CI evidence. Before adding or measuring another control, declare this addendum: run a separately labelled matched-preparation eager control on the identical 512 cases and two frozen checkpoints. It will own a copied eval-mode model, precompute only input-independent positions, cache the native checker class, and retain the historical Python recurrent graph. Compare it against prepared native dispatch over seven seeded interleaved rounds, checking every answer, validity and work dictionary. Retain complete rows and separately report the candidate/eager ratio, paired problem-cluster interval and regressions. This ablation must not replace or redefine the already declared four-arm primary; it tests whether dispatch elimination adds value beyond simply hoisting eval/position setup. No new trained capability, classical-solver superiority, or memory/energy claim is introduced. |
…te-solve evidence Own and validate the audited FP32 n=T=1 4x4 TRM graph; dispatch the same ATen operators from a C++ context without changing weights, recurrence, per-step semantic checks or the historical implementations. Explicit PyTorch 2.10.x environment and graph constraints; snapshot ownership, input validation, trace and concurrency contracts. Two existing hash-bound checkpoints, 256 previously observed development puzzles, 512 model/input cases. Separate four-arm primary (14,336 observations) and predeclared matched-preparation eager ablation (7,168 observations). Preserve 478/512 neural valid outcomes, every answer and work record, and 2,048 full-budget recurrent state/logit steps. The exact symbolic control remains faster and solves all 512 cases. Final local source tree matches this tree. 1,532 fast tests pass, 16 historical slow tests excluded. Actual sdist-built wheel passes 11 public installation checks and fresh native execution outside checkout. Primary ratio 0.586054 [0.583107,0.588916]; matched-preparation eager ratio 0.641995 [0.637490,0.646411]. No case-median regressions, but 12/23 descriptive case-P95 regressions remain visible. Preparation/cold compilation and copied-weight storage are separate; no memory, energy, learned-generalization or external-solver-superiority claim. Historical 11,520-answer verification and 3,840 checkpoint comparisons pass. Earlier development and pre-audit measurements remain retained; final raw evidence is generated/replayed by the new two-version trained-FP workflow. Local validation is not final-head CI completion. No main merge or release publication.
tugrapaydiner
changed the title
Final trained-FP delivery receipt — all ten workflows passHead: All 10 final-head workflows completed successfully: repository audit Trained-task result and stronger controlTwo existing accepted FP32 checkpoints, 256 distinct puzzle arrays from previously observed ordinary/shifted DEVELOPMENT surfaces, 512 model/puzzle cases. The new performance experiment does not execute confirmation/reserve surfaces; unchanged historical validation may replay previously recorded confirmation evidence without generating/selecting a new result.
Ratios are sums of per-case medians. Intervals are conditional paired problem-cluster bootstraps retaining both fixed checkpoints, not population-level or cross-host intervals. Warm complete solve starts from a pretokenized input; it includes fresh checker/state, input embedding, recurrence, decoding/checking and returned work. JIT loading/compilation, model preparation, input-tensor construction and the additional external audit are separate. All neural paths preserve 478/512 valid outcomes, every answer, depth and work dictionary. These FP checkpoints already achieved those outcomes; this is NOT an accuracy increase from the earlier packed fixture benchmark. The packed and FP paths are different formats; their speedups do not multiply. The symbolic solver still achieves 512/512 and is faster. Local charged time per verified answer, including failed attempts: native 0.845 ms, prepared 0.507 ms, symbolic 0.104 ms. Each final measurement environment retains 14,336 primary calls + 7,168 stronger-control calls, with 2,048 primary execution replays, 2,048 full-budget recurrent-step replays, and 1,024 stronger-control execution replays. Every within-environment check passes. No per-case median regressions occur. Local empirical P95 regressions remain 12 primary / 23 control; CI Python3.11 has 0 / 1, Python3.13 0 / 0. Seven repetitions do not establish production-tail guarantees. No failed row was removed. Tests, installation, preservation and independent auditsThe exact final tree passes 1,532 fast tests locally and in both full CPU CI environments. Sixteen historical slow tests are deselected, not counted as passes; the 136 new tests are included in the total. Each new trained-FP CI environment separately passes all 136 focused tests, 11 dependency-free installed-wheel checks, and fresh native compilation/execution from an isolated installed wheel. Final wheel source audit matches 126 packaged source members to the checkout, including all three new runtime/analysis/C++ members. Both trained-FP CI ZIPs were downloaded and verified against their GitHub SHA256 values, ZIP CRC and all 578 source-member bytes. Their source trees equal the pushed final tree (CI synthetic merge commit The independent standard-library audit checks every original-clue answer, full case/arm/round/order coverage, strict counter types, complete repeated work records, median ratios and charged-time arithmetic without importing the benchmark analysis. Earlier development/pre-audit runs and frozen source snapshots remain retained. The full historical check still verifies 11,520 stored answers and 3,840 checkpoint comparisons with zero mismatches. Relative to previous branch head Strict portability failure — retained, not waivedBoth CI artifacts pass their own strict trajectory replay. Replaying their stored internal-state hashes on local EPYC9V74 fails strict bitwise matching. The complete separate diagnosis confirms candidate/reference equality within the local environment on all 512 cases/2,048 steps; against each CI archive all 512 trajectory hashes and 2,048 combined-step hashes differ, while embedding hashes match. Across 2,048 complete-arm comparisons per archive there are zero decoded-answer, validity or work differences. CI3.11 and CI3.13 trajectory hashes match each other. Both failed strict logs and the full diagnostic are retained; the verifier was not weakened. No cross-hardware bitwise guarantee or numerical-difference magnitude bound is claimed. Setup and scopeFresh native compilation plus preparation takes 49.84 seconds locally, about 41.22/41.25 seconds in CI. The separate cached-library preparation break-even is approximately 1,193 local calls, not a cold-compiler break-even or first-call speedup. Owned tensor payload is 206,620 bytes per model, not peak RSS; retained source models duplicate storage. No memory/energy advantage or new learned/generalization/classical-solver-superiority claim. Delivered filesThe accompanying ChatGPT delivery provides
The branch implementation and final-head validation are complete. Main is intentionally unchanged. |
1 participant
Status: implemented and fully checked; not merged
Head:
b28e01b85bd95b78f291e466c25c13bfa2fc5b46Source tree:
0d00d6e01c5aca76e60bc6651a9d5ffdc90c00f7— 578 filesAll 10 final-head workflows pass. Main remains
9fd6682345b34e992b3cab7e55750e9ef4b04e87.The final source, CI, performance, failure and delivery receipt records exact run IDs, artifact hashes, counts and limitations. No main merge, release publication, permissions change or unrelated branch retirement is performed.
What changes
This PR contains two separate, opt-in efficiency improvements. Their model formats and speedups must not be conflated or multiplied.
Trained FP execution:
PreparedFPSudokuowns a snapshot of the accepted CPU FP32, n=T=1, 4x4 Sudoku graph and dispatches the same ATen native MHA, RMSNorm, linear, exact GELU and ordered residual operations from C++. Only immutable weights and input-independent positions are prepared. Each solve creates a fresh exact checker and zero recurrent state, retains every recurrent step and per-step validity check, and stops on the same answer/depth. This is reduced Python dispatch/setup overhead, not a new GEMM/attention algorithm, quantization or model training result. Unsupported module classes/overrides/hooks, nonfinite or unfrozen weights, dtype/shape/budget mismatches, autocast and incompatible attention dispatch are rejected. The internal ATen dependency is explicitly scoped to PyTorch 2.10.x.Earlier packed execution: compose final-only output with historical checked, validated-owned and four-vector-blocked packed runtimes while preserving ordered FP32 accumulation. Arbitrary runtime subclasses remain rejected. The original diagnostic forward and historical operators remain available. Packaging-only edits no longer attempt republication of an existing version; 11 tests exercise the actual version-transition guard against disposable Git histories.
Trained-model result
Two existing accepted FP checkpoints (seeds 1401/2402), 256 distinct ordinary/shifted development puzzles, 512 model/puzzle cases. The performance experiment uses previously observed development inputs, not new confirmation/reserve cases. No checkpoint is retrained, quantized, replaced or selected using the new outcomes.
Ratios use sums of per-case medians. Every declared primary gate passes; conditional problem-cluster intervals, all strata and all raw observations are in the final receipt/artifacts. Warm complete solve begins with a pretokenized tensor and includes fresh checker/state, embeddings, recurrence, decoding/checking and returned work. JIT/model preparation/input-tensor construction and the extra external audit are separately scoped.
The stronger eager control also owns copied weights and precomputes positions without repeated eval setup. It isolates the additional dispatch benefit rather than comparing only against a weak setup-heavy reference. The primary protocol was committed before implementation/measurement at
3cb7073; the ablation was separately declared in comment5648284600before implementation/measurement.All neural paths preserve 478/512 valid outcomes, every answer, depth and work record. These FP models already achieved those outcomes. This is not an accuracy increase from the earlier packed fixtures' zero-valid result. The symbolic solver still solves 512/512 and is faster. Local total charged time per verified output, including failed attempts: 0.845 ms native / 0.507 ms prepared / 0.104 ms symbolic.
Each environment retains 14,336 four-arm timing observations plus 7,168 two-arm control observations. Seven repeats are not independent new tasks. No case-median regressions occur; local empirical case-P95 regressions remain 12 primary / 23 control, and one CI control case also regresses. No production-tail guarantee is made.
Validation and preservation
Compared with preceding branch head
8c1abb3, 565/567 files are byte-identical. Only two navigation guides change; eleven files are added. All old research implementations, configurations, tests and raw evidence remain unchanged. Earlier development/pre-audit measurements and their exact source snapshots remain retained. An initially incomplete later-round work-record audit was fixed, adversarially tested and followed by a separately frozen final run, not an overwrite of old observations.Important retained limitations
Cross-host bits: both CI environments pass their own strict replay, but their stored internal-state hashes fail strict replay on local EPYC9V74. A complete diagnostic finds candidate/reference bit equality locally on all 512 cases/2,048 steps; against each CI archive, all trajectory hashes differ but all 2,048 complete-arm answers, validity flags and work records agree. Failed strict logs remain retained; no verifier is weakened and no cross-hardware bitwise guarantee is claimed.
Startup/storage: fresh compilation plus preparation takes 49.84 seconds locally and about 41.22/41.25 seconds in CI. The separate cached-library setup break-even is approximately 1,193 local calls, not a cold-compiler or first-call gain. Each runtime owns 206,620 bytes of tensor payload; source retention duplicates storage. Payload bytes are not peak RSS. No memory or energy reduction is established.
Earlier packed result: the retained packed fixture comparison reports 2.47x local and 2.66x/2.19x earlier CI gains, but every fixture output is invalid. A separate dense FP control is faster but nonbitwise; earlier packed cross-hardware strict failures are also retained. None of that evidence is relabeled as trained FP solving.
This is useful existing trained execution becoming cheaper, not new learned reasoning, broader generalization, superiority to classical solvers or a hiring claim. See
docs/TRAINED_FP_GUIDE.md,docs/TRAINED_FP_PROTOCOL.mdand the separatedocs/RUNTIME_GUIDE.mdfor interfaces and reproduction. The final receipt links exact evidence and the accompanying ChatGPT delivery; no new GitHub release is claimed.