Benchmark trained FP execution against real regional Inductor controls

[tugrapaydiner]

Implemented and validated; open and unmerged

Head 4cb091653bd720d36f36bccecdae7781824f4985; exact source tree cefba7e47a82aafaa24296cc06c6dcff32e21b5a (588 files).

Both Python 3.11/3.13 jobs in final-head workflow 34719785871 pass. Their downloaded artifacts were checked against GitHub SHA256, ZIP CRC, all 588 source members and independent raw-data analysis. This is a separate stacked PR on PR27 (b28e01b), not a merge into main. All 578 base files remain byte-identical; ten files and 1,326 lines are added, zero deleted. Main, PR27, old exact APIs and historical scientific outcomes are unchanged.

Main finding: use the stronger compiler baseline

Frozen/max-autotuned Inductor outperforms the custom prepared FP runtime in the measured aggregate, while preserving every observed answer, validity flag, stopping depth and work record. The advantage is hardware-dependent; default Inductor actually loses on Intel.

Environment	Default / prepared latency ratio	Frozen / prepared ratio	Frozen latency reduction
Local EPYC 9V74 / Python3.13.5	0.947538	0.732840	26.7%
CI EPYC 7763 / Python3.11.16	0.918498	0.731177	26.9%
CI Xeon 6973P-C / Python3.13.15	1.079033	0.924016	7.6%

Ratios are sums of per-case medians against prepared execution within each environment. Conditional paired problem-cluster intervals for frozen are respectively [0.728216, 0.737636], [0.729004, 0.733532], and [0.920016, 0.928232]. They retain the two fixed checkpoints and are not population/model-seed/cross-host confidence intervals.

Two accepted FP32 checkpoint seeds, 256 distinct previously observed ordinary/shifted DEVELOPMENT puzzles, 512 model/puzzle cases. No training, filtering, checkpoint conversion, new confirmation/reserve experiment or answer cache. All four arms retain 478/512 valid outcomes in every environment, with zero changed answers, depth/work differences, lost valid outputs or nonfinite audited states.

Neither compiler is bitwise reference-equivalent: each differs in internal trajectory bits on all 512 cases. Maximum observed absolute logit differences range from 2.15769e-5 to 3.50475e-5 across modes/environments. Both pass the separate bounded task-output check, not the exact track. No original exact tolerance is weakened.

What changes

• Experimental CompilerFPSudoku: owning frozen FP32 models, fixed positions, fresh checker/state per request, unchanged four-step budget. Compile one recurrent cycle plus output head with actual Inductor, fullgraph=True and dynamic=False. Original native checking and data-dependent exit remain outside the compiled region.
• Explicit default and max-autotune/freezing controls; error suppression is rejected and compile errors propagate. No silent eager fallback or automatic production-default switch.
• Complete source-bound four-arm measurement and fresh-compilation replay, independent original-clue validation, strict repeated work/counter audits, full-budget state/logit hashes and numerical differences, setup/cache/compiler/generated-source evidence.
• Adapter/corruption tests, actual compiler execution from an sdist-built wheel outside checkout with fresh native/Inductor caches, and dedicated CPU CI.

The protocol was committed before implementation/measurement at 8b35c0879864fa8fd958d457d2442d173a32d784. All old executable files, tests, configurations and raw evidence are unchanged.

Actual validation

• 1,591 fast tests pass locally and in both final CI jobs; 16 historical slow tests excluded, two historical warnings. The 59 new tests are included, not additional.
• 11 dependency-free installed-wheel checks and two real installed compiler modes pass in each checked environment; all three built wheels match 128 packaged source members.
• Per environment: 14,336 timing observations, 2,048 fresh-compilation complete execution replays, 8,192 full-budget recurrent-step replays. No new graph capture during warm timing (four graphs before/after).
• Historical checks verify 11,520 retained answers and 3,840 checkpoint answer/validity/work comparisons with zero mismatches per environment.
• A separate standard-library delivery auditor, importing no SPECTRA code, recomputes all matrix/order coverage, original-clue answers, repeated work, medians, charged costs, tails and bootstrap intervals.

CI synthetic merge a09b0d16fecb0767f30692d0f23891c3b73758b0 has the exact branch tree. Artifacts: Python3.11 10305228330, SHA256 07140d98b26af1a1ac6941b72a68f302183d2310aea13f434d82fb314ca3c9a3; Python3.13 10305449770, SHA256 75dd9589f543a7c7650816910d04c2e2e50e36d749411d0fd81f51694e42b7a2.

Regressions, costs and retained failures

Frozen case-median/empirical-P95 regressions vs prepared are 0/47 local, 0/5 AMD CI, 34/48 Intel CI. Default regressions are 88/156, 2/27, 485/443. Seven repeats do not support production-tail guarantees. No unfavorable row or case was removed.

Warm timing includes embedding, fresh state/checker, every executed recurrence, decode/check, and returned work. Compilation/model loading/input construction/extra external audit are separate. An initially empty Inductor cache is shared within a run; preparation order permits reuse, so per-arm setup receipts are not independent cold-install costs. Local first full traces took 20.826/11.921/0.159/4.627 seconds in default/frozen seed1401 then default/frozen seed2402 order. The separate installed-wheel probe uses fresh caches and is explicitly an untrained execution fixture.

Whole-solve strict capture fails at the native checker binding; regional controls succeed. Initial option-only freezing did not establish packed frozen parameters; the corrected startup configuration and generated sources do. An over-restrictive guard aborted the first formal attempt before timing. Initial malformed-input test failures, the first complete measurement, and the separately frozen final run are retained with their applicable logs/source snapshots. No unsuccessful attempt is counted as a pass.

The exact runtime remains available. These results do not establish generalization, new learned reasoning, superiority to classical solvers, cross-hardware bitwise portability, or memory/energy gains. The historical symbolic comparator remains stronger on its separate experiment and was not retimed here. Torch is the tested 2.10.0 CPU stack, not a claim about the latest compiler.

See docs/COMPILER_BASELINE_GUIDE.md, docs/COMPILER_BASELINE_PROTOCOL.md, results/compiler_baseline/local_receipt.json and the accompanying source/evidence delivery. Any later batching experiment should compete against the strongest applicable compiler control rather than only our historical custom path.