Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
5069116
research: restructure manuscript after review and add CPU/evidence CI
0z5a Oct 14, 2026
0b3fbd3
ci: make package CPU tests blocking after first green run
0z5a Oct 14, 2026
d510441
research: add accepted Ethier-Steinman 3D solves (protocol v4)
0z5a Oct 14, 2026
b76074e
research: qualify Ethier attribution and record run software
0z5a Oct 14, 2026
808792e
research: freeze C36 v2 development protocol and batch sources
0z5a Oct 14, 2026
5443d6d
research: add C36 v2 development results (budget-only, both engines)
0z5a Oct 14, 2026
9bb74d7
research: state the C36 v2 decision rule exactly and correct its caveats
0z5a Oct 14, 2026
dc4e59b
research: freeze C36 all-planes observation and smoothness-schedule b…
0z5a Oct 14, 2026
6393255
ci: run the schedule-family tests with the C36 CPU regression
0z5a Oct 14, 2026
e9789fb
research: keep the C36 v3 analytic engine alive with its captured graph
0z5a Oct 14, 2026
aa6407b
research: observe every C36 checkpoint on all planes; add the smoothn…
0z5a Oct 15, 2026
7771e39
research: quote the v3 rule verbatim, publish the rank correlations, …
0z5a Oct 15, 2026
286be99
research: pre-register the C36 v4 combination study (C2 start plus lo…
0z5a Oct 15, 2026
6e35a5c
research: record the cancelled C36 v4 attempt and why
0z5a Oct 15, 2026
a072ba1
research: pre-register the C36 v5 budget study (v2 recipe, 4,800 L-BF…
0z5a Oct 15, 2026
df4caa5
research: add the C36 v5 budget study (registered outcome E5) and the…
0z5a Oct 16, 2026
ffb3c48
research: record the C36 v5 observation archive on persistent storage
0z5a Oct 16, 2026
9270f13
research: pre-register the 2D representation/execution ablation (H100)
0z5a Oct 16, 2026
ef4e730
research: fix the 2D ablation start-up (pinned git checkout, fail-fas…
0z5a Oct 16, 2026
45efbce
research: add the H100 representation/execution ablation (Section 5.6)
0z5a Oct 16, 2026
170412b
research: pre-register the 2D H100 solve-scale study
0z5a Oct 16, 2026
7ff3e60
research: pin the 2,048-point Adam/L-BFGS phase split
0z5a Oct 16, 2026
1f3be04
research: add the H100 solve-scale results to Section 5.6
0z5a Oct 16, 2026
de0b9b5
research: state the setup cache effect precisely
0z5a Oct 16, 2026
c5de0ad
research: pre-register the C36 zero-gate folding GPU batch
0z5a Oct 16, 2026
2b614f6
research: add the H100 zero-gate folding results
0z5a Oct 16, 2026
85500b3
research: state the folding-factor reference as an expectation
0z5a Oct 16, 2026
ffd597e
research: pre-register the 2D H100 retest under the H20 protocol
0z5a Oct 16, 2026
dbb6d33
research: pre-register the 2D FP64/FP32/TF32 precision study
0z5a Oct 16, 2026
e2e5108
research: pre-register external 3D derivative baselines (Ethier)
0z5a Oct 16, 2026
a9909a6
research: add the precision study and the external 3D baselines
0z5a Oct 16, 2026
35ca59b
research: state the 3D and precision results without over-claiming
0z5a Oct 16, 2026
4909b95
research: add the 2D H100 retest under the H20 protocol
0z5a Oct 16, 2026
89b2c1b
research: tighten the abstract and the retest and 3D caveats
0z5a Oct 16, 2026
7e803c0
research: pre-register compiled external 3D baselines (Ethier)
0z5a Oct 17, 2026
72ca71f
research: add the compiled 3D external baselines (C17 loses its lead)
0z5a Oct 17, 2026
b391841
research: pre-register attempt 2 of the compiled 3D batch (C17 repair)
0z5a Oct 17, 2026
4142016
research: add attempt 2 of the compiled 3D batch (compiled C17)
0z5a Oct 17, 2026
4d17f8a
research: state two verified properties of the C36 source instance
0z5a Oct 18, 2026
69ee10b
research: cite the rewritten pre-registration commits
0z5a Oct 18, 2026
32af58d
research: locate the remaining C36 pressure error (post-hoc, CPU)
0z5a Oct 18, 2026
7671c3f
research: pre-register C36 v6, a wider-cutoff instance (eta 1/16)
0z5a Oct 18, 2026
c6acd08
research: add the C36 v6 results (wider cutoff, outcome F3)
0z5a Oct 18, 2026
0db002b
research: attribute the C36 pressure error (post-hoc, CPU)
0z5a Oct 19, 2026
4e0f669
research: pre-register C36 v7, a dense self-scaled Broyden optimizer
0z5a Oct 19, 2026
72474fb
research: amend C36 v7 for a 2 h allocation (before any run)
0z5a Oct 19, 2026
c34ed33
research: qualify GH200 and register two-seed C36 v7 fresh continuation
0z5a Oct 21, 2026
5083508
Preregister C36 new-seed time-to-accuracy validation
0z5a Oct 21, 2026
070c274
research: add the C36 v7 GH200 results (F1) and the unqualified TTA c…
0z5a Oct 21, 2026
1a059c3
research: state how the GH200 packets were frozen; record the PDF pag…
0z5a Oct 21, 2026
1fd2bc6
research: pre-register a second C36 TTA cohort with a rounding-bounde…
0z5a Oct 21, 2026
e7d5941
research: the second C36 TTA cohort qualifies (5/5, median 5,074 s)
0z5a Oct 21, 2026
023fdab
research: add the paired C36 engine trainer and its GH200 qualificati…
0z5a Oct 21, 2026
aaea603
research: register the paired C36 engine cohort (analytic vs paramete…
0z5a Oct 21, 2026
1024da8
research: paired C36 engines to verified accuracy (analytic 1.257x fa…
0z5a Oct 22, 2026
e5aa482
research: register the C36 external-backend throughput comparison
0z5a Oct 22, 2026
3b86067
research: keep CUDA Graph owners alive in the external throughput dri…
0z5a Oct 22, 2026
a3d748b
research: post-hoc r3 protocol for the external throughput rerun
0z5a Oct 22, 2026
d9fac9c
research: C36 against external derivative backends (torch.func, jet, …
0z5a Oct 22, 2026
5618da0
research: add the analytic vs torch.func paired trainer and its GH200…
0z5a Oct 22, 2026
8180b07
research: trace the torch.func evaluation per chunk so its memory fol…
0z5a Oct 22, 2026
92a527b
research: capture the eager torch.func evaluation directly for the pa…
0z5a Oct 22, 2026
0062255
research: register the paired C36 cohort analytic engine vs torch.func
0z5a Oct 22, 2026
1835026
research: register the 2D L-BFGS execution study on GH200 (device-sid…
0z5a Oct 22, 2026
0b95936
research: 2D L-BFGS GH200 study v2 protocol pins the aarch64 initiali…
0z5a Oct 22, 2026
3dca6ac
research: 2D L-BFGS GH200 wrapper binds torch.optim.lbfgs at import (…
0z5a Oct 22, 2026
ac4253c
research: 2D L-BFGS execution study on GH200 (36/36 identical, 2.21/1…
0z5a Oct 22, 2026
832f108
research: paired C36 cohort against torch.func (analytic faster 3.92x…
0z5a Oct 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
53 changes: 53 additions & 0 deletions .github/workflows/cpu-regression.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: CPU regression and evidence hashes

on:
push:
branches: [main]
pull_request:

permissions:
contents: read

jobs:
evidence-hashes:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Verify manuscript evidence index
run: python scripts/verify_evidence_index.py

cpu-tests:
runs-on: ubuntu-latest
timeout-minutes: 45
env:
PYTHONDONTWRITEBYTECODE: "1"
OMP_NUM_THREADS: "2"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install CPU dependencies
run: |
python -m pip install --upgrade pip
python -m pip install "torch>=2.8,<3" --index-url https://download.pytorch.org/whl/cpu
python -m pip install "numpy>=2,<3" "scipy>=1.14,<2" "sympy>=1.13,<2" "mpmath>=1.3,<2" "pytest>=8,<10"
- name: Package CPU tests
run: PYTHONPATH=src python -m pytest -q
- name: C36 CPU regression
working-directory: research/c36-potential-proof
run: >
PYTHONPATH=src python -m unittest -v
test_c36_unpack test_c36_packed
box376.test_coordinate_oracle box376.test_potential_geometry
box376.test_potential_ad_objective box376.test_potential_audit
box376.test_fold_zero_gate box376.test_schedule_family
box376.test_decide_c36_v4 box376.test_decide_c36_v5 box376.test_decide_c36_v6
box376.test_decide_c36_v7 box376.test_ssbroyden box376.test_tta_control box376.test_audit_tta_v2 box376.test_decide_c36_pair box376.test_pair_control box376.test_external_c36 box376.test_decide_c36_pair_func box376.test_pair_func_control
- name: 2D L-BFGS GH200 study CPU checks
working-directory: research/2d-lbfgs-gh200
run: PYTHONPATH=src python -m unittest -v test_lbfgs_study
47 changes: 46 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,4 +78,49 @@ The [tested source manifest](docs/tested-source-sha256.json) records the unchang

## Research manuscript

The [October 13 research draft](research/manuscript/README.md) retains the accepted 2D H20 solves and adds the full C36 derivative qualifications, matched H100/CPU timing, and completed 3D training and physical acceptance failures. The new CPU analytic-transpose result is separate from GPU and complete-solver performance.
The [October 14 research draft](research/manuscript/README.md) keeps the accepted 2D H20 solves as the main result. It adds the first accepted 3D solve comparison and reports the C36 extension (full derivative qualification, matched H100/CPU objective-gradient timing, and the failed 3D training and physical acceptance). It is restructured after an external review; no earlier measured value changed.

**Accepted 3D solves ([research/ethier-3d](research/ethier-3d/README.md)).** On the transient Ethier–Steinman benchmark (FP64, H100), the 17-channel C17 contract passes all eight gates in 18 of 18 solves, together with the internal baselines. Paired setup-plus-optimization speedups are **3.602×** over the A24 full spatial-jet reference (range 3.353–4.284) and **6.957×** over a naive 35-monomial ablation (range 6.750–7.436), six seed pairs each. Both baselines are internal, and the network is small (4,740 parameters).

**Representation versus execution on H100 ([details](research/2d-ablation/README.md)).** A pre-registered ablation separates GL8's derivative state from its execution on the same 2D objective (FP64, H100).
- **Execution.** Native pointwise kernels make the same GL8 computation 1.53–1.83× faster at every size; the cuBLAS GEMM time is identical.
- **Representation.** The GL8 state alone, against compiled Q10, is small and unresolved at the original 2,048 points (1.09, where the A/A control failed), and 1.63–1.66× from 16,384 points on.
- **Explicit transpose.** Against generic autograd on the same state it gives no gain at 2,048 points and 1.31× from 16,384 points on.
- **Whole solves ([details](research/2d-solve-scale/README.md)).** Each L-BFGS evaluation costs about 6–7 ms beyond the gradient at every measured size.
- At the original 2,048 points this cost absorbs the 1.71× faster gradient: optimization alone is 0.95×.
- A second pre-registered study ran 36 complete solves at 16,384 and 65,536 points, all accepted. Native GL8 shortens optimization by **1.48×** and **1.88×** over compiled Q10, and the registered rule finds a solve-level gain for every pair at both sizes.
- The H100 solve-level gain is therefore a matter of problem size.
- **The H20 campaign on H100 ([details](research/2d-h100-retest/README.md)).** All 78 solves pass. GL8 still beats every baseline, but the ratios compress: Q10 1.66 → 1.22, collapsed Taylor 1.73 → 1.37, ED/torch AD/STDE 2.31–3.37 → 1.29–1.90. The cause is the same fixed L-BFGS cost per evaluation.
- **Precision ([details](research/2d-precision/README.md)).** At these 1–2% gates, FP32 training passes the unchanged FP64 acceptance (24/24 solves). At 65,536 points it shortens optimization by 1.53× (Q10) and 1.36× (GL8), and GL8 stays 1.67× faster than Q10. TF32 training fails every solve (0/24).

**External derivative baselines in 3D ([details](research/ethier-3d/README.md#external-derivative-baselines-pre-registered-2026-10-16)).** On Ethier–Steinman, all 18 solves are accepted. Against nested PyTorch autograd and the public collapsed Taylor-mode `jet` package, C17's paired ratios are 1.90× and 1.08× (range 0.98–1.19). No arm was compiled. A fixed L-BFGS cost per evaluation compresses a 1.37× faster gradient phase. The C36 external-baseline item stays open.

**Compiled 3D variants (October 17, [details](research/ethier-3d/README.md#result-of-the-compiled-batch-2026-10-17)).** C17's compiled variant failed CUDA Graph capture, so it ran graph-only. Against compiled nested autograd, C17 is 1.59× faster. Against compiled collapsed Taylor mode there is no decided difference in whole solves (1.05×), and that baseline is faster in optimization alone (C17 takes 1/0.81 ≈ 1.23× as long). Its compiled gradient phase is about 2.6× faster than eager C17, which has no native execution layer.

**Compiled C17, second attempt ([details](research/ethier-3d/README.md#result-of-attempt-2-2026-10-17)).** A repaired compile route lets every method run compiled. In whole solves C17 is 1.68× faster than nested AD and 1.13× faster than collapsed Taylor mode; both are registered "C17 faster". In optimization alone the jet is faster: C17 takes 1/0.96 ≈ 1.05× as long. C17's lead therefore rests on cheaper setup.

**C36 development run ([details](research/c36-potential-proof/README.md#development-run-with-a-larger-l-bfgs-budget-v2)).** The C36 problem has the same data, model and gates as before; only the L-BFGS budget grows 40×. All four runs (two engines × two seeds) pass the training gate. In the two fully observed final checkpoints (analytic engine), every physical gate passes except stratified corrected pressure (about 2.5× over its limit) and, for one seed, the trajectory endpoint (7% over). Under its pre-registered rule the outcome is mechanically branch (b). The analytic engine reaches each loss threshold 1.30–1.33× sooner than parameter AD. These are development seeds, so no formal time-to-accuracy claim is made.

**C36 checkpoints on every data plane, and a smoothness continuation ([details](research/c36-potential-proof/README.md#all-plane-observations-along-the-training-trajectory)).**
- **Observations.** 72 observations cover every saved development checkpoint and 26 continuation checkpoints. Each is checked on all data planes and re-read by the independent reader (8,424 checks, all pass). No checkpoint passes physical acceptance.
- **Errors still falling.** The physical errors fall with the loss and are still falling at the end of the budget. The remaining failure, stratified pressure inside the moving box, falls more slowly than the loss. So branch (b)'s "errors stop falling" does not hold either.
- **Smoothness continuation.** A continuation that starts with a slower, smoother box motion (transition width w = 1, then 3/4, then the original 1/2) changes where training meets the loss gate:
- One schedule reaches the gate with 0.6–0.7× the baseline's stratified pressure error.
- Another reaches loss thresholds in 8–36% fewer evaluations.
- Neither changes which acceptance test fails. Under the registered rule, the continuation does not help: the slower schedule is harmful and the other fits no branch cleanly.

**More optimizer budget does not close the gap ([details](research/c36-potential-proof/README.md#budget-study-v5-pre-registered-outcome-e5)).** The same recipe with six times the L-BFGS budget (4,800 calls, both engines, two seeds) stalls soon after call 800. Over the extra 4,000 calls the loss falls 3–12%, and stratified pressure changes by under 4%, ending at 2.2–3.0× its limit. The registered outcome is E5: no pass, and the errors stopped falling. The remaining gap therefore needs a changed problem definition, not more budget.
- An earlier resumed design (v4) was cancelled before any physical observation, because a fresh L-BFGS history at loss ~1e-6 stalls. It is recorded and archived.

### Naming

| Name | Where | Meaning |
| --- | --- | --- |
| Q10 / Q20 | paper, code | Complete third-order Taylor state in 2D (10 channels) / 3D (20 channels) |
| GL8 | paper | 2D consumer state: value, gradient, packed Hessian, gradient of the Laplacian (8 channels) |
| C36 | paper, `research/c36-potential-proof` | 3D transient curl-potential consumer state (36 channels) |
| C17 / A24 | `research/c36-potential-proof/src` | Earlier 3D velocity-model contract (17 channels) and its full-spatial-third-order reference (24 channels) |
| B1 | `docs/cuda-*` | Library GEMM plus separate native jet forward/VJP (the library-GEMM jet baseline) |
| U / F | `docs/cuda-*` | Same CUTLASS main loop with a separate VJP (U) or fused dgrad/VJP (F) |
| B1+ / F+ | `docs/cuda-h100-results.md` | B1 / F with a finite Torch/cuBLASLt GEMM plan search |
| G1 / G2 | `docs/validation.md` | SM120 first-layer coordinate-gradient kernel (G1) and its fused VJP/wgrad variant (G2) |
6 changes: 5 additions & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,8 @@ PYTHONPATH=src uv run pytest -q

## 研究论文

[2026-10-13 研究稿](research/manuscript/README.md)保留已通过验收的 2D H20 求解实验,并加入完整 C36 导数检查、H100/CPU 配对性能、已完成的 3D 训练与物理验收失败结果。CPU 解析转置加速与 GPU 性能、完整求解器加速分别报告。
[2026-10-14 研究稿](research/manuscript/README.md)以已通过验收的 2D H20 求解实验为主体,新增第一个通过验收的 3D 求解对比,C36 作为三维推广:完整导数检查、H100/CPU 目标与梯度配对吞吐,以及已完成但未通过验收的 3D 训练和物理检验。本版按外部评审意见重新组织结构,原有实测数值均未改动。命名对照见英文 README 的 Naming 表。

**通过验收的 3D 求解([research/ethier-3d](research/ethier-3d/README.md))**:在瞬态 Ethier–Steinman 基准上(FP64,H100),17 通道 C17 契约与内部基线共 18 次求解全部通过 8 项门限。C17 的配对加速比为:相对 A24 完整空间 jet 参考 **3.602×**(3.353–4.284),相对朴素 35 单项式消融 **6.957×**(6.750–7.436),各 6 对。两条基线都是内部实现,网络规模较小(4,740 参数)。

**C36 开发性运行([详情](research/c36-potential-proof/README.md#development-run-with-a-larger-l-bfgs-budget-v2))**:数据、模型、门限都不变,只把 L-BFGS 预算加大 40 倍。四个 run(两种引擎 × 两个种子)全部通过训练门限。完整观测的两个最终检查点(analytic 引擎)中,除分层修正压力(约为门限的 2.5 倍)和一个种子的轨迹终点(超 7%)外,其余物理门限全部通过。按预先登记的规则,机械判定为情形 (b),但该情形要求的趋势条件无法评估。analytic 引擎到达各 loss 阈值比参数 AD 快 1.30–1.33×。种子在 v1 已观测过,因此不作正式的到达精度时间结论。
Loading
Loading