Repository navigation
Consolidation of #122, #124, #125 and #119: device symbolic maps, ordering knobs, division-free Cholesky, GV100 record - #126
Merged
Merged
Conversation
On a GPU backend the analysis builds three data-parallel pieces through KernelAbstractions kernels with element-identical results to the host originals: the SymmetricPattern adjacency (packed-key sort + dedup), the assembly map (binary search per stored entry), and the unsigned amap grouping (packed-key sort reproducing the host (abs, neg, p) order). Gated by device_maps_supported (symmetric structures, triangular stored views, packed-key size bounds); everything else keeps the host route. Measured on a 674k condensed ACOPF KKT (GV100): assembly map 0.37 -> 0.03 s, grouping 0.68 -> 0.04 s, pattern build 0.83 -> 0.21 s; MadNLP SparseCondensedKKTSystem init 6.4 -> 3.3 s with a given permutation, same iterations and objective. Tests: test/test_symbolic_device.jl asserts host/device equality for all three routes on every backend (sizes 0..1500, views L/U, structures SPD/S) plus an end-to-end analyze/factorize/solve. CPU suite 41389 pass; CUDA leg of the new file 121 pass locally (GV100). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ime-B front kernels The fused subtree kernel and the regime-B panel helper performed the rank-1 trailing update with the UNSCALED column, dividing both factors by the pivot inside the inner loop: two f64 divisions per updated element, which on the 78k-bus ACOPF condensed KKT (pglib_opf_case78484_epigrids, n = 674562) is ~1.2e9 divisions per refactorization, each an order of magnitude costlier than the fused multiply-add. Restructured both column steps as scale-first: the first half checks the pivot and scales the column below the diagonal by one reciprocal (recomputed per work item from local memory, so no extra barrier and no shared pivot cell), the second half does the trailing rank-1 update with multiply-adds only and writes the diagonal (the diagonal is untouched in the first half so no work item reads a value another one is overwriting). Same barrier count, same pivot sequence, same failure semantics (info = first non-positive column, NaN included). Measured on a GV100 (fused refactorization of the 78k KKT, correctness-gated against the reference solve): regime-A subtree phase 10.3 -> 6.2 ms, fused panel/mega phase 16.4 -> 11.6 ms, end-to-end refactorization 29.3 -> 25.7 ms (cuDSS with its own reordering: 24.3 ms). Tests: full CPU suite 9813 pass (all four element types); CUDA suite pending CI.
The rank-1 trailing-update loops of the regime-A and regime-B column steps iterated the full r x r square and discarded the upper half with a guard, wasting half the work items (divergent lanes idle through the local-memory traffic of their warp). _tri_decode inverts the packed lower-triangle enumeration (one float sqrt plus two exact fixups, verified exhaustively for r = 1..700), so the loops now launch r(r+1)/2 work items with no guard. GV100, 78k ACOPF condensed KKT: fused refactorization 25.3 -> 24.4 ms (regime-A subtree phase 6.2 -> 5.1 ms); cuDSS with its own reordering on the same device: 24.3 ms. CPU suite green (cholesky + refactorization selections); CUDA suite pending.
One command on a clean NVIDIA box: installs Julia 1.13 via juliaup, clones the branch, runs the CPU and CUDA test suites, then the end-to-end MadNLP ACOPF solve (78k-bus pglib case) with cuDSS and with the SDS prototype wrapper on the same device. SDS_BRANCH/SDS_DIR/SKIP_TESTS/CPU_ONLY_TESTS knobs documented in the header.
…GPU suite, run_amd.jl)
…d, nohup/sbatch usage in the header)
A readable alternative to fresh_machine.sh for the workflow itself: a single julia file that bootstraps its own environment (registered packages plus this repository at the tested rev), then reads as the workflow does — using SparseDirectSolver, MadNLP, MadNLPGPU, ExaModels, ExaModelsPower; build the 78k-bus ACOPF; solve with cuDSS and with SDS on the same device. The shell script remains for the full test-suite pass.
…-split environment)
…/.julia/dev) instead of a frozen add
…der SparseCondensedKKTSystem)
…ng T23 The previous commits copied subtree.jl/front.jl wholesale from a pre-T23 checkout, silently reverting the per-backend local-memory machinery (the 5-entry SUBTREE_LOCAL_SIZES and the matching _with_local_bytes), which made every analysis fail with 'update _with_local_bytes' on a clean clone. Both files are rebuilt from origin/main with only the two optimization hunks applied (division-free scale-first column step; packed-triangle enumeration of the trailing update).
…sion requires it to activate the GPU condensed-KKT path)
METIS_OPTION_NSEPS (candidate separators per split) is the one NodeND knob that reproducibly improves the ACOPF KKT orderings: on kkt_pglib_opf_case78484_epigrids_condensed_10 (n = 157k), nd_nseps = 4 gives nnz(L) 14.42M vs 14.76M (-2.3%), factorization flops -6.7%, and the PR #107 fused refactorization 30.1 ms vs 32.1 ms (-6.2%, interleaved A/B on a GV100, correctness-gated) at the same schedule depth. The ordering itself runs ~2x slower, so the default stays at the METIS default (-1); the option pays off when one symbolic analysis is reused across many factorizations and is exposed like nd_ubfactor. ND_PROVIDER gains a third argument (values < 1 fall back to the library default); validated like the other config ints and covered in test_options and test_symbolic_etree.
…ring nd_seed mirrors nd_ubfactor/nd_nseps (ND_PROVIDER gains a fourth argument, the Metis extension passes it to CliqueTrees.METIS; values < 0 fall back to the library default). The MadNLP wrapper (bench/e2e/MadNLPSDS.jl) gains sds_nd_nseps/sds_nd_seed/sds_amalgamation with the 78k-ACOPF champion defaults (nseps = 4, seed = 3, max_width = 48: refactorization 24.4 ms vs 30.6 ms on the untuned ordering, GV100) and sds_user_perm, which bypasses the built-in reordering with a user permutation. fresh_machine.jl targets this branch and documents both hooks.
The tuned METIS ordering (nd_nseps = 4) costs ~2x the default ordering at this size, every solver construction. The wrapper now caches the computed fill-reducing permutation (perm_reorder_row after the analysis) keyed by the matrix pattern hash + ordering knobs under ~/.julia/sds_perm_cache and reloads it as a user permutation on any later construction with the same structure: the first solve pays METIS once, every later init loads the permutation in milliseconds. sds_perm_cache = "" disables.
…hout a driver chose cuda on AMD nodes); document --startup-file=no everywhere
… clean nvidia-smi exit
…ts/SparseDirectSolver.jl into custom-reordering
The complete bench/compare.jl run of PR #107's harness state on a second device — every done-task feature, all harness matrices including the case14/118/1354 dumps (iterations 1, 10, 20) and the 78k-bus epigrids dumps, both solvers, plus the rendered report. Kept beside bench/comparison so the RTX 4080 record stands; the CSVs carry the device per row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Oct 10, 2026
Atomix.@atomic x[i] += v returns the new value, so 'left == 1' in the TRSM and TILE completion branches fired on the second-to-last finisher: cb_done, arrived[parent] and the write-through ea_left were signaled while one block of the front was still writing its contribution. A fence cannot repair a logically early signal; architectures with 2048 threads/SM (sm_70/80/90) hid it behind co-scheduling latency, 1536-thread parts (sm_86/89, Blackwell) and gfx906 exposed it. Found by SASS-and-protocol audit (the wide-front roles already used the safe spin-to-zero design, narrow roles kept the last-decrementer shortcut with the wrong-value test). left == 0 is the true last finisher. Measured: L40S race_repro 20/20 bad -> 0/20; AMD Radeon VII merged-launch stress 3/12 bad -> 0/24; GV100/H200 unchanged-clean.
… last finisher signals) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MetisExact.jl: bitwise-faithful Julia translation of METIS 5.1.0's NodeND path (all-default options, the PR #107 configuration), validated perm- and iperm-identical to libmetis on lap2d/lap3d and the case78484 KKT dumps up to n = 2.09M. GpuND.jl: KernelAbstractions kernels for the pipeline stages (hashed-handshake HEM matching with 2-hop, device-resident contraction), stage-validated against the exact port. Hybrid.jl: nodend_hybrid = device coarsening chain + exact host separator/refinement/MMD, threaded sub-threshold subtrees; fill 1.003-1.015x exact on the benchmark set. Smoke: bench/ordering/nodend_smoke.jl (lap2d60: ratio 1.0000, SMOKE OK). Base for the one-shot ordering-bottleneck work: parallel node refinement and level-batched recursion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…f by default Round-based parallel approximation of FM_2WayNodeRefine1Sided + FM_2WayNodeBalance: fixed-direction wavefront micro-rounds (gain, local-max conflict filter, budgeted selection, apply, pull), light-side balance rounds with the serial guards, pass-level global-best tracking. Hybrid.jl walks the uncoarsening chain with exact host FM below REFINE_HOST_BELOW (50k) and device rounds above (full-device rounds left a coarse level at sides 90k/55k; the split fixed it). Measured on the 78k-bus condensed KKT (session GV100, interleaved): split mode nnz_L 15.245M vs host-FM 14.796M (1.030x), flops 1.661G vs 1.398G (1.19x), schedule 28 vs 24 levels, t_ord at parity under load ~90. lap2d 300^2 smoke: 0.987x exact. GPU_REFINE stays FALSE: on the runtime gate the device top-chain refinement saves only the ~0.36s host-FM share while charging the numeric phase, so it loses net. The kernels are groundwork for level-batched sub-bisections, where projection/balance at scale is the missing piece and refinement stays host-exact per level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…o src - src/fused.jl: the fused factorization (Cholesky + no-pivot LDLt) and the partitioned-inverse/fused solve as the package submodule SparseDirectSolver.Fused, together with the SDSTuning bundle and the per-architecture default_tuning table (they parameterize these kernels, so they live here, not in the MadNLP glue). - ext/SparseDirectSolverMadNLPExt.jl: the MadNLP.AbstractLinearSolver implementation (SDSSolver, SDSSolverOptions: reordering, tuning, user permutation, on-disk permutation cache), loading when MadNLP is present (new weakdep, compat 0.8). Works against both the registered MadNLP 0.8 and the dev branch (solve_linear_system! defined conditionally). Accessors SparseDirectSolver.madnlp_solver() / madnlp_solver_options() are the stable entry points. - bench/e2e/MadNLPSDS.jl and bench/e2e/SDSFused.jl retire; the drivers and benches use the extension and SparseDirectSolver.Fused. Gates: full suite green (CPU + CUDA); extension loads against registered MadNLP (SDSSolver <: AbstractLinearSolver); end-to-end 78k ACOPF on the GV100 through the extension: SOLVE_SUCCEEDED, iterations and objective identical to cuDSS (101, 1.53139916e7), wall 18.95 s vs 24.24 s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…real blocks Block-diagonal batching of independent sub-bisection graphs: one device CSR carries a whole recursion level; GpuND matching/contraction apply unchanged (blocks are disconnected components, so proposals cannot cross blocks), and per-block CoarsenTo freezing rides on the per-vertex matching-cap vector (cap 0 = identity). Replaces the 2026-10-09 host-resliced dead-end: the batch stays device-resident across levels; the host sees nb counters per level. Validated on the 20 real sub-50k blocks of the 78k-bus condensed KKT recursion (671947 vertices total, median 31.5k): the full batched coarsening chain to per-block CoarsenTo (coarse n = 1875, 20/20 frozen) runs in 0.31 s warm on the GV100 (two consistent runs: 0.31/0.35 s), with no cross-block edges and per-block weight conservation checked on device before and after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Metis is a dependency: the Metis extension moves into src/symbolic/ordering.jl as `nd_algorithm`; `ND_PROVIDER`, `nd_available` and the `nd_available` ordering-stats field are gone, with the tests, the test environment and the docs following. * bench/ordering (the GPU NodeND prototype, referenced by nothing) is removed. * Fused: no module-global plan state (the backend comes from the solver, the inverse buffers are arguments of the sweeps, the merged tier's subtree list lives in the solve plan); the element type comes from the factor instead of hard-coded Float64; stdout prints go through the package logger; dead code removed (`algo1_solve!`, `waiter_kernel!` with its undefined `scap`, the `if true` wrapper, the unused `nstreams`/`debug`/`s2`/`nblocks`); the per-architecture tuning keys on a backend hook (`max_threads_per_multiprocessor`, defined by the CUDA extension) instead of module reflection. * The MadNLP extension, the e2e bench scripts and the bench write-ups follow (retired file names, the `main` branch in the reproduction command). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XYWifvvbufNiXqVYuvCrtp
…ype T The cleanup (0cf1b7c) introduced T = eltype(s.numeric.factor) at the top of build_fused_plan while the emit_split closure already assigned T = cld(m, TILE) in the same function scope, so any plan with host-wide fronts (the GV100/A100/ H200 tuning profile) reached the return with T bound to an Int and crashed in KA.zeros. The merged profile (L40S, AMD) never calls emit_split, which is why the suite and small matrices pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
The cleanup commit 0cf1b7c crashed |
Contributor
Author
|
Correction to the previous comment, verified on an L40S run: the 0cf1b7c crash is NOT limited to the host-wides profile. |
The cleanup pointed the reproduction one-liner, fresh_machine.sh's default branch and fresh_machine.jl's pinned ref at main ahead of the merge, but PR 126 has not merged: the doc's download URL 404s (fresh_machine.jl does not exist on main) and the .sh default would benchmark the pre-campaign stock solver. Point all three at consolidated-prs, marked to flip to main when the PR merges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
nseps = 4 (the 78k ordering champion) costs ~2x the reordering time (11.5 s versus 5 s on the 78k condensed KKT, same machine) for a 6% refactorization gain. The owner prefers the fast first touch as the default (2026-10-12); nseps = 4 stays one sds_tuning knob away, and the performance write-up records that its scoreboard numbers were measured with nseps = 4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d solve The one-liner used to end silently after the solve. It now prints status, iterations, objective and every MadNLP time counter (init = reordering + analysis + first factorization, linear_solver_time, total) for the first solve, then runs a second solve in the same process so compilation is separated from the real speed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…flow.jl runner The shell route now generates ~/.julia/environments/sds/Project.toml from bench/e2e/Project.toml (SparseDirectSolver's [sources] entry rewired to the local checkout) and Pkg.instantiate()s it -- no ad-hoc Pkg.add anywhere (owner request). The 78k workflow moves into bench/e2e/workflow.jl (backend autodetected through device files, first solve with the full time-counter split, warm second solve) and runs with --project=@sds; the environment is reusable afterwards from anywhere on the machine. The test suites become the opt-in RUN_TESTS=1 instead of the default. The reproduction section of the performance write-up shows the shell route first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…curl|bash truncation Two changes. (1) exec < /dev/null is gone: under curl | bash the script itself arrives on stdin, so redirecting stdin away discards everything past bash's first read-ahead buffer -- the piped invocation executed a few header lines and exited 0 (measured with a padded script; a short script fits one buffer and masks the bug). Children that might read stdin get per-command < /dev/null. (2) With a terminal attached the workflow step execs julia -i reading from /dev/tty, so the run ends in a REPL with m, sol, sol2 and solve() defined (owner request); detached runs (nohup, sbatch) behave as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…inertia :ldl builds the solver as structure "S" with pivot_type 'N' and runs the fused pipeline in LDL mode; host-wide fronts take the potrf-derived path (valid while a front's pivots share a sign; a sign flip fails potrf, info != 0, and MadNLP regularizes). MadNLP.inertia reports the true inertia from the signs of D, so SparseKKTSystem-style formulations get real curvature information instead of the found-ok/failed surrogate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ow.jl With the cache on (the default), only the first solve on a pattern pays METIS; SDS_PERM_CACHE=0 makes every solve pay the full reordering, which is what you want when timing it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner call (2026-10-12): no caching of any kind. The sds_perm_cache option, the on-disk store under ~/.julia/sds_perm_cache and the read/write plumbing are gone; every solver instance recomputes its ordering (nseps = 1 default: ~5 s on the 78k), and sds_user_perm remains for pinning one explicitly. workflow.jl loses its SDS_PERM_CACHE knob (nothing left to switch); the write-ups record that the campaign's numbers were produced with the cache. An existing ~/.julia/sds_perm_cache directory is now inert and can be deleted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… the extension) The madnlp_solver()/madnlp_solver_options() accessors are replaced by the idiom MadNLPGPU uses for CUDSSSolver: the parent declares and exports the SDSSolver and SDSSolverOptions globals, and the extension assigns them in its __init__ with setglobal!. With MadNLP loaded, users write madnlp(model; linear_solver = SDSSolver, sds_factorization = :ldl, ...); before that the names are unassigned, as MadNLPGPU.CUDSSSolver is without CUDA. The bench scripts and write-ups follow. MadNLP compat widens to 0.8-0.10 (the extension's interface surface is unchanged across them; load verified on 0.8 here and on 0.10.1 downstream). Load-smoke validated: SDSSolver is unassigned before MadNLP loads, and is the extension's type with working options after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The exported SDSSolver/SDSSolverOptions are declared globals the MadNLP extension assigns, so Aqua's undefined-exports check fails on the bare package. The Aqua workflow now loads MadNLP first, which makes that check validate the assigned exports for real; the in-suite Aqua run excludes only undefined_exports (MadNLP is not a test dependency) and keeps everything else. Both configurations run green locally against this tree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ots, knob validation Findings of the PR-wide adversarial review, each verified at source: * LDL refactorization never reduced the per-front statuses into the slot _numeric_info! reads (cholesky_stats! was skipped in LDL mode), so after the constructor's one stock factorization every refact_fused! reported that stale success forever -- a zero pivot mid-IPM left info == 0 and MadNLP consumed 'true inertia' from a corrupt factor. The stats kernel now runs unconditionally (its positive-pivot fields are unused garbage in LDL mode; the reduction is what matters). * Per-front info is zeroed at the top of every refactorization: the host-wide LDL kernels set info[v] only on failure, so one failed refactorization stayed failed through every later (genuinely successful) one. * The last-TRSM completion check read the LIVE tiles counter, leaving a window where both the last TRSM and the last tile signal cb_done/arrived -- the consumers' != spin loops then never terminate. The check now reads the plan constant tiles_base (the static 'front has no tiles' predicate). * All four no-pivot LDL pivot checks accepted NaN/Inf pivots (!iszero), unlike the Cholesky paths' potrf semantics; they now require isfinite as well. * build_fused_plan validates wgf (>= 128, multiple of 32): the tile roles keep cld(1024, wgf) partial sums in exactly 8 registers, so smaller workgroups silently dropped/reused accumulator slots. * emit_wide rejects fronts whose tile count overflows the 10-bit WTILE rc encoding instead of silently decoding tiles into the wrong panel. * MadNLP ext: the LDL failure inertia was (0, 1, n), which sums to n + 1; it is (n - 2, 1, 1) now, like the Cholesky branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…g control) * ldl_fused_race.jl exited 0 with 20/20 BAD runs; it errors on any bad run now. Its 'same-strategy' Cholesky control used in-kernel wides against the LDL plan's host-potrf wides -- the control is the host-wides Cholesky plan. * ldl_fused_test.jl gated the fused solve at relres < 1.0, which a zero vector nearly meets; the gate is now 10x the stock solve's residual on the same system. * Docs de-staled: ARCH-TUNING names the committed gate scripts (the campaign's one-off sweep drivers were never shipped) and the SDSTuning knobs that replaced the SDS_WGF/SDS_SPIN_CAP env vars; PERFORMANCE/README state the MA86 numbers' one-off-driver provenance; flip-at-merge marker on the reproduction URLs; fresh_machine.jl's stale perm-cache comment removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, kernel Float64s * device_symmetric_pattern validates its input with the same checks (and the same InvalidValueError messages) as the host route, on the host copies it already makes -- malformed device input previously reached the kernels (BoundsError/trap at best, in-range garbage from the unzeroed keys buffer at worst). * device_maps_supported admits only what the maps' hard-coded Int32 storage holds (n + 1, nnz + 1, factor_len + 1 <= typemax(Int32)); the 2^31..2^32 band previously died in a raw InexactError mid-analysis. * The device amap kernel writes a typemax sentinel for an entry outside the symbolic factor (the dropped host-route guard) and the wrapper raises the host route's exact error once, instead of silently producing a wrong offset. * No Float64 left in kernels (Metal has no fp64): _tri_decode is integer isqrt-based (verified equal to the float version exhaustively for r <= 300 plus big-r boundaries), and the spin-backoff scratch/accumulators are Int32. * nd_nseps = 0 is rejected loudly instead of silently coerced to the library default (-1 and >= 1 remain the documented domain). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uction slowdown Measured on the augmented 78k KKT (n = 2.09M, GV100, nseps = 1): analysis is 316 s with the default "S"-structure pair detection, 64 s with pivot_pairs = "none", 27 s for the SPD control. A saddle system's huge structurally-zero diagonal block is exactly what zero_pivot_pairs! rounds chew on, and the no-pivot factorization never consumes the pairs. This was the dominant cost of the ~6-minute per-instance MadNLP init observed on the stabilized augmented benchmarks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The reproduction one-liners, fresh_machine.sh's default branch and fresh_machine.jl's pinned ref move from consolidated-prs to main, per the PR's merging checklist. Note the one-liner tests main's HEAD, so it serves the consolidated work only once the PR lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sshin23
force-pushed
the
consolidated-prs
branch
from
October 12, 2026 23:33
47c3dc1 to
e86447d
Compare
github-actions Bot
pushed a commit
that referenced
this pull request
Oct 13, 2026
…and task-parallel recursion (#130) Adds bench/ordering (the exact METIS 5.1.0 NodeND port, the device matching and contraction kernels, and the hybrid ordering developed alongside #126) with two performance changes: the contraction groups edges with one radix sort! of (crow<<44 | ccol<<24 | edge index) instead of sortperm + gathers (element-identical output, 1.8x on the GV100 coarsening chain, guarded with the sortperm path as fallback for larger graphs), and the sub-threshold recursion spawns tasks above SPAWN_MIN instead of running one serial walk per top subtree. Opt-in PARALLEL_TOP overlaps the top sibling bisections' host work; on multi-GPU hosts TASK_SETUP must pin each task's device (documented). Case78484 (n = 674,562, GV100, 16 Julia threads): MadNLP init_time 8.62s -> 7.42s (packed contraction) -> 5.64s (+ parallel top), identical convergence (101 iterations, same objective) and fill (1.48e7 nnz_L vs METIS 1.476e7). Gates: lap2d fill 1.017x vs the exact port; 78k fill 1.003x/1.012x, valid permutations. The recursion gains require a multi-threaded Julia session; single-threaded sessions should keep algo4. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
One PR carrying the four open PRs, as requested. Base content is the integration branch that merged them (
custom-reordering), plus the #119 record commit cherry-picked on top. Constituents:SymmetricPattern/assembly_map/_group_amapbuilt on the GPU backend behind adevice_maps_supportedgate, element-exact vs the host originals (kernel-level on the 78k-bus condensed KKT, n = 674k: assembly map 12x, amap grouping 16x, pattern build 4x; MadNLP init on that case 6.4 -> 3.3 s).nd_nseps: METIS_OPTION_NSEPS as a cuDSS-style config parameter (nd_nseps = 4 on the 78k condensed KKT: nnz(L) -2.3%, flops -6.7%). Plusnd_seedand the e2e wrapper's custom-reordering hook with an on-disk permutation cache (validated across fresh processes: init 189.6 -> 171.5 s, identical solution).bench/compare.jl+compare_report.jlrecord vs cuDSS 0.8.0 (bench artifacts only).Performance highlights
Refactorization of the 78k-bus ACOPF condensed KKT (the IPM's repeated
operation), medians of gated runs; "before" is the stock solver on
mainorthe first fused measurement on that device, as noted.
main)Gating already run on this content (GV100, Julia 1.13)
madnlp(m; linear_solver=SDSProtoSolver, tol=1e-8), 78k-bus ACOPF condensed KKT) from a clean environment: Optimal Solution Found.CI on this PR re-runs the suites from scratch.
Notes
Addenda (synced from the integration branch after opening)
left == 0signals on the true last finisher. Validated: L40S repro 20/20 bad -> 0/20 clean; Radeon VII merged-launch stress 3/12 bad -> 0/24 clean.default_tuning(backend)table in the wrapper, per-plan workgroup size and spin-backoff cap as a kernel argument (with a fix for the cap leaking into solve kernels without the binding), procedure and champions inbench/ARCH-TUNING.md; e2e-gated on GV100 + Radeon VII (AMD linsolve 36.5 -> 34.0 s from the new defaults).bench/PERFORMANCE-2026-10.md(cross-platform results incl. the FP64-tensor-core ceiling measurement: role-ablation on A100/H200 bounds the realistic DMMA gain at ~10-12%, so no DMMA kernel), the fused solver becomesSDSSolver(SDSProtoSolveraliased), concurrent-checkout hardening infresh_machine.jl.inkernel_wides = false(correct, measured slower: ~5 launches per 64-column block where potrf is one call).bench/e2e/SDSCPU.jl, a task-parallel CPU refactorization over the stock schedule (subtree chunks in plain scalar Julia; per-front assembly + LAPACK/BLAS tasks for regime B/C; a no-task single-thread specialization; CPU-specific amalgamation 96). 78k KKT on a 2x24-core Xeon 8160, gated: 449 / 106 / ~85 ms at 1 / 8 / 24 threads, vs HSL MA86 963 / 151 / 76.5, CHOLMOD 966 (serial), MA57 1909, MA27 2798. Negative results recorded in the log: DAG scheduling (correct, 6x slower - the CB stack's slot reuse is the cache locality) and single-socket NUMA pinning (2.5x slower).ma27ad_returns success with zeroed output on any input; the subset build returns -26), which is the real cause of MadNLPHSL'sMa27SolverSymbolicException.src/fused.jl(submoduleSparseDirectSolver.Fused, carrying theSDSTuningbundle and the per-architecturedefault_tuningtable);ext/SparseDirectSolverMadNLPExt.jlimplementsMadNLP.AbstractLinearSolveron a new MadNLP weakdep (compat 0.8; works against the registered release and the dev branch), reached viaSparseDirectSolver.madnlp_solver(). The bench prototypesMadNLPSDS.jl/SDSFused.jlretire. Gated: full CPU+CUDA suite green; end-to-end 78k ACOPF on the GV100 through the extension matches cuDSS exactly (101 iterations, objective 1.53139916e7) at 18.95 s wall vs 24.24 s.bench/PERFORMANCE-2026-10.mdextended with the LDL, CPU and one-row-per-hardware summary sections.Addenda 2 (the 2026-10-12 consolidation-finishing work)
nd_algorithmin src, the Metis extension retires); the fused module drops all module-global plan state, takes its element type from the factor, logs through the package logger; dead code removed.T = eltype(...)collided with the pre-existingT = cld(m, TILE)assigned by theemit_splitclosure in the same function scope, sobuild_fused_plancrashed on every GPU for any problem with a front wider than 64 rows below the diagonal. The test suite's matrices are too small to have one, which is why it stayed green; a regression test should build a fused plan with one such front.nd_nsepsdefault 4 -> 1 (owner call: first-touch reordering ~2x faster, 5 s vs 11.5 s on the 78k; nseps = 4, the -6% refactorization champion, stays onesds_tuningknob away; the scoreboard numbers above were measured with nseps = 4).sds_perm_cacheis gone and every solver instance recomputes its ordering;sds_user_permpins one explicitly.madnlp_solver()is replaced by the MadNLPGPU idiom — the package declares and exportsSDSSolver/SDSSolverOptions, the extension assigns them in its__init__viasetglobal!; usage ismadnlp(model; linear_solver = SDSSolver, ...). MadNLP compat widens to 0.8-0.10 (load-verified on 0.8 and 0.10.1). The Aqua workflow loads MadNLP so the undefined-exports check validates the assigned exports; the in-suite Aqua excludes only that check.sds_factorization = :ldlthrough the extension (8b02065): no-pivot LDLt with TRUE inertia from the signs of D (host-wide fronts on the potrf-derived path; a sign flip fails potrf and MadNLP regularizes). End-to-end on an L40S: cholesky/ldl x cold/warm all SOLVE_SUCCEEDED at 101 iterations (cuDSS parity), objectives identical to 13 digits, warm linear-solve 7.1 s (ldl) vs 6.3 s (cholesky).fresh_machine.shbuilds the@sdsnamed project environment frombench/e2e/Project.toml(no ad-hocPkg.add), runsbench/e2e/workflow.jl(first solve with the full MadNLP time-counter split, then a warm second solve), and ends in ajulia -iREPL when a terminal is attached. Fixed en route:exec < /dev/nullundercurl | bashsilently discards the script past bash's first read-ahead buffer.Merging checklist
flip to main when PR 126 merges:bench/e2e/fresh_machine.sh(header URL +SDS_BRANCHdefault),bench/e2e/fresh_machine.jl(SDS_REF),bench/PERFORMANCE-2026-10.md(reproduction URLs).🤖 Generated with Claude Code