DArray: Add sparse matrix support and solvers and preconditioners

[jpsamaroo]

Adds a DSparseMatrix type for sparse DArray tiles, that allows in-place modifications, necessary for Datadeps algorithms. Then this builds algorithms like GEMM, GEMV, iterative solvers, and preconditioners on top that allow the DArray to work with sparse matrix and vector data effectively.

Written by Claude Opus

[github-actions]

Dagger benchmarks: dirty vs master

	master	dirty	master / dirty
array/dagger/N=1024 (block 512)/add (X + X)	4.29 ± 0.17 ms	4.89 ± 0.23 ms	0.877 ± 0.055
array/dagger/N=1024 (block 512)/alloc (rand)	3.23 ± 0.33 ms	3.56 ± 0.58 ms	0.907 ± 0.17
array/dagger/N=1024 (block 512)/broadcast (X .+ 1)	3.02 ± 0.14 ms	3.09 ± 0.13 ms	0.977 ± 0.06
array/dagger/N=1024 (block 512)/map (sin.(X))	7.24 ± 0.7 ms	7.33 ± 0.029 ms	0.988 ± 0.095
array/dagger/N=1024 (block 512)/norm	1.24 ± 0.25 ms	1.33 ± 0.086 ms	0.937 ± 0.2
array/dagger/N=1024 (block 512)/reduce (sum)	2.34 ± 0.36 ms	2.28 ± 0.35 ms	1.03 ± 0.22
array/dagger/N=1024 (block 512)/transpose (permutedims)	7.06 ± 1.1 ms	6.84 ± 0.4 ms	1.03 ± 0.17
array/dagger/N=256 (block 256)/add (X + X)	1.45 ± 0.69 ms	1.77 ± 0.19 ms	0.82 ± 0.4
array/dagger/N=256 (block 256)/alloc (rand)	1.1 ± 0.2 ms	1.18 ± 0.18 ms	0.93 ± 0.22
array/dagger/N=256 (block 256)/broadcast (X .+ 1)	0.688 ± 2 ms	0.494 ± 0.1 ms	1.39 ± 4
array/dagger/N=256 (block 256)/map (sin.(X))	1.19 ± 0.14 ms	1.15 ± 0.27 ms	1.03 ± 0.27
array/dagger/N=256 (block 256)/norm	0.635 ± 0.082 ms	0.56 ± 0.049 ms	1.14 ± 0.18
array/dagger/N=256 (block 256)/reduce (sum)	1.25 ± 0.095 ms
array/dagger/N=256 (block 256)/transpose (permutedims)	1.02 ± 0.082 ms	1.14 ± 0.36 ms	0.899 ± 0.29
linalg/dagger/N=1024 (block 512)/cholesky	21.7 ± 4.9 ms	21.1 ± 0.17 ms	1.02 ± 0.23
linalg/dagger/N=1024 (block 512)/lu	0.0438 ± 0.0029 s	0.0449 ± 0.0022 s	0.977 ± 0.081
linalg/dagger/N=1024 (block 512)/matmul (A*A)	0.0574 ± 0.012 s	0.077 ± 0.0054 s	0.745 ± 0.16
linalg/dagger/N=1024 (block 512)/matvec (A*x)	2.95 ± 0.053 ms	3.07 ± 0.12 ms	0.963 ± 0.042
linalg/dagger/N=1024 (block 512)/qr	0.121 ± 0.0012 s	0.121 ± 0.012 s	0.996 ± 0.1
linalg/dagger/N=1024 (block 512)/solve (A\b via lu)	0.0546 ± 0.011 s	0.0551 ± 0.0022 s	0.991 ± 0.2
linalg/dagger/N=1024 (block 512)/syrk (A'*A)	0.033 ± 0.0025 s	0.0727 ± 0.0023 s	0.454 ± 0.037
linalg/dagger/N=256 (block 256)/cholesky	3.6 ± 2.7 ms	3.35 ± 0.55 ms	1.08 ± 0.83
linalg/dagger/N=256 (block 256)/lu	4.54 ± 0.38 ms	5.29 ± 0.42 ms	0.858 ± 0.1
linalg/dagger/N=256 (block 256)/matmul (A*A)	2.36 ± 0.063 ms	4.21 ± 0.42 ms	0.561 ± 0.057
linalg/dagger/N=256 (block 256)/matvec (A*x)	1.82 ± 0.38 ms	1.45 ± 0.15 ms	1.25 ± 0.29
linalg/dagger/N=256 (block 256)/qr	5.3 ± 0.71 ms	6.69 ± 2.1 ms	0.793 ± 0.27
linalg/dagger/N=256 (block 256)/solve (A\b via lu)	8.88 ± 0.37 ms	8.56 ± 1.1 ms	1.04 ± 0.14
linalg/dagger/N=256 (block 256)/syrk (A'*A)	4.24 ± 0.43 ms	5.34 ± 0.077 ms	0.794 ± 0.081
stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap)	25.5 ± 2.8 ms	27.3 ± 2.6 ms	0.934 ± 0.14
stencil/dagger/N=1024 (block 512)/assign (const)	1.5 ± 0.045 ms	1.51 ± 0.051 ms	0.997 ± 0.045
stencil/dagger/N=1024 (block 512)/multi-expr	4.41 ± 0.37 ms	3.78 ± 0.34 ms	1.17 ± 0.14
stencil/dagger/N=1024 (block 512)/neighbors (Clamp)	24.8 ± 0.52 ms	24.3 ± 6 ms	1.02 ± 0.25
stencil/dagger/N=1024 (block 512)/neighbors (Pad)	24.8 ± 0.35 ms	23.7 ± 3.2 ms	1.05 ± 0.14
stencil/dagger/N=1024 (block 512)/neighbors (Reflect)	23.9 ± 2.3 ms	23.8 ± 5.1 ms	1.01 ± 0.24
stencil/dagger/N=1024 (block 512)/neighbors (Wrap)	25.4 ± 0.64 ms	25.3 ± 1.2 ms	1 ± 0.055
stencil/dagger/N=1024 (block 512)/update (+)	2.77 ± 0.41 ms	2.64 ± 1.1 ms	1.05 ± 0.45
stencil/dagger/N=256 (block 256)/alloc (neighbors Wrap)	5.45 ± 0.33 ms	5.4 ± 0.24 ms	1.01 ± 0.075
stencil/dagger/N=256 (block 256)/assign (const)	0.74 ± 0.031 ms	0.837 ± 0.11 ms	0.885 ± 0.13
stencil/dagger/N=256 (block 256)/multi-expr	1.55 ± 0.077 ms	1.62 ± 0.29 ms	0.962 ± 0.18
stencil/dagger/N=256 (block 256)/neighbors (Clamp)	4.92 ± 0.3 ms	4.93 ± 0.33 ms	0.999 ± 0.089
stencil/dagger/N=256 (block 256)/neighbors (Pad)	4.85 ± 0.27 ms	4.92 ± 0.13 ms	0.986 ± 0.061
stencil/dagger/N=256 (block 256)/neighbors (Reflect)	4.76 ± 0.4 ms	4.71 ± 0.18 ms	1.01 ± 0.095
stencil/dagger/N=256 (block 256)/neighbors (Wrap)	4.76 ± 0.19 ms	4.89 ± 0.25 ms	0.973 ± 0.063
stencil/dagger/N=256 (block 256)/update (+)	0.896 ± 0.037 ms	0.873 ± 0.025 ms	1.03 ± 0.052
sparse/dagger/N=1024 (block 64)/cg solve (laplacian)		1.09 ± 0.11 s
sparse/dagger/N=256 (block 16)/cg solve (laplacian)		0.985 ± 0.0044 s
sparse/dagger/N=256 (block 16)/spmv (S*x)		0.0637 ± 0.002 s
sparse/dagger/N=1024 (block 64)/spgemm (S*S)		1.09 ± 0.023 s
sparse/dagger/N=256 (block 16)/spgemm (S*S)		0.984 ± 0.06 s
sparse/dagger/N=1024 (block 64)/spmv (S*x)		0.0641 ± 0.0017 s
time_to_load	1.05 ± 0.013 s	1.09 ± 0.0017 s	0.965 ± 0.012

⚠️ Regressions (> 25.0%)

• linalg/dagger/N=1024 (block 512)/syrk (A'*A): +120.4%
• linalg/dagger/N=256 (block 256)/matmul (A*A): +78.2%
• linalg/dagger/N=1024 (block 512)/matmul (A*A): +34.2%
• linalg/dagger/N=256 (block 256)/qr: +26.2%
• linalg/dagger/N=256 (block 256)/syrk (A'*A): +26.0%

Improvements (> 25.0% faster)

• array/dagger/N=256 (block 256)/broadcast (X .+ 1): -28.1%

Full results and plots (download the benchmark-results artifact).