Skip to content

Remove EAGER_ID_MAP - #728

Merged
jpsamaroo merged 1 commit into
masterfrom
jps/stream-key-err
Jul 27, 2026
Merged

Remove EAGER_ID_MAP#728
jpsamaroo merged 1 commit into
masterfrom
jps/stream-key-err

Conversation

@jpsamaroo

Copy link
Copy Markdown
Member

The logic around EAGER_ID_MAP wasn't perfect and was causing streaming test failures, but thankfully we didn't need this logic anymore since task TID == UID. This should improve performance especially for multi-threaded submission, as we no longer need to take this lock again and again.

Written by Cursor Grok

@github-actions

Copy link
Copy Markdown
Contributor

Dagger benchmarks: dirty vs master

master dirty master / dirty
array/dagger/N=1024 (block 512)/add (X + X) 4.09 ± 0.72 ms 4.41 ± 0.41 ms 0.927 ± 0.19
array/dagger/N=1024 (block 512)/alloc (rand) 3.25 ± 0.76 ms 3.51 ± 0.17 ms 0.925 ± 0.22
array/dagger/N=1024 (block 512)/broadcast (X .+ 1) 3.03 ± 0.18 ms 2.92 ± 0.34 ms 1.04 ± 0.14
array/dagger/N=1024 (block 512)/map (sin.(X)) 7.32 ± 0.27 ms 6.47 ± 2 ms 1.13 ± 0.35
array/dagger/N=1024 (block 512)/norm 1.19 ± 0.038 ms 1.22 ± 0.027 ms 0.977 ± 0.038
array/dagger/N=1024 (block 512)/reduce (sum) 2.5 ± 0.18 ms 2.42 ± 0.15 ms 1.04 ± 0.097
array/dagger/N=1024 (block 512)/transpose (permutedims) 6.88 ± 0.11 ms 7.11 ± 0.22 ms 0.968 ± 0.034
array/dagger/N=256 (block 256)/add (X + X) 1.29 ± 0.16 ms 1.19 ± 0.07 ms 1.09 ± 0.15
array/dagger/N=256 (block 256)/alloc (rand) 1.29 ± 0.081 ms 1.09 ± 0.033 ms 1.18 ± 0.082
array/dagger/N=256 (block 256)/broadcast (X .+ 1) 0.514 ± 0.095 ms 0.569 ± 0.12 ms 0.903 ± 0.26
array/dagger/N=256 (block 256)/map (sin.(X)) 1.02 ± 0.24 ms 1.06 ± 0.15 ms 0.954 ± 0.26
array/dagger/N=256 (block 256)/norm 0.623 ± 0.12 ms 0.582 ± 0.13 ms 1.07 ± 0.31
array/dagger/N=256 (block 256)/reduce (sum) 1.01 ± 0.07 ms 0.999 ± 0.71 ms 1.01 ± 0.72
array/dagger/N=256 (block 256)/transpose (permutedims) 1.15 ± 0.033 ms 1.12 ± 0.17 ms 1.03 ± 0.16
linalg/dagger/N=1024 (block 512)/cholesky 23 ± 0.11 ms 18.8 ± 2.8 ms 1.23 ± 0.18
linalg/dagger/N=1024 (block 512)/lu 0.044 ± 0.00085 s 0.0431 ± 0.0039 s 1.02 ± 0.094
linalg/dagger/N=1024 (block 512)/matmul (A*A) 0.0492 ± 0.0059 s 0.0519 ± 0.011 s 0.947 ± 0.23
linalg/dagger/N=1024 (block 512)/matvec (A*x) 3.03 ± 0.61 ms 3.19 ± 0.2 ms 0.949 ± 0.2
linalg/dagger/N=1024 (block 512)/qr 0.112 ± 0.0023 s 0.112 ± 0.0035 s 0.998 ± 0.037
linalg/dagger/N=1024 (block 512)/solve (A\b via lu) 0.0527 ± 0.0017 s 0.0529 ± 0.004 s 0.998 ± 0.082
linalg/dagger/N=1024 (block 512)/syrk (A'*A) 0.0327 ± 0.0021 s 0.0381 ± 0.0044 s 0.858 ± 0.11
linalg/dagger/N=256 (block 256)/cholesky 3.07 ± 0.083 ms 4.78 ± 1.9 ms 0.644 ± 0.26
linalg/dagger/N=256 (block 256)/lu 4.43 ± 0.42 ms 4.61 ± 0.15 ms 0.963 ± 0.097
linalg/dagger/N=256 (block 256)/matmul (A*A) 2.36 ± 0.22 ms 2.46 ± 0.58 ms 0.961 ± 0.24
linalg/dagger/N=256 (block 256)/matvec (A*x) 1.58 ± 0.25 ms 1.41 ± 0.22 ms 1.12 ± 0.25
linalg/dagger/N=256 (block 256)/qr 5.21 ± 0.4 ms 5.46 ± 0.73 ms 0.954 ± 0.15
linalg/dagger/N=256 (block 256)/solve (A\b via lu) 9.76 ± 1.8 ms 11.2 ± 3.6 ms 0.874 ± 0.33
linalg/dagger/N=256 (block 256)/syrk (A'*A) 3.6 ± 0.52 ms 4.36 ± 1.3 ms 0.826 ± 0.27
stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap) 26.3 ± 0.61 ms 23.2 ± 0.91 ms 1.13 ± 0.052
stencil/dagger/N=1024 (block 512)/assign (const) 1.43 ± 0.1 ms 1.45 ± 0.026 ms 0.984 ± 0.074
stencil/dagger/N=1024 (block 512)/multi-expr 4.68 ± 0.72 ms 4.06 ± 0.19 ms 1.15 ± 0.18
stencil/dagger/N=1024 (block 512)/neighbors (Clamp) 24.9 ± 0.98 ms 24.1 ± 2.9 ms 1.04 ± 0.13
stencil/dagger/N=1024 (block 512)/neighbors (Pad) 24.1 ± 1.2 ms 24.8 ± 1.6 ms 0.975 ± 0.08
stencil/dagger/N=1024 (block 512)/neighbors (Reflect) 23.6 ± 3.5 ms 24 ± 1.6 ms 0.986 ± 0.16
stencil/dagger/N=1024 (block 512)/neighbors (Wrap) 24.2 ± 1.6 ms 22.1 ± 0.25 ms 1.1 ± 0.074
stencil/dagger/N=1024 (block 512)/update (+) 2.83 ± 0.67 ms 2.93 ± 0.82 ms 0.967 ± 0.35
stencil/dagger/N=256 (block 256)/alloc (neighbors Wrap) 5.03 ± 0.21 ms 5.2 ± 0.23 ms 0.968 ± 0.059
stencil/dagger/N=256 (block 256)/assign (const) 0.654 ± 0.062 ms 0.719 ± 0.14 ms 0.909 ± 0.2
stencil/dagger/N=256 (block 256)/multi-expr 1.53 ± 0.18 ms 1.4 ± 0.035 ms 1.09 ± 0.13
stencil/dagger/N=256 (block 256)/neighbors (Clamp) 4.55 ± 0.31 ms 4.98 ± 0.62 ms 0.914 ± 0.13
stencil/dagger/N=256 (block 256)/neighbors (Pad) 4.78 ± 0.19 ms 4.64 ± 0.18 ms 1.03 ± 0.058
stencil/dagger/N=256 (block 256)/neighbors (Reflect) 4.61 ± 0.018 ms 4.6 ± 0.091 ms 1 ± 0.02
stencil/dagger/N=256 (block 256)/neighbors (Wrap) 4.57 ± 0.52 ms 4.57 ± 0.1 ms 1 ± 0.12
stencil/dagger/N=256 (block 256)/update (+) 0.901 ± 0.075 ms 0.845 ± 0.034 ms 1.07 ± 0.099
time_to_load 1.02 ± 0.0083 s 1.02 ± 0.0028 s 0.998 ± 0.0086

Plots

⚠️ Regressions (> 25.0%)

  • linalg/dagger/N=256 (block 256)/cholesky: +55.4%

Full results and plots (download the benchmark-results artifact).

@jpsamaroo
jpsamaroo merged commit 9567a1f into master Jul 27, 2026
22 of 36 checks passed
@jpsamaroo
jpsamaroo deleted the jps/stream-key-err branch July 27, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant