Multithreading with `RecursiveArrayTools.VectorOfArray` is broken

[ranocha]

Describe the bug 🐞

Multi-threading applied to a RecursiveArrayTools.VectorOfArray is broken. Each thread appears to perform the update.

Expected behavior

Multithreading works as expected (and as it did on version 0.3 of FastBroadcast.jl).

Minimal Reproducible Example 👇

julia> import Pkg; Pkg.activate(temp = true); Pkg.add(["FastBroadcast", "Polyester", "RecursiveArrayTools", "StaticArrays"])
  Activating new project
   Resolving package versions...
    Updating
  [7034ab61] + FastBroadcast v1.3.1
  [f517fe37] + Polyester v0.7.19
⌃ [731186ca] + RecursiveArrayTools v3.52.0
  [90137ffa] + StaticArrays v1.9.18
    Updating
  [7d9f7c33] + Accessors v0.1.44
  [79e6a3ab] + Adapt v4.5.0
  [4fba245c] + ArrayInterface v7.23.0
  [62783981] + BitTwiddlingConvenienceFunctions v0.1.6
  [2a0fbf3d] + CPUSummary v0.2.7
  [fb6a15b2] + CloseOpenIntervals v0.1.13
  [f70d9fcc] + CommonWorldInvalidations v1.0.0
  [34da2185] + Compat v4.18.1
  [a33af91c] + CompositionsBase v0.1.2
  [187b0558] + ConstructionBase v1.6.0
  [adafc99b] + CpuId v0.3.1
  [ffbed154] + DocStringExtensions v0.9.5
  [e2ba6199] + ExprTools v0.1.10
  [7034ab61] + FastBroadcast v1.3.1
  [46192b85] + GPUArraysCore v0.2.0
  [615f187c] + IfElse v0.1.1
  [3587e190] + InverseFunctions v0.1.17
  [10f19ff3] + LayoutPointers v0.1.17
  [1914dd2f] + MacroTools v0.5.16
  [d125e4d3] + ManualMemory v0.1.8
  [f517fe37] + Polyester v0.7.19
  [1d0040c9] + PolyesterWeave v0.2.2
⌅ [aea7be01] + PrecompileTools v1.2.1
  [21216c6a] + Preferences v1.5.2
  [3cdcf5f2] + RecipesBase v1.3.4
⌃ [731186ca] + RecursiveArrayTools v3.52.0
  [ae029012] + Requires v1.3.1
  [7e49a35a] + RuntimeGeneratedFunctions v0.5.17
  [94e857df] + SIMDTypes v0.1.0
  [431bcebd] + SciMLPublic v1.0.1
  [aedffcd0] + Static v1.3.1
  [0d7ed370] + StaticArrayInterface v1.9.0
  [90137ffa] + StaticArrays v1.9.18
  [1e83bf80] + StaticArraysCore v1.4.4
  [7792a7ef] + StrideArraysCore v0.5.8
  [2efcf032] + SymbolicIndexingInterface v0.3.46
  [8290d209] + ThreadingUtilities v0.5.5
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [d6f4376e] + Markdown
  [de0858da] + Printf
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [fa267f1f] + TOML v1.0.3
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
  [4536629a] + OpenBLAS_jll v0.3.23+5
  [8e850b90] + libblastrampoline_jll v5.11.0+0
        Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`

julia> using FastBroadcast, Polyester, RecursiveArrayTools, StaticArrays

julia> u = VectorOfArray(fill(SVector(1.0, 1.0), 2, 2))
VectorOfArray{Float64,2}:
2×2 Matrix{SVector{2, Float64}}:
 [1.0, 1.0]  [1.0, 1.0]
 [1.0, 1.0]  [1.0, 1.0]

julia> v = copy(u)
VectorOfArray{Float64,2}:
2×2 Matrix{SVector{2, Float64}}:
 [1.0, 1.0]  [1.0, 1.0]
 [1.0, 1.0]  [1.0, 1.0]

julia> @.. thread=true v = v + u
VectorOfArray{Float64,2}:
2×2 Matrix{SVector{2, Float64}}:
 [3.0, 3.0]  [3.0, 3.0]
 [3.0, 3.0]  [3.0, 3.0]

julia> Threads.nthreads()
2

The output clearly shows that the broadcasting update was performed twice (once for each thread). The correct output with one thread is

julia> @.. thread=true v = v + u
VectorOfArray{Float64,2}:
2×2 Matrix{SVector{2, Float64}}:
 [2.0, 2.0]  [2.0, 2.0]
 [2.0, 2.0]  [2.0, 2.0]

julia> Threads.nthreads()
1

Error & Stacktrace ⚠️

See above

Environment (please complete the following information):

See above

julia> versioninfo()
Julia Version 1.10.11
Commit a2b11907d7b (2026-03-09 14:59 UTC)
Build Info:
  Official https://julialang.org/ release

Additional context

This appears to be the root cause of SciML/OrdinaryDiffEq.jl#3331

[ChrisRackauckas]

Multithreading works as expected (and as it did on version 0.3 of FastBroadcast.jl).

Interesting. I looked at that code when doing the update and it just had a non-multithreading fallback IIRC. This is probably a good time to just make the extension handle this case appropriately, though it might need to be a double extend.

Either way, wrong repo though.

[ChrisRackauckas]

Can you let me know if this approach is sufficient? It's a bit ill-defined what to do here. The previous code just did a fallback to serial. We're trying to make this more exacting, but cannot always take a Polyester dep (it has been fully removed from the stack) so it now does Base multithreading. But Base multithreading can be bad. So... the two other options if this is too slow is to either create a RATFastBroadcastPolyester 3 way extension that invalidates this one when polyester is loaded in order to do the actual fast multithreading for users who are okay with the high invalidations, or potentially doing the v0.3 behavior of just falling back to serial in this case.

[ranocha]

Sorry for posting this at the wrong repository 🤦 But thanks for working on it so quickly 🙏

We have observed issues when mixing Polyester.jl threads and Base multithreading. Would you mind looking into the other option to create a threading extension based on Polyester.jl and RecursiveArrayTools.jl in FastBroadcast.jl?

This would also be consistent with how threading is handled in general in FastBroadcast.jl: taking the example from the README (https://github.com/SciML/FastBroadcast.jl?tab=readme-ov-file#threading), it throws an error (ERROR: MethodError: no method matching fast_materialize_threaded!(::Vector{…}, ::Base.Broadcast.Broadcasted{…})) if Polyester.jl is not loaded. Thus, it makes sense from my point of view to use either Polyester.jl or no multithreading at all.

[ChrisRackauckas]

okay it should all be polyester via extensions now.