Repository navigation
Conversation
lock_managed sorted every launch's memories by objectid to lock them in
a
global order. objectid and the sort cost ~2 µs per launch with 16 array
arguments, more with more. Take the locks with trylock in argument order
instead (it never blocks, so it cannot deadlock), and fall back to the
sorted, blocking order only when a lock is held elsewhere. Repeated
memories are dropped by identity first, so each is still locked and
owned
once.
using CUDA, BenchmarkTools
kernel(xs...) = return
xs = ntuple(_ -> CUDA.zeros(1), 16)
@Btime @cuda kernel($xs...)
# before: 5.5 µs, after: 3.5 µs (32 arrays: 9.5 → 4.9 µs)
The managed memories are held as an abstract Vector{Managed}, so each
take_ownership! call was dispatched dynamically and boxed its keyword
arguments: one allocation per array argument. Branch on the common
Managed{DeviceMemory} case to call it statically.
using CUDA, BenchmarkTools
kernel(xs...) = return
xs = ntuple(_ -> CUDA.zeros(1), 32)
@Btime @cuda kernel($xs...)
# before: 4.9 µs (39 allocations: 3.02 KiB)
# after: 4.1 µs (7 allocations: 528 bytes)
Author
|
@maleadt just a ping, would appreciate your feedback, if it's good to be merged. |
Member
|
Yeah sorry I wanted the huge stack of launch-related PRs to merge first. I'll take a look. |
Member
|
Should be included in #3362. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
8151595 introduced locking for the duration of the launch, which costs microseconds. Contention is rare, so we try to minimise lock as much as possible. In my testing a launch with 32 array arguments drops from 9.5 microseconds (39 allocations) to 4.1 microseconds (7 allocations).