Skip to content

add tmc::qu_mpsc_bounded#236

Merged
tzcnt merged 3 commits into
mainfrom
qu_mpsc_bounded
Jun 6, 2026
Merged

add tmc::qu_mpsc_bounded#236
tzcnt merged 3 commits into
mainfrom
qu_mpsc_bounded

Conversation

@tzcnt

@tzcnt tzcnt commented Jun 6, 2026

Copy link
Copy Markdown
Owner

tmc::qu_mpsc_bounded is a bounded MPSC queue. It's the worst-performing TMC queue overall due to the combination of multi-producer atomic interactions and bounded queue behavior. Still the performance is respectable. Benchmarks in SPSC mode:

  • tmc::qu_mpsc_bounded: 26M elems/sec, using capacity = 16384
  • tmc::qu_spsc_bounded: 70M elems/sec, or 90M elems/sec in polling-only mode, using capacity = 16384
  • tmc::qu_spsc_unbounded: 200M elems/sec
  • tmc::qu_mpsc_unbounded: 45M elems/sec (underperforms in SPSC scenarios due to its reclamation scheme)
  • tmc::channel: 60M elems/sec

push() is an awaitable that will suspend the producer if the queue is full, until a slot becomes available.
try_push() and push_bulk() are not provided, due to the 2-stage producer commit protocol making them impossible to safely implement.

Compared to tmc::channel:

Advantages of tmc::qu_mpsc_bounded:

  • Hazard pointers are not required. This makes it suitable for use inside of a data structure which may have an unknown number of producers, such as an "actor" type.
  • Zero-copy API by default (less bloated API)
  • No dependency on std::optional / std::variant. The zero-copy scope objects handle the optionality.
  • Bounded capacity for when you need it
  • 1 allocation during construction. Zero-allocation afterward.

Advantages of tmc::channel:

  • Supports multi-producer / multi-consumer
  • Automatically reference-counted lifetime (this may or may not be an advantage)

Consumer functions (can only be called by the single consumer):

  • co_await pull()
  • try_pull()
  • empty()

Producer functions (can be called by anyone):

  • co_await push()
  • close()
  • close_resume_inline() - When a queue and executor are destroyed nearly simultaneously, and the consumer is running on the executor that's going out of scope, there would be a race condition where close() / destructor post the consumer back to the (soon to be destroyed) executor and then return before it actually finishes. close_resume_inline() solves this by setting the close signal and then resuming the awaiter immediately inline. This is a special-purpose tool that should only be used when you can be sure that the coroutine will exit promptly when it receives the close signal. This is recommended for use when building an "actor" class.

It also supports these configurable features:

  • ConsumerCanSuspend enables the pull() operation. This is enabled by default, but can be disabled to slightly improve performance.
  • PackingLevel - by default, queue elements are padded to avoid false sharing. If this is set to 1 then no padding will be applied.

Capacity is set at runtime as a constructor parameter.


The implementation of the consumer / multi-producer suspension protocol is fairly complicated due to the possibility of multiple producers waiting on the same slot due to wraparound. e.g. if the queue capacity is 2, and 6 producers try to enqueue, producer 0 can produce to slot 0, producer 2 will suspend on slot 0, and producer 4 will also suspend on slot 0. Then, a consumer can come and consume from producer 0+1, wake producer 2, then suspend waiting for producer 2 to produce data. At this point we have both producer 4 and a consumer suspended on the same slot, and an in-flight producer 2 that will come produce to that slot. This all works, but requires careful handling.

I did also test a simpler version that just used a qu_mpsc_unbounded wrapped with a semaphore, but it was slower across all combinations of producer count and capacity, was not allocation-free, and didn't offer any additional APIs (push_bulk / try_push were still unimplementable with the current version of tmc::semaphore).


Other changes in this PR:

  • All of the TMC queues use the same uninitialized storage type. Moved it into tmc/detail/qu_storage.hpp.
  • Moved std::atomic<T>::is_always_lock_free asserts into compat.hpp
  • Standardized doc comment structures across all the newly added qu_* types.

@tzcnt tzcnt merged commit 68532d7 into main Jun 6, 2026
46 checks passed
@tzcnt tzcnt deleted the qu_mpsc_bounded branch June 6, 2026 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant