# ThreadSanitizer reports an `Awaiter` / `WaitListLight` race during shutdown

[gorundebug]

Add a description

ThreadSanitizer reports an Awaiter / WaitListLight race during shutdown

Summary

An orderly shutdown of a userver service can produce a ThreadSanitizer report
whose conflicting paths are the construction/bookkeeping of an
engine::impl::Awaiter and
engine::impl::WaitListLight::SetSignalAndNotifyOne().

The report is observed after the service has started successfully, processed a
request, received SIGTERM, and begun normal component shutdown. It is not a
forced SIGKILL, a shutdown-timeout failure, or a report from application
business code.

This is intentionally tracked separately from the uGRPC
AsyncMethodInvocation::Notify issue. The uGRPC report requires multiple gRPC
completion queues; this report belongs to the core task/awaiter shutdown path.

Environment

• userver: v3.1, commit c9f77729c0edce7e423def2d4a4450aa7fc9d259
• compiler: Clang 18.1.3
• C++ standard: C++20
• build type: optimized Release with debug information and frame pointers
• sanitizer: ThreadSanitizer
• userver build option: -DUSERVER_SANITIZE=thread
• coroutine implementation: userver's upstream-supported
  USERVER_FEATURE_UBOOST_CORO=ON
• container OS: Ubuntu 24.04, Linux arm64
• host: Apple Silicon macOS with Docker Desktop

The process uses userver's official runtime suppression file:

TSAN_OPTIONS=halt_on_error=1:\
ignore_interceptors_accesses=1:\
suppressions=/opt/userver/cmake/tsan.suppressions.txt:\
allow_addr2line=1:\
external_symbolizer_path=/usr/bin/addr2line

No third-party source is patched.

Trigger

The smallest application-level sequence currently known to reproduce the
report is:

1. start a TSan-instrumented userver service;
2. wait until all components report ready;
3. complete one ordinary request;
4. send SIGTERM immediately after the response;
5. let userver perform its normal component and task shutdown.

The report is timing-sensitive, so the lifecycle is repeated. The wider
verification also reproduces it by stopping the service while bounded
concurrent traffic is active.

Conflicting userver paths

The normalized report identifies userver core code on both sides:

WARNING: ThreadSanitizer: data race

  one access:
    userver::engine::impl::Awaiter ...
    core/include/userver/engine/impl/awaiter.hpp
    ...
    core/src/engine/task/task_base.cpp

  conflicting access:
    userver::engine::impl::WaitListLight::SetSignalAndNotifyOne()
    core/src/engine/impl/wait_list_light.cpp

SUMMARY: ThreadSanitizer: data race

The relevant shutdown-side code atomically replaces the wait-list state and
then notifies the extracted awaiter:

void WaitListLight::SetSignalAndNotifyOne() {
    const auto old_awaiter = state_.exchange<std::memory_order_seq_cst>(
        AwaiterWithContext{kSignaled, {}});
    if (old_awaiter.awaiter == nullptr || old_awaiter.awaiter == kSignaled) {
        return;
    }
    DoNotify(old_awaiter);
}

The task-side path creates and appends a non-cancellable awaiter while waiting
for a task to finish, including TaskBase::BlockingWait() in
core/src/engine/task/task_base.cpp.

Current narrow verification rule

Until the upstream status is known, the local TSan build uses one
source-scoped compile-time ignorelist entry:

[thread]
src:*userver/engine/impl/awaiter.hpp
mainfile:*core/src/engine/task/task_base.cpp
fun:*WaitListLight*SetSignalAndNotifyOne*

This entry does not suppress all userver races and does not match application
code. It exists only to keep the remainder of userver and the generated service
instrumented while this report is investigated. It should be removed when an
upstream fix or official upstream suppression is available.

Expected behavior

An ordinary SIGTERM after a completed request should allow userver to stop
all components without a TSan report. Waiting for a task to finish and waking
that waiter during shutdown should have a synchronization relationship that
TSan can observe, or userver should provide an official narrowly scoped
suppression if the report is a known coroutine instrumentation artifact.

Questions

1. Is this a known false positive caused by the coroutine/context-switching
   implementation, or a real lifetime/synchronization race?
2. Is TaskBase::BlockingWait() expected to overlap
   WaitListLight::SetSignalAndNotifyOne() during component shutdown?
3. Does a newer userver revision contain a synchronization or lifetime fix for
   this path?
4. If it is an expected TSan artifact, can an official narrow rule be added to
   userver's cmake/tsan.suppressions.txt or sanitize.blacklist.txt?
5. Is there an upstream native test that should be used as the minimal
   reproducer for this shutdown sequence?

Reproducer status

The report is reproducible in the generated service lifecycle gate. A
ServiceLib-free native reproducer has not yet been reduced for this second
issue; unlike the separate uGRPC completion-queue issue, this document therefore
does not claim that application lifecycle ownership has already been excluded.
The next reduction step is to repeat the same start/request/SIGTERM sequence
with a minimal userver-only task created and joined during component shutdown.