Report EINTR from one waiter per signal

[xalestar]

kill(2) queues one signal for the whole thread group, and Linux complete_signal() wakes only the thread it picks to run the handler. On elfuse, several threads blocked in read, ppoll, pselect6, epoll_pwait or flock could each return EINTR for that one signal, while only one of them ran the handler.

These waits tested the shared pending set with signal_pending_interruption() and never took the signal. One byte on the wakeup pipe returns every thread parked in host poll() at the same time, so each saw the same bit and reported EINTR, and whichever reached signal_deliver() first ran the handler. The others handed their guests an EINTR that no handler explains. nanosleep had the same defect and claims the signal since #377; this moves the other waits onto that claim: io_wait_fd_timed_or_interrupted, tty_drain_interruptible, io_retry_backoff, ppoll, pselect6 and epoll_pwait, and in the third commit futex_os_sync_wait, futex_poll_signal_relock and rt_sigtimedwait.

The second commit fixes an older defect cubic pointed out in review. ppoll, pselect6 and epoll_pwait put the caller's mask back before the syscall epilogue delivers, and signal_deliver() tests that restored mask, so a signal that only the wait's sigmask unblocks ended the wait with EINTR and was never delivered: a guest that blocks the signal everywhere else and unblocks it only inside the wait got EINTR on every retry and no handler. Linux keeps the temporary mask until delivery and restores the original at sigreturn. These waits now do the same when they return EINTR for a signal they claimed, through the saved_blocked path rt_sigsuspend already uses, and the syscall epilogue puts back a saved mask that no handler frame took, as restore_saved_sigmask() does. epoll_pwait still claims only when kevent produced nothing, so ready events keep outranking the signal.

The third commit changes what a claim is, after review. It used to move the signal into the claimer's private set, which made it thread-directed for good: a mask restored over it, or the claimer exiting, stranded it where no other thread could take it. A claim now leaves the signal in the shared set and records the thread, and the other threads leave it alone only while that thread can still deliver it; any dequeue, thread_deactivate(), exec and a fork snapshot release it. With a wait's saved mask pending, delivery takes the claimed signal first, so the frame carrying that mask is its own. The fourth commit lets ready descriptors outrank a pending signal in ppoll and pselect6, as epoll_pwait already did, and has all three make one non-blocking pass when a signal is already pending, so a finite pselect6 or epoll_pwait no longer sits out its whole timeout for it.

FUSE stays on signal_pending_interruption(), which it needs for restart_out. The comment in io_retry_backoff() said that function filtered SA_RESTART; it never did, and the rewritten comment no longer says so.

Reproduction

tests/test-wait-process-signal.c parks four threads in each of read, ppoll, pselect6, epoll_pwait, flock and FUTEX_WAIT, blocks the signal in the main thread, and sends the group one kill(), up to four rounds per wait. It fails if more than one waiter returns EINTR, or if no round caught a waiter parked. The futex kind returned 3 EINTRs for one kill() on the second commit and passes with the third.

On 4872bb3 it failed 5 of 5 runs, with every wait failing at least once. With this change it passes 10 of 10, and 3 of 3 under the QEMU reference lane.

tests/test-wait-sigmask-signal.c waits in each of ppoll, pselect6 and epoll_pwait under a sigmask that is the only one unblocking SIGUSR1, and checks that the handler runs, that it runs under the wait's mask, and that the original mask is back after both an EINTR and a ready return. Without the second commit 8 of those 12 checks fail. Two more checks come with the fourth commit: a signal already pending at entry ends a 5 s wait at once, and a ready descriptor beats a pending signal, which stays pending. On the second commit they fail for pselect6 and epoll_pwait (5005 and 5001 ms spent) and for ppoll and pselect6 (EINTR instead of the ready count). With all four commits the 18 checks pass, as they do under QEMU.

Measurements

A separate program with the same shape, 30 rounds each, SA_RESTART off:

EINTR per kill() at 4 waiters	before	after	Linux 6.12
read	1 to 3 (42 over 30)	1 (30)	1 (30)
ppoll	1 to 4 (80 over 30)	1 (30)	1 (30)
epoll_pwait	1 to 3 (48 over 30)	1 (30)	1 (30)

The handler ran exactly once per round in every row.

The temporary-mask case: every thread blocks SIGUSR1, and the main thread loops for 1 s on each wait with an empty sigmask after one kill(). On 4872bb3 and on the first commit the handler never runs and the waits return EINTR 4 (pselect6), 7 (ppoll) and 4 (epoll_pwait) times. With the second commit, as under Linux 6.12, the handler runs once on the waiting thread and each wait returns EINTR once.

Not in this change

• SA_RESTART is still not honored for a signal-interrupted wait, so the thread that does run the handler still sees EINTR from read(). tests/test-poll.c already documents that.
• A signal that arrives during the wait still reaches an indefinite epoll_pwait only on its 200 ms kevent slice (about 96 ms on average here), and a finite pselect6 or epoll_pwait only at its deadline, since none of them joins the wakeup. That is A guest signal does not interrupt futex, epoll_wait or a finite select #378, which will add the wake on top of this claim.
• With two signals pending that only a wait's mask unblocks, elfuse builds one frame per epilogue, so unless the first handler makes a syscall the second stays pending under the restored mask instead of nesting under the first as on Linux. The reply on the review thread at signal.c describes the two ways to close it; it is left for a follow-up.

Commands

• make check: exit 0, all 99 unit tests pass. An earlier run failed only test-shim-futex-toctou with the -38 from rt_sigreturn overwrites the restored X8, so a signal taken on an svc re-enters it as syscall 2 #379; looped 300 times it fails 19 on 4872bb3 and 9 with this change, so it is not caused here
• make check-tsan: exit 0, all 72 tests pass, no ThreadSanitizer reports
• test-nanosleep-process-signal, test-sigsuspend, test-sigtimedwait, test-signalfd, test-poll, test-epoll, test-epoll-mt, test-epoll-edge, test-signal, test-signal-thread, test-tgkill-directed: pass with all four commits
• Each commit builds on its own
• make check-format, make lint: exit 0
• make check-ascii: clean across 438 source files

Rebased on 4872bb3.

[jserv]

The claim makes the converted waits agree with each other, but the waits that were not converted still test the shared set non-destructively, so one kill(2) can still hand out two EINTRs. futex_os_sync_wait returns -LINUX_EINTR on a bare signal_pending(), futex_poll_signal_relock wakes a bucket waiter the same way, and rt_sigtimedwait decides on signal_set_would_wake_locked(). A guest with one thread in futex_wait and one in ppoll still sees exactly the defect this fixes. Either move those onto the claim as a follow-up, or say in the commit that the invariant holds only among the waits listed here.

[xalestar]

Moved them onto the claim in 8dee3fb. futex_os_sync_wait, futex_poll_signal_relock and the rt_sigtimedwait interrupt test, which claims outside its own wait set, now take the signal the way the other waits do; each site tests teardown before it claims. tests/test-wait-process-signal.c gains a futex kind: four threads in FUTEX_WAIT returned 3 EINTRs for one kill() on 3c9c38c, and exactly one now, as under QEMU.

[jserv]

deliver_signal_locked hands saved to whichever signal it delivers first, not to the one the wait claimed, and a claimed process-directed signal is now indistinguishable from a thread-directed one in tpending. Thread A blocks SIGUSR2 outside the wait, unblocks it in the wait's mask, and claims a process-directed SIGUSR2; a SIGUSR1 aimed at A before the epilogue is lower-numbered, so signal_deliver_one picks it, stamps uc_sigmask with saved, and rt_sigreturn restores the mask that blocks SIGUSR2. That SIGUSR2 is left in A's private set where no other thread can take it, and it dies with A. A reservation on the shared set, released on mask restore or teardown, would keep process-directed signals out of tpending.

[xalestar]

Done as you suggest, in 8dee3fb. A claim no longer moves the signal into tpending: it stays in the shared set with the claiming thread recorded, and every other thread's view of the shared set (delivery, rt_sigtimedwait, signalfd, the claim itself) leaves it out for as long as the claimer does not block it. Once a mask is restored over it, or the claimer exits (thread_deactivate releases its claims and raises attention), it is an ordinary shared signal again. Any dequeue from the shared set releases the claim, and exec and a fork snapshot start with none. When a wait has left its saved mask, delivery takes the signal this thread claimed before a thread-directed one, so the frame that carries saved belongs to the claimed signal.

I could not make the interleaving you describe, a SIGUSR1 landing between the claim and the epilogue, deterministic from a guest, so no test pins that order; the wait, signal, sigtimedwait and signalfd tests pass with the change, and make check-tsan is clean.

One difference from Linux is left for a follow-up.

The case: two signals are pending that only the wait's mask unblocks.

• Linux delivers both before returning to user space, nesting a frame for each.
• elfuse builds one frame per return. The first handler's frame carries the saved mask, so its rt_sigreturn restores the original mask over the second signal. Unless that handler makes a syscall, the second stays pending: a process-directed one in the shared set, where another thread can still take it, a thread-directed one in tpending.

The one-frame limit is syscall_return_epilogue's. Its comment calls signal_deliver a once-per-epilogue call by contract, since a second call "would stack a frame on the handler the first one just installed". That rule came in with the ptrace-stop work in 348e0e5, and its message gives no correctness reason against nesting.

Two ways to close it:

1. Nest frames as Linux does. This changes the one-frame rule, and has to get the X8 frame-drop marker (rt_sigreturn overwrites the restored X8, so a signal taken on an svc re-enters it as syscall 2 #379), where a ptrace stop is taken, and the altstack check right.
2. Chain at rt_sigreturn: when the frame being restored carries a wait's saved mask, keep the handler's mask for one more delivery and pass the saved mask to that frame through the same saved_blocked path. Both handlers run, but in the opposite order from Linux.

I'd rather do 2 as a separate PR after this one than grow this one further.

[jserv]

Thank @xalestar for contributing!