From a83f704ab98960a5a519faf0b216141379bf2919 Mon Sep 17 00:00:00 2001 From: jotpalch Date: Sun, 13 Sep 2026 22:11:13 +0800 Subject: [PATCH] Restore the guest X8 when the shim drops its frame The EL1 shim is told to drop the exception frame it is holding by a marker the host writes into X8, and exec_drop_frame consumes that marker by popping the frame and ERETing without restoring a register. X8 is therefore the one register that cannot carry guest state across that return, and X8 is where an aarch64 SVC takes its syscall number. A signal delivered while the vCPU is at EL0 with PC on an SVC that has not executed yet is saved with that PC, correctly and without the marker. The handler returns, signal_rt_sigreturn restores all 31 GPRs from the frame and then writes the marker over X8, and the ERET resumes on the SVC, which runs as syscall 2. Dispatch finds nothing and answers ENOSYS to a call the guest made as something else. sysprog21/elfuse#379. No second register can carry the value instead: every GPR on that tail is meant to be final EL0 state, which is why X7's ptrace request is already left unwritten there. The value goes in the frame's own X8 slot, and exec_drop_frame reloads X8 from it before the pop. That slot is EL1-only memory on this vCPU's own exception stack, unreachable from EL0, unshared with any other vCPU, holding nothing live once the frame is being dropped, and retired by the pop, so it cannot collide with the marker the way a register does. rt_sigreturn is the only publisher: it is the one rebuilder whose X8 differs from the one the frame was entered with, and the only one that can promise the frame is live, since it always arrives through HVC #5. No other path needs a publisher: the slot already holds the X8 its exception was taken with, which is the guest's own. The write is bounded at the EL1 exception-stack region, not at the 2 MiB shim data block that contains it. Only the top MAX_THREADS slots of that block are stack; everything below is the shim-globals cache, the identity slots, the urandom ring and the attention bitmask. A bound at the block accepts an SP_EL1 that is wrong but still inside it and rewrites eight bytes of that cache instead of taking the log_error path, which is the reverse of what the check is for. thread_sp_el1_region derives the slot region from the same constants thread_alloc_sp_el1 carves it with, so the two cannot drift. Probed by forcing SP_EL1 to block+0x100, inside the urandom ring: the block bound writes the published X8 over a ring word, 0xbd0f39f71fe88c59 to 0xac, and logs nothing; the region bound logs and does not write, and the first address it accepts is block+0x1c0000 exactly. The window that closes is the low 1835008 bytes of the block, every byte of it cache. The frame slot alone does not close the window, which was measured rather than argued. A signal delivered after the rt_sigreturn but before the vCPU is resumed snapshots the live registers, and the live X8 there is the marker while ELR_EL1 is the instruction about to resume, which is where an unexecuted SVC sits; the frame that delivery builds records the marker as the guest's X8 and its own rt_sigreturn hands it back. With only the slot in place, the 32 nested rounds of the test below all still fail and the 32 plain ones all pass, so the second hand-off stays: rt_sigreturn parks the value it published and deliver_signal_locked takes it in place of the register. What makes that substitution safe is that the parked value cannot outlive the epilogue that parked it. signal_forget_sigreturn_x8 drops the record, and the run loop calls it before every hv_vcpu_run, so the only reader that can ever see one is a delivery between the rt_sigreturn that wrote it and the guest running again. Without that, a record sits until something consumes it and is then taken by any later delivery that merely lands on the same ELR: a SIGSEGV on an instruction the guest returned to has the faulting PC in ELR_EL1 and a live X8, and got the parked value instead, which is silent register corruption on exactly the fault-driven lazy-JIT shape this epilogue exists to keep honest. Pairing the record with its ELR is kept as an identity check, not as the bound, and decoding the parked ELR to require an SVC there was weighed and dropped: a fault can be taken on an SVC too, so it narrows the stale window without closing it, and it buys nothing once the record cannot survive a resume. The one path that can move the guest inside the epilogue is where that check needs help rather than widening, and it gets it at the end. tests/test-shim-sigreturn-x8.c drives both routes without a race, and then holds the record to its window. A SIGUSR1 handler rewrites its own ucontext so the return lands on a stub that reads X8 and then issues the SVC that X8 names, and every second round raises a SIGUSR2 that the handler's sa_mask holds pending until rt_sigreturn unblocks it. A third phase returns to a stub the same way, lets the guest run on, makes the page the stub loads from unreadable, and branches back with a different X8 live, so the fault that follows is an ordinary delivery landing on the PC the earlier return came back to. A fourth phase leaves the syscall route entirely: it branches to a page with no PROT_EXEC and stores to a page with no PROT_WRITE, both of which reach the same drop tail from HVC #9, and asserts the handler enters with the guest's X8. A fifth does the same for BRK, asserting both halves: the X8 the SIGTRAP handler enters with, and the X8 the guest resumes with once the handler has stepped the saved PC past the trap and returned. All 64 rounds fail on afdcfce, the third phase passes there, having nothing to go stale, and the two fourth-phase rounds fail, entering with 0x2 where the guest held 0xa5; with only the frame slot in place the 32 nested rounds fail and nothing else does; with the slot and the park but no forget, the rounds pass and the third phase fails, reporting 0x55 where the guest faulted with 0x99; the fifth phase fails with handle_brk in its old shape, reading 0x2 where the guest held 0xc3; all five phases pass here. Every assertion is plain Linux signal ABI, so the test is registered in tests/test-matrix.sh rather than exempted, and the comment above run_unit_tests now records it there beside test-shim-futex-fast, each with its reason. That comment read as though futex-fast were the only test-shim-* in the lane, which would have had the next reader hold this one out; two recorded exceptions read as a rule. It was run against the qemu reference lane by hand, on the fixture kernel of the day, Alpine 6.18.52-0-virt, and passes there. No EXPECTED_BASELINES floor moves: a full lane was not run, and an unobserved floor asserts a run that did not happen. The reproduction the issue carries, a raw FUTEX_WAIT loop with a signal landing on it, was run interleaved so both binaries met the same machine: 5 failures in 400 runs on afdcfce against 0 in 400 here, and 161 in 400 against 0 in 400 under eight concurrent compile jobs. This machine is shared, so the first column is a floor rather than a rate: the same pair read 53 in 400 when another build was running. The loaded column is the one to read. check-svc-tails.py gains the two rules that hold the hand-off in place. A frame-dropping tail must load X8 from the frame before it pops, at the offset the host publishes to, which the gate reads out of signal.c rather than trusting the two sides to match by eye, and it must still pop: dropping the load and dropping the pop both leave a tail that assembles and ERETs. Both exec_drop_frame and handle_brk are held to that rule, handle_brk because the same deletion there is the bug it arrived with. And every hv_vcpu_run call site must forget the parked X8 first, since deleting that call is as invisible as deleting the load and leaves a worse symptom behind it; a call commented out is a call deleted, so the search that looks for one skips comments the way the resume scan already does. Fifteen self-test cases cover the two, including deleting the load, moving it below the pop, pointing it at the wrong slot, deleting the pop, and the same four on the BRK tail plus restoring its frame before the HVC again, and deleting the forget, commenting it out, moving it below the resume, and adding a second resume without one. The substitution itself is held by the test and not by a gate. Deleting it, or reading the live register in its place, reads to a text gate like any other assignment, and the test settles it anyway: the 32 nested rounds fail on that change every round, with no race to lose, and the test runs in the matrix. The forget is listed in the gate for the one failure no test can reach, a resume added later somewhere else with nothing above it. What that listing establishes is textual and no more: a call is written above every resume, not that the call runs. Wrapping the existing call in a condition leaves the gate at rc=0 with its summary line unchanged, while the record outlives the resume; the test is what fails there, its third phase reporting the same 0x55. The gate's docstring now says that, rather than claiming the forget is on the path to the resume, which no text scan of C can show. The rule stays, because a call that was never written is how it actually breaks. Three shim tails branch to that drop tail, not one, so this repairs more than the syscall path and docs/internals.md now says which. Both HVC #9 W^X tails branch there, because the host answers a flip request with a SIGSEGV when the region never had the permission asked for, and a guest holding 0xa5 in X8 entered its handler with 0x2 on afdcfce and with 0xa5 here, on each of the two. A fourth tail needed the same repair and reaches EL0 without passing through that label: handle_brk. It popped its frame before HVC #10 and ERETed bare, so the marker a SIGTRAP delivery leaves in X8 reached the handler verbatim, making it the one marker consumer that never looked at the marker. JIT translators use BRK as a patching trampoline and read X8 in the handler, which is exactly the caller that notices. It now takes the handle_el0_fault shape: LOAD_GPRS in place of RESTORE_GPRS so the frame outlives the HVC, then the same ldr x8, [sp, #64] and add sp, sp, #256 after it. Measured on this head with a guest holding 0xc3 in X8 across a BRK #0 with a SIGTRAP handler installed: handler entry reads 0x2 with the old shape and 0xc3 with this one, while the X8 the guest resumes with once its rt_sigreturn has run is 0xc3 either way, that half having gone through HVC #5 like any other. What this tail still shadows on the ordinary syscall-return path is left for its own change, and docs/internals.md now measures it rather than excusing it. A delivery there snapshots the TLBI wire value as the guest's X8 and hands X9 through X11 out as TLBI operands, which syscall/proc.h already names for X8. The excuse this first wrote, that the saved PC is past the SVC, does not survive being run: the instruction after an SVC can be another SVC. Two adjacent SVCs with the first an mprotect, under a 200 us itimer over 60000 rounds, interleaved so both binaries met the same machine: all 707 deliveries here left the second SVC running as syscall 0 or 3 rather than the 226 the guest held, against 703 of 704 running as syscall 2 on afdcfce. The handler's ucontext reads X8 = 0x3 on both. So it is pre-existing, it is not the register this commit moves, and afdcfce is not better; the repair is to substitute the saved frame's X8 on that path too, which is a second publisher and the X9 through X11 shadowing beside it, and belongs in its own commit rather than under this one's claim. Since #379 closes here and that does not, it is tracked as its own issue rather than living only in a source comment; the number goes into the comment once it is filed. Two neighbors turned up while measuring the above. The first is recorded rather than fixed: it is pre-existing, it measures identically on afdcfce, and it is not a register this moves. The frame an HVC #9 SEGV_ACCERR delivery builds records X0 and X1 as the shim left them for the HVC, so a guest faulting with x0=0xc7 and x1=0xb6 reads FAR_EL1 and the W^X type out of its ucontext instead. And PTRACE_INTERRUPT is taken inline on this same tail, where the live registers are final except for X8, so a self-signaling tracee kicked 4000 times reports the marker through PTRACE_GETREGSET at 3015 to 3458 of its stops on afdcfce and 2677 to 3124 here, over three interleaved runs on each. The second is the ptrace stop, and it is where the parked record needs help rather than only a note, so the record now follows the guest through it. A tracer that writes a new PC at such a stop and resumes with an injected signal makes the delivery's ELR differ from the one the record was parked for, so the identity check misses the one delivery the record is genuinely for and the frame takes the marker. signal_repark_sigreturn_x8 re-keys the record on the PC the stop left behind, which is all that moved: same guest, same epilogue, same X8 owed. Deleting the comparison instead would have closed this half too, at 0 of 122 and 0 of 215 landings, and it stays, because it is what keeps a record off every delivery that did not come through that stop. Interleaved on the same machine, the same tracee: 312 of 542, 494 of 986 and 431 of 776 stub landings read the marker on afdcfce, 1 of 251, 1 of 612 and 1 of 378 with the frame slot and the park but no re-key, and 0 of 672, 0 of 134 and 0 of 291 with it. No test drives it, because the landing that matters is one stop in a few hundred and the shape is a race; the probe is what measures it. Two halves of that stop stay. PTRACE_GETREGSET still reports the marker where the guest's X8 belongs, measured above. And an X8 the tracer writes back is not merely dropped: the shim dispatches on that register after HVC #5, so a value that is neither the marker nor a TLBI kind takes the conservative tail, which restores the stale frame and then tests the X7 this tail leaves holding guest state. The same tracee editing X8 to 0x99 at every stop kills the vCPU with an HVC #13 and no stop armed, on afdcfce and here alike. Honoring such an edit means taking it out of the live register as well as into the frame slot, which is a change to the dispatch and not to this record. --- .claude/skills/elfuse-guest-abi/SKILL.md | 22 +- docs/internals.md | 163 +++++-- scripts/check-svc-tails.py | 412 +++++++++++++++- src/core/guest.h | 4 +- src/core/shim.S | 89 +++- src/runtime/thread.c | 17 +- src/runtime/thread.h | 30 +- src/syscall/proc.c | 33 +- src/syscall/proc.h | 9 +- src/syscall/signal.c | 155 +++++- src/syscall/signal.h | 35 ++ tests/manifest.txt | 1 + tests/test-matrix.sh | 17 +- tests/test-shim-futex-toctou.c | 4 +- tests/test-shim-sigreturn-x8.c | 576 +++++++++++++++++++++++ 15 files changed, 1475 insertions(+), 92 deletions(-) create mode 100644 tests/test-shim-sigreturn-x8.c diff --git a/.claude/skills/elfuse-guest-abi/SKILL.md b/.claude/skills/elfuse-guest-abi/SKILL.md index b347a33f..94d743ef 100644 --- a/.claude/skills/elfuse-guest-abi/SKILL.md +++ b/.claude/skills/elfuse-guest-abi/SKILL.md @@ -63,21 +63,25 @@ the block size the shim assumes. |----|------|-------------| | 0 | TLBI_NONE | skip flush | | 1 | TLBI_BROADCAST | TLBI VMALLE1IS + DSB ISH + ISB | -| 2 | drop-frame | host rebuilt EL0 state; discard saved frame on ERET | +| 2 | drop-frame | host rebuilt EL0 state; discard saved frame on ERET, restoring no register except X8, reloaded from the frame's own X8 slot (`[sp, #64]`) | | 3 | TLBI_RANGE | loop TLBI VAE1IS, X9=VA, X10=page count | | 4 | TLBI_RANGE_LARGE | single RVAE1IS, encoded operand in X9 | X11=1 is the icache-flush hint: set when a page transitions to executable, and the shim then issues an IC invalidate alongside whichever TLBI it picked. The -shim restores X11 from the saved frame before ERET, so EL0 never observes it. +restoring tails reload X11 from the saved frame before ERET, so EL0 never +observes it. The X8 = 2 tail restores nothing but X8, so a delivery that follows +a page-table-modifying syscall in the same epilogue hands X9 through X11 out to +EL0 in place of the guest's. X7 is the ptrace-stop request on the same return, and it obeys a rule the TLBI codes do not. The shim reads it only after restoring the saved frame, so the tracer snapshots the guest's architectural registers rather than shim scratch; non-zero means take HVC #13 before the ERET. That makes X7 unusable on the one -tail that never restores the frame, X8 = 2, where the live registers already -are the final EL0 state and a host write to X7 would land in EL0 as guest -state. The host takes that stop inline in the epilogue instead and leaves X7 +tail that never restores the frame, X8 = 2, where the live registers are the +final EL0 state with one exception -- X8 holds the marker, which is why that +tail reloads it from the frame -- and a host write to X7 would land in EL0 as +guest state. The host takes that stop inline in the epilogue instead and leaves X7 alone, and an `execve` re-entry, which has no tail at all, leaves the stop owed for the new image. @@ -123,7 +127,13 @@ your path goes through the dispatch epilogue at all: - Inside the epilogue, set X8=2. The shim reads it as the drop-frame marker and discards the saved GPR frame. Signal delivery on the syscall-return path works this way. It does not need the marker when EL0 was preempted rather - than returning from a syscall, because there is no shim frame to drop. + than returning from a syscall, because there is no shim frame to drop. If + the X8 you want EL0 to see differs from the one the frame was entered with, + publish it into the frame's X8 slot with the marker: the marker occupies the + register, and the tail reloads X8 from that slot alone. Bound that write at + the EL1 stack region (`thread_sp_el1_region`), not at the shim data block: + the block's low end is the shim-globals cache, and only its top + `MAX_THREADS` slots are stack. - Bypass the epilogue by returning `SYSCALL_EXEC_HAPPENED`. The epilogue returns early, before it writes X0 or X8. `sys_execve` works this way, and the normal X0 writeback is exactly what it needs to avoid. It must also skip diff --git a/docs/internals.md b/docs/internals.md index 2ada3b61..204af26f 100644 --- a/docs/internals.md +++ b/docs/internals.md @@ -285,16 +285,18 @@ X8 == 1 TLBI_BROADCAST TLBI VMALLE1IS + DSB ISH + ISB -> restore GPRs (keep X0); ERET X8 == 2 drop-frame discard the saved GPR frame (`add sp, sp, #256`) and ERET on the rebuilt - EL0 register state. Set by `execve` and - `rt_sigreturn` (which write the whole frame - directly into the vCPU) and by - `signal_deliver()` on the syscall-return + EL0 register state, except for `X8` itself, + which the shim reloads from the frame's own + `X8` slot (`[sp, #64]`) because the marker + arrived in that register. Set by + `rt_sigreturn` (which writes the whole + register set directly into the vCPU, and + publishes its `X8` into that slot) and by + `deliver_signal_locked` on the syscall-return path (so handler PC/SP/LR/args installed by the host are not overwritten by the stale - shim frame on ERET). `execve` additionally - issues `IC IALLU` because the new program - text may live in pages that previously held - the old text. + shim frame on ERET). The flush is + unconditional, `IC IALLU` included. X8 == 3 TLBI_RANGE loop TLBI VAE1IS over `X9` (start VA), `X10` (page count); 4 KiB granule. Used for up to `TLBI_SELECTIVE_MAX_PAGES = 16` pages. @@ -317,25 +319,125 @@ separate broadcast after the split lands. `X8 == 2` is the generic drop-saved-frame marker: the host has rebuilt EL0 register state directly into the vCPU and the saved syscall frame on the EL1 stack is stale, so the shim drops the frame -and `ERET`s without restoring GPRs. Three call sites use it: - -- `sys_execve` (`src/syscall/exec.c:785, 1093`) after the ELF reload. -- `signal_rt_sigreturn` (`src/syscall/signal.c:1710`) after restoring - the saved sigframe. -- `signal_deliver` (`src/syscall/signal.c:1594`) when a signal is - delivered on the syscall-return path; without the marker the shim - would overwrite the handler PC, SP, LR, and arg-register state with - the stale syscall frame on `ERET`. - -`X8` (the syscall-number register) and `X9`/`X10` are already considered -clobbered by the Linux syscall ABI, so callers never expect them to be -preserved across SVC. - -Important: the first two paths (`sys_execve` and -`signal_rt_sigreturn`) return `SYSCALL_EXEC_HAPPENED` to bypass the -normal syscall dispatch epilogue. `signal_deliver` runs from inside -the epilogue. Any future code path that rebuilds EL0 register state -on the syscall-return path must write `X8 = 2` the same way. +and `ERET`s without restoring GPRs, except `X8`, which it reloads from +the frame's own `X8` slot because the marker arrived in that register. +Two call sites write it, both in `src/syscall/signal.c`: + +- `signal_rt_sigreturn`, after restoring the saved sigframe. +- `deliver_signal_locked`, when a signal is delivered on the + syscall-return path; without the marker the shim would overwrite the + handler PC, SP, LR, and arg-register state with the stale syscall + frame on `ERET`. + +`sys_execve` rebuilds EL0 state too and writes no marker: it re-enters +through the shim's MMU-off `_start`, which never pops a frame. + +Three shim tails branch to `exec_drop_frame`, not one. Besides the +`HVC #5` epilogue, both `HVC #9` W^X tails test `X8` against 2 and +branch there (`handle_inst_abort` and `handle_data_abort` in +`src/core/shim.S`), because the host answers a flip request with a +`SIGSEGV` delivery when the region never had the permission asked for. +Nothing publishes an `X8` on that route, so the reload hands EL0 the +value the exception was taken with, which is the guest's own. Measured +with a guest holding `0xa5` in `X8` across a branch to a page with no +`PROT_EXEC` and across a store to a page with no `PROT_WRITE`: the +handler enters with `X8 = 0x2` on the tree before the reload and with +`0xa5` after it, on both tails. `tests/test-shim-sigreturn-x8` covers +the pair. + +A fourth tail carries the same reload in its own body. `handle_brk` +forwards a `BRK` from EL0 to the host through `HVC #10`, and the host +delivers `SIGTRAP` there the way it delivers on any other path, +marker included. The tail used to pop its frame before the `HVC` and +`ERET` bare, so the marker was the `X8` the handler entered with: the +one marker consumer that never looked at it. It now takes the +`handle_el0_fault` shape, loading the GPRs without popping so the +frame outlives the `HVC`, then reloading `X8` from `[sp, #64]` and +popping after it. Nothing publishes on this route either, so what the +handler gets is the `X8` the `BRK` was taken with. JIT translators use +`BRK` as a patching trampoline and read that register, which is what +makes it worth the two instructions. Measured with a guest holding +`0xc3` in `X8` across a `BRK #0` with a `SIGTRAP` handler installed: +handler entry reads `X8 = 0x2` before the change and `0xc3` after, +while the value the guest resumes with once its `rt_sigreturn` has run +is `0xc3` on both, since that half goes through `HVC #5` like any +other. `tests/test-shim-sigreturn-x8` asserts both halves and +`scripts/check-svc-tails.py` holds both tails to the reload. + +Linux preserves `X1`-`X30` across `SVC #0` and modifies only `X0`, so +none of these registers may carry a host-to-shim value out to EL0. `X8` +is the one the marker occupies, which is why the drop tail reloads it +from the frame slot the host publishes into; a resumed `SVC` that has +not executed yet takes its syscall number from that register, so a +marker left there runs the call as syscall 2 +(sysprog21/elfuse#379). The ordinary syscall-return tail is a different +matter and this reload does not reach it. `X8` carries the TLBI kind +there and `X9`-`X11` its operands and the icache hint, so a signal +delivered in the epilogue of a page-table syscall records those wire +values as the guest's registers and its own `rt_sigreturn` hands them +back. The saved PC being past the `SVC` does not make that free: the +instruction after an `SVC` can be another `SVC`. Measured with two +adjacent `SVC`s under a 200 us itimer over 60000 rounds, the first of +them `mprotect`: all 707 deliveries left the second running as syscall +0 or 3 rather than the 226 the guest held, and the handler's +`ucontext` read `X8 = 0x3` at the same time. Pre-existing, and not the +register this reload moves: interleaved on the same machine, afdcfce +read 703 of its 704 as syscall 2, the marker rather than the wire +value. It is left for its own change, which has to substitute the +saved frame's `X8` on that path the way `deliver_signal_locked` takes +the parked value on this one. Since #379 closes here and this does +not, it is tracked as its own issue, #384. + +The frame slot covers the return itself. It does not cover a signal +delivered after the `rt_sigreturn` but before the vCPU is resumed: +that delivery snapshots the live registers, where `X8` is still the +marker, and the frame it builds would hand the marker back on its own +return. `signal_rt_sigreturn` therefore also parks the value in a +per-vCPU record that `deliver_signal_locked` reads, and the run loop +calls `signal_forget_sigreturn_x8()` before every `hv_vcpu_run()`, so +the record cannot be read by anything but a delivery in the epilogue +that wrote it. A record that outlived the resume would be handed to a +later delivery that merely lands on the same PC, a fault on an +instruction the guest returned to among them. + +One path can move the guest inside the epilogue, and the record +follows it there. `PTRACE_INTERRUPT` is consumed inline on this tail +rather than deferred to `HVC #13`, because the live registers there +are already the architectural EL0 set. A tracer that writes a new PC +and resumes with an injected signal makes the delivery's `ELR_EL1` +differ from the ELR the record was parked for, so the identity check +would miss the one delivery the record is genuinely for and the frame +would record the marker as the guest's `X8`. +`signal_repark_sigreturn_x8()` re-keys the record on the PC the stop +left behind, which is all that changed: the guest, its epilogue and +the `X8` owed to it are the same. Measured on a self-signaling tracee +kicked 4000 times, interleaved so the binaries met the same machine: +312 of 542, 494 of 986 and 431 of 776 stub landings came back with +`X8 = 2` on afdcfce, 1 of 251, 1 of 612 and 1 of 378 with the frame +slot and the park but no re-key, and 0 of 672, 0 of 134 and 0 of 291 +with it. + +Two halves of that stop are pre-existing and stay. `X8` holds the +marker while the tracer reads it, so `PTRACE_GETREGSET` reports 2 +where the guest's `X8` belongs, on both trees. And a tracer that +writes some other `X8` back does not get it honored: the shim +dispatches on that register after `HVC #5`, so a value that is neither +the marker nor a TLBI kind takes the conservative tail, which restores +the stale frame and then tests the `X7` this tail leaves holding guest +state, and the vCPU dies on the `HVC #13` that follows with no stop +armed. +Measured with the same tracee editing `X8` to `0x99` at every stop: +`FATAL ... HVC #13 with no ptrace stop armed` on this tree and on the +tree before the reload alike. Honoring such an edit means taking it +out of the live register as well as into the frame slot, which is a +change to the dispatch and not to this record. + +Important: `signal_rt_sigreturn` returns `SYSCALL_EXEC_HAPPENED` to +bypass the normal syscall dispatch epilogue, as `sys_execve` does. +`deliver_signal_locked` runs from inside the epilogue. Any future code +path that rebuilds EL0 register state on the syscall-return path must +write `X8 = 2` the same way, and publish the guest's `X8` with it if +that value differs from the one the frame was entered with. ## EL1 Shim And HVC Protocol @@ -348,10 +450,10 @@ aligned address from the `Rt` register); HVF traps DC ZVA via `HCR_EL2.TDZ=1`. | #0 | Normal exit | `X0` = exit code | | #2 | Bad exception | `X0`=ESR, `X1`=FAR, `X2`=ELR, `X3`=SPSR, `X5`=vector | | #4 | Set boot system register | `X0` = reg ID (0–8), `X1` = value (used by the shim during boot to install RES1 bits and enable the MMU) | -| #5 | Syscall forward | `X0`–`X5` = args, `X8` = syscall number on entry; on return `X8` carries the TLBI kind (`0` = none, `1` = broadcast, `3` = selective range with `X9` = VA + `X10` = page count, `4` = single-shot `TLBI RVAE1IS` with encoded operand in `X9`). `X8 = 2` is the generic drop-saved-frame marker -- set when the host has rebuilt EL0 state directly (by `execve`, `rt_sigreturn`, and `signal_deliver()` on the syscall-return path) so the shim discards the saved syscall frame on ERET. `X11` is the icache-flush hint (set to `1` when the request transitions a page to executable, so the shim issues `IC` alongside the chosen TLBI) | +| #5 | Syscall forward | `X0`–`X5` = args, `X8` = syscall number on entry; on return `X8` carries the TLBI kind (`0` = none, `1` = broadcast, `3` = selective range with `X9` = VA + `X10` = page count, `4` = single-shot `TLBI RVAE1IS` with encoded operand in `X9`). `X8 = 2` is the generic drop-saved-frame marker -- set when the host has rebuilt EL0 state directly, by two writers both in `src/syscall/signal.c` (`signal_rt_sigreturn`, and `deliver_signal_locked` on the syscall-return path; `sys_execve` rebuilds EL0 state too and writes no marker), so the shim discards the saved syscall frame on ERET and restores no register except `X8`, which it reloads from the frame's own `X8` slot (`[sp, #64]`) because the marker arrived in that register. `X11` is the icache-flush hint (set to `1` when the request transitions a page to executable, so the shim issues `IC` alongside the chosen TLBI) | | #6 | Embedder extension | `X8` = call number, `X0`–`X7` = args; routed to `g->hvc6_handler` if set, no-op otherwise. Handler may request a vCPU yield via `proc_request_hvc6_yield()` | | #7 | MRS trap (read sysreg) | host reads register from ESR ISS; returns value in `X0` | -| #9 | W^X toggle | `X0` = FAR, `X1` = type (0 = exec→RX, 1 = write→RW) | +| #9 | W^X toggle | in: `X0` = FAR, `X1` = type (0 = exec→RX, 1 = write→RW); out: `X8` = 2 when the host answered with a `SIGSEGV` delivery instead of a flip, which sends the shim to `exec_drop_frame`, and 0 on a completed flip. No kind is dispatched on this route: both tails test `cmp x8, #2` and otherwise branch to `tlbi_restore_eret`, which issues a single-page `TLBI VAE1IS` on `FAR_EL1` plus `IC IALLU` whatever `X8` held. The frame that delivery builds records `X0` and `X1` as the shim left them, so a handler reading `regs[0]`/`regs[1]` out of its `ucontext` sees `FAR_EL1` and the W^X type rather than the guest's values; pre-existing, and the same family as the `X8` marker above | | #10 | BRK from EL0 | SIGTRAP delivery / ptrace-stop; GPRs in frame | | #11 | EL0 fault | SIGSEGV/SIGILL delivery; GPRs in frame | | #12 | EL0 system-instruction trap | cache maintenance logging (DC CVAU, IC IVAU, …) and `MSR TPIDR_EL0` emulation | @@ -796,7 +898,8 @@ In `src/syscall/proc.c`: `hv_vcpus_exit()`. A stop taken on a syscall return whose tail restores the saved SVC frame goes through HVC #13, so ptrace snapshots the architectural GPR set rather than shim scratch. The tails that rebuild EL0 state instead - (`X8 = 2`) already hold that set live, so the host stops on them directly; + (`X8 = 2`) already hold that set live, bar the `X8` the shim reloads from + the saved frame, so the host stops on them directly; an `execve` re-entry leaves the stop owed for the new image's first syscall. - `PTRACE_GETREGSET` / `PTRACE_SETREGSET` (`NT_PRSTATUS`) -- read or write diff --git a/scripts/check-svc-tails.py b/scripts/check-svc-tails.py index ee4608fd..0618cf73 100644 --- a/scripts/check-svc-tails.py +++ b/scripts/check-svc-tails.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Hold every HVC #5 return tail to the X7 ptrace test. +"""Hold every HVC #5 return tail to the X7 ptrace test, and the X8 tails to X8. The host encodes a ptrace-stop request in X7 on the HVC #5 return and the shim reads it in svc_hvc_restore_eret. A tail that reaches EL0 without passing @@ -11,8 +11,9 @@ comment: exec_drop_frame never tested X7. That one is now deliberate (the host leaves - X7 alone on the X8 == 2 tail, whose live registers are already the final EL0 - state) and is the single allowed exception below. + X7 alone on the X8 == 2 tail, whose live registers are the final EL0 state + except for X8, which holds the marker and is reloaded from the frame by the + second rule below) and is the single allowed exception below. tlbi_selective's defensive zero-count exit was "cbz x10, 1f", and 1f resolved to a numeric label sitting inside svc_restore_eret, past the test. A numeric @@ -23,6 +24,40 @@ svc_restore_eret. Every tail the host can select lives there. Numeric labels inside that region are fine, and the selective-TLBI loop uses one; what is not fine is a reference that resolves past the end of it. + +The same tail carries a second rule, for the same reason: it restores nothing, +so whatever is in a register when it runs reaches EL0. X8 is where the marker +that selected the tail arrived, so the host publishes the guest's X8 in the +frame's own X8 slot and exec_drop_frame reloads it from there before the pop. +Deleting that one load is invisible in review and turns an SVC the guest has +not executed yet into syscall 2 (sysprog21/elfuse#379), so the offset is +checked here against the C side that writes it rather than left to match by +eye. + +handle_brk is held to the same rule and checked the same way. It is not an +HVC #5 tail, but it ends the same: the host delivers a signal out of HVC #10 +and leaves the marker in X8, and the tail reloads the guest's value from the +frame slot before dropping the frame. A SIGTRAP handler reading X8 is the +ordinary case there, since JIT translators use BRK as a trampoline. + +The host half of the same hand-off is checked too. A signal delivered after an +rt_sigreturn but before the vCPU is resumed cannot read the guest's X8 out of +the register either, so rt_sigreturn parks it and that delivery takes the parked +value. The park is correct only for that one epilogue, and what keeps it from +reaching an unrelated later delivery is that the run loop forgets it before +every resume. A resume added later somewhere else would not forget, and no test +driving the resume that exists today can see that, which is why the call sites +are enumerated here. + +What that check establishes is textual and narrower than the rule it stands for: +a call to signal_forget_sigreturn_x8() is written above every hv_vcpu_run(). +It does not establish that the call runs. Narrowing it in place, by wrapping the +existing call in a condition, leaves this gate at rc=0 with its summary line +unchanged while the record outlives the resume. Only +tests/test-shim-sigreturn-x8 answers that, and it does: its stale phase fails on +a narrowed forget, reporting the X8 the earlier rt_sigreturn returned with. +Read this gate as the answer to a resume added without a forget, and the test as +the answer to a forget that does not run. """ import argparse @@ -60,6 +95,28 @@ GATE = "svc_hvc_restore_eret" FORBIDDEN = "svc_restore_eret" +DROP_TAIL = "exec_drop_frame" +BRK_TAIL = "handle_brk" + +# Each X8-reloading tail's one restore, and the pop it has to precede. +DROP_RELOAD = re.compile(r"^\s*ldr\s+x8,\s*\[sp,\s*#(\d+)\]") +DROP_POP = re.compile(r"^\s*add\s+sp,\s*sp,\s*#(\d+)\b") +ERET = re.compile(r"^\s*eret\b") + +# The C side of the same frame slot. +SIGNAL_C = ROOT / "src" / "syscall" / "signal.c" +C_DEFINE = re.compile(r"^#define\s+(SHIM_FRAME_OFF_X8|SHIM_FRAME_BYTES)\s+(\d+)\s*$") + +# The host side of the hand-off: the resume, and the call that has to precede +# it. RESUME_LOOKBACK is generous on purpose -- a forget separated from its +# resume by a few lines of bookkeeping is the shape the run loop already has, +# and demanding the line above would reject it. What the window buys is +# tolerance, not proof: see the module docstring for what this does not answer. +SRC = ROOT / "src" +RESUME = re.compile(r"\bhv_vcpu_run\s*\(") +FORGET = re.compile(r"\bsignal_forget_sigreturn_x8\s*\(\s*\)") +COMMENT = re.compile(r"^\s*(?:/\*|\*|//)") +RESUME_LOOKBACK = 12 # The one tail allowed to skip the test, and why. Keep the reason with the name: # an exception added later without one is the bug this gate exists to stop. @@ -74,7 +131,121 @@ def branch_target(operands): return tail.split()[0] if tail else "" -def check(lines, path, counted=None): +def frame_constants(lines, path): + """The C-side SHIM_FRAME_* values, or a problem list.""" + found = {} + for line in lines: + if m := C_DEFINE.match(line): + found[m.group(1)] = int(m.group(2)) + missing = {"SHIM_FRAME_OFF_X8", "SHIM_FRAME_BYTES"} - set(found) + if missing: + return None, [ + f"{path}: no #define for {', '.join(sorted(missing))}; the host no " + f"longer names the frame slot the drop tail reloads X8 from" + ] + return found, [] + + +def check_reload_tail(lines, path, tail, gone, leaks, frame=None): + """A frame-dropping tail must put the frame's X8 back before it pops it. + + @tail is the label, @gone what a missing label means, and @leaks what EL0 + is handed when the load is not there. Both callers below restore no other + register, so the load is the whole of what stands between the host's X8 and + the guest's. + """ + at = next( + (i for i, l in enumerate(lines) if LABEL.match(l) and l.startswith(tail + ":")), + None, + ) + if at is None: + return [f"{path}: no '{tail}:' label; {gone}"] + + reload_at = reload_off = pop_at = pop_size = None + for i in range(at + 1, len(lines)): + if reload_at is None and (m := DROP_RELOAD.match(lines[i])): + reload_at, reload_off = i, int(m.group(1)) + if pop_at is None and (m := DROP_POP.match(lines[i])): + pop_at, pop_size = i, int(m.group(1)) + if ERET.match(lines[i]): + break + + problems = [] + if reload_at is None: + problems.append( + f"{path}:{at + 1}: '{tail}' does not reload X8 from the frame. " + f"It restores no register, so {leaks}" + ) + elif pop_at is not None and reload_at > pop_at: + problems.append( + f"{path}:{reload_at + 1}: the X8 reload is below the pop, so it " + f"reads past the frame it was meant to read." + ) + if pop_at is None: + problems.append( + f"{path}:{at + 1}: '{tail}' ERETs without popping the frame " + f"it was told to drop. SAVE_GPRS subtracted those bytes from " + f"SP_EL1 and nothing else gives them back, so the exception stack " + f"walks down one frame per drop until it leaves the shim block." + ) + if frame: + if reload_off is not None and reload_off != frame["SHIM_FRAME_OFF_X8"]: + problems.append( + f"{path}:{reload_at + 1}: reloads X8 from [sp, #{reload_off}], " + f"but the host publishes it at " + f"{frame['SHIM_FRAME_OFF_X8']} (SHIM_FRAME_OFF_X8)." + ) + if pop_size is not None and pop_size != frame["SHIM_FRAME_BYTES"]: + problems.append( + f"{path}:{pop_at + 1}: pops {pop_size} bytes, but the host " + f"bounds-checks a {frame['SHIM_FRAME_BYTES']}-byte frame " + f"(SHIM_FRAME_BYTES)." + ) + return problems + + +def check_resume_barrier(lines, path): + """Every vCPU resume must have a forget written above it. + + Textual, and deliberately reported as such: this finds a resume that no + forget precedes, which is the way the call goes missing when a resume is + added elsewhere. A forget that is present but does not run reads the same + here; tests/test-shim-sigreturn-x8 is what catches that. + """ + problems = [] + sites = 0 + for i, line in enumerate(lines): + if COMMENT.match(line) or not RESUME.search(line): + continue + sites += 1 + lo = max(0, i - RESUME_LOOKBACK) + covered = False + for j in range(i - 1, lo - 1, -1): + # Prose is not code on either side of this walk. A commented-out + # forget is exactly the deletion this rule exists to catch, and + # counting it as coverage would hide it. + if COMMENT.match(lines[j]): + continue + # A second resume between the two is the one that is uncovered: + # the forget above it belongs to the first. + if RESUME.search(lines[j]): + break + if FORGET.search(lines[j]): + covered = True + break + if covered: + continue + problems.append( + f"{path}:{i + 1}: resumes the vCPU without calling " + f"signal_forget_sigreturn_x8() first. The X8 an rt_sigreturn " + f"parked for a delivery in its own epilogue then outlives the " + f"guest running, and a later delivery that lands on the same PC " + f"is handed it in place of the live register." + ) + return problems, sites + + +def check(lines, path, counted=None, frame=None): """Return a list of problem strings. Empty means the tails are sound. @counted, when a list, receives the number of tails reaching the test, so @@ -92,12 +263,31 @@ def check(lines, path, counted=None): if end is None: return [f"{path}: no '{FORBIDDEN}:' after the HVC #5; the tail moved"] + problems_drop = check_reload_tail( + lines, + path, + DROP_TAIL, + "the X8 == 2 tail is gone", + "the drop-frame marker itself reaches EL0 in X8, and a frame whose " + "saved PC is on an SVC issues that SVC as syscall 2.", + frame, + ) + problems_drop += check_reload_tail( + lines, + path, + BRK_TAIL, + "the BRK tail is gone", + "the marker a SIGTRAP delivery leaves behind reaches the handler in " + "X8, in place of the X8 the guest hit the BRK with.", + frame, + ) + named = {m.group(1): i for i, l in enumerate(lines) if (m := LABEL.match(l))} numeric = [ (int(m.group(1)), i) for i, l in enumerate(lines) if (m := NUM_LABEL.match(l)) ] - problems = [] + problems = list(problems_drop) # The whole gate rests on this label testing X7. Renaming or emptying it # would leave every tail branching somewhere that no longer checks. @@ -234,8 +424,31 @@ def check(lines, path, counted=None): return problems -def _shim(tail): - """Wrap a dispatch tail in the minimum surrounding shape.""" +# A sound BRK tail, and a sound drop tail. Both are appended to every fixture +# below: each rule runs against the whole file, so a fixture aimed at one of +# them has to keep the other intact or every case inherits its failure. +GOOD_BRK = [ + "handle_brk:", + " LOAD_GPRS", + " hvc #10", + " ldr x8, [sp, #64]", + " add sp, sp, #256", + " eret", +] +GOOD_DROP = [ + "exec_drop_frame:", + " ldr x8, [sp, #64]", + " add sp, sp, #256", + " eret", +] + + +def _shim(tail, brk=None): + """Wrap a dispatch tail in the minimum surrounding shape. + + The drop tail goes last so a case aiming at it can slice its body off with + [:-3] and write its own. + """ return ( ["handle_svc_0:", " hvc #5", " cbz x8, svc_hvc_restore_eret"] + tail @@ -246,9 +459,9 @@ def _shim(tail): "svc_hvc_restore_eret:", " cbz x7, svc_restore_eret", " b svc_restore_eret", - "exec_drop_frame:", - " eret", ] + + (GOOD_BRK if brk is None else brk) + + GOOD_DROP ) @@ -272,7 +485,9 @@ def _shim(tail): "svc_hvc_restore_eret:", " cbz x7, svc_restore_eret", " eret", - ], + ] + + GOOD_BRK + + GOOD_DROP, 2, ), ( @@ -313,7 +528,9 @@ def _shim(tail): "svc_hvc_restore_eret:", " cbz x7, svc_restore_eret", " eret", - ], + ] + + GOOD_BRK + + GOOD_DROP, 1, ), ( @@ -323,7 +540,8 @@ def _shim(tail): ), ( "branch out to an unrelated handler", - _shim(["tlbi_full:", " b handle_brk"]) + ["handle_brk:", " eret"], + _shim(["tlbi_full:", " b handle_unrelated"]) + + ["handle_unrelated:", " eret"], 1, ), ("no dispatch at all", ["_start:", " ret"], 1), @@ -360,8 +578,134 @@ def _shim(tail): " eret", "svc_hvc_restore_eret:", " b svc_restore_eret", + ] + + GOOD_BRK + + GOOD_DROP, + 1, + ), + ( + "drop tail no longer reloads X8", + _shim(["tlbi_full:", " b svc_hvc_restore_eret"])[:-3] + + [" add sp, sp, #256", " eret"], + 1, + ), + ( + "drop tail reloads X8 after the pop", + _shim(["tlbi_full:", " b svc_hvc_restore_eret"])[:-3] + + [" add sp, sp, #256", " ldr x8, [sp, #64]", " eret"], + 1, + ), + ( + "drop tail reloads X8 from the wrong slot", + _shim(["tlbi_full:", " b svc_hvc_restore_eret"])[:-3] + + [" ldr x8, [sp, #72]", " add sp, sp, #256", " eret"], + 1, + ), + ( + "drop tail no longer pops the frame", + _shim(["tlbi_full:", " b svc_hvc_restore_eret"])[:-3] + + [" ldr x8, [sp, #64]", " eret"], + 1, + ), + ( + "BRK tail no longer reloads X8", + _shim( + ["tlbi_full:", " b svc_hvc_restore_eret"], + brk=["handle_brk:", " LOAD_GPRS", " hvc #10", + " add sp, sp, #256", " eret"], + ), + 1, + ), + ( + "BRK tail reloads X8 after the pop", + _shim( + ["tlbi_full:", " b svc_hvc_restore_eret"], + brk=["handle_brk:", " LOAD_GPRS", " hvc #10", + " add sp, sp, #256", " ldr x8, [sp, #64]", " eret"], + ), + 1, + ), + ( + "BRK tail reloads X8 from the wrong slot", + _shim( + ["tlbi_full:", " b svc_hvc_restore_eret"], + brk=["handle_brk:", " LOAD_GPRS", " hvc #10", + " ldr x8, [sp, #72]", " add sp, sp, #256", " eret"], + ), + 1, + ), + ( + "BRK tail popped its frame before the HVC again", + _shim( + ["tlbi_full:", " b svc_hvc_restore_eret"], + brk=["handle_brk:", " RESTORE_GPRS", " hvc #10", " eret"], + ), + 2, + ), + ( + "BRK tail gone", + _shim(["tlbi_full:", " b svc_hvc_restore_eret"], brk=[]), + 1, + ), +] + + +SELF_TEST_FRAME = {"SHIM_FRAME_OFF_X8": 64, "SHIM_FRAME_BYTES": 256} + +RESUME_CASES = [ + ( + "resume behind the forget", + [ + " signal_forget_sigreturn_x8();", + "", + " HV_CHECK_CTX(hv_vcpu_run(vcpu), vcpu, g);", + ], + 1, + 0, + ), + ( + "forget deleted", + [" HV_CHECK_CTX(hv_vcpu_run(vcpu), vcpu, g);"], + 1, + 1, + ), + ( + "forget moved below the resume", + [ + " HV_CHECK_CTX(hv_vcpu_run(vcpu), vcpu, g);", + " signal_forget_sigreturn_x8();", + ], + 1, + 1, + ), + ( + "a second resume added without one", + [ + " signal_forget_sigreturn_x8();", + " HV_CHECK_CTX(hv_vcpu_run(vcpu), vcpu, g);", + " if (retry)", + " hv_vcpu_run(vcpu);", + ], + 2, + 1, + ), + ( + "prose naming the resume is not a call site", + [ + " /* kick the vCPUs out of hv_vcpu_run() before the unmap */", + " * ordered by the first hv_vcpu_run and the release here", + ], + 0, + 0, + ), + ( + "forget commented out above the resume", + [ + " // signal_forget_sigreturn_x8();", + " HV_CHECK_CTX(hv_vcpu_run(vcpu), vcpu, g);", ], 1, + 1, ), ] @@ -370,14 +714,23 @@ def self_test(): print(" SVCTAIL self-test", flush=True) failures = 0 for name, lines, expected in CASES: - got = len(check(lines, "")) + got = len(check(lines, "", frame=SELF_TEST_FRAME)) if got != expected: print(f" FAIL {name}: expected {expected} problem(s), got {got}") failures += 1 + for name, lines, want_sites, expected in RESUME_CASES: + got, sites = check_resume_barrier(lines, "") + if len(got) != expected or sites != want_sites: + print( + f" FAIL {name}: expected {expected} problem(s) over " + f"{want_sites} site(s), got {len(got)} over {sites}" + ) + failures += 1 + total = len(CASES) + len(RESUME_CASES) if failures: - print(f" self-test: {failures} of {len(CASES)} cases failed") + print(f" self-test: {failures} of {total} cases failed") return 1 - print(f" self-test: {len(CASES)} cases, all pass") + print(f" self-test: {total} cases, all pass") return 0 @@ -390,21 +743,42 @@ def main(): return self_test() counted = [] - problems = check(SHIM.read_text().splitlines(), str(SHIM), counted) + frame, problems = frame_constants( + SIGNAL_C.read_text().splitlines(), str(SIGNAL_C) + ) + problems += check(SHIM.read_text().splitlines(), str(SHIM), counted, frame) + + resumes = 0 + for src in sorted(SRC.rglob("*.c")): + found, sites = check_resume_barrier( + src.read_text().splitlines(), str(src.relative_to(ROOT)) + ) + problems += found + resumes += sites + if not resumes: + problems.append( + "src: no hv_vcpu_run() call site found; the vCPU resume moved, and " + "with it the point the parked X8 has to be forgotten at" + ) + print(f" SVCTAIL {SHIM.relative_to(ROOT)}", flush=True) if problems: for p in problems: print(f" {p}", file=sys.stderr) print( - f"\n {len(problems)} HVC #5 tail(s) can reach EL0 without the X7 " - f"ptrace test.", + f"\n {len(problems)} problem(s): a tail that can reach EL0 without " + f"the X7 ptrace test, a frame-dropping tail that hands EL0 the " + f"marker instead of the guest's X8 or leaves the frame it was told " + f"to drop, or a resume with no forget written above it.", file=sys.stderr, ) return 1 print( f" {counted[0]} HVC #5 tail(s) reach the X7 test, " - f"{len(ALLOWED_SKIP)} documented exception" + f"{len(ALLOWED_SKIP)} documented exception; {DROP_TAIL} and {BRK_TAIL} " + f"both reload X8 from [sp, #{frame['SHIM_FRAME_OFF_X8']}] before the " + f"pop; {resumes} vCPU resume(s) have a forget written above them" ) return 0 diff --git a/src/core/guest.h b/src/core/guest.h index f265fabd..a4e14a97 100644 --- a/src/core/guest.h +++ b/src/core/guest.h @@ -301,8 +301,8 @@ typedef struct { * TLBI_BROADCAST -> X8 = 1 (TLBI VMALLE1IS, broadest) * TLBI_RANGE -> X8 = 3, X9 = start VA, X10 = page count * (TLBI VAE1IS loop preserves unrelated TLB entries) - * X8 = 2 is reserved for the execve drop-frame marker the shim handles - * separately; it is never produced by the accumulator. + * X8 = 2 is reserved for the drop-frame marker the shim handles separately; it + * is never produced by the accumulator. */ typedef enum { TLBI_NONE = 0, diff --git a/src/core/shim.S b/src/core/shim.S index b473a2da..a7a1b525 100644 --- a/src/core/shim.S +++ b/src/core/shim.S @@ -35,9 +35,13 @@ * X8 on return from #5, the post-syscall request: * 0 no flush * 1 broadcast TLBI VMALLE1IS - * 2 host replaced the EL0 register state (execve, rt_sigreturn, signal - * delivery on the syscall-return path): full flush, drop the saved frame, - * ERET without restoring GPRs + * 2 host replaced the EL0 register state (rt_sigreturn, signal delivery on + * the syscall-return path): full flush, drop the saved frame, ERET + * without restoring GPRs, with one exception. X8 is the register the + * marker itself arrives in, so it cannot also carry the guest value the + * host wants EL0 to see. The host leaves that value in the frame's own X8 + * slot and exec_drop_frame reloads X8 from there; see the note on that + * label. * 3 TLBI VAE1IS over X10 pages from the page-aligned VA in X9 * 4 single-shot TLBI RVAE1IS (FEAT_TLBIRANGE), operand pre-encoded in X9 as * baddr | NUM<<39 | SCALE<<44 | TTL<<37 | ASID<<48 (SCALE, TTL and ASID @@ -48,15 +52,20 @@ * take HVC #13 after the restore, so the tracer sees the architectural GPR set * rather than shim scratch. The host leaves X7 alone on the X8 == 2 tail, which * never restores the frame and would carry the flag into EL0 as guest state; it - * takes that stop itself, where the live registers are already final. A tail + * takes that stop itself, where the live registers are already final bar X8, + * which holds the marker and is reloaded from the frame by the tail. A tail * that skips the X7 test drops a stop the host has already consumed, so new * exits from the dispatch below go through svc_hvc_restore_eret, never straight * to svc_restore_eret. * * X11 is the I-cache hint, read for X8 in {1, 3, 4}: 1 issues IC IALLU after * the TLBI because the change made new content executable, 0 skips it. X8 == 2 - * always flushes, since execve loads new code. The shim restores X11 from the - * saved frame before ERET, so EL0 never sees the hint. + * flushes unconditionally, IC IALLU included, because the host has rebuilt the + * EL0 state and the shim cannot tell what changed. The restoring tails reload + * X11 from the saved frame before ERET, so EL0 never sees the hint. The X8 == 2 + * tail restores nothing but X8, so a delivery that follows a page-table + * modifying syscall in the same epilogue hands X9 through X11 out to EL0 in + * place of the guest's; see the note on exec_drop_frame. * * Two constraints on everything below. macOS 'as' treats ';' on AArch64 as a * comment, not a statement separator, so every instruction needs its own line. @@ -1669,7 +1678,7 @@ restore_and_bad: /* handle_brk: BRK instruction from EL0 (EC=0x3C) * * JIT translators use BRK instructions as patching trampolines and debug hooks. - * The guest registers a SIGTRAP handler via rt_sigaction. The shim restores all + * The guest registers a SIGTRAP handler via rt_sigaction. The shim loads all * GPRs and forwards to the host via HVC #10 for signal delivery. * * Host reads ESR_EL1 (BRK immediate), ELR_EL1 (BRK PC), and current GPRs from @@ -1678,15 +1687,27 @@ restore_and_bad: * SIGTRAP action (terminate). */ handle_brk: - /* Restore all GPRs from stack frame (host needs current state) */ - RESTORE_GPRS + /* Load the EL0 GPRs from the saved frame WITHOUT popping it, the shape + * handle_el0_fault uses and for the same reason: the host needs the guest's + * register state, and the frame has to outlive the HVC so the tail below + * can read out of it. + */ + LOAD_GPRS /* Forward to host for SIGTRAP delivery */ hvc #10 /* Host has set up signal frame (if handler registered) or flagged for - * termination. Either way, ERET to new PC. - */ + * termination. A delivery leaves the drop-frame marker in X8, which is + * guest state on this path: the SIGTRAP handler would enter with 2 in place + * of whatever the guest was carrying, and a JIT translator using BRK as a + * trampoline is the caller most likely to read that register. Reload it + * from the frame's own X8 slot before the pop, the way exec_drop_frame + * does; nothing publishes there on this path, so the value is the one the + * BRK was taken with. Then drop the frame and ERET to the new PC. + */ + ldr x8, [sp, #64] + add sp, sp, #256 eret /* handle_svc_0: Linux syscall forwarding @@ -1710,7 +1731,9 @@ handle_svc_0: * 4. Set X8 to indicate the post-syscall request: * 0 = no TLB flush * 1 = broadcast TLBI VMALLE1IS - * 2 = execve replaced register state (drop frame + flush) + * 2 = host replaced register state (rt_sigreturn, signal + * delivery): drop frame + flush, and reload X8 from the + * frame's own slot; see exec_drop_frame * 3 = selective TLBI VAE1IS over X10 pages starting at X9 * 5. Resume vCPU (execution continues below) */ @@ -1844,6 +1867,48 @@ svc_hvc_restore_eret: eret exec_drop_frame: + /* This tail restores nothing from the frame because the host has written + * the EL0 state it wants straight into the vCPU. X8 is the one register + * that cannot arrive that way: it carries the drop-frame marker in, so the + * value the host wants EL0 to see has to travel out of band. + * + * It travels in the frame's own X8 slot, [sp+64] per SAVE_GPRS, filled by + * the host before it writes the marker. That slot cannot collide with the + * marker the way a second register would: it is EL1-only memory on this + * vCPU's exception stack, unreachable from EL0 and unshared with any other + * vCPU, and the pop below retires it. Putting it back here is what keeps a + * frame whose saved PC sits on an SVC from issuing that SVC as syscall 2. + * + * Three tails branch here and only the HVC #5 one has a publisher. The two + * W^X permission faults reach it when the host answers the flip request + * with the marker, which it does when the region never had the permission + * and the fault is a real SIGSEGV. Nothing publishes on that route, so the + * reload hands EL0 the X8 the exception was taken with, which is the + * guest's own and is what the handler then enters with. Before this reload + * existed those two entered their handler with the marker. handle_brk has + * the same problem and does not branch here, since it has its own flush to + * skip and no X8 dispatch to run; it carries its own copy of the reload. + * + * The ordinary syscall-return tail above is not covered, and this reload + * does not reach it. A signal delivered in the epilogue of a page-table + * syscall snapshots the live registers there, where X8 is the TLBI kind and + * X9 through X11 are its operands and the icache hint, so the frame that + * delivery builds records the wire values as the guest's and its own + * rt_sigreturn hands them back. The saved PC being past the SVC does not + * make that free: the instruction after an SVC can be another SVC, and the + * guest then issues the wire value as a syscall number. Measured with two + * adjacent SVCs under a 200us itimer over 60000 rounds, the first of them + * mprotect: all 707 deliveries left the second running as syscall 0 or 3 + * rather than the 226 the guest held. It is pre-existing and not the + * register this tail moves. The same probe on afdcfce, interleaved so both + * binaries met the same machine, read 703 of its 704 as syscall 2, the + * marker rather than the wire value. syscall/proc.h names the same + * shadowing on the frame a handler returns through. + * + * It survives the change that closes #379, so it is tracked as its own + * issue rather than living only here: #384. + */ + ldr x8, [sp, #64] tlbi vmalle1is dsb ish isb diff --git a/src/runtime/thread.c b/src/runtime/thread.c index 17ebf792..6b3ea185 100644 --- a/src/runtime/thread.c +++ b/src/runtime/thread.c @@ -518,10 +518,10 @@ uint64_t thread_alloc_sp_el1(const guest_t *g, thread_entry_t *t) int slot = bit_ctz64(free_mask); /* Main thread's SP_EL1 sits at the top of the shim data block. Each - * subsequent thread is 4KiB below. + * subsequent thread is one slot below. */ uint64_t top = sp_el1_top(g); - sp = top - (uint64_t) slot * 4096; + sp = top - (uint64_t) slot * SP_EL1_SLOT_BYTES; sp_el1_allocated |= BIT64(slot); t->sp_el1 = sp; t->sp_el1_slot = slot; @@ -532,6 +532,19 @@ uint64_t thread_alloc_sp_el1(const guest_t *g, thread_entry_t *t) return sp; } +void thread_sp_el1_region(const guest_t *g, uint64_t *lo, uint64_t *hi) +{ + /* Derived from what thread_alloc_sp_el1 above hands out, not from the block + * it hands it out of: slot 0 starts at the top and slot MAX_THREADS - 1 is + * the last one, so the region is the top MAX_THREADS slots and everything + * under it belongs to the shim-globals cache. + */ + uint64_t top = sp_el1_top(g); + + *hi = top; + *lo = top - (uint64_t) MAX_THREADS * SP_EL1_SLOT_BYTES; +} + void thread_for_each(void (*fn)(thread_entry_t *t, void *ctx), void *ctx) { pthread_mutex_lock(&thread_lock); diff --git a/src/runtime/thread.h b/src/runtime/thread.h index 2423ca76..92f153a8 100644 --- a/src/runtime/thread.h +++ b/src/runtime/thread.h @@ -29,6 +29,13 @@ /* Maximum number of concurrent guest threads in one VM. */ #define MAX_THREADS 64 + +/* One EL1 exception stack per thread. Named because the region these slots + * occupy is a bound as well as an allocation: the host writes into a live shim + * frame on that stack, and anything below the lowest slot is the shim-globals + * cache rather than stack. + */ +#define SP_EL1_SLOT_BYTES 4096 #define MAX_DEFERRED_STACK_UNMAPS 8 /* Per-thread state. One entry per guest thread (main + workers). Tagged (struct @@ -353,15 +360,28 @@ uint64_t thread_pending_union(void); int thread_is_single_active(void); /* Allocate a per-thread SP_EL1 stack and record both the IPA and the slot index - * into t. Thread N gets the Nth 4KiB slot counting down from the top of the - * shim data block (g->shim_data_base + 2MiB). The shim block lives at high IPA - * computed by guest_init, so callers must pass g; the slot index is stored in - * t->sp_el1_slot so the free path (which is reached from teardown contexts that - * lack g) can clear the bitmask directly. + * into t. Thread N gets the Nth SP_EL1_SLOT_BYTES slot counting down from the + * top of the shim data block (g->shim_data_base + 2MiB). The shim block lives + * at high IPA computed by guest_init, so callers must pass g; the slot index is + * stored in t->sp_el1_slot so the free path (which is reached from teardown + * contexts that lack g) can clear the bitmask directly. * Returns the SP_EL1 IPA, or 0 on slot exhaustion. */ uint64_t thread_alloc_sp_el1(const guest_t *g, thread_entry_t *t); +/* The IPA range the slots above are carved from: *lo is the bottom of the + * lowest slot, *hi one past the top of the highest, so a live SP_EL1 is in + * [*lo, *hi). + * + * The shim data block holds two unrelated things. These slots sit at the top of + * it, and the shim-globals cache -- identity slots, urandom ring, attention + * bitmask -- starts at the bottom. A host that validates an SP_EL1 against the + * whole block therefore accepts addresses that name cache and not stack, and + * writing a shim frame at one of those corrupts the cache instead of reporting + * the bad SP_EL1. Callers that write through an SP_EL1 bound it with this. + */ +void thread_sp_el1_region(const guest_t *g, uint64_t *lo, uint64_t *hi); + /* Iterate over all active threads, calling fn(entry, ctx) for each. Holds the * thread table lock during iteration. */ diff --git a/src/syscall/proc.c b/src/syscall/proc.c index fe8d1946..0014cea1 100644 --- a/src/syscall/proc.c +++ b/src/syscall/proc.c @@ -1915,9 +1915,10 @@ int64_t sys_ptrace(guest_t *g, * whatever the tracer writes back, since the shim restores its own * frame over it. The HVC #5 epilogue consumes the flag and then either * stops right there, on the tails whose live registers are already the - * final EL0 set, or asks the shim through X7 to restore the frame and - * come back at HVC #13. The canceled-exit handler consumes it once it - * has established the vCPU is at EL0. + * final EL0 set bar the X8 the tail reloads from the frame, or asks the + * shim through X7 to restore the frame and come back at HVC #13. The + * canceled-exit handler consumes it once it has established the vCPU is + * at EL0. * * Attention goes up before the kick, the same order * shim_globals_raise_attention uses and for the same reason: a fast @@ -4029,6 +4030,13 @@ static bool ptrace_take_stop(guest_t *g, hv_vcpu_t vcpu, int *exit_code) if (cont_sig > 0) signal_queue(cont_sig); + /* The stop above returns with whatever registers the tracer wrote, PC among + * them. An X8 parked by an rt_sigreturn earlier in this same epilogue is + * still the X8 the guest is owed at the PC it now resumes from, so the + * record follows it before the delivery below reads it. + */ + signal_repark_sigreturn_x8(vcpu); + /* One delivery covers both the injected resume signal and anything that * arrived while the tracee was stopped, so neither caller repeats it. */ @@ -4074,7 +4082,8 @@ static bool syscall_return_epilogue(guest_t *g, * host-only. The vector entry clobbers no GPR below X9, so the live set at * HVC #5 is still the guest's, and only that restore puts a host write to * X7 back. Two tails skip it: X8 == 2, where the host has rebuilt EL0 state - * and the live registers are already final, and an execve re-entry, which + * and the live registers are already final bar X8, which carries the marker + * and is reloaded from the frame by the tail, and an execve re-entry, which * goes through the MMU-off _start with no tail at all. * * Only the exec-happened return has to ask the vCPU which of those it is. @@ -4099,7 +4108,11 @@ static bool syscall_return_epilogue(guest_t *g, ptrace_consume_owed_stop(g)) { if (regs_final) { /* Live registers are already the architectural EL0 set, which is - * what the detour exists to produce. Stop here instead. + * what the detour exists to produce. Stop here instead. One + * register is not the guest's: X8 still holds the drop-frame + * marker, which the tail reloads from the frame after this stop, so + * PTRACE_GETREGSET here reports 2 where the guest's X8 belongs. + * Pre-existing and unchanged; docs/internals.md measures it. */ running = ptrace_take_stop(g, vcpu, exit_code); stop_taken = true; @@ -4675,8 +4688,8 @@ int vcpu_run_loop_with_hooks(hv_vcpu_t vcpu, * so it is the only thread that can survive one. Running it at the top * of the loop puts the rebuilt EL0 state in place before the vCPU is * resumed, whether this thread was preempted in guest code or is - * returning from its own syscall (sys_execve sets the X8=2 frame-drop - * marker either way). + * returning from its own syscall (sys_execve writes no frame-drop + * marker either way: its MMU-off _start re-entry never pops a frame). */ if (thread_current_is_leader() && thread_leader_work_pending()) exec_run_handoff(vcpu, g, verbose); @@ -4708,6 +4721,12 @@ int vcpu_run_loop_with_hooks(hv_vcpu_t vcpu, atomic_store_explicit(&g_vcpu_progress, iter * 2 + 1, memory_order_relaxed); + /* The guest owns X8 again from here, so the X8 an rt_sigreturn parked + * for a delivery in its own epilogue must not survive into the next + * exception. See signal_forget_sigreturn_x8 in syscall/signal.h. + */ + signal_forget_sigreturn_x8(); + HV_CHECK_CTX(hv_vcpu_run(vcpu), vcpu, g); if (is_main) diff --git a/src/syscall/proc.h b/src/syscall/proc.h index f71663ad..e1ae0c6c 100644 --- a/src/syscall/proc.h +++ b/src/syscall/proc.h @@ -368,8 +368,13 @@ const char *proc_resolve_sysroot_create_path(const char *path, * a signal really did arrive. And the frame the handler returns through carries * the live X8, which by then is the TLBI wire value the shim epilogue wrote * rather than the syscall number the shim would have restored from its own - * saved frame. That is harmless while the saved PC is past the SVC, but a - * rewound PC would make rt_sigreturn re-execute the SVC as the wrong call. + * saved frame. A rewound PC makes rt_sigreturn re-execute that SVC as the call + * the wire value names. Leaving the PC past the SVC narrows that rather than + * closing it, because the instruction after an SVC can be another SVC; that + * residue is pre-existing, measured beside exec_drop_frame in core/shim.S, and + * out of reach of this cancel. The drop-frame marker is the other value that + * displaces the guest's X8, and signal.c keeps that one out of both the frame + * and the ERET on its own. * * Cancel is a no-op unless ELR_EL1 still holds the value the arm wrote, so a * later delivery on an unrelated path cannot disturb a guest that has moved on. diff --git a/src/syscall/signal.c b/src/syscall/signal.c index cd5f2df9..45693fee 100644 --- a/src/syscall/signal.c +++ b/src/syscall/signal.c @@ -32,7 +32,10 @@ * * rt_sigreturn always has a frame to drop, having entered through one. * deliver_signal_locked and signal_rt_sigreturn each say why at the point they - * write the marker. sys_execve is a fourth rebuilder, in exec.c, and takes + * write the marker. The marker occupies X8, and the tail that consumes it + * restores no register, so the guest's own X8 has to reach EL0 another way: + * shim_publish_frame_x8 puts it in the shim frame's X8 slot, and the shim + * reloads it from there. sys_execve is a fourth rebuilder, in exec.c, and takes * neither shape: it re-enters through the shim's MMU-off _start with the GPRs * zeroed, so there is no frame to drop and no marker to write. * @@ -2203,6 +2206,121 @@ static void build_sigcontext_reserved(uint8_t *reserved, memset(reserved + off, 0, 8); } +/* The EL1 shim's exception frame: 256 bytes, saved X8 at byte 64. SAVE_GPRS in + * src/core/shim.S lays it out and exec_drop_frame reloads X8 from that slot; + * scripts/check-svc-tails.py holds the two sides to the same number. + */ +#define SHIM_FRAME_BYTES 256 +#define SHIM_FRAME_OFF_X8 64 + +/* The guest's own X8 while the host borrows the register, handed from the + * rt_sigreturn that borrowed it to a signal delivered later in the same host + * epilogue. + * + * The shim is told to drop its saved exception frame by a marker the host puts + * in X8, and the tail that consumes the marker restores no register, so X8 is + * the one register that cannot hold guest state across that return. Publishing + * the value into the frame slot covers the return itself. It does not cover a + * signal delivered before the vCPU is resumed: that delivery snapshots the live + * registers, and the live X8 is still the marker, so the frame it builds would + * record the marker as the guest's X8 and its rt_sigreturn would hand that + * back. + * + * What makes substituting this safe is that it cannot outlive that window. + * signal_forget_sigreturn_x8 drops the record at every vCPU resume, so the only + * reader that can ever see one is a delivery between the rt_sigreturn that + * wrote it and the guest running again, which is exactly the stretch where the + * live register is not the guest's. Consumption drops it too, so a second + * delivery in the same epilogue reads the register. + * + * The ELR it was parked for is kept and compared, the way syscall_restart_arm + * recognizes its own rewind, but it is an identity check and not the bound: one + * thing moves the guest inside an epilogue and it re-keys the record as it + * does, so a delivery that still finds a different ELR is on a path this did + * not anticipate and reads the register instead. + * + * One path moves the guest inside the epilogue: a ptrace stop consumed on this + * same tail, where the tracer writes a new PC before the epilogue delivers a + * signal. ELR_EL1 is then the tracer's PC, and the comparison would miss the + * one delivery this record is genuinely for, leaving it to read the marker out + * of the register. signal_repark_sigreturn_x8 re-keys the record on the PC the + * stop left behind instead of widening the comparison away, because only the + * address moved: the guest, its epilogue and the X8 owed to it are the same, + * and every delivery that did not come through that stop still has to match. + * What the stop hands the tracer through PTRACE_GETREGSET, and what the shim + * does with an X8 the tracer writes back, are pre-existing and out of this + * record's reach; docs/internals.md measures both. + * + * Per-vCPU: TLS, like cpu_tlbi_req and cpu_restart_req. + */ +static _Thread_local struct { + bool valid; + uint64_t elr; /* ELR_EL1 the value belongs to */ + uint64_t x8; /* what the guest had in X8 at that ELR */ +} sigreturn_x8; + +void signal_forget_sigreturn_x8(void) +{ + sigreturn_x8.valid = false; +} + +void signal_repark_sigreturn_x8(hv_vcpu_t vcpu) +{ + if (!sigreturn_x8.valid) + return; + sigreturn_x8.elr = vcpu_get_sysreg(vcpu, HV_SYS_REG_ELR_EL1); +} + +/* Hand the shim the X8 that EL0 must see when it drops the frame it is holding, + * by writing it into the frame's own X8 slot. + * + * That slot cannot collide with the marker the way a second register would: it + * is EL1-only memory on this vCPU's exception stack, unreachable from EL0 and + * unshared with any other vCPU, it holds nothing live once the frame is being + * dropped, and the shim's pop retires it. A drop that reaches the tail without + * a value published here gets the X8 the exception was taken with, which is + * guest state rather than a wire value. + * + * Only rt_sigreturn publishes, and only because it is the one rebuilder whose + * X8 differs from the one the frame was entered with. It is also the only + * caller that can promise the frame is live: it always arrives through HVC #5, + * where the vector entry has saved the frame and SP_EL1 still points at it. + * + * The bound is the EL1 stack region, not the shim data block that contains it. + * The block holds the shim-globals cache at the bottom -- identity slots, + * urandom ring, attention bitmask -- and only the slots thread_alloc_sp_el1 + * carves from the top are stack. An SP_EL1 that is wrong but still inside the + * block would otherwise pass, and the write would land in that cache: silent + * corruption of live shim state where the whole point of the check is to + * report. thread_sp_el1_region derives the slots from the constants the + * allocator hands them out with. + */ +static void shim_publish_frame_x8(hv_vcpu_t vcpu, + const guest_t *g, + uint64_t x8_for_el0) +{ + uint64_t sp_el1 = vcpu_get_sysreg(vcpu, HV_SYS_REG_SP_EL1); + uint64_t lo, hi; + + thread_sp_el1_region(g, &lo, &hi); + + if (sp_el1 < lo || sp_el1 > hi - SHIM_FRAME_BYTES) { + /* Nothing that took an exception through the shim can be here. Report + * it rather than write; the shim then hands EL0 the X8 the frame was + * entered with. + */ + log_error( + "rt_sigreturn: SP_EL1 0x%llx is outside the EL1 stack region " + "[0x%llx, 0x%llx)", + (unsigned long long) sp_el1, (unsigned long long) lo, + (unsigned long long) hi); + return; + } + + uint8_t *frame = (uint8_t *) g->host_base + sp_el1; + memcpy(frame + SHIM_FRAME_OFF_X8, &x8_for_el0, sizeof(x8_for_el0)); +} + /* Build and install the rt_sigframe for `signum` on the current thread, with * sig_lock held on entry and released on every return path. Shared by * signal_deliver() (signal selected from the process-wide pending set) and @@ -2304,6 +2422,20 @@ static int deliver_signal_locked(hv_vcpu_t vcpu, saved_pstate = vcpu_get_sysreg(vcpu, HV_SYS_REG_SPSR_EL1); } + /* X8 is not the guest's while the drop-frame marker occupies it, and an + * rt_sigreturn earlier in this same epilogue leaves the marker live with + * ELR_EL1 on the instruction the guest is about to resume, which can be an + * SVC it has not executed yet. Snapshotting the register there would record + * the marker as the guest's X8, and the handler's own rt_sigreturn would + * hand it back, so that SVC would run as syscall 2. Take what rt_sigreturn + * parked instead. A record exists only inside the epilogue that wrote it, + * so there is no later delivery for this to reach; see sigreturn_x8. + */ + if (!el0_preempt && sigreturn_x8.valid && sigreturn_x8.elr == saved_pc) { + saved_regs[8] = sigreturn_x8.x8; + sigreturn_x8.valid = false; + } + /* 1b. rseq abort: if the thread is in a restartable sequence critical * section, abort it. Linux does this on every signal delivery. */ @@ -2527,6 +2659,12 @@ static int deliver_signal_locked(hv_vcpu_t vcpu, * consume it. The EL0-preemption path resumes straight into the handler at * EL0 with no shim frame to drop, so the marker is neither needed nor * consulted. + * + * The marker is all this writes. What EL0 sees in X8 on that drop is + * whatever stands in the frame's X8 slot: the value the exception was taken + * with, or the one an rt_sigreturn published earlier in this same epilogue. + * Neither is the marker, which is the whole point; shim_publish_frame_x8 + * says why the slot and not a register. */ if (!el0_preempt) hv_vcpu_set_reg(vcpu, HV_REG_X8, 2); @@ -2828,7 +2966,20 @@ int signal_rt_sigreturn(hv_vcpu_t vcpu, guest_t *g) * restored the complete guest register state here; letting the shim restore * X1-X30 from the rt_sigreturn syscall entry would corrupt the interrupted * context. - */ + * + * X8 is restored above with the other 30 GPRs and then handed over again, + * because the marker is written into the same register. Without that second + * hand-off the marker is what the ERET delivers, and a frame whose saved PC + * sits on an SVC issues it as syscall 2. The same value is parked for a + * signal delivered later in this epilogue, which snapshots the register + * itself and would otherwise write the marker into the frame it builds. The + * park dies at the next vCPU resume whether or not anything took it. + */ + uint64_t restored_x8 = frame.uc.uc_mcontext.regs[8]; + shim_publish_frame_x8(vcpu, g, restored_x8); + sigreturn_x8.valid = true; + sigreturn_x8.elr = restored_pc; + sigreturn_x8.x8 = restored_x8; hv_vcpu_set_reg(vcpu, HV_REG_X8, 2); /* Return SYSCALL_EXEC_HAPPENED to skip the normal X0 writeback, since diff --git a/src/syscall/signal.h b/src/syscall/signal.h index b0068fc2..61b38bad 100644 --- a/src/syscall/signal.h +++ b/src/syscall/signal.h @@ -518,6 +518,41 @@ int signal_deliver_fault(hv_vcpu_t vcpu, */ int signal_rt_sigreturn(hv_vcpu_t vcpu, guest_t *g); +/* Drop the guest X8 that rt_sigreturn parked for a signal delivered later in + * the same host epilogue. + * + * The drop-frame marker occupies X8, so a delivery that follows an rt_sigreturn + * before the guest runs again cannot read the guest's X8 out of the register; + * signal_rt_sigreturn parks it and that delivery takes the parked value. The + * park is only ever correct for that stretch. Once the vCPU is resumed the + * guest owns X8 again, and a record left behind would be handed to some later + * delivery that merely resumes at the same PC -- a fault on an instruction the + * guest returned to, say, whose ELR_EL1 is the faulting PC and whose X8 is + * live. + * + * The vCPU run loop therefore calls this immediately before every resume, and + * scripts/check-svc-tails.py holds every hv_vcpu_run() call site to it. + */ +void signal_forget_sigreturn_x8(void); + +/* Re-key that record on the PC a ptrace stop left the guest at. + * + * Nothing moves the guest between the rt_sigreturn that parks an X8 and the + * resume that ends the record's life, with one exception: a PTRACE_INTERRUPT + * stop is taken inline on that same tail, and the tracer can write a new PC + * before it resumes the tracee with an injected signal. The delivery that + * follows then lands on an ELR_EL1 the record was not parked for, so the ELR + * check that keeps a stale record from being read misses the one delivery the + * record is genuinely for, and the frame it builds records the drop-frame + * marker as the guest's X8. + * + * Called with the vCPU still inside that epilogue, where the guest is the same + * guest with the same X8 owed to it and only the address it resumes from has + * moved. Nothing to do when no value is parked, which is every stop taken + * anywhere else. + */ +void signal_repark_sigreturn_x8(hv_vcpu_t vcpu); + /* Handle rt_sigaction (SYS 134). */ int64_t signal_rt_sigaction(guest_t *g, int signum, diff --git a/tests/manifest.txt b/tests/manifest.txt index 8ad6cce6..eea47d91 100644 --- a/tests/manifest.txt +++ b/tests/manifest.txt @@ -115,6 +115,7 @@ test-mprotect-mt # diff=skip [section] Signal + thread tests test-signal-thread test-signal-in-shim # diff=skip +test-shim-sigreturn-x8 test-sigsuspend test-fault-signal-mt # diff=skip test-exit-group-worker diff --git a/tests/test-matrix.sh b/tests/test-matrix.sh index 8f850b53..a9c495f8 100755 --- a/tests/test-matrix.sh +++ b/tests/test-matrix.sh @@ -646,10 +646,17 @@ test_pipe() # check") binary except the handful that assert elfuse-internal implementation # details with no meaningful counterpart on a real kernel (most of the EL1 shim # fast-path suite -- test-shim-* and test-shim-cred-race, which probe elfuse's -# own shim_data block and identity cache; test-shim-futex-fast is the exception -# and does run here, because every assertion in it is plain Linux futex ABI that -# a real kernel adjudicates (unlike test-mremap-infra, which guards elfuse's -# guest-IPA infra reserve, and test-oom-proc, documented in its own header). +# own shim_data block and identity cache). Two test-shim-* binaries are listed +# below rather than held out, each because every assertion in it is ABI a real +# kernel adjudicates: test-shim-futex-fast, which is plain Linux futex ABI, and +# test-shim-sigreturn-x8, which is plain Linux signal ABI throughout -- what +# rt_sigreturn restores, and what a handler enters with after a W^X permission +# fault or a BRK. Both are named here on purpose: a reader who finds one +# exception recorded takes it for the whole rule and holds the next one out. +# test-shim-sigreturn-x8 was run against this lane by hand when it was +# registered, on the fixture kernel of the day (Alpine 6.18.52-0-virt), and +# passes there. Contrast test-mremap-infra, which guards elfuse's guest-IPA +# infra reserve, and test-oom-proc, documented in its own header. # test-mremap-tail-emfile is listed here as an elfuse-lane regression and marked # QEMU_SKIP because its host-reserve assertion has no Linux analogue. There is # no "core" vs "extended" split here; everything below runs in both @@ -728,6 +735,8 @@ run_unit_tests() "$bindir/test-wait-process-signal" test_check "$runner" "test-wait-sigmask-signal" " - PASS" \ "$bindir/test-wait-sigmask-signal" + test_check "$runner" "test-shim-sigreturn-x8" "0 failed" \ + "$bindir/test-shim-sigreturn-x8" test_check "$runner" "test-ptrace-interrupt" "OK: ptrace-stop reports EL0" \ "$bindir/test-ptrace-interrupt" test_check "$runner" "test-sigsuspend" "PASS|0 failed" "$bindir/test-sigsuspend" diff --git a/tests/test-shim-futex-toctou.c b/tests/test-shim-futex-toctou.c index e86fe294..996cf78d 100644 --- a/tests/test-shim-futex-toctou.c +++ b/tests/test-shim-futex-toctou.c @@ -100,7 +100,9 @@ static void *spin_waiter(void *arg) /* Print the first one. This phase used to count these and say * nothing else, and the counter alone cannot tell a wrong errno * from an SVC that re-executed as a different syscall, which is - * what the open Tier A item about X8=2 turns out to be. + * what the Tier A item about X8=2 turned out to be + * (sysprog21/elfuse#379, now closed; tests/test-shim-sigreturn-x8 + * reaches the same window without a race). */ if (spin_other == 0) fprintf(stderr, "FAIL: unexpected spin rc %ld (round %d)\n", rc, diff --git a/tests/test-shim-sigreturn-x8.c b/tests/test-shim-sigreturn-x8.c new file mode 100644 index 00000000..3929ac2e --- /dev/null +++ b/tests/test-shim-sigreturn-x8.c @@ -0,0 +1,576 @@ +/* + * test-shim-sigreturn-x8.c -- rt_sigreturn hands EL0 the X8 the frame records. + * + * Copyright 2026 elfuse contributors + * SPDX-License-Identifier: Apache-2.0 + * + * A signal handler rewrites its own ucontext so the return lands on a stub that + * reads X8 and then issues an SVC with it. The kernel contract is that + * rt_sigreturn restores all 31 GPRs from the frame, so the stub must see the X8 + * the handler wrote and the SVC must run as that syscall. + * + * elfuse has one more thing to get right there. Its EL1 shim is told to drop + * the exception frame it is holding by a marker the host writes into X8, and + * the tail that consumes the marker restores no register, so the marker itself + * used to reach EL0 in place of the restored X8. A guest whose resumed PC sat + * on an SVC then issued it as syscall 2 and got ENOSYS out of a call it had + * made as something else, which is what tests/test-shim-futex-toctou caught by + * racing. This reaches the same window without a race: the handler puts the + * resume PC on the stub, so every round runs with a live X8 across the return. + * + * Rounds alternate, because the marker reaches the resumed SVC by two routes. + * Plain: the rt_sigreturn itself hands X8 to EL0. Nested: a second signal, + * raised inside the first handler and blocked until rt_sigreturn unblocks it, + * is delivered on the way out, and the frame that delivery builds is the one + * that records X8 for the return that follows. + * + * The nested route needs the host to remember, across the rt_sigreturn, what X8 + * the guest is owed, and a third phase holds that memory to its window. It + * returns to a stub, lets the guest run, and then faults on the same PC with a + * different X8 live. The fault is an ordinary delivery, not a continuation of + * the rt_sigreturn, so what the SIGSEGV handler sees has to be the register the + * guest faulted with and not anything the earlier return left parked. + * + * A fourth phase leaves the syscall route entirely. The same drop tail is + * branched to from the two W^X permission-fault handlers, where the host + * answers a flip request with the marker because the region never had the + * permission and the fault is a real SIGSEGV. Nothing publishes an X8 there, so + * the reload hands back the value the exception was taken with, and the two + * rounds assert that the handler enters with the guest's X8 rather than the + * marker. + * + * A fifth phase does the same for BRK, which carries the marker out of its own + * tail rather than exec_drop_frame's. JIT translators use BRK as a trampoline + * and read X8 in the SIGTRAP handler, so both halves are asserted: the X8 the + * handler enters with, and the X8 the resumed guest carries once its + * rt_sigreturn has run. + * + * Plain Linux signal ABI throughout, so the reference kernel adjudicates it and + * the test is registered in tests/test-matrix.sh rather than exempted from it. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +/* aarch64 __NR_getpid. Spelled out rather than taken from a header because the + * stub below has to load the same number into X8 from assembly. + */ +#define NR_GETPID 172 + +#define ROUNDS 32 + +/* aarch64 __NR_mprotect, and the two X8 values the stale-park phase tells + * apart. Spelled out because stale_stub below loads all three from assembly. + */ +#define NR_MPROTECT 226 +#define STALE_PARKED 0x55 +#define STALE_LIVE 0x99 + +/* Written by the stub, read by main. Not static: the stub addresses them by + * name through adrp/add. + */ +volatile uint64_t resumed_x8; +volatile int64_t resumed_rc; + +static sigjmp_buf resume; +static int failures; +static volatile sig_atomic_t nest; /* raise the second signal in handler */ +static volatile sig_atomic_t nested; /* the second handler ran */ + +void sigreturn_finish(void); +void sigreturn_stub(void); + +/* Runs at EL0 with the register state rt_sigreturn restored. Records X8, then + * issues the SVC that X8 names without touching it, so a corrupted X8 shows up + * twice: in the recorded value and in the syscall that ran. + */ +__asm__( + ".text\n" + ".globl sigreturn_stub\n" + ".type sigreturn_stub, %function\n" + "sigreturn_stub:\n" + " adrp x9, resumed_x8\n" + " add x9, x9, :lo12:resumed_x8\n" + " str x8, [x9]\n" + " svc #0\n" + " adrp x9, resumed_rc\n" + " add x9, x9, :lo12:resumed_rc\n" + " str x0, [x9]\n" + " b sigreturn_finish\n" + ".size sigreturn_stub, .-sigreturn_stub\n"); + +void sigreturn_finish(void) +{ + siglongjmp(resume, 1); +} + +static void handler(int sig, siginfo_t *info, void *ctx) +{ + (void) sig; + (void) info; + ucontext_t *uc = ctx; + + /* Resume on the stub with getpid live in X8. Everything else the frame + * holds is left alone, including the stack pointer, so the stub returns + * through siglongjmp on the interrupted thread's own stack. + */ + uc->uc_mcontext.pc = (uint64_t) (uintptr_t) sigreturn_stub; + uc->uc_mcontext.regs[8] = NR_GETPID; + + /* SIGUSR2 is in this handler's sa_mask, so it stays pending until + * rt_sigreturn restores the mask and is then delivered on the way out, with + * the resume PC and X8 already set above. + */ + if (nest) + raise(SIGUSR2); +} + +static void nested_handler(int sig) +{ + (void) sig; + nested = 1; +} + +/* Reached twice at the same PC, with X19 holding a writable page. + * + * First pass: rt_sigreturn resumes here with X8 = STALE_PARKED, the load + * succeeds, and the guest runs on. Second pass: the page has been made + * unreadable and X8 is STALE_LIVE, so the load faults with the fault PC equal + * to the PC the earlier return came back to. + */ +void stale_stub(void); + +__asm__( + ".text\n" + ".globl stale_stub\n" + ".type stale_stub, %function\n" + "stale_stub:\n" + " ldr x1, [x19]\n" + " mov x0, x19\n" + " mov x1, #4096\n" + " mov x2, #0\n" + " mov x8, #226\n" /* NR_MPROTECT: PROT_NONE over the page */ + " svc #0\n" + " mov x8, #0x99\n" /* STALE_LIVE */ + " b stale_stub\n" + ".size stale_stub, .-stale_stub\n"); + +static uint8_t *stale_page; +static sigjmp_buf stale_back; +static volatile uint64_t stale_x8; +static volatile uint64_t stale_pc; +static volatile sig_atomic_t stale_faulted; + +static void stale_redirect(int sig, siginfo_t *info, void *ctx) +{ + (void) sig; + (void) info; + ucontext_t *uc = ctx; + uc->uc_mcontext.pc = (uint64_t) (uintptr_t) stale_stub; + uc->uc_mcontext.regs[8] = STALE_PARKED; + uc->uc_mcontext.regs[19] = (uint64_t) (uintptr_t) stale_page; +} + +static void stale_fault(int sig, siginfo_t *info, void *ctx) +{ + (void) sig; + (void) info; + ucontext_t *uc = ctx; + stale_faulted = 1; + stale_x8 = uc->uc_mcontext.regs[8]; + stale_pc = uc->uc_mcontext.pc; + siglongjmp(stale_back, 1); +} + +static int check_stale_park(void) +{ + static char altbuf[SIGSTKSZ * 4]; + struct sigaction sa; + stack_t ss; + int failures = 0; + + stale_page = mmap(NULL, 4096, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (stale_page == MAP_FAILED) { + perror("mmap"); + return 1; + } + + /* The fault is taken on stale_stub, which runs on the interrupted thread's + * own stack; an alternate stack keeps the handler off it. + */ + ss.ss_sp = altbuf; + ss.ss_size = sizeof(altbuf); + ss.ss_flags = 0; + if (sigaltstack(&ss, NULL) < 0) { + perror("sigaltstack"); + return 1; + } + + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = stale_redirect; + sa.sa_flags = SA_SIGINFO; + sigemptyset(&sa.sa_mask); + if (sigaction(SIGUSR1, &sa, NULL) < 0) { + perror("sigaction SIGUSR1"); + return 1; + } + + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = stale_fault; + sa.sa_flags = SA_SIGINFO | SA_ONSTACK; + sigemptyset(&sa.sa_mask); + if (sigaction(SIGSEGV, &sa, NULL) < 0) { + perror("sigaction SIGSEGV"); + return 1; + } + + if (sigsetjmp(stale_back, 1) == 0) + raise(SIGUSR1); + + if (!stale_faulted) { + fprintf(stderr, "FAIL stale park: the second pass never faulted\n"); + return 1; + } + if (stale_pc != (uint64_t) (uintptr_t) stale_stub) { + fprintf(stderr, + "FAIL stale park: faulted at 0x%llx, want the stub at " + "0x%llx\n", + (unsigned long long) stale_pc, + (unsigned long long) (uintptr_t) stale_stub); + failures++; + } + if (stale_x8 != STALE_LIVE) { + fprintf(stderr, + "FAIL stale park: the fault reports X8 = 0x%llx, want the live " + "0x%x; 0x%x is the value the earlier rt_sigreturn returned " + "with\n", + (unsigned long long) stale_x8, STALE_LIVE, STALE_PARKED); + failures++; + } + return failures; +} + +/* The same drop tail is reached from the W^X permission faults, not only from a + * syscall: the shim asks the host to flip a page and branches to the tail when + * the host answers with the marker, which it does when the region never had the + * permission and the fault is a real SIGSEGV. Nothing publishes an X8 on that + * route, so what the tail reloads is the value the exception was taken with, + * and that reload is the last write to X8 before EL0: signal delivery installs + * the handler's PC, SP, LR and argument registers, and X8 is none of those. X8 + * at handler entry is therefore the guest's own on both routes, which is what + * these two rounds assert; before the reload existed they both entered with the + * marker. + * + * Recorded in assembly at the handler's first instruction. The ucontext cannot + * answer this: the frame the host builds for these two deliveries records the + * shim's HVC arguments in X0 and X1 rather than the guest's, so reading X8 out + * of it would be reading a snapshot taken on the same terms. + * + * The value is spelled twice, here and in the two stubs below, which load it + * from assembly the way stale_stub does. + */ +#define WX_X8 0xa5 + +/* Written by the handler stub, read by the checker. Not static: the stub + * addresses it by name through adrp/add. + */ +volatile uint64_t wx_entry_x8; + +void wx_entry(int sig, siginfo_t *info, void *ctx); +void wx_entry_c(int sig, siginfo_t *info, void *ctx); +void wx_exec(uint64_t page); +void wx_write(uint64_t page); + +__asm__( + ".text\n" + ".globl wx_entry\n" + ".type wx_entry, %function\n" + "wx_entry:\n" + " adrp x9, wx_entry_x8\n" + " add x9, x9, :lo12:wx_entry_x8\n" + " str x8, [x9]\n" + " b wx_entry_c\n" + ".size wx_entry, .-wx_entry\n" + ".globl wx_exec\n" + ".type wx_exec, %function\n" + "wx_exec:\n" + " mov x8, #0xa5\n" + " br x0\n" + ".size wx_exec, .-wx_exec\n" + ".globl wx_write\n" + ".type wx_write, %function\n" + "wx_write:\n" + " mov x8, #0xa5\n" + " str xzr, [x0]\n" + " ret\n" + ".size wx_write, .-wx_write\n"); + +static sigjmp_buf wx_back; + +void wx_entry_c(int sig, siginfo_t *info, void *ctx) +{ + (void) sig; + (void) info; + (void) ctx; + siglongjmp(wx_back, 1); +} + +/* The X8 the handler entered with, through *taken. A stub that returns instead + * of faulting leaves it false, which is a host that granted the permission the + * page was not mapped with rather than an X8 worth reporting. + */ +static uint64_t wx_take_fault(void *page, int write, int *taken) +{ + wx_entry_x8 = 0; + *taken = 1; + if (sigsetjmp(wx_back, 1) == 0) { + if (write) + wx_write((uint64_t) (uintptr_t) page); + else + wx_exec((uint64_t) (uintptr_t) page); + *taken = 0; + return 0; + } + return wx_entry_x8; +} + +static int check_wx_tails(void) +{ + struct sigaction sa; + int failures = 0; + uint32_t ret_insn = 0xd65f03c0; /* ret, so a page that is executable runs */ + + /* A page with no PROT_EXEC, branched to: an instruction permission fault. A + * page with no PROT_WRITE, stored to: a write permission fault. + */ + uint8_t *noexec = mmap(NULL, 4096, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + uint8_t *nowrite = mmap(NULL, 4096, PROT_READ | PROT_EXEC, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (noexec == MAP_FAILED || nowrite == MAP_FAILED) { + perror("mmap"); + return 1; + } + memcpy(noexec, &ret_insn, sizeof(ret_insn)); + + /* Both faults are taken outside any handler and neither faulting stub has + * touched the stack, so the handler runs on the interrupted stack and needs + * no alternate one. check_stale_park, which does need one, runs after this + * and installs it then. + */ + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = wx_entry; + sa.sa_flags = SA_SIGINFO; + sigemptyset(&sa.sa_mask); + if (sigaction(SIGSEGV, &sa, NULL) < 0) { + perror("sigaction SIGSEGV"); + return 1; + } + + int taken = 0; + uint64_t got = wx_take_fault(noexec, 0, &taken); + if (!taken) { + fprintf(stderr, + "FAIL W^X exec tail: branching to a page with no PROT_EXEC " + "did not fault\n"); + failures++; + } else if (got != WX_X8) { + fprintf(stderr, + "FAIL W^X exec tail: the handler entered with X8 = 0x%llx, " + "want the guest's 0x%x\n", + (unsigned long long) got, WX_X8); + failures++; + } + + got = wx_take_fault(nowrite, 1, &taken); + if (!taken) { + fprintf(stderr, + "FAIL W^X write tail: storing to a page with no PROT_WRITE " + "did not fault\n"); + failures++; + } else if (got != WX_X8) { + fprintf(stderr, + "FAIL W^X write tail: the handler entered with X8 = 0x%llx, " + "want the guest's 0x%x\n", + (unsigned long long) got, WX_X8); + failures++; + } + return failures; +} + +/* BRK reaches EL0 through a tail of its own, not exec_drop_frame's, and has the + * same problem to solve there. The host delivers SIGTRAP out of HVC #10 and + * leaves the drop-frame marker in X8 as it does on every other delivery, so + * without a reload the handler enters with 2 in place of the guest's X8. A JIT + * translator patching through BRK trampolines is precisely the caller that + * reads that register, which is why the tail loads it back from the frame's own + * X8 slot before dropping the frame. + * + * Both halves are asserted. Handler entry is recorded in assembly, for the + * reason the W^X phase gives: the ucontext is a snapshot the host took on its + * own terms. The resumed value is read after the handler steps the saved PC + * past the BRK and returns, which covers the rt_sigreturn out of a BRK- + * delivered signal, a different tail from the one the entry tests. + * + * The value is spelled twice, here and in brk_fire below, which loads it from + * assembly the way the W^X stubs do. + */ +#define BRK_X8 0xc3 + +/* Written by the handler stub and by brk_fire, read by the checker. Not static: + * both address them by name through adrp/add. + */ +volatile uint64_t brk_entry_x8; +volatile uint64_t brk_resumed_x8; + +void brk_entry(int sig, siginfo_t *info, void *ctx); +void brk_entry_c(int sig, siginfo_t *info, void *ctx); +void brk_fire(void); + +__asm__( + ".text\n" + ".globl brk_entry\n" + ".type brk_entry, %function\n" + "brk_entry:\n" + " adrp x9, brk_entry_x8\n" + " add x9, x9, :lo12:brk_entry_x8\n" + " str x8, [x9]\n" + " b brk_entry_c\n" + ".size brk_entry, .-brk_entry\n" + ".globl brk_fire\n" + ".type brk_fire, %function\n" + "brk_fire:\n" + " mov x8, #0xc3\n" + " brk #0\n" + " adrp x9, brk_resumed_x8\n" + " add x9, x9, :lo12:brk_resumed_x8\n" + " str x8, [x9]\n" + " ret\n" + ".size brk_fire, .-brk_fire\n"); + +void brk_entry_c(int sig, siginfo_t *info, void *ctx) +{ + (void) sig; + (void) info; + ucontext_t *uc = ctx; + + /* Step the saved PC past the BRK so the return lands on the store below it + * rather than re-executing the trap. Nothing else in the frame is touched, + * so the X8 the return delivers is the one the frame recorded. + */ + uc->uc_mcontext.pc += 4; +} + +static int check_brk_tail(void) +{ + struct sigaction sa; + int failures = 0; + + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = brk_entry; + sa.sa_flags = SA_SIGINFO; + sigemptyset(&sa.sa_mask); + if (sigaction(SIGTRAP, &sa, NULL) < 0) { + perror("sigaction SIGTRAP"); + return 1; + } + + brk_entry_x8 = 0; + brk_resumed_x8 = 0; + brk_fire(); + + if (brk_entry_x8 != BRK_X8) { + fprintf(stderr, + "FAIL BRK tail: the handler entered with X8 = 0x%llx, want " + "the guest's 0x%x\n", + (unsigned long long) brk_entry_x8, BRK_X8); + failures++; + } + if (brk_resumed_x8 != BRK_X8) { + fprintf(stderr, + "FAIL BRK tail: the resumed guest carries X8 = 0x%llx, want " + "the guest's 0x%x\n", + (unsigned long long) brk_resumed_x8, BRK_X8); + failures++; + } + return failures; +} + +int main(void) +{ + struct sigaction sa; + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = handler; + sa.sa_flags = SA_SIGINFO; + sigemptyset(&sa.sa_mask); + sigaddset(&sa.sa_mask, SIGUSR2); + if (sigaction(SIGUSR1, &sa, NULL) < 0) { + perror("sigaction SIGUSR1"); + return 1; + } + + struct sigaction sa2; + memset(&sa2, 0, sizeof(sa2)); + sa2.sa_handler = nested_handler; + sigemptyset(&sa2.sa_mask); + if (sigaction(SIGUSR2, &sa2, NULL) < 0) { + perror("sigaction SIGUSR2"); + return 1; + } + + int64_t want_pid = (int64_t) getpid(); + + for (int round = 0; round < ROUNDS * 2; round++) { + resumed_x8 = 0; + resumed_rc = 0; + nest = round & 1; + nested = 0; + + if (sigsetjmp(resume, 1) == 0) { + raise(SIGUSR1); + fprintf(stderr, "FAIL round %d: handler did not redirect\n", round); + failures++; + continue; + } + + if (resumed_x8 != NR_GETPID) { + fprintf(stderr, + "FAIL round %d: X8 after rt_sigreturn is %llu, " + "want %d\n", + round, (unsigned long long) resumed_x8, NR_GETPID); + failures++; + } + if (resumed_rc != want_pid) { + fprintf(stderr, + "FAIL round %d: the resumed SVC returned %lld, " + "want the pid %lld\n", + round, (long long) resumed_rc, (long long) want_pid); + failures++; + } + if (nest && !nested) { + fprintf(stderr, "FAIL round %d: the nested signal never ran\n", + round); + failures++; + } + } + + failures += check_wx_tails(); + failures += check_brk_tail(); + failures += check_stale_park(); + + if (!failures) + printf( + "PASS: %d rt_sigreturn rounds kept X8, a later fault on the same " + "PC saw its own, and the W^X and BRK tails handed theirs to the " + "handler\n", + ROUNDS * 2); + printf("%d failed\n", failures); + return failures ? 1 : 0; +}