diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 359f619..e1c92d6 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,62 +1,55 @@ # Architecture This document describes pvKernel's current architecture as implemented in -version 0.0.8. It does not describe planned or aspirational architecture; see +version 0.0.9. It does not describe planned or aspirational architecture; see [ROADMAP.md](ROADMAP.md) for that. ## Components -``` +```text UEFI firmware (OVMF) - │ - ▼ -rust-osdev bootloader (external crate, not part of pvKernel) - │ loads the pvKernel ELF, exits UEFI boot services, - │ builds an initial page table mapping (incl. a physical-memory - │ offset mapping), and passes BootInfo - ▼ -pvKernel ELF (kernel_main) - │ - ├── drivers::serial – COM1 UART diagnostics (kprint!/kprintln!) - ├── arch::x86_64 – GDT, TSS, IDT, exception + IRQ handlers, PIC, PIT - ├── arch::x86_64::memory – physical frame allocator, active page table - │ access, map/unmap/permission operations - ├── arch::x86_64::syscall – provisional int 0x80 syscall ABI - ├── arch::x86_64::userspace – test-only CPL0 → CPL3 (iretq) entry - ├── graphics::framebuffer – centered blue "P" logo - └── halt – fatal halt vs. interrupt-enabled idle loop + | +rust-osdev bootloader + | upper-half dynamic mapping arena, BootInfo, framebuffer, memory map + v +pvKernel kernel_main + |-- drivers::serial, graphics::framebuffer, halt + |-- arch::x86_64 + | |-- GDT/TSS/RSP0, IDT, PIC/PIT, exception and timer entry + | |-- memory: physical allocator, audited kernel root, AddressSpace + | |-- syscall: experimental scalar int 0x80 ABI + |-- task: bounded scheduler, kernel/user Task contexts, SwitchPlan + |-- process: ProcessId/PID table, lifecycle, ELF64 loader, snapshots ``` - ## Responsibility boundary -The root `task` module owns the fixed task table and runnable ring. -`arch::x86_64::task` owns task stack mapping, the saved frame layout, and the -small timer-entry/trampoline assembly. Policy stays in Rust; CR3 remains -shared across all kernel tasks. +The root `task` module owns the bounded Task table, runnable ring, context +selection, and kernel/user execution-context state. `process` owns generation- +safe ProcessId identity, boot-lifetime PIDs, process lifecycle, AddressSpace +ownership, and process metadata. `arch::x86_64::memory` owns the kernel root, +inactive process-root construction, mapping audit, and CR3 activation. +`arch::x86_64::task` owns task-stack mapping, the saved frame layout, and the +timer-entry/trampoline assembly. Scheduler policy remains separate from +process policy, and the PIT hot path does not allocate or parse executable data. * **UEFI firmware**: powers on the machine, initializes basic platform devices, and runs the boot manager. -* **rust-osdev bootloader**: locates and loads the pvKernel ELF binary, - establishes an initial 4-level page table mapping and stack, collects the - boot memory map and framebuffer information into `BootInfo`, and transfers - control to `kernel_main`. pvKernel does not currently own or replace this - page table mapping. -* **pvKernel**: everything from `kernel_main` onward. This includes serial - diagnostics, the GDT/TSS, the IDT and its exception/IRQ handlers, the - legacy 8259 PIC and PIT drivers, the physical frame allocator and active - page-table access, a provisional `int 0x80` syscall ABI, and (test-only) - CPL0 → CPL3 userspace entry, and framebuffer rendering. -* **Kernel tasks**: 0.0.8 adds bounded, preemptive, single-core ring-0 tasks - with separate guarded stacks. It does not schedule CPL3 payloads or switch - CR3; see [docs/architecture/kernel-tasks.md](docs/architecture/kernel-tasks.md). -* **Future pvOS userspace**: 0.0.6 adds only a minimal, test-only CPL0 → - CPL3 transition and bootstrap syscall ABI (see - [docs/architecture/userspace-entry.md](docs/architecture/userspace-entry.md) - and [docs/architecture/syscalls.md](docs/architecture/syscalls.md)); a - process model and ELF userspace loading remain future roadmap items (see - [ROADMAP.md](ROADMAP.md)); pvKernel does not - currently support them. - +* **rust-osdev bootloader**: loads the pvKernel ELF, exits UEFI boot services, + creates the initial 4-level mappings and stack, and passes `BootInfo`. + pvKernel audits required mappings and constrains dynamic boot mappings to + the upper canonical half before it can activate process CR3 roots. +* **Kernel Tasks**: bounded preemptive single-core ring-0 contexts with + dedicated supervisor-only guarded stacks. +* **Process Tasks**: one CPL3 Task per process in 0.0.9, each with an isolated + lower-half address space, dedicated USER RW/NX stack and supervisor kernel + entry stack. Timer IRQs can preempt CPL3 execution and switch CR3/RSP0. +* **Userspace ABI**: experimental `int 0x80` scalar `getpid` and terminal + `exit`. The checked static ELF64 loader accepts only validated `ET_EXEC` + `PT_LOAD` images. CPL3 #PF/#GP/#UD terminate the affected process and retain + scalar fault metadata. No stable ABI is promised before 0.1.0. +* **Future work**: VFS/initrd, `/bin/init`, FDs, fork/general exec, dynamic + linking, user heap/mmap, signals, IPC, general frame reclamation, storage, + networking, SMP, and GUI remain out of scope. ## Initialization order 1. `kernel_main` receives `BootInfo` from the bootloader. @@ -72,9 +65,10 @@ shared across all kernel tasks. (`x86_64::instructions::interrupts::disable()`). 2. The kernel-owned GDT and TSS are built and loaded (`arch::x86_64::gdt`), including the dedicated double-fault IST stack, - ring-3 (CPL3) code/data segment descriptors, and a dedicated - TSS.RSP0 ring-transition kernel stack (see - [docs/architecture/userspace-entry.md](docs/architecture/userspace-entry.md)). + ring-3 (CPL3) code/data segment descriptors, and a kernel-lifetime + fallback TSS.RSP0 stack. User Task kernel-stack tops are installed + before returning to CPL3 (see + [docs/architecture/processes.md](docs/architecture/processes.md)). 3. The kernel-owned IDT is built and loaded (`arch::x86_64::interrupts`), including handlers for all 16 remapped IRQ vectors (32-47) and the `int 0x80` syscall gate (see @@ -121,13 +115,17 @@ shared across all kernel tasks. 12. If `supervisor-write-protection-test` is enabled, a CPL0 write to a supervisor-only read-only page verifies CR0.WP behavior through an expected protection-violation page fault. -13. If exactly one `userspace-*-test` Cargo feature is enabled, a hand-written - test payload is mapped into the reserved userspace range and entered at - CPL3 via `iretq`; the scenario ends deterministically (see - [docs/development/userspace-testing.md](docs/development/userspace-testing.md)). -14. The timer entry saves the complete interrupt context, schedules CPL0 - contexts only, and resumes bootstrap after both smoke tasks exit. -15. The CPU enters an interrupt-enabled idle loop (`halt::idle_loop`), unless +13. The process subsystem audits required boot mappings, initializes its + bounded table, and can construct inactive process address spaces while + preserving audited supervisor-only kernel mappings. +14. Process-owned tasks use checked ELF loading, dedicated user and kernel + stacks, and timer-driven CPL3 preemption. On ordinary boots an embedded + static smoke ELF exercises CPL3 entry, `getpid`, and terminal `exit`; + QEMU features select deterministic process and legacy boundary scenarios. +15. The timer entry saves the complete interrupt context and can schedule + both CPL0 kernel tasks and process-owned CPL3 tasks, switching CR3 and + TSS.RSP0 according to the selected context. +16. The CPU enters an interrupt-enabled idle loop (`halt::idle_loop`), unless the `qemu-test` Cargo feature is enabled, in which case QEMU exits deterministically instead (see [docs/development/testing.md](docs/development/testing.md)). @@ -152,10 +150,11 @@ items. * a TSS descriptor, wrapping a kernel-owned `TaskStateSegment`; * a dedicated double-fault stack (5 × 4 KiB), referenced by IST index `DOUBLE_FAULT_IST_INDEX`; -* ring-3 (CPL3) code and data segment descriptors, and a dedicated, - writable TSS.RSP0 ring-transition kernel stack (4 × 4 KiB), used only - when entering CPL3 (see - [docs/architecture/userspace-entry.md](docs/architecture/userspace-entry.md)). +* ring-3 (CPL3) code and data segment descriptors, and a kernel-lifetime + fallback RSP0 stack for bootstrap/kernel execution. Before returning to + CPL3, the scheduler installs the selected user Task's guarded supervisor + kernel-stack top in TSS.RSP0 (see + [docs/architecture/processes.md](docs/architecture/processes.md)). The GDT and TSS are stored in `spin::Once` cells with `'static` storage duration, loaded once, and never replaced during the kernel's lifetime. @@ -173,9 +172,9 @@ handler-level detail. * `BootFrameAllocator` — a monotonic physical frame allocator built once from the bootloader's memory map; -* `KernelAddressSpace` — the single active x86_64 page table (an - `OffsetPageTable` over the CR3-resident level-4 table, reached through the - bootloader's physical-memory offset mapping), paired with the frame +* the active kernel root (`KernelAddressSpace`) and process-owned inactive + `AddressSpace` roots. Page-table frames are reached through the audited + bootloader physical-memory offset mapping and paired with the frame allocator. Both are stored together behind a `spin::Mutex`, itself behind a @@ -202,22 +201,24 @@ the heap for the kernel lifetime and are not reclaimed by `Box`, `Vec`, or [docs/architecture/virtual-memory.md](docs/architecture/virtual-memory.md) for the layout and safety invariants. -## Syscall and userspace entry ownership (0.0.6) +## Syscall and userspace entry ownership (0.0.9) -`arch::x86_64::syscall` owns a minimal, provisional, scalar-only `int 0x80` -syscall ABI: a naked-asm entry stub installed into the IDT's vector `0x80`, -a `SyscallFrame` register-frame layout built on the RSP0 stack, and a -Rust-side dispatcher. `arch::x86_64::userspace` owns the (test-only, behind -Cargo features) code that maps a hand-written payload into the reserved -userspace virtual-address range and transitions into it via `iretq`. See +`arch::x86_64::syscall` owns the experimental scalar `int 0x80` ABI: a +naked-asm entry stub at vector `0x80`, the `SyscallFrame` saved on the +selected user's supervisor RSP0 stack, and a Rust dispatcher for `getpid` +and terminal `exit`. The separate legacy `arch::x86_64::userspace` module +retains bootstrap CPL3 boundary-test payloads. Process-owned user Tasks use +the normal checked ELF loader and synthetic scheduler context; ordinary +boots run an embedded static smoke executable until VFS/initrd is available. +See [docs/architecture/syscalls.md](docs/architecture/syscalls.md) and -[docs/architecture/userspace-entry.md](docs/architecture/userspace-entry.md) -for the full design. +[docs/architecture/processes.md](docs/architecture/processes.md) for the +current details. ## Fatal exception behavior -Breakpoint exceptions log the event and return normally. Invalid opcode, -general protection fault, page fault, and double fault are treated as fatal: +Breakpoint exceptions log the event and return normally. CPL0 invalid opcode, +general protection fault, page fault, and double fault remain kernel-fatal: they log structured diagnostics (exception name, error code where relevant, faulting address for page faults, the CPL the fault originated from, and the interrupt stack frame) and then call `halt::halt_forever` directly (or, @@ -225,6 +226,9 @@ under the `qemu-test` feature, exit QEMU with a failure status instead), without invoking `panic!`. The exceptions are deliberately triggered faults under the `fault-tests`, `userspace-privilege-fault-test`, and `userspace-supervisor-fault-test` features: when one matches its expected -signature, it is treated as a passed test rather than a fatal condition (see +signature, it is treated as a passed test rather than a fatal condition. +CPL3 #UD/#GP/#PF from a process-owned Task instead capture scalar fault +metadata and use the terminal scheduler handoff to terminate that process; +the kernel remains active. See [docs/development/testing.md](docs/development/testing.md) and -[docs/development/userspace-testing.md](docs/development/userspace-testing.md)). +[docs/development/userspace-testing.md](docs/development/userspace-testing.md). diff --git a/CHANGELOG.md b/CHANGELOG.md index 98ac061..1c4a58d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,36 @@ # Changelog +## [0.0.9] - Unreleased + +### Added + +* Generation-safe `ProcessId` handles distinct from monotonic boot-lifetime + PIDs, a bounded process table, checked parent references, and a Creating / + Alive / Exiting / Zombie lifecycle with internal reap. +* Per-process four-level address spaces built from audited upper-half kernel + mappings, inactive-root construction, and CR3-aware task switching. +* Dedicated 32 KiB user stacks with unmapped guards, supervisor-only guarded + kernel entry stacks per user Task, and checked TSS.RSP0 updates. +* True PIT preemption while CPL3 processes run, with a fixed scheduler/task + capacity and explicit terminal exit/fault handoffs. +* A checked static ELF64 little-endian x86_64 `ET_EXEC` loader for `PT_LOAD` + segments, with W^X enforcement and zeroed BSS/new user pages. +* Experimental `getpid` and non-returning `exit` syscalls plus central, + page-chunked `copy_from_user`/`copy_to_user` APIs. +* CPL3 #PF/#GP/#UD/#NM containment with scalar fault metadata, deferred Zombie + completion after stack handoff, process CPU/page accounting, and a bounded + process snapshot API. +* Embedded static userspace smoke execution on ordinary boots and deterministic + QEMU scenarios for process preemption, same-VA isolation, capacity, exit, + re-use, ELF validation, user-copy, guard faults, and fault containment. + +Until per-task floating-point state is implemented, x87/legacy SIMD and AVX +are disabled, and inherited SYSCALL/SYSENTER entry targets are cleared. This +remains an experimental single-core userspace foundation. VFS, initrd, +`/bin/init`, file descriptors, fork/general exec, dynamic linking, signals, +IPC, user heap/mmap, general physical-frame reclamation, storage, networking, +SMP and GUI support are not included. + ## [0.0.8] - 2026-10-24 ### Added diff --git a/Cargo.lock b/Cargo.lock index e642fa9..4dac244 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -340,7 +340,7 @@ dependencies = [ [[package]] name = "kernel" -version = "0.0.8" +version = "0.0.9" dependencies = [ "bootloader_api", "spin", @@ -453,7 +453,7 @@ dependencies = [ [[package]] name = "pvkernel-runner" -version = "0.0.8" +version = "0.0.9" dependencies = [ "bootloader", "kernel", diff --git a/Cargo.toml b/Cargo.toml index 417a13a..39b9b30 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -9,7 +9,7 @@ members = ["kernel"] # docs/security/vulnerability-management.md and the release workflows for # why that consistency matters at release time. [workspace.package] -version = "0.0.8" +version = "0.0.9" license = "MIT OR Apache-2.0" edition = "2024" repository = "https://github.com/ultronstudio/pvkernel" @@ -56,6 +56,18 @@ heap-oom-test = ["kernel/heap-oom-test"] heap-guard-fault-test = ["kernel/heap-guard-fault-test"] scheduler-test = ["kernel/scheduler-test"] scheduler-preemption-test = ["kernel/scheduler-preemption-test"] +process-preemption-test = ["kernel/process-preemption-test"] +process-smoke-test = ["kernel/process-smoke-test"] +process-exit-test = ["kernel/process-exit-test"] +process-id-reuse-test = ["kernel/process-id-reuse-test"] +user-copy-test = ["kernel/user-copy-test"] +user-fault-containment-test = ["kernel/user-fault-containment-test"] +address-space-isolation-test = ["kernel/address-space-isolation-test"] +process-capacity-test = ["kernel/process-capacity-test"] +user-stack-guard-fault-test = ["kernel/user-stack-guard-fault-test"] +user-kernel-isolation-test = ["kernel/user-kernel-isolation-test"] +elf-loader-test = ["kernel/elf-loader-test"] +elf-malformed-test = ["kernel/elf-malformed-test"] task-stack-guard-fault-test = ["kernel/task-stack-guard-fault-test"] task-capacity-test = ["kernel/task-capacity-test"] diff --git a/README.md b/README.md index 0e01682..0ff2db7 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ written in Rust and developed as the foundation of pvOS. ## Current milestone -Version: `0.0.8` — Scheduler + Kernel Tasks +Version: `0.0.9` — Processes + Address Spaces + Userspace Foundation Implemented: @@ -42,10 +42,10 @@ Implemented: * required NXE and CR0.WP paging invariants, plus a deterministic CPL0 supervisor write-protection QEMU test * deterministic, headless QEMU tests behind the `qemu-test` Cargo feature -* dedicated ring-3 (CPL3) code/data GDT descriptors and a dedicated - TSS.RSP0 ring-transition kernel stack -* the first CPL0 → CPL3 transition, via `iretq` (test-only, behind Cargo - features) +* dedicated ring-3 (CPL3) code/data GDT descriptors, a kernel-lifetime RSP0 + fallback, and per-user-task supervisor kernel-entry stacks +* validated synthetic CPL3 context entry via `iretq`, including both legacy + boundary tests and process-owned userspace Tasks * a minimal, provisional, scalar-only `int 0x80` syscall ABI * `USER_DATA`/`USER_CODE` page permissions, reusing the existing W^X mapping API @@ -64,9 +64,22 @@ Implemented: * eight centrally reserved 16 KiB kernel task stacks, supervisor-only RW/NX, with unmapped guard pages above and below * dedicated timer IRQ context switching with complete general-purpose - register and interrupt-frame preservation; CPL3 timer IRQs resume in place + register and interrupt-frame preservation; process-owned CPL3 frames are + preempted while legacy bootstrap CPL3 tests keep their boundary-test path * deterministic QEMU scheduler, CPU-bound preemption, stack-guard, capacity, and stale-ID/reuse coverage +* generation-safe ProcessId handles, monotonic PIDs, and a bounded process + lifecycle with Zombie/reap semantics +* audited per-process address spaces, inactive-root construction, CR3 switching, + and same-VA process isolation +* dedicated user stacks and per-task supervisor kernel-entry stacks, with + TSS.RSP0 updates and CPL3 timer preemption +* checked static ELF64 loading with W^X, zeroed BSS/pages, and an embedded + userspace smoke executable on normal boot +* provisional `getpid`/`exit`, checked user-copy APIs, and process-level + #PF/#GP/#UD containment +* deterministic QEMU process, isolation, capacity, ELF, user-copy, and fault + containment scenarios * structured serial diagnostics * panic handler * an interrupt-enabled idle loop (plus a separate fatal halt path) @@ -75,23 +88,24 @@ Not implemented: * APIC initialization * keyboard or mouse input -* general physical-frame deallocation/reuse (heap backing frames stay owned - by the heap for the kernel lifetime) +* general physical-frame deallocation/reuse (heap and process backing frames + remain kernel-lifetime owned) * dynamic heap growth, per-process heaps, or a slab allocator * dynamic task stack growth, task join, blocking, sleep, or priorities * APIC, SMP, and multicore scheduling -* userspace process scheduling, processes, or per-process address spaces -* `SYSCALL`/`SYSRET`, a process model, or general (non-test) userspace - execution -* ELF userspace loading -* `copy_from_user`/`copy_to_user` or any user-pointer dereferencing -* filesystems -* networking +* VFS, initrd, filesystem, `/bin/init`, or file descriptors +* fork, general exec/spawn syscall, dynamic linking, shared libraries, PIE, or + relocations +* userspace thread API, signals, pipes, PTY, shared memory, or general IPC +* user heap, `brk`, `mmap`, demand paging, copy-on-write, swap, or general + physical-frame reclamation +* APIC/SMP/multicore scheduling, storage, networking, users/groups, + capabilities, or GUI/desktop * device drivers beyond serial, framebuffer, PIC, and PIT -pvKernel does not claim production readiness, general hardware support, -process isolation, a stable ABI, or readiness as a reusable kernel or pvOS -foundation. +pvKernel's 0.0.9 process and syscall interfaces are experimental. It does not +claim production readiness, general hardware support, a stable userspace ABI, +or readiness as a general-purpose kernel. ## Build @@ -164,9 +178,11 @@ kernel_main ├── kernel heap mappings + global allocator ├── PIC remap + PIT configuration ├── memory-mapping smoke test + ├── scheduler and process tables + ├── audited address spaces and userspace smoke ELF ├── framebuffer ├── controlled breakpoint test - ├── timer interrupt verification + ├── timer interrupt verification and process/task preemption └── interrupt-enabled idle loop ``` diff --git a/ROADMAP.md b/ROADMAP.md index a348533..336dfcc 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -108,13 +108,22 @@ remain future work. * QEMU coverage for scheduler operation, true timer preemption, stack guards, and capacity/reuse -## 0.0.9 — Processes and Userspace Address Spaces - -* process model and process lifecycle foundation -* per-process address spaces and CR3 switching -* userspace ELF execution foundation -* builds on the existing provisional CPL3 entry/syscall boundary; does not - imply a stable userspace ABI +## 0.0.9 — Processes and Userspace Address Spaces (implemented; unreleased) + +* generation-safe ProcessId handles, monotonic PIDs, bounded process table, + parent identity, Zombie/reap lifecycle, and process snapshots +* audited per-process address spaces and CR3 switching while preserving + supervisor-only kernel-global mappings +* dedicated user and kernel-entry stacks, TSS.RSP0 switching, and true PIT + preemption from CPL3 +* checked static ELF64 `ET_EXEC` loading with W^X, BSS/page zeroing, and an + embedded smoke ELF before VFS/initrd exists +* experimental `getpid`/`exit`, central checked user-copy, and CPL3 + #PF/#GP/#UD containment +* deterministic QEMU tests for process preemption, address-space isolation, + capacity/reuse, ELF validation, user-copy, guard faults, and containment +* no stable userspace ABI is implied; first meaningful compatibility + checkpoint remains 0.1.0 ## 0.0.10 — VFS and Initial Ramdisk diff --git a/docs/architecture/processes.md b/docs/architecture/processes.md index 7e74752..2f93336 100644 --- a/docs/architecture/processes.md +++ b/docs/architecture/processes.md @@ -1,8 +1,29 @@ # pvKernel 0.0.9 process architecture -**Status:** design for human review; no process implementation is present in -pvKernel 0.0.8. This document describes the stable repository at -`2e99b6a16e3c3bec7fdde88ae580c1d76cba8e56`. +**Status:** authoritative 0.0.9 design and current implementation model on +the unmerged implementation branch; 0.0.9 is not released. The stable- +baseline sections describe pvKernel 0.0.8 at +`2e99b6a16e3c3bec7fdde88ae580c1d76cba8e56`. Phase A process identity, +Phase B boot-mapping audit/inactive-root construction, Phase C user-stack/ +RSP0 support, Phase D checked static ELF loading, and the Phase E prepared +user-task/process-resource model are implemented on the 0.0.9 branch. Phase F +has an audited CR3 activation primitive, a Rust scheduler SwitchPlan, and a +QEMU runtime root-switch/RSP0 smoke. Phase G now schedules process-owned CPL3 +timer frames and has a deterministic two-process preemption test; legacy +direct-from-bootstrap CPL3 boundary tests retain their non-scheduling path. +Phase H adds process-aware `getpid` and terminal `exit`, with deferred Zombie +publication after the outgoing kernel stack becomes inactive; an embedded +ELF smoke checks PID identity and non-returning exit. Phase I adds central +page-chunked `copy_from_user`/`copy_to_user` APIs with full-range preflight; +the dedicated QEMU test covers same/cross-page copies and hostile ranges. +Phase J routes CPL3 #PF/#GP/#UD/#NM from process Tasks into the terminal exit +handoff and defers Zombie publication until a later PIT runs on an adopted +stack; its QEMU scenario observes #PF/#GP/#UD/#NM, stack-guard and supervisor- +mapping faults, then reaps their Zombies. Phase K adds saturating PIT-based +process CPU accounting, a fixed-size `snapshot_all` API, real same-VA/ +different-frame user-process switching, and mixed scheduler/process capacity +validation. Ordinary boots now run an embedded static userspace smoke ELF +through the same process and loader path. ## 1. Scope and goals @@ -11,7 +32,7 @@ isolated userspace processes. Its central rule is: **Process** owns an address space and process-wide resources; **Task** is a schedulable execution context. The first implementation may have one user task per process, while leaving process identity independent of that initial one-to-one relationship. -This is design only; it does not change v0.0.8 or implement any feature below. +The detailed decisions below remain the design contract for the implementation. ## 2. Current 0.0.8 baseline @@ -86,22 +107,37 @@ interrupt entry uses the current RSP, not RSP0. Add a narrow checked `set_current_rsp0(top)` operation later. Immediately before every `iretq` that returns to CPL3, RSP0 must contain the validated kernel-stack top of that exact userspace Task. Do not treat RSP0 as the current kernel task's -stack pointer. When switching from a user Task to a kernel Task/bootstrap, restore the safe fallback before the outgoing user stack can become reusable. +Implementation note: the TSS now has a single-core `UnsafeCell` wrapper for +RSP0 updates. `set_current_rsp0` rejects calls while IF is set and accepts +only an aligned top matching an initialized guarded task-stack slot. +`set_kernel_fallback_rsp0` restores the static syscall stack. QEMU verifies +both programmed values. The setter does not select tasks or change the +context-switch path. + ## 6. Current paging, userspace and syscall facts `memory::init` rejects `CR4.LA57`, verifies NX/write-protect prerequisites, -reads active CR3 and builds one `OffsetPageTable` using the bootloader's -physical-memory offset. The active PML4 is reached at offset + CR3 physical -address. `KernelAddressSpace` and monotonic `BootFrameAllocator` live behind -one `Once>`. `map_new_page`, unmap, permission update and translate -operate on that active root only. Missing intermediate tables use the frame -allocator. Mapping/permission changes flush the changed active page; unmap -flushes but frames are not reusable. There is no inactive-root API, page -table ownership registry or recursive mapping. The bootloader's physical -mapping covers usable physical memory at a dynamically chosen offset and is -needed to access page-table frames. +reads active CR3 and builds the kernel `OffsetPageTable` using the +bootloader's physical-memory offset. `KernelAddressSpace` and monotonic +`BootFrameAllocator` live behind one `Once>`. The runtime boot audit +checks the dynamic mapping inventory, kernel ELF `PT_LOAD` permissions, and +every present upper-half page-table entry for USER clear. `create_address_space` +allocates a zeroed inactive root, copies only audited PML4 entries 256..511, +and never copies lower-half entries. Inactive `OffsetPageTable` operations +allocate intermediate tables through the boot allocator and never switch +CR3. `AddressSpace` maps zeroed user frames only in the accepted lower-half +range and tracks mapped-user-page count; page-table leaves are the mapping +inventory. Frames remain monotonic and are not reclaimed. The bootloader's +physical map is used to reach inactive page-table frames. + +QEMU's address-space smoke creates two roots, maps the same user VA to distinct +frames, writes distinct values, and verifies those values through the +supervisor physical map. It also checks that a kernel task-stack VA translates +to the same physical frame in both roots with USER clear, and that low and +upper-bound user pages remain unmapped. This validates root construction, not +CR3 switching or scheduled userspace execution. The active root starts with bootloader mappings. Kernel image, boot stack, boot info, framebuffer and physical-memory mapping use dynamic placement; @@ -118,6 +154,14 @@ GPRs on RSP0, calls the scalar dispatcher, writes its return to saved RAX, restores registers and `iretq`s. As an interrupt gate, it clears IF; PIT cannot preempt a syscall body. Keep this non-preemptible syscall policy. +The 0.0.9 context frame does not save x87, MMX, SSE, or AVX state. Before +userspace runs, the kernel therefore sets CR0.EM, clears CR4.OSXSAVE and +EFER.SCE, and zeros IA32_SYSENTER_CS/ESP/EIP. This contains x87/legacy-SIMD +and AVX use as process faults and prevents inherited firmware SYSCALL or +SYSENTER targets from bypassing the `int 0x80` boundary. Saving floating-point +state and enabling those instruction families requires a later explicit +per-task state design. + ## 7. Current virtual-address map Four-level canonical ranges are lower `0x0000_0000_0000_0000` through @@ -127,9 +171,11 @@ as constants from current source. | Region | Current address/range | Notes | |---|---|---| -| Kernel image, boot stack, boot info | Bootloader-selected dynamic mappings | Exact ranges not retained in kernel metadata | -| Physical-memory map | BootInfo-provided dynamic base; maps usable physical memory | Required by OffsetPageTable; varies by boot | -| Framebuffer | Bootloader-selected dynamic mapping | Dimensions retained; mapping range not tracked | +| Kernel image | Bootloader dynamic arena; current QEMU base `0xffff800000000000` (PML4 256) | ELF `PT_LOAD` pages are audited from the loaded kernel ELF; `BootInfo.kernel_len` is the ELF backing length, not a contiguous virtual extent | +| Bootloader kernel stack | Current QEMU `0xffff808000001000..0xffff808000015000` (PML4 257) | Guard page immediately below; runtime range is audited | +| Framebuffer | Current QEMU `0xffff810000000000..0xffff8100003e8000` (PML4 258) | Retained by `kernel_main`; exact byte range is registered and audited | +| Physical-memory map | Current QEMU `0xffff818000000000..0xffff828000000000` (PML4 259–260) | 1 TiB physical-address coverage in this QEMU memory map, including address holes/MMIO; not 1 TiB of RAM | +| BootInfo + memory-region slice | Current QEMU `0xffff828000000000..0xffff828000000bb0` (PML4 261) | The allocator retains the memory-region slice; both ranges are audited | | Scratch | `0xffff_9000_0000_0000` | Test-only; checked unmapped before use | | Kernel heap | `0xffff_9000_1000_0000..0xffff_9000_1010_0000` | 1 MiB supervisor RW/NX; guard pages at `...0ffff000` and `...10100000` | | Kernel task stacks | `0xffff_9000_2000_0000..0xffff_9000_2003_0000` | 8 x 16 KiB RW/NX supervisor stacks; one unmapped page on each side; 24 KiB stride | @@ -140,11 +186,15 @@ as constants from current source. | Other reservations | No global reservation registry | Collision checks are local to mapping callers | The explicit scratch, heap and task-stack ranges are all in PML4 index 288; -the current test user pages are in lower-half PML4 index 192. The dynamic -bootloader mappings' indices are not established by current code. - -No numeric address is claimed for dynamic bootloader mappings. A future -mapping audit must record actual entries before a process root is activated. +the current test user pages are in lower-half PML4 index 192. QEMU confirmed +that the bootloader's default dynamic mapping previously placed the kernel at +`0x0000_0100_0000_0000` (lower-half PML4 2). The implementation now configures +the exact bootloader 0.11.17 dynamic allocator to use a dedicated upper-half +arena, and runtime inspection confirms the required mappings in PML4 indices +256–261. Bootloader identity mappings in lower-half PML4 entries remain in the +kernel's initial root, but are not required after the kernel replaces the +bootloader GDT and begins using the audited upper-half code, stacks, and +physical-memory map; process roots do not copy them. ## 8. Proposed user virtual layout @@ -152,7 +202,9 @@ Use only the lower canonical half (`USER_MIN=0x0000_0000_0001_0000`, exclusive `USER_MAX=0x0000_8000_0000_0000`). Keep addresses 0 through 64 KiB unmapped, including null and low aliases. USER_MAX is an exclusive bound, not dereferenceable. All arithmetic and rounding is checked. This layout is -compatible with 48-bit canonical addressing and `LA57=0`. +compatible with 48-bit canonical addressing and `LA57=0`. All retained +bootloader mappings are now constrained to the upper half, so no lower-half +PML4 slot is reserved for kernel mappings. | Region | Proposed layout | |---|---| @@ -233,6 +285,15 @@ publication itself is an infallible state write after validation; if that assumption changes, add rollback while locks and interrupt masking still prevent observation. +Implementation note: each process slot now owns an optional `AddressSpace` +and primary `TaskId` while Creating. Scheduler slots distinguish Kernel and +User(ProcessId) tasks and include a non-queued `Prepared` state. Synthetic +user contexts are initialized with user selectors, checked entry/RSP bounds, +safe RFLAGS, and zero GPRs. QEMU verifies a Prepared task remains non-runnable +while its process is Creating, and that cancellation plus Creating rollback +leaves no schedulable task. Final publication will be exercised together +with CR3-aware scheduling in later phases. + ## 13. Lifecycle, Zombie and reap Legal lifecycle: `Vacant -> Creating -> Alive -> Exiting -> Zombie -> reap @@ -265,27 +326,18 @@ memory to report a fault. ## 15. Fault containment -Current #UD, #GP and #PF handlers log and invoke global fatal handling. -#GP/#PF report saved CS privilege; #PF reads CR2 and error bits. Expected -user-fault QEMU tests exit the test successfully, not contain a process. - -Later, classify by interrupted frame's saved CS RPL: CPL0 fault remains -kernel-fatal; CPL3 #PF/#GP/#UD records structured metadata and transitions -that process to Exiting. PF USER_MODE is a useful consistency check, not the -only classifier. #UD has no error code, #GP has one, and #PF also records -CR2. Existing Rust interrupt handlers return through the faulting frame or -halt the kernel. They cannot safely mark a process dead and return. Add a -common non-returning exception-to-scheduler handoff that captures the frame, -marks Process and Task Exiting, and releases all locks while IF remains -clear. Only after that state is safe may the terminal handoff deliberately -enable maskable timer progress (for this single-core PIT design, a loop -equivalent to the proven `sti; hlt` task-exit wait). The next timer interrupt -must save the exiting context and select away; it must never requeue or -return to the faulting RIP. Do not reuse the outgoing kernel stack while -the timer/assembly path is still executing on it. Record pending Zombie -completion at scheduling time, but finalize Zombie only from a post-stack- -adoption hook or continuation on the selected task's stack, after the old -stack is inactive. No process or scheduler lock crosses this handoff. +Exception classification uses saved CS RPL as the primary source. CPL0 +faults retain existing kernel-fatal behavior. CPL3 #UD/#GP/#PF from a +process-owned Running Task capture only vector, optional hardware error code, +CR2 for #PF, and saved RIP. #PF USER_MODE remains consistency information, +not the sole classifier. The handler calls the common non-returning terminal +handoff while IF is clear; it marks Process and Task Exiting and releases all +locks before the `sti; hlt`-equivalent wait permits PIT delivery. Timer saves +the CPL0 handoff context on that process's kernel stack, selects another +context, and never restores the faulting user frame. Zombie is finalized on a +later PIT after a different stack has been adopted. The QEMU scenario runs +separate #PF/#GP/#UD/#NM processes, checks their scalar metadata and Zombie +states, then reaps them while the kernel continues. ## 16. AddressSpace and frame ownership @@ -312,27 +364,55 @@ addresses. PIC/PIT are currently port-I/O devices and need no virtual mapping; future MMIO must be inventoried. All kernel leaves and parent entries must have USER clear. -The proposed split uses lower PML4 indices 0..255 for user and upper indices -256..511 for kernel. Do not blindly copy the upper half based on current -code: kernel image, boot stack, framebuffer, boot info and physical map use -dynamic bootloader placement, and pvKernel has no manifest proving all live -dependencies are in the upper half. Before activating another root, inspect -the active PML4 and recursively audit present entries (including huge pages), -record actual required ranges, prove user ranges clear of bootloader-owned -state, and verify effective USER permissions. If a required kernel mapping -lies in the user half, relocate/rebuild it or fail boot clearly. If the -audit proves all required mappings are upper-half, cloning entries 256..511 -is the compact kernel-global rule. Never silently omit a dependency or expose -it to CPL3. - -New roots must remain reachable through the shared physical mapping. The -current active-only mapper cannot do this; inactive-root construction and -ownership-safe mapping are prerequisites. CR3 switching correctness requires +The lower canonical half (PML4 0..255) is exclusively userspace; PML4 +256..511 is kernel-only. Bootloader 0.11.17's `dynamic_range_start` and +`dynamic_range_end` are applied by its `UsedLevel4Entries` allocator at PML4 +granularity to every `Mapping::Dynamic` allocation, including an ET_DYN +kernel image. pvKernel constrains that arena to +`0xffff800000000000..0xffff900000000000` (PML4 256..287), immediately below +its explicit index-288 mappings. The API treats the configured end as an +address whose containing PML4 entry remains eligible; pvKernel passes the +last page in index 287 (`0xffff8ffffffff000`) so index 288 onward is excluded. +This arena has 32 PML4 entries / 16 TiB of virtual capacity. The QEMU boot +uses six entries (kernel image, stack, framebuffer, two for the 1 TiB physical +map, and BootInfo/memory map), leaving 26 entries / 13 TiB before index 288. +Bootloader 0.11.17 panics with `no usable level 4 entries found` if a dynamic +mapping cannot obtain enough contiguous entries; arena exhaustion is a boot +failure, not a recoverable kernel allocation error. + +The default dynamic placement was observed to put the image in lower PML4 2. +That placement is not retained as a 0.0.9 compatibility model: bootloader +mapping policy moves all retained dynamic mappings into the dedicated upper +arena. The kernel verifies actual BootInfo ranges and all loaded kernel +`PT_LOAD` pages, checks the physical-map extent, confirms each required VA is +canonical and upper-half, and recursively audits every present upper-half +page-table entry (including huge pages) for USER clear. Process roots copy +only audited upper-half PML4 entries. Lower-half bootloader identity mappings +are not required by live kernel state and are deliberately omitted. Any +required lower-half mapping or USER bit in the kernel half fails closed before +a process root can be activated. + +New roots remain reachable through the shared physical mapping. Inactive-root +construction is implemented and covered by the QEMU smoke test. CR3 switching +correctness requires the executing kernel stack, switch assembly, scheduler state, GDT, IDT and TSS to remain mapped at identical VAs in the target. The mapping audit and clone verification establish that invariant; failure to establish it blocks process scheduling. +Implementation note: `KernelAddressSpace::activate_address_space` checks IF=0, +compares every upper PML4 entry with the kernel root (ignoring only the +hardware-managed Accessed bit), and only then loads the process CR3. The full +recursive boot/live mapping audit runs at boot and during root construction, +not on the PIT hot path; scheduler-time verification is bounded to the 512 +top-level entries. The scheduler constructs a `SwitchPlan` under scheduler +state protection, releases locks, acknowledges PIT, then applies the root and +selected-task RSP0 before returning the context pointer to assembly. The QEMU +runtime smoke alternates two roots whose +`USER_MIN` pages contain `0xAA` and `0xBB`, reads through the active VA each +time, then restores the kernel root. This proves root-load continuity and +same-VA translation isolation; it does not yet prove scheduler handoff. + ## 18. User and kernel stacks Give every process a dedicated 32 KiB user stack (8 pages), USER, writable, @@ -340,6 +420,12 @@ NX; one downward guard page remains unmapped. Initial RSP is the exclusive page-aligned USER_STACK_TOP. Use checked stack arithmetic and zero frames before USER publication. Do not overlap ELF segments, stack or guard. +Implementation note: `UserStack::map` applies this fixed layout to an +inactive AddressSpace, rejects collisions, initializes every page through +the zeroed-user-page mapping path, and rolls back installed leaves on error. +The QEMU stack test verifies all eight effective USER/RW/NX mappings and the +unmapped lower guard. It does not yet enter CPL3 using this stack. + User stack is for ordinary CPL3 execution. Kernel stack is for CPL3→CPL0 timer/syscall/exception entry and handler execution. A process still needs a scheduler-owned supervisor-only RW/NX kernel stack with guards; never use @@ -381,11 +467,19 @@ are released, it explicitly permits the PIT handoff as described below. | user A → user A | unchanged | restore A top | | user A → user B | A → B | set B top | -The CPL3 timer early return must be removed only after a user's full context -is saved on its own kernel stack and associated with a live task. The same -Context should represent kernel and user timer returns. The syscall interrupt -gate keeps the handler non-preemptible; pending PIT delivery occurs after -iretq restores IF. Do not switch processes mid-syscall. +Implementation note: `timer_schedule` now classifies the saved CS, requires it +to match the current TaskKind, and schedules a CPL3 frame when that frame +belongs to a process-owned Running user Task. The existing timer Context is +saved on that task's RSP0 kernel stack; the selected process root and RSP0 are +applied before assembly restores the selected frame. Direct CPL3 boundary +tests that run from bootstrap with no current Task retain the legacy return- +same-context behavior. The feature-gated `process-preemption-test` starts two +non-yielding CPL3 loops and succeeds only after A and B each set an in-page +flag after reading the peer's counter under PIT-driven task switching. It uses +a PIT-tick watchdog and no host wall-clock criterion. The shared page mapping +exists only in this QEMU test harness. Ordinary int 0x80 handlers remain non-preemptible; +pending PIT delivery occurs after `iretq` restores IF. Do not switch processes +mid-syscall. ## 20. Context representation @@ -401,22 +495,28 @@ handoff. ## 21. Syscall policy and initial surface Keep provisional `int 0x80`, with no SYSCALL/SYSRET migration. Interrupt-gate -entry clears IF, so the maskable PIT cannot preempt syscall Rust code; this -already gives the recommended non-preemptible syscall policy. Preserve it. -Only initial process-aware calls are `exit(status)` (does not return) and -`getpid()`. Process creation stays an internal kernel/test operation. Do -not add fork, exec/spawn, open/read/write, mmap, brk, signals or filesystem -calls. +entry clears IF, so the maskable PIT cannot preempt ordinary syscall Rust +code. `getpid()` returns the PID only when the current scheduler Task is a +Running user Task; bootstrap CPL3 tests get `InvalidContext`. `exit(status)` +marks the Process and Task Exiting, releases locks, then uses the explicit +STI;HLT handoff. The next PIT saves the outgoing frame and switches away; +Zombie is published only on a later interrupt after the old kernel stack is +inactive. The embedded smoke ELF verifies getpid and that exit does not +return. Process creation stays an internal kernel/test operation. Do not add +fork, exec/spawn, open/read/write, mmap, brk, signals or filesystem calls. ## 22. User-copy design -Centralize `copy_from_user`/`copy_to_user` around an AddressSpace handle. -Validate low/null policy, 48-bit canonicality, full range in `[USER_MIN, -USER_MAX)`, checked `start + len`, every touched page present, effective USER -at all levels, and writable destination for copy-to-user. Handle unaligned -and cross-page ranges; define zero-length semantics (recommended no -deref, with syscall pointer policy explicit). Return typed errors for bad -range, overflow, missing page and permissions. +The implementation centralizes `copy_from_user`/`copy_to_user` in +`process::user_copy` and resolves only the current Running user Task's +generation-safe ProcessId. It validates low/null policy, 48-bit canonicality, +the full range in `[USER_MIN, USER_MAX)`, checked `start + len`, every touched +page's effective USER permission, and writable destination for copy-to-user. +Unaligned and cross-page ranges are copied in page-bounded chunks through the +supervisor physical map. The entire range is preflighted before any copy, so +an unmapped later page cannot partially update a kernel output buffer or user +destination. Zero-length copies succeed without dereferencing or validating +the pointer. Do not validate then raw-memcpy user VA. Copy in page-sized chunks through the kernel physical-memory mapping after translation. This is safe only @@ -443,6 +543,15 @@ alignment/flags, and W+X rejection. Entry must be canonical, user-range, mapped, and inside an executable segment. Malformed data returns typed loader errors, never panic. +Implementation note: the byte parser uses checked field decoding and +validates the complete image before allocating. To keep hostile-input work +bounded, this implementation accepts at most 16 load segments and 4096 +mapped image pages (16 MiB). It rejects page-rounded overlap and stack/guard +overlap. QEMU loads a two-segment test image, checks RX/RW/NX permissions, +zero BSS and page padding, and rejects malformed identity/type/range/overlap/ +W+X/entry cases. This test validates loading and mapping only; it does not +execute the ELF entry point. + ## 24. W^X, BSS and user-page publication Map `PF_X` executable, `!PF_X` as NX, `PF_W` writable; reject PF_W+PF_X. @@ -493,9 +602,10 @@ User faults follow the same terminal sequence: capture trusted scalar `Fault(...)` metadata, mark Process and Task Exiting, release locks, then permit the timer handoff. No path returns to the faulting RIP. The scheduler records pending completion when it saves the exiting context, but Process -becomes Zombie only after a post-stack-adoption hook/continuation confirms -the old kernel stack is inactive. Do not release its root or stack before -then. An internal reaper accepts only Zombie with no executing task, changes +becomes Zombie only when a subsequent timer entry runs on the newly adopted +context's stack; this deferred completion proves the old kernel stack is +inactive. Do not release its root or stack before then. An internal reaper +accepts only Zombie with no executing task, changes it to Vacant (making the old ProcessId invalid immediately), and removes metadata. Generation advances only on a later reservation; PID is never recycled. Public waitpid is not required now. @@ -558,12 +668,12 @@ continued kernel activity are observed. | Scenario | Setup / purpose | Required success; failure | |---|---|---| -| `process-smoke-test` | Embedded ELF enters CPL3, getpid, exit | deterministic entered/PID/exited markers; timeout/panic fails | -| `process-preemption-test` | Two CPU-bound user tasks wait on each other's progress without yielding | both progress and finish before tick watchdog | +| `process-smoke-test` | Embedded ELF enters CPL3, checks getpid, calls exit, then contains UD2 sentinel | Zombie retains status 0, sentinel never executes, kernel reaps, QEMU passes; PIT watchdog fails otherwise | +| `process-preemption-test` | Two non-yielding CPL3 loops increment separate counters on a test-only shared page and set peer-observed flags | PIT handler passes only after both peer-observed flags are set; tick watchdog fails otherwise | | `address-space-isolation-test` | A/B map same VA to different frames with AA/BB and alternate repeatedly | each always sees own value over A→B→A; tests CR3/TLB/scheduler | | `user-stack-guard-fault-test` | User writes below mapped stack | process fault recorded, kernel continues | | `user-kernel-isolation-test` | CPL3 reads/writes supervisor mapping | controlled fault/denial; kernel data unchanged | -| `user-fault-containment-test` | Separate #PF/#GP/#UD processes | metadata and Zombie observed, other work continues, no panic/return | +| `user-fault-containment-test` | Separate #PF/#GP/#UD/#NM processes | metadata and Zombie observed, other work continues, no panic/return | | `process-exit-test` | Known exit status then sentinel user instruction | sentinel never runs; Zombie retains status | | `process-capacity-test` | Fill shared slots and request one more | typed error; no panic/partial publication | | `process-id-reuse-test` | Reap and reuse slot | new generation/PID; old ProcessId rejected | @@ -620,44 +730,43 @@ manager, GUI/desktop, APIC/x2APIC, SMP, NUMA or multicore scheduling. | frame reclamation | ownership metadata exists although allocator remains monotonic | | SMP | explicit per-task ownership helps, but locks/TLB shootdowns/AP bring-up must change | -## 34. Implementation phases - -1. **A:** ProcessId reservation-generation semantics, checked monotonic PID, lifecycle, bounded table, snapshots. -2. **B:** mapping inventory/audit, inactive-root AddressSpace construction, kernel mapping clone; fail boot if invariant is unproven. -3. **C:** per-task kernel stack and checked RSP0 update; user stack layout. -4. **D:** hostile-input ELF validation, mapping permissions and zeroing. -5. **E:** TaskKind/process handle, kernel stack top and synthetic CPL3 context (not yet runnable). -6. **F:** CR3 switch backend and lock-free SwitchPlan handoff; kernel/user root transition tests. -7. **G:** CPL3 timer preemption for all transition cases. -8. **H:** non-preemptible ordinary syscall path, getpid and non-returning exit policy, explicit IF-safe exit handoff and deferred Zombie completion. -9. **I:** checked page-chunk user-copy and negative pointer tests. -10. **J:** exception scheduler handoff with explicit IF enable after Exiting publication, fault containment, deferred Zombie completion and internal reap. -11. **K:** accounting and deterministic QEMU scenarios. -12. **L:** security review of mappings, transitions, locks, ownership and capacity. -13. **M:** separate implementation PR and review. -14. **N:** formal pre-release audit. -15. **O:** RC, published-artifact verification, then stable only after review. - -Mapping audit precedes CR3 work; exception handoff precedes fault containment. -These dependencies avoid assuming the difficult part is just loading ELF. +## 34. Implementation status and remaining phases + +Implementation phases A through L are present on the 0.0.9 branch: bounded +identity/lifecycle, audited inactive roots, stack/RSP0 support, checked ELF, +prepared tasks/publication, CR3 switching, CPL3 PIT preemption, getpid/exit, +checked user-copy, fault containment/Zombie reaping, accounting/QEMU scenarios, +and version/docs updates. The remaining project gates are: + +* **M:** separate implementation PR and human review. +* **N:** formal pre-release audit against the exact merged main and artifacts. +* **O:** RC, published-artifact verification, then stable release only after + review. No RC or release is part of this implementation branch. + +The dependency order remains significant: runtime mapping audit precedes CR3 +activation, and the non-returning exception handoff precedes claims of user +fault containment. ## 35. Open questions / blockers -No stable scheduler contradiction blocks this design. These are mandatory -implementation gates: - -* Dynamic bootloader ranges are not recorded. Inspect actual PML4 mappings - and prove every live kernel dependency is preserved supervisor-only before - activating another root. If any needed mapping conflicts with user ranges, - relocation or explicit reconstruction is required; otherwise CR3 execution - is blocked. -* Current paging API mutates only active root. Inactive root construction, - physical access and frame ownership must be added before process creation. -* Current Rust exception handlers are fatal/returning, not scheduler exits. - A non-returning user-fault handoff is required before claiming containment. -* Eight process slots do not imply eight simultaneous processes: the current - eight task slots are shared with kernel tasks. Preserve this bound initially - and test mixed occupancy. - -No stable ABI is promised for 0.0.9. Embedded smoke ELF and ABI details need -implementation review. +No unresolved correctness blocker is known from the implementation and local +QEMU review. Keep these constraints visible in future work: + +* Re-run the dynamic boot-mapping audit whenever bootloader/configuration or + required kernel mappings change; process-root activation fails closed if + the runtime inventory cannot prove the required supervisor mappings. +* AddressSpace roots and mappings are torn down from process metadata, but + the monotonic physical-frame allocator does not reclaim process backing + frames or intermediate table frames. This is an intentional 0.0.9 limit. +* User-copy safety relies on single-core execution, IF-clear syscalls, + immutable Alive mappings, no user remapping, no user threads and no SMP. + mmap, concurrent threads or SMP require a stronger pin/locking/fault-safe + copy design before they are added. +* Eight process slots share the existing eight scheduler Task slots with + kernel Tasks; process table capacity does not imply eight simultaneously + runnable user processes. +* The embedded static smoke ELF is a temporary provider until VFS/initrd + supplies bytes; the loader API remains byte-slice based. + +The `int 0x80` process ABI remains experimental. No ABI compatibility promise +is made before 0.1.0. diff --git a/docs/architecture/syscalls.md b/docs/architecture/syscalls.md index d7bc7b0..6093f31 100644 --- a/docs/architecture/syscalls.md +++ b/docs/architecture/syscalls.md @@ -1,18 +1,17 @@ # Provisional bootstrap syscall ABI -This document describes pvKernel 0.0.6's provisional, scalar-only `int 0x80` +This document describes pvKernel's provisional, scalar-only `int 0x80` syscall ABI, implemented in [kernel/src/arch/x86_64/syscall/](../../kernel/src/arch/x86_64/syscall/). See [docs/architecture/userspace-entry.md](userspace-entry.md) for how CPL3 code capable of issuing this instruction gets there in the first place. -## Status: provisional +## Status: provisional and experimental in 0.0.9 This ABI is **not stable** and not borrowed from `errno.h`/POSIX/Linux -numbering. It exists to let 0.0.6's deterministic tests exercise a real -CPL3 → CPL0 → CPL3 round trip and a minimal set of bootstrap operations; it -is expected to be extended or replaced as pvKernel gains a process model -(see [ROADMAP.md](../../ROADMAP.md)). +numbering. It began as the 0.0.6 bootstrap boundary and is now extended with +the first process-aware calls in 0.0.9. It remains experimental; the first +meaningful compatibility checkpoint is 0.1.0 (see [ROADMAP.md](../../ROADMAP.md)). ## Entry mechanism: `int 0x80` only @@ -39,7 +38,9 @@ from CPL0 — without this, a CPL3 `int 0x80` would itself raise `#GP`. the entry stub saves and restores all 15 GPRs around the dispatch. No syscall in this milestone accepts or dereferences a user pointer — -every argument is treated as an opaque scalar integer. +every argument is treated as an opaque scalar integer. Central +`copy_from_user`/`copy_to_user` primitives exist for future pointer-taking +syscalls, but current `getpid` and `exit` calls do not use them. ## Result encoding @@ -65,6 +66,8 @@ Always compiled in (`syscall::numbers`): | 0 | `ABI_VERSION_QUERY` | Returns the current bootstrap ABI version (`1`). | | 1 | `DEBUG_WRITE_BYTE` | Logs the byte in `RDI` to serial output; returns `0`. | | 2 | `TIMER_TICKS` | Returns the current PIT tick counter (`pit::ticks()`). | +| 3 | `GETPID` | Returns the current process PID; returns `InvalidContext` when no process-owned user Task is current. | +| 4 | `EXIT` | Terminates the current process with `RDI as i32`; never returns to userspace. | Additionally, under the `userspace-test` Cargo feature, a small test-only namespace (`>= TEST_NAMESPACE_BASE`, `0x1000`) exists solely for the @@ -104,20 +107,21 @@ privilege-level-changing interrupt. `syscall::dispatch::pvkernel_syscall_dispatch` first checks `frame.caller_privilege_level()` (derived from the saved `CS`'s RPL bits). -Nothing in pvKernel 0.0.6 legitimately invokes `int 0x80` from CPL0; a CPL0 -origin is treated as a kernel bug, logged loudly, and reported through the -normal error-return path (`SyscallError::InvalidContext`) rather than -panicking — a stray kernel-mode `int 0x80` should not itself crash the -kernel. Otherwise, it matches `frame.rax` against the documented syscall -numbers above (and, under `userspace-test`, the test-only namespace), -falling back to `SyscallError::Unsupported` for anything else. +A CPL0 origin is treated as a kernel bug, logged loudly, and reported through +the normal error-return path (`SyscallError::InvalidContext`) rather than +panicking. For CPL3, `GETPID` checks that the scheduler has a current +process-owned user Task. `EXIT` validates that same context, records Process +and Task as Exiting, releases locks while IF is clear, then deliberately +executes an STI;HLT terminal handoff so PIT can switch away. It never writes a +result or returns to the saved user RIP. Zombie publication is deferred until +a later timer entry runs on another stack. Other calls match the documented +numbers above (and, under `userspace-test`, the test-only namespace), falling +back to `SyscallError::Unsupported`. ## What this ABI does not provide * No `SYSCALL`/`SYSRET`. -* No user-pointer arguments, and no `copy_from_user`/`copy_to_user`. -* No per-process or per-thread state — there is no process model yet. +* No user-pointer arguments yet; central user-copy primitives exist, but no current syscall accepts a pointer. +* No stable ABI guarantee; the 0.0.9 process-aware extension remains experimental. * No signal delivery, no blocking syscalls, no I/O beyond the debug byte-logging syscall. -* No stability guarantee: every number, encoding, and behavior described - here may change as pvKernel gains a real process model. diff --git a/docs/development/testing.md b/docs/development/testing.md index 8d4def6..ac1e244 100644 --- a/docs/development/testing.md +++ b/docs/development/testing.md @@ -85,6 +85,45 @@ timeout 60 cargo run --features "qemu-test scheduler-test" -- --headless-test # Two CPU-bound tasks that can finish only after IRQ-driven preemption. timeout 60 cargo run --features scheduler-preemption-test -- --headless-test +# Two non-yielding CPL3 processes must each observe the peer's shared test +# counter; the only path between them is PIT-driven scheduler preemption. +timeout 60 cargo run --features process-preemption-test -- --headless-test + +# Embedded ELF checks getpid and must not return after exit; the kernel +# observes deferred Zombie publication and reaps the process. +timeout 60 cargo run --features process-smoke-test -- --headless-test + +# Page-chunked user copies, including cross-page transfer and hostile-pointer +# rejection before any partial destination update. +timeout 60 cargo run --features user-copy-test -- --headless-test + +# Three process-owned CPL3 tasks trigger #PF, #GP, and #UD; the kernel checks +# saved fault metadata and deferred Zombie completion before reaping them. +timeout 60 cargo run --features user-fault-containment-test -- --headless-test + +# User stack guard and supervisor mapping denial are included in the process +# fault-containment scenario. +timeout 60 cargo run --features user-stack-guard-fault-test -- --headless-test +timeout 60 cargo run --features user-kernel-isolation-test -- --headless-test + +# Two process roots map the same private VA to different frames and values; +# user code checks its value while PIT scheduling switches between them. +timeout 60 cargo run --features address-space-isolation-test -- --headless-test + +# Two kernel Tasks plus six prepared process Tasks exhaust the shared task +# table; a seventh process gets a typed failure and process-table capacity is +# checked independently. +timeout 60 cargo run --features process-capacity-test -- --headless-test + +# Embedded ELF exit status, non-returning exit, Zombie/reap and stale-ID +# rejection after slot reuse. +timeout 60 cargo run --features process-exit-test -- --headless-test +timeout 60 cargo run --features process-id-reuse-test -- --headless-test + +# Valid static ELF permissions/BSS and malformed-input rejection. +timeout 60 cargo run --features elf-loader-test -- --headless-test +timeout 60 cargo run --features elf-malformed-test -- --headless-test + # Expected task stack lower-guard page fault. timeout 60 cargo run --features task-stack-guard-fault-test -- --headless-test diff --git a/kernel/Cargo.toml b/kernel/Cargo.toml index 39e6492..dca5d9b 100644 --- a/kernel/Cargo.toml +++ b/kernel/Cargo.toml @@ -65,6 +65,18 @@ heap-guard-fault-test = ["heap-test"] # exercises two tasks, while this feature gives it a dedicated QEMU job. scheduler-test = [] scheduler-preemption-test = ["qemu-test"] +process-preemption-test = ["qemu-test"] +process-smoke-test = ["qemu-test"] +process-exit-test = ["process-smoke-test"] +process-id-reuse-test = ["process-smoke-test"] +user-copy-test = ["qemu-test"] +user-fault-containment-test = ["qemu-test"] +address-space-isolation-test = ["qemu-test"] +process-capacity-test = ["qemu-test"] +user-stack-guard-fault-test = ["user-fault-containment-test"] +user-kernel-isolation-test = ["user-fault-containment-test"] +elf-loader-test = ["qemu-test"] +elf-malformed-test = ["qemu-test"] task-stack-guard-fault-test = ["qemu-test"] task-capacity-test = [] diff --git a/kernel/src/arch/x86_64/gdt.rs b/kernel/src/arch/x86_64/gdt.rs index 32c6b4f..885ad9a 100644 --- a/kernel/src/arch/x86_64/gdt.rs +++ b/kernel/src/arch/x86_64/gdt.rs @@ -55,9 +55,9 @@ static DOUBLE_FAULT_STACK: DoubleFaultStack = /// a CPL3-to-CPL0 interrupt or exception occurs (in particular, the `int /// 0x80` syscall gate; see `arch::x86_64::syscall`). /// -/// 0.0.6 has no thread or process model, so there is exactly one such stack -/// for the whole kernel (the boot CPU's transition stack), not a per-thread -/// kernel stack. Sized generously (16 KiB) for the hardware interrupt frame, +/// This static stack is the bootstrap/kernel-task fallback. User tasks select +/// their scheduler-owned kernel stack through the checked RSP0 setter below. +/// It is sized generously (16 KiB) for the hardware interrupt frame, /// the syscall entry stub's saved general-purpose registers, the Rust /// dispatcher's own stack usage, and a nested hardware interrupt (e.g. the /// PIT timer) firing while the syscall path is on this stack; there is no @@ -92,7 +92,15 @@ struct Selectors { user_data_selector: SegmentSelector, } -static TSS: Once = Once::new(); +struct TssCell(UnsafeCell); + +// SAFETY: pvKernel is single-core. The CPU reads RSP0 only on a privilege +// transition from CPL3; updates are restricted to CPL0 with IF clear, when no +// such transition can concurrently use the TSS. Other TSS fields are +// initialized once and never mutated after `ltr`. +unsafe impl Sync for TssCell {} + +static TSS: Once = Once::new(); static GDT: Once<(GlobalDescriptorTable, Selectors)> = Once::new(); fn double_fault_stack_top() -> VirtAddr { @@ -195,13 +203,16 @@ pub fn init() { let mut tss = TaskStateSegment::new(); tss.interrupt_stack_table[DOUBLE_FAULT_IST_INDEX as usize] = double_fault_stack_top(); tss.privilege_stack_table[0] = syscall_stack_top(); - tss + TssCell(UnsafeCell::new(tss)) }); let (gdt, selectors) = GDT.call_once(|| { let mut gdt = GlobalDescriptorTable::new(); let code_selector = gdt.append(Descriptor::kernel_code_segment()); - let tss_selector = gdt.append(Descriptor::tss_segment(tss)); + // SAFETY: this is the one initialized TSS cell, and its address is + // stable for kernel lifetime; the GDT descriptor stores that address. + let tss_ref = unsafe { &*tss.0.get() }; + let tss_selector = gdt.append(Descriptor::tss_segment(tss_ref)); // `Descriptor::user_code_segment()` / `user_data_segment()` already // encode DPL 3; `GlobalDescriptorTable::append` derives the // returned `SegmentSelector`'s RPL from that same descriptor DPL, @@ -253,6 +264,70 @@ pub fn init() { ); } +/// Installs the kernel-entry stack belonging to one allocated scheduler +/// stack slot as the next CPL3-to-CPL0 RSP0 destination. +/// +/// The caller must keep interrupts disabled through the later context +/// adoption and `iretq`. Only exact tops returned by the task stack layout +/// are accepted; those stacks were mapped supervisor RW/NX with guards by +/// `task::init_stacks`. +pub fn set_current_rsp0(stack_top: u64) -> Result<(), Rsp0Error> { + if x86_64::instructions::interrupts::are_enabled() { + return Err(Rsp0Error::InterruptsEnabled); + } + let top = VirtAddr::try_new(stack_top).map_err(|_| Rsp0Error::InvalidAddress)?; + if stack_top % 16 != 0 { + return Err(Rsp0Error::InvalidAddress); + } + if !crate::arch::x86_64::task::is_validated_stack_top(stack_top) { + return Err(Rsp0Error::NotTaskStackTop); + } + write_rsp0(top) +} + +/// Restores the static, kernel-lifetime privilege-transition stack for +/// bootstrap and kernel-task execution. It must be selected before a user +/// task's kernel stack becomes inactive or reusable. +pub fn set_kernel_fallback_rsp0() -> Result<(), Rsp0Error> { + if x86_64::instructions::interrupts::are_enabled() { + return Err(Rsp0Error::InterruptsEnabled); + } + write_rsp0(kernel_fallback_rsp0()) +} + +pub fn kernel_fallback_rsp0() -> VirtAddr { + syscall_stack_top() +} + +/// Reads the programmed privilege-transition stack top. Intended for +/// diagnostics and deterministic architecture tests; scheduler callers still +/// select only validated tops through the setters above. +#[cfg(feature = "qemu-test")] +pub fn current_rsp0() -> Result { + let tss = TSS.get().ok_or(Rsp0Error::TssUnavailable)?; + // SAFETY: RSP0 is a scalar read. Single-core execution and the existing + // IF-masked setter prevent a concurrent Rust update; hardware reads it + // only during a CPL3 privilege transition. + Ok(unsafe { (*tss.0.get()).privilege_stack_table[0].as_u64() }) +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum Rsp0Error { + InterruptsEnabled, + InvalidAddress, + NotTaskStackTop, + TssUnavailable, +} + +fn write_rsp0(top: VirtAddr) -> Result<(), Rsp0Error> { + let tss = TSS.get().ok_or(Rsp0Error::TssUnavailable)?; + // SAFETY: called only on the single CPU with IF clear. The CPU is in CPL0 + // and cannot be performing a CPL3 RSP0 transition concurrently; only the + // RSP0 field changes, while the TSS/GDT storage remains kernel-lifetime. + unsafe { (*tss.0.get()).privilege_stack_table[0] = top }; + Ok(()) +} + /// Returns the ring-3 user code segment selector (RPL 3) installed by /// [`init`]. Used only when constructing an `iretq` frame for entering /// CPL3; see `arch::x86_64::userspace`. diff --git a/kernel/src/arch/x86_64/interrupts.rs b/kernel/src/arch/x86_64/interrupts.rs index 2cec4a4..e71b3ca 100644 --- a/kernel/src/arch/x86_64/interrupts.rs +++ b/kernel/src/arch/x86_64/interrupts.rs @@ -34,6 +34,8 @@ pub fn init() { idt.breakpoint.set_handler_fn(breakpoint_handler); idt.invalid_opcode.set_handler_fn(invalid_opcode_handler); + idt.device_not_available + .set_handler_fn(device_not_available_handler); idt.general_protection_fault .set_handler_fn(general_protection_fault_handler); idt.page_fault.set_handler_fn(page_fault_handler); @@ -290,17 +292,54 @@ extern "x86-interrupt" fn breakpoint_handler(stack_frame: InterruptStackFrame) { } extern "x86-interrupt" fn invalid_opcode_handler(stack_frame: InterruptStackFrame) { + let cpl = stack_frame.code_segment.rpl(); + if cpl == x86_64::PrivilegeLevel::Ring3 && crate::task::current_user_process_id().is_some() { + crate::task::terminate_current_user_fault( + 6, + None, + None, + stack_frame.instruction_pointer.as_u64(), + ); + } kprintln!("[exception] invalid opcode"); kprintln!("{:#?}", stack_frame); fatal_exit(); } +/// x87 execution raises #NM while CR0.EM is set. This kernel does not emulate +/// or context-switch floating-point state, so reject it in CPL3 processes and +/// keep unexpected kernel use fatal. +extern "x86-interrupt" fn device_not_available_handler(stack_frame: InterruptStackFrame) { + let cpl = stack_frame.code_segment.rpl(); + if cpl == x86_64::PrivilegeLevel::Ring3 && crate::task::current_user_process_id().is_some() { + crate::task::terminate_current_user_fault( + 7, + None, + None, + stack_frame.instruction_pointer.as_u64(), + ); + } + kprintln!("[exception] device not available (#NM)"); + kprintln!("origin: CPL{}", cpl as u8); + kprintln!("{:#?}", stack_frame); + fatal_exit(); +} + extern "x86-interrupt" fn general_protection_fault_handler( stack_frame: InterruptStackFrame, error_code: u64, ) { let cpl = stack_frame.code_segment.rpl(); + if cpl == x86_64::PrivilegeLevel::Ring3 && crate::task::current_user_process_id().is_some() { + crate::task::terminate_current_user_fault( + 13, + Some(error_code), + None, + stack_frame.instruction_pointer.as_u64(), + ); + } + #[cfg(all(feature = "qemu-test", feature = "userspace-privilege-fault-test"))] if take_expected_privilege_fault_if_matches(cpl) { kprintln!("[exception] general protection fault"); @@ -396,6 +435,15 @@ extern "x86-interrupt" fn page_fault_handler( qemu::exit_success(); } + if cpl == x86_64::PrivilegeLevel::Ring3 && crate::task::current_user_process_id().is_some() { + crate::task::terminate_current_user_fault( + 14, + Some(error_code.bits()), + Some(faulting_address), + stack_frame.instruction_pointer.as_u64(), + ); + } + kprintln!("[exception] page fault"); kprintln!("faulting address (CR2): {:#x}", faulting_address); kprintln!("origin: CPL{}", cpl as u8); diff --git a/kernel/src/arch/x86_64/memory/address_space.rs b/kernel/src/arch/x86_64/memory/address_space.rs index 36b288c..0b2cee9 100644 --- a/kernel/src/arch/x86_64/memory/address_space.rs +++ b/kernel/src/arch/x86_64/memory/address_space.rs @@ -1,18 +1,20 @@ //! Ownership of the active x86_64 page table and the boot frame allocator. //! -//! There is exactly one [`KernelAddressSpace`] for the whole kernel in this -//! milestone (pvKernel 0.0.5 does not create per-process address spaces). -//! It is built once during [`super::init`] and afterward is only reachable -//! through [`with_kernel_address_space`], which serializes access behind a +//! The single [`KernelAddressSpace`] owns the kernel root and allocator. It +//! is built once during [`super::init`] and afterward is reachable through +//! [`with_kernel_address_space`], which serializes access behind a //! [`spin::Mutex`] and disables maskable interrupts for the duration of the -//! call. This keeps a single owner of the mapper at all times: no code path -//! can obtain two simultaneous mutable references to the page tables. +//! call. Process [`AddressSpace`] values own inactive roots whose audited +//! upper-half subtrees are shared with the kernel root. use spin::{Mutex, Once}; use x86_64::{ PhysAddr, VirtAddr, - registers::control::Cr3, - structures::paging::{FrameAllocator, OffsetPageTable, Page, PageTable, PhysFrame, Size4KiB}, + registers::control::{Cr3, Cr3Flags}, + structures::paging::{ + FrameAllocator, OffsetPageTable, Page, PageSize, PageTable, PageTableFlags, PhysFrame, + Size4KiB, + }, }; use super::frame_allocator::BootFrameAllocator; @@ -20,15 +22,1171 @@ use super::paging::{self, MemoryError, Permissions, Translation}; static KERNEL_ADDRESS_SPACE: Once> = Once::new(); +#[derive(Clone, Copy, Debug)] +pub(super) struct MappingRange { + pub(super) start: u64, + pub(super) len: u64, +} + +#[derive(Clone, Copy)] +pub(super) struct BootMappingInventory { + pub(super) kernel_image_base: u64, + pub(super) kernel_elf_physical: u64, + pub(super) kernel_elf_len: u64, + pub(super) kernel_stack: MappingRange, + pub(super) boot_info: MappingRange, + pub(super) memory_regions: MappingRange, + pub(super) physical_memory: MappingRange, + pub(super) framebuffer: Option, + pub(super) ramdisk: Option, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum AddressSpaceError { + FrameAllocationFailed, + InvalidRoot, + UnsafeKernelMapping, + KernelMappingMissing, + OutsideUserRange, + NonUserPermissions, + UserPageAlreadyMapped, + UserPageNotMapped, + StackCollision, + Mapping(MemoryError), + AccountingOverflow, + InvalidCopyRange, + InvalidUserContext, +} + +impl From for AddressSpaceError { + fn from(error: MemoryError) -> Self { + Self::Mapping(error) + } +} + +/// One independent four-level root. Its lower-half page tables and user +/// frames are process-owned; audited upper-half kernel PML4 entries share the +/// kernel's supervisor-only subtrees. Frames remain monotonic/kernel-lifetime +/// owned until general reclamation exists. +pub struct AddressSpace { + root: PhysFrame, + mapped_user_pages: u64, +} + +/// Opaque, copyable identity for a process root used only by the architecture +/// switch boundary. It exposes neither a naked CR3 integer nor page-table +/// mutation to generic scheduler code. The owning Process must stay Alive (or +/// Exiting while its task is active) until another context has been adopted. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct AddressSpaceHandle(PhysFrame); + +/// A root whose required shared kernel mappings were checked while holding +/// the kernel address-space lock. Activation is deliberately a separate +/// operation so that no page-table lock is held across the CR3 load. +pub(crate) struct VerifiedRootSwitch { + root: PhysFrame, + flags: Cr3Flags, +} + +impl VerifiedRootSwitch { + /// Loads the root after the caller has left the kernel page-table lock. + /// Interrupts must remain disabled until the selected context is adopted. + pub(crate) fn activate(self) -> Result<(), AddressSpaceError> { + if x86_64::instructions::interrupts::are_enabled() { + return Err(AddressSpaceError::UnsafeKernelMapping); + } + // SAFETY: this token is only constructed after auditing that the + // target root contains the identical supervisor kernel mappings. + // Process roots are kernel-lifetime allocated and cannot be reclaimed + // during this IF-masked single-CPU handoff. + unsafe { Cr3::write(self.root, self.flags) }; + Ok(()) + } +} + +/// User entry/stack pair checked against one specific inactive root. Fields +/// remain private so callers cannot fabricate a context from raw addresses. +#[derive(Clone, Copy)] +pub struct ValidatedUserContext { + root: PhysFrame, + entry: u64, + user_rsp: u64, +} + +impl ValidatedUserContext { + pub const fn entry(self) -> u64 { + self.entry + } + pub const fn user_rsp(self) -> u64 { + self.user_rsp + } + pub(crate) const fn root_frame(self) -> PhysFrame { + self.root + } +} + +impl AddressSpace { + pub const fn mapped_user_pages(&self) -> u64 { + self.mapped_user_pages + } + + pub(crate) fn root_frame(&self) -> PhysFrame { + self.root + } + + pub(crate) const fn handle(&self) -> AddressSpaceHandle { + AddressSpaceHandle(self.root) + } + + pub fn validate_user_context( + &mut self, + kernel: &mut KernelAddressSpace, + entry: u64, + user_rsp: u64, + ) -> Result { + if !(super::USER_MIN..super::USER_MAX).contains(&entry) + || user_rsp <= super::USER_MIN + || user_rsp > super::USER_MAX + || user_rsp % 16 != 0 + || VirtAddr::try_new(entry).is_err() + || VirtAddr::try_new(user_rsp).is_err() + { + return Err(AddressSpaceError::InvalidUserContext); + } + match self.translate(kernel, VirtAddr::new(entry))? { + Translation::Mapped { flags, .. } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) + && !flags.contains(PageTableFlags::WRITABLE) + && !flags.contains(PageTableFlags::NO_EXECUTE) => {} + _ => return Err(AddressSpaceError::InvalidUserContext), + } + let stack_byte = user_rsp + .checked_sub(1) + .ok_or(AddressSpaceError::InvalidUserContext)?; + match self.translate(kernel, VirtAddr::new(stack_byte))? { + Translation::Mapped { flags, .. } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) + && flags.contains(PageTableFlags::WRITABLE) + && flags.contains(PageTableFlags::NO_EXECUTE) => {} + _ => return Err(AddressSpaceError::InvalidUserContext), + } + Ok(ValidatedUserContext { + root: self.root, + entry, + user_rsp, + }) + } + + pub fn map_new_user_page( + &mut self, + kernel: &mut KernelAddressSpace, + page: Page, + permissions: Permissions, + ) -> Result, AddressSpaceError> { + validate_user_page(page, permissions)?; + if !matches!( + self.translate(kernel, page.start_address())?, + Translation::NotMapped + ) { + return Err(AddressSpaceError::UserPageAlreadyMapped); + } + let next_count = self + .mapped_user_pages + .checked_add(1) + .ok_or(AddressSpaceError::AccountingOverflow)?; + let frame = kernel.allocate_zeroed_frame()?; + kernel.with_inactive_mapper(self, |mapper, allocator| { + paging::map_page(mapper, allocator, page, frame, permissions) + })?; + self.mapped_user_pages = next_count; + match self.translate(kernel, page.start_address())? { + Translation::Mapped { flags, .. } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) => + { + Ok(frame) + } + _ => { + let _ = self.unmap_user_page(kernel, page); + Err(AddressSpaceError::NonUserPermissions) + } + } + } + + /// Installs a supervisor-only PTE inside the user range for the + /// user-copy negative test. This mapping is never exposed to a process + /// task and is not part of the production mapping API. + #[cfg(feature = "user-copy-test")] + pub(crate) fn map_test_supervisor_page( + &mut self, + kernel: &mut KernelAddressSpace, + page: Page, + ) -> Result<(), AddressSpaceError> { + validate_user_address(page.start_address())?; + if !matches!( + self.translate(kernel, page.start_address())?, + Translation::NotMapped + ) { + return Err(AddressSpaceError::UserPageAlreadyMapped); + } + let frame = kernel.allocate_zeroed_frame()?; + kernel.with_inactive_mapper(self, |mapper, allocator| { + paging::map_page(mapper, allocator, page, frame, Permissions::KERNEL_DATA) + })?; + Ok(()) + } + + /// Test-only shared user page mapping for the CPL3 preemption handshake. + /// This is not a production shared-memory API; it maps only the frame + /// allocated by the paired deterministic QEMU test. + #[cfg(feature = "process-preemption-test")] + pub(crate) fn map_test_shared_user_page( + &mut self, + kernel: &mut KernelAddressSpace, + page: Page, + frame: PhysFrame, + ) -> Result<(), AddressSpaceError> { + validate_user_page(page, Permissions::USER_DATA)?; + if !matches!( + self.translate(kernel, page.start_address())?, + Translation::NotMapped + ) { + return Err(AddressSpaceError::UserPageAlreadyMapped); + } + let next_count = self + .mapped_user_pages + .checked_add(1) + .ok_or(AddressSpaceError::AccountingOverflow)?; + kernel.with_inactive_mapper(self, |mapper, allocator| { + paging::map_page(mapper, allocator, page, frame, Permissions::USER_DATA) + })?; + self.mapped_user_pages = next_count; + Ok(()) + } + + pub fn update_user_permissions( + &mut self, + kernel: &mut KernelAddressSpace, + page: Page, + permissions: Permissions, + ) -> Result<(), AddressSpaceError> { + validate_user_page(page, permissions)?; + match self.translate(kernel, page.start_address())? { + Translation::Mapped { flags, .. } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) => {} + _ => return Err(AddressSpaceError::UserPageNotMapped), + } + kernel.with_inactive_mapper(self, |mapper, _| { + paging::update_permissions(mapper, page, permissions) + })?; + Ok(()) + } + + /// Copies into one already-mapped, writable user page through the + /// supervisor physical map. Callers split multi-page data into page + /// chunks and keep the process Creating until initialization is complete. + pub fn write_user_page( + &mut self, + kernel: &mut KernelAddressSpace, + page: Page, + offset: usize, + bytes: &[u8], + ) -> Result<(), AddressSpaceError> { + validate_user_address(page.start_address())?; + let end = offset + .checked_add(bytes.len()) + .ok_or(AddressSpaceError::InvalidCopyRange)?; + if end > Size4KiB::SIZE as usize { + return Err(AddressSpaceError::InvalidCopyRange); + } + if bytes.is_empty() { + return Ok(()); + } + let virtual_address = page + .start_address() + .as_u64() + .checked_add(offset as u64) + .ok_or(AddressSpaceError::InvalidCopyRange)?; + let physical = match self.translate(kernel, VirtAddr::new(virtual_address))? { + Translation::Mapped { phys_addr, flags } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) + && flags.contains(PageTableFlags::WRITABLE) => + { + phys_addr + } + _ => return Err(AddressSpaceError::UserPageNotMapped), + }; + let target = kernel.physical_alias(physical)?; + // SAFETY: the destination PTE belongs to this inactive AddressSpace, + // was checked user-accessible and writable above, and the matching + // physical frame is mapped supervisor-only through the audited + // physical map. The input slice is valid for `bytes.len()`. + unsafe { + core::ptr::copy_nonoverlapping(bytes.as_ptr(), target.as_mut_ptr::(), bytes.len()); + } + Ok(()) + } + + /// Copies bytes from one mapped user page through its supervisor-only + /// physical alias. The requested range must remain within that page. + #[cfg(feature = "qemu-test")] + pub fn read_user_page( + &mut self, + kernel: &mut KernelAddressSpace, + page: Page, + offset: usize, + output: &mut [u8], + ) -> Result<(), AddressSpaceError> { + validate_user_address(page.start_address())?; + let end = offset + .checked_add(output.len()) + .ok_or(AddressSpaceError::InvalidCopyRange)?; + if end > Size4KiB::SIZE as usize { + return Err(AddressSpaceError::InvalidCopyRange); + } + if output.is_empty() { + return Ok(()); + } + let virtual_address = page + .start_address() + .as_u64() + .checked_add(offset as u64) + .ok_or(AddressSpaceError::InvalidCopyRange)?; + let physical = match self.translate(kernel, VirtAddr::new(virtual_address))? { + Translation::Mapped { phys_addr, flags } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) => + { + phys_addr + } + _ => return Err(AddressSpaceError::UserPageNotMapped), + }; + let source = kernel.physical_alias(physical)?; + // SAFETY: translation proved that the user mapping resolves to the + // checked page, physical_alias proved a supervisor-readable mapping, + // and output is a valid uniquely borrowed destination slice. + unsafe { + core::ptr::copy_nonoverlapping( + source.as_ptr::(), + output.as_mut_ptr(), + output.len(), + ); + } + Ok(()) + } + + /// Copies a complete userspace range into a kernel buffer without ever + /// dereferencing the userspace virtual address. The full range is + /// validated before bytes are copied, so an invalid later page cannot + /// produce a partial kernel buffer update. + /// + /// This is safe for the 0.0.9 single-core model: user mappings cannot be + /// changed while a syscall runs (IF is clear), and there is no user + /// remapping, userspace threading, or SMP. Those assumptions must be + /// revisited before adding any such feature. + // SAFETY/API NOTE: 0.0.9 deliberately ships the checked copy primitives + // before adding a syscall that consumes user buffers. Keep these methods + // available in normal builds for the first buffer-bearing syscall. + #[allow(dead_code)] + pub fn copy_from_user( + &mut self, + kernel: &mut KernelAddressSpace, + user_address: u64, + output: &mut [u8], + ) -> Result<(), AddressSpaceError> { + let range = validate_copy_range(user_address, output.len())?; + if output.is_empty() { + return Ok(()); + } + self.validate_copy_pages(kernel, range, false)?; + self.copy_user_chunks( + kernel, + range, + output.len(), + output.as_ptr(), + output.as_mut_ptr(), + CopyDirection::FromUser, + ) + } + + /// Copies a complete kernel buffer into a writable userspace range + /// through supervisor-only physical aliases. No raw user virtual pointer + /// is passed to memcpy/copy_nonoverlapping. + #[allow(dead_code)] + pub fn copy_to_user( + &mut self, + kernel: &mut KernelAddressSpace, + user_address: u64, + input: &[u8], + ) -> Result<(), AddressSpaceError> { + let range = validate_copy_range(user_address, input.len())?; + if input.is_empty() { + return Ok(()); + } + self.validate_copy_pages(kernel, range, true)?; + self.copy_user_chunks( + kernel, + range, + input.len(), + input.as_ptr(), + core::ptr::null_mut(), + CopyDirection::ToUser, + ) + } + + #[allow(dead_code)] + fn validate_copy_pages( + &mut self, + kernel: &mut KernelAddressSpace, + range: CopyRange, + writable: bool, + ) -> Result<(), AddressSpaceError> { + let mut address = range.start; + while address < range.end { + let page = Page::::containing_address(VirtAddr::new(address)); + match self.translate(kernel, VirtAddr::new(address))? { + Translation::Mapped { flags, .. } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) + && (!writable || flags.contains(PageTableFlags::WRITABLE)) => {} + _ => return Err(AddressSpaceError::InvalidCopyRange), + } + address = page + .start_address() + .as_u64() + .checked_add(Size4KiB::SIZE) + .ok_or(AddressSpaceError::InvalidCopyRange)?; + } + Ok(()) + } + + #[allow(dead_code)] + fn copy_user_chunks( + &mut self, + kernel: &mut KernelAddressSpace, + range: CopyRange, + buffer_len: usize, + source: *const u8, + destination: *mut u8, + direction: CopyDirection, + ) -> Result<(), AddressSpaceError> { + let mut address = range.start; + let mut offset = 0usize; + while address < range.end { + let page_offset = (address as usize) & (Size4KiB::SIZE as usize - 1); + let page_remaining = Size4KiB::SIZE as usize - page_offset; + let remaining = buffer_len - offset; + let chunk_len = core::cmp::min(page_remaining, remaining); + let physical = match self.translate(kernel, VirtAddr::new(address))? { + Translation::Mapped { phys_addr, flags } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) + && (direction == CopyDirection::FromUser + || flags.contains(PageTableFlags::WRITABLE)) => + { + phys_addr + } + _ => return Err(AddressSpaceError::InvalidCopyRange), + }; + let alias = kernel.physical_alias(physical)?; + // SAFETY: the preceding translation is for this page chunk, + // validates effective USER access (and WRITE for ToUser), and + // physical_alias validates a supervisor-only mapped physical + // address. The kernel buffer is a valid uniquely borrowed slice. + unsafe { + match direction { + CopyDirection::FromUser => core::ptr::copy_nonoverlapping( + alias.as_ptr::(), + destination.add(offset), + chunk_len, + ), + CopyDirection::ToUser => core::ptr::copy_nonoverlapping( + source.add(offset), + alias.as_mut_ptr::(), + chunk_len, + ), + } + } + offset = offset + .checked_add(chunk_len) + .ok_or(AddressSpaceError::InvalidCopyRange)?; + address = address + .checked_add(chunk_len as u64) + .ok_or(AddressSpaceError::InvalidCopyRange)?; + } + Ok(()) + } + + pub fn unmap_user_page( + &mut self, + kernel: &mut KernelAddressSpace, + page: Page, + ) -> Result, AddressSpaceError> { + validate_user_address(page.start_address())?; + let next_count = self + .mapped_user_pages + .checked_sub(1) + .ok_or(AddressSpaceError::AccountingOverflow)?; + match self.translate(kernel, page.start_address())? { + Translation::Mapped { flags, .. } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) => {} + _ => return Err(AddressSpaceError::UserPageNotMapped), + } + let frame = + kernel.with_inactive_mapper(self, |mapper, _| paging::unmap_page(mapper, page))?; + self.mapped_user_pages = next_count; + Ok(frame) + } + + pub fn translate( + &mut self, + kernel: &mut KernelAddressSpace, + address: VirtAddr, + ) -> Result { + if VirtAddr::try_new(address.as_u64()).is_err() { + return Err(AddressSpaceError::Mapping(MemoryError::NonCanonicalAddress)); + } + kernel.with_inactive_mapper(self, |mapper, _| Ok(paging::translate(mapper, address))) + } + + /// Returns the supervisor physical-map alias for a mapped user page. + /// This narrow helper exists only for the deterministic preemption test, + /// whose trusted payload increments a private counter page. + #[cfg(feature = "process-preemption-test")] + pub(crate) fn test_user_page_kernel_alias( + &mut self, + kernel: &mut KernelAddressSpace, + page: Page, + ) -> Result { + validate_user_address(page.start_address())?; + match self.translate(kernel, page.start_address())? { + Translation::Mapped { phys_addr, flags } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) + && flags.contains(PageTableFlags::WRITABLE) => + { + kernel.physical_alias(phys_addr) + } + _ => Err(AddressSpaceError::UserPageNotMapped), + } + } +} + +#[allow(dead_code)] // Used by the checked user-copy API, currently no buffer syscall consumes it. +#[derive(Clone, Copy)] +struct CopyRange { + start: u64, + end: u64, +} + +#[allow(dead_code)] // Used by the checked user-copy API, currently no buffer syscall consumes it. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum CopyDirection { + FromUser, + ToUser, +} + +#[allow(dead_code)] // Used by the checked user-copy API, currently no buffer syscall consumes it. +fn validate_copy_range(start: u64, len: usize) -> Result { + if len == 0 { + return Ok(CopyRange { start, end: start }); + } + let end = start + .checked_add(u64::try_from(len).map_err(|_| AddressSpaceError::InvalidCopyRange)?) + .ok_or(AddressSpaceError::InvalidCopyRange)?; + if VirtAddr::try_new(start).is_err() + || VirtAddr::try_new(end - 1).is_err() + || start < super::USER_MIN + || end > super::USER_MAX + { + return Err(AddressSpaceError::InvalidCopyRange); + } + Ok(CopyRange { start, end }) +} + +fn validate_user_address(address: VirtAddr) -> Result<(), AddressSpaceError> { + if address.as_u64() < super::USER_MIN || address.as_u64() >= super::USER_MAX { + return Err(AddressSpaceError::OutsideUserRange); + } + Ok(()) +} + +fn validate_user_page( + page: Page, + permissions: Permissions, +) -> Result<(), AddressSpaceError> { + validate_user_address(page.start_address())?; + let end = page + .start_address() + .as_u64() + .checked_add(Size4KiB::SIZE) + .ok_or(AddressSpaceError::OutsideUserRange)?; + if end > super::USER_MAX { + return Err(AddressSpaceError::OutsideUserRange); + } + if !permissions.user_accessible { + return Err(AddressSpaceError::NonUserPermissions); + } + if permissions.writable && permissions.executable { + return Err(AddressSpaceError::Mapping(MemoryError::InvalidPermissions)); + } + Ok(()) +} + /// The kernel's active page-table mapper plus the boot-time physical frame /// allocator, held behind one lock so the two can never fall out of sync /// with each other. pub struct KernelAddressSpace { mapper: OffsetPageTable<'static>, frame_allocator: BootFrameAllocator, + physical_memory_offset: VirtAddr, + boot_mappings: BootMappingInventory, + kernel_root: PhysFrame, + kernel_root_flags: Cr3Flags, + boot_mapping_audited: bool, } impl KernelAddressSpace { + /// Prepares a process-root switch only after proving that its kernel-half + /// PML4 entries exactly match the currently audited kernel root. The + /// returned token must be activated after this lock guard is released. + #[cfg(feature = "qemu-test")] + pub(crate) fn prepare_address_space_activation( + &mut self, + address_space: &AddressSpace, + ) -> Result { + self.prepare_handle(address_space.handle()) + } + + /// Loads a previously issued opaque process-root handle after validating + /// that it is still represented by the audited shared kernel-half tree. + pub(crate) fn prepare_handle( + &mut self, + address_space: AddressSpaceHandle, + ) -> Result { + if x86_64::instructions::interrupts::are_enabled() { + return Err(AddressSpaceError::UnsafeKernelMapping); + } + if !self.boot_mapping_audited { + return Err(AddressSpaceError::UnsafeKernelMapping); + } + self.verify_kernel_clone(address_space)?; + Ok(VerifiedRootSwitch { + root: address_space.0, + flags: self.kernel_root_flags, + }) + } + + /// Restores the audited kernel root. Used on the return-to-kernel side of + /// a switch plan and by root-switch validation tests. + pub(crate) fn prepare_kernel_root_activation( + &mut self, + ) -> Result { + if x86_64::instructions::interrupts::are_enabled() { + return Err(AddressSpaceError::UnsafeKernelMapping); + } + Ok(VerifiedRootSwitch { + root: self.kernel_root, + flags: self.kernel_root_flags, + }) + } + + fn verify_kernel_clone( + &self, + address_space: AddressSpaceHandle, + ) -> Result<(), AddressSpaceError> { + let alias = self.physical_alias(address_space.0.start_address())?; + // SAFETY: the root frame is privately owned by this AddressSpace, + // mapped through the audited supervisor physical map, and read-only + // for this verification while the kernel address-space lock is held. + let process_root = unsafe { &*alias.as_ptr::() }; + let kernel_root = self.mapper.level_4_table(); + for index in 256..512 { + let expected = &kernel_root[index]; + let actual = &process_root[index]; + let expected_present = expected.flags().contains(PageTableFlags::PRESENT); + let actual_present = actual.flags().contains(PageTableFlags::PRESENT); + let hardware_flags = !PageTableFlags::ACCESSED; + if expected_present != actual_present + || expected_present + && (expected.addr() != actual.addr() + // Accessed is hardware-managed independently in each + // root even when the two entries name the same tree. + || expected.flags() & hardware_flags + != actual.flags() & hardware_flags + || actual.flags().contains(PageTableFlags::USER_ACCESSIBLE)) + { + return Err(AddressSpaceError::UnsafeKernelMapping); + } + } + Ok(()) + } + + /// Creates an inactive address-space root by cloning the already-audited + /// supervisor-only upper half. Lower-half bootloader mappings are never + /// copied into a process root. + pub fn create_address_space(&mut self) -> Result { + if !self.boot_mapping_audited { + return Err(AddressSpaceError::UnsafeKernelMapping); + } + self.verify_live_kernel_mappings() + .map_err(|_| AddressSpaceError::KernelMappingMissing)?; + + let root_frame = self + .frame_allocator + .allocate_frame() + .ok_or(AddressSpaceError::FrameAllocationFailed)?; + let root_alias = self.physical_alias(root_frame.start_address())?; + // SAFETY: the allocated frame is uniquely owned by the monotonic + // allocator, its complete page is reachable through the audited + // supervisor physical map, and no other reference to this new root + // exists. The closure is IRQ-masked by with_kernel_address_space. + unsafe { + core::ptr::write_bytes(root_alias.as_mut_ptr::(), 0, Size4KiB::SIZE as usize) + }; + + let kernel_root_alias = self.physical_alias(self.kernel_root.start_address())?; + // SAFETY: kernel_root is the active root captured at initialization; + // physical_alias proved its mapped supervisor alias. This immutable + // reference is used only while the address-space lock is held. + let kernel_root = unsafe { &*kernel_root_alias.as_ptr::() }; + // SAFETY: root_frame is fresh, zeroed, and not active; it has no other + // references. The mutable reference is scoped to this operation. + let process_root = unsafe { &mut *root_alias.as_mut_ptr::() }; + for index in 256..512 { + let entry = &kernel_root[index]; + if !entry.flags().contains(PageTableFlags::PRESENT) { + continue; + } + if entry.flags().contains(PageTableFlags::USER_ACCESSIBLE) { + return Err(AddressSpaceError::UnsafeKernelMapping); + } + process_root[index] = entry.clone(); + } + Ok(AddressSpace { + root: root_frame, + mapped_user_pages: 0, + }) + } + + pub(super) fn physical_alias(&self, physical: PhysAddr) -> Result { + let virtual_address = self + .physical_memory_offset + .as_u64() + .checked_add(physical.as_u64()) + .ok_or(AddressSpaceError::InvalidRoot)?; + let alias = + VirtAddr::try_new(virtual_address).map_err(|_| AddressSpaceError::InvalidRoot)?; + self.verify_kernel_root_mapping(alias)?; + Ok(alias) + } + + pub(super) fn verify_kernel_root_mapping( + &self, + address: VirtAddr, + ) -> Result<(), AddressSpaceError> { + match paging::translate(&self.mapper, address) { + Translation::Mapped { flags, .. } + if !flags.contains(PageTableFlags::USER_ACCESSIBLE) => + { + Ok(()) + } + // The audited bootloader physical map may use 2 MiB/1 GiB leaf + // mappings. Its upper-half tree was recursively checked for + // USER bits before this alias is accepted. + Translation::HugePage { .. } => Ok(()), + _ => Err(AddressSpaceError::UnsafeKernelMapping), + } + } + + fn allocate_zeroed_frame(&mut self) -> Result, AddressSpaceError> { + let frame = self + .frame_allocator + .allocate_frame() + .ok_or(AddressSpaceError::FrameAllocationFailed)?; + let alias = self.physical_alias(frame.start_address())?; + // SAFETY: the monotonic allocator uniquely returned this full frame; + // its physical-map alias was verified supervisor-mapped and no user + // mapping exposes it before initialization completes. + unsafe { core::ptr::write_bytes(alias.as_mut_ptr::(), 0, Size4KiB::SIZE as usize) }; + Ok(frame) + } + + fn with_inactive_mapper( + &mut self, + address_space: &mut AddressSpace, + f: impl FnOnce(&mut OffsetPageTable<'_>, &mut BootFrameAllocator) -> Result, + ) -> Result { + let alias = self.physical_alias(address_space.root.start_address())?; + // SAFETY: this AddressSpace exclusively owns its root frame, which + // is not loaded in CR3. The kernel address-space mutex serializes all + // page-table editing and interrupts are disabled, so this is the only + // mutable root reference and all child frames remain allocator-owned. + let root = unsafe { &mut *alias.as_mut_ptr::() }; + // SAFETY: physical_memory_offset is the bootloader mapping audited at + // init and retained for kernel lifetime; `root` is the inactive root. + let mut mapper = unsafe { OffsetPageTable::new(root, self.physical_memory_offset) }; + f(&mut mapper, &mut self.frame_allocator).map_err(AddressSpaceError::from) + } + + fn verify_live_kernel_mappings(&self) -> Result<(), &'static str> { + self.audit_upper_pml4()?; + let heap_end = super::heap::HEAP_END; + let mut address = super::heap::HEAP_START; + while address < heap_end { + self.verify_kernel_rw_nx(address)?; + address = address + .checked_add(Size4KiB::SIZE) + .ok_or("heap range overflow")?; + } + for slot in 0..crate::arch::x86_64::task::MAX_KERNEL_TASKS { + let (low, high) = + crate::arch::x86_64::task::stack_bounds(slot).ok_or("invalid task stack slot")?; + let mut address = low; + while address < high { + self.verify_kernel_rw_nx(address)?; + address = address + .checked_add(Size4KiB::SIZE) + .ok_or("task stack range overflow")?; + } + for guard in [ + low.checked_sub(Size4KiB::SIZE) + .ok_or("task stack guard underflow")?, + high, + ] { + if !matches!( + paging::translate(&self.mapper, VirtAddr::new(guard)), + Translation::NotMapped + ) { + return Err("task stack guard unexpectedly mapped"); + } + } + } + Ok(()) + } + + fn verify_kernel_rw_nx(&self, address: u64) -> Result<(), &'static str> { + match paging::translate(&self.mapper, VirtAddr::new(address)) { + Translation::Mapped { flags, .. } + if flags.contains(PageTableFlags::WRITABLE | PageTableFlags::NO_EXECUTE) + && !flags.contains(PageTableFlags::USER_ACCESSIBLE) => + { + Ok(()) + } + _ => Err("kernel heap/task stack is not supervisor RW/NX"), + } + } + + fn verify_boot_mapping_policy(&self) -> Result<(), &'static str> { + let required = [ + ("kernel stack", Some(self.boot_mappings.kernel_stack)), + ("BootInfo", Some(self.boot_mappings.boot_info)), + ("memory regions", Some(self.boot_mappings.memory_regions)), + ("physical map", Some(self.boot_mappings.physical_memory)), + ("framebuffer", self.boot_mappings.framebuffer), + ("ramdisk", self.boot_mappings.ramdisk), + ]; + + for (name, range) in required { + let Some(range) = range else { continue }; + self.verify_dynamic_mapping(name, range)?; + } + + self.verify_kernel_image_segments()?; + + let current_rsp: u64; + // SAFETY: reads the current stack pointer without changing it. + unsafe { + core::arch::asm!("mov {}, rsp", out(reg) current_rsp, options(nomem, nostack, preserves_flags)); + } + self.verify_upper_address("current kernel RSP", current_rsp)?; + + let (root_frame, _) = Cr3::read(); + let root_virtual = self.physical_memory_offset + root_frame.start_address().as_u64(); + self.verify_upper_address("active PML4 physical-map alias", root_virtual.as_u64())?; + self.verify_supervisor_address(root_virtual.as_u64())?; + + self.audit_upper_pml4()?; + Ok(()) + } + + fn verify_kernel_image_segments(&self) -> Result<(), &'static str> { + let base = self.boot_mappings.kernel_image_base; + self.verify_upper_address("kernel image base", base)?; + if !(super::BOOTLOADER_DYNAMIC_START..super::BOOTLOADER_DYNAMIC_END).contains(&base) { + return Err("kernel image base lies outside the bootloader dynamic arena"); + } + + let elf_end = self + .boot_mappings + .kernel_elf_physical + .checked_add(self.boot_mappings.kernel_elf_len) + .ok_or("kernel ELF physical range overflow")?; + if elf_end > self.boot_mappings.physical_memory.len { + return Err("kernel ELF bytes are outside the physical-memory mapping"); + } + let elf_virtual = self.physical_memory_offset + self.boot_mappings.kernel_elf_physical; + self.verify_supervisor_address(elf_virtual.as_u64())?; + + // SAFETY: the bootloader's physical map is audited before this call; + // checked arithmetic above proves the entire kernel ELF byte range is + // within the mapped physical extent. This creates a read-only slice + // over the bootloader-owned image for bounded field decoding only. + let bytes = unsafe { + core::slice::from_raw_parts( + elf_virtual.as_ptr::(), + usize::try_from(self.boot_mappings.kernel_elf_len) + .map_err(|_| "kernel ELF length does not fit usize")?, + ) + }; + if bytes.get(0..4) != Some(b"\x7fELF") + || bytes.get(4) != Some(&2) + || bytes.get(5) != Some(&1) + || read_u16(bytes, 18)? != 62 + || read_u16(bytes, 16)? != 3 + { + return Err("bootloader kernel ELF identity is unsupported"); + } + let entry = read_u64(bytes, 24)?; + let phoff = usize::try_from(read_u64(bytes, 32)?) + .map_err(|_| "kernel program-header offset does not fit usize")?; + let phentsize = usize::from(read_u16(bytes, 54)?); + let phnum = usize::from(read_u16(bytes, 56)?); + if phentsize != 56 { + return Err("kernel program-header entry size is invalid"); + } + let table_len = phentsize + .checked_mul(phnum) + .ok_or("kernel program-header table size overflow")?; + let table_end = phoff + .checked_add(table_len) + .ok_or("kernel program-header table range overflow")?; + if table_end > bytes.len() { + return Err("kernel program-header table is truncated"); + } + + let entry_virtual = base.checked_add(entry).ok_or("kernel entry overflow")?; + self.verify_upper_address("kernel entry", entry_virtual)?; + let mut entry_in_executable_segment = false; + let mut load_segments = 0u16; + for index in 0..phnum { + let header = phoff + .checked_add( + index + .checked_mul(phentsize) + .ok_or("program-header offset overflow")?, + ) + .ok_or("program-header address overflow")?; + if read_u32(bytes, header)? != 1 { + continue; + } + let flags = read_u32(bytes, header + 4)?; + let writable = flags & 2 != 0; + let executable = flags & 1 != 0; + if writable && executable { + return Err("bootloader kernel PT_LOAD violates W^X"); + } + let vaddr = read_u64(bytes, header + 16)?; + let memsz = read_u64(bytes, header + 40)?; + if memsz == 0 { + continue; + } + let start = base + .checked_add(vaddr) + .ok_or("kernel segment VA overflow")?; + let end = start + .checked_add(memsz) + .ok_or("kernel segment end overflow")?; + self.verify_upper_address("kernel PT_LOAD start", start)?; + self.verify_upper_address("kernel PT_LOAD end", end - 1)?; + if start < super::BOOTLOADER_DYNAMIC_START || end > super::BOOTLOADER_DYNAMIC_END { + return Err("kernel PT_LOAD segment lies outside the dynamic arena"); + } + if flags & 1 != 0 && entry_virtual >= start && entry_virtual < end { + entry_in_executable_segment = true; + } + let first_page = start & !(Size4KiB::SIZE - 1); + let rounded_end = end + .checked_add(Size4KiB::SIZE - 1) + .ok_or("kernel segment page-rounding overflow")? + & !(Size4KiB::SIZE - 1); + let mut page = first_page; + while page < rounded_end { + let relro = kernel_page_is_relro(bytes, phoff, phentsize, phnum, base, page)?; + self.verify_kernel_segment_page(page, writable && !relro, executable)?; + page = page + .checked_add(Size4KiB::SIZE) + .ok_or("kernel segment page iteration overflow")?; + } + load_segments = load_segments + .checked_add(1) + .ok_or("kernel PT_LOAD count overflow")?; + crate::kprintln!( + "[memory] audited kernel PT_LOAD: {start:#x}..{end:#x} PML4 {}..{} flags={flags:#x}", + pml4_index(start), + pml4_index(end - 1) + ); + } + if load_segments == 0 || !entry_in_executable_segment { + return Err("kernel ELF has no mapped executable entry segment"); + } + crate::kprintln!( + "[memory] kernel ELF entry {entry_virtual:#x}; {load_segments} PT_LOAD segments audited" + ); + Ok(()) + } + + fn verify_kernel_segment_page( + &self, + address: u64, + writable: bool, + executable: bool, + ) -> Result<(), &'static str> { + match paging::translate(&self.mapper, VirtAddr::new(address)) { + Translation::Mapped { flags, .. } => { + if flags.contains(PageTableFlags::USER_ACCESSIBLE) + || flags.contains(PageTableFlags::WRITABLE) != writable + || flags.contains(PageTableFlags::NO_EXECUTE) == executable + { + crate::kprintln!( + "[memory] kernel PT_LOAD permission mismatch at {address:#x}: {flags:?}" + ); + return Err("kernel PT_LOAD permissions do not match ELF flags"); + } + if flags.contains(PageTableFlags::WRITABLE) + && !flags.contains(PageTableFlags::NO_EXECUTE) + { + return Err("kernel PT_LOAD mapping violates W^X"); + } + Ok(()) + } + Translation::HugePage { .. } => { + Err("kernel image PT_LOAD unexpectedly uses a huge page") + } + Translation::NotMapped | Translation::Unsupported => { + Err("kernel PT_LOAD page is absent or unsupported") + } + } + } + + fn verify_dynamic_mapping(&self, name: &str, range: MappingRange) -> Result<(), &'static str> { + if range.len == 0 { + crate::kprintln!("[memory] required boot mapping has zero length: {name}"); + return Err("required boot mapping has zero length"); + } + let end = range + .start + .checked_add(range.len) + .ok_or("required boot mapping range overflow")?; + self.verify_upper_address(name, range.start)?; + self.verify_upper_address(name, end - 1)?; + if range.start < super::BOOTLOADER_DYNAMIC_START || end > super::BOOTLOADER_DYNAMIC_END { + crate::kprintln!( + "[memory] boot mapping outside dynamic arena: {name} {:#x}..{:#x}", + range.start, + end + ); + return Err("required boot mapping lies outside the reserved dynamic arena"); + } + if range.start < super::USER_MAX && end > super::USER_MIN { + crate::kprintln!( + "[memory] boot mapping overlaps userspace VA policy: {name} {:#x}..{:#x}", + range.start, + end + ); + return Err("required boot mapping overlaps userspace"); + } + + let audit_stride = if name == "physical map" { + 2 * 1024 * 1024 + } else { + Size4KiB::SIZE + }; + let mut address = range.start; + while address < end { + self.verify_supervisor_address(address)?; + address = address + .checked_add(audit_stride) + .ok_or("required mapping audit address overflow")?; + } + self.verify_supervisor_address(end - 1)?; + crate::kprintln!( + "[memory] audited {name}: {:#x}..{:#x} (PML4 {}..{})", + range.start, + end, + pml4_index(range.start), + pml4_index(end - 1) + ); + Ok(()) + } + + fn verify_upper_address(&self, name: &str, address: u64) -> Result<(), &'static str> { + if VirtAddr::try_new(address).is_err() || pml4_index(address) < 256 { + crate::kprintln!( + "[memory] required mapping is not canonical upper-half: {name} {address:#x}" + ); + return Err("required kernel mapping is not in the upper canonical half"); + } + Ok(()) + } + + fn verify_supervisor_address(&self, address: u64) -> Result<(), &'static str> { + match paging::translate( + &self.mapper, + VirtAddr::try_new(address).map_err(|_| "noncanonical mapping")?, + ) { + Translation::Mapped { flags, .. } => { + if flags.contains(PageTableFlags::USER_ACCESSIBLE) { + crate::kprintln!( + "[memory] required kernel mapping is USER at {address:#x}: {flags:?}" + ); + Err("required kernel mapping is user-accessible") + } else { + Ok(()) + } + } + Translation::HugePage { .. } => Ok(()), + Translation::NotMapped | Translation::Unsupported => { + crate::kprintln!("[memory] required kernel mapping missing at {address:#x}"); + Err("required kernel mapping is absent") + } + } + } + + fn audit_upper_pml4(&self) -> Result<(), &'static str> { + let root = self.mapper.level_4_table(); + for index in 256..512 { + let entry = &root[index]; + let flags = entry.flags(); + if !flags.contains(PageTableFlags::PRESENT) { + continue; + } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) { + crate::kprintln!("[memory] USER bit set in kernel PML4 entry {index}"); + return Err("upper-half PML4 entry is user-accessible"); + } + self.audit_table(entry.addr(), 3)?; + } + Ok(()) + } + + fn audit_table(&self, frame: PhysAddr, level: u8) -> Result<(), &'static str> { + let virtual_address = self.physical_memory_offset + frame.as_u64(); + self.verify_supervisor_address(virtual_address.as_u64())?; + // SAFETY: `frame` is reached through the verified supervisor-only + // physical map; this walk is read-only and IRQs are disabled while + // the single kernel address-space lock is held during initialization. + let table = unsafe { &*virtual_address.as_ptr::() }; + for entry in table.iter() { + let flags = entry.flags(); + if !flags.contains(PageTableFlags::PRESENT) { + continue; + } + if flags.contains(PageTableFlags::USER_ACCESSIBLE) { + return Err("upper-half page-table entry is user-accessible"); + } + if level > 1 && !flags.contains(PageTableFlags::HUGE_PAGE) { + self.audit_table(entry.addr(), level - 1)?; + } + } + Ok(()) + } + /// Allocates a fresh physical frame and maps `page` to it with /// `permissions`. Returns the frame that was mapped. pub fn map_new_page( @@ -36,6 +1194,18 @@ impl KernelAddressSpace { page: Page, permissions: Permissions, ) -> Result, MemoryError> { + if permissions.user_accessible { + let start = page.start_address().as_u64(); + let end = start + .checked_add(Size4KiB::SIZE) + .ok_or(MemoryError::NonCanonicalAddress)?; + if start < super::USER_MIN + || end > super::USER_MAX + || permissions.writable && permissions.executable + { + return Err(MemoryError::InvalidPermissions); + } + } let frame = self .frame_allocator .allocate_frame() @@ -63,6 +1233,18 @@ impl KernelAddressSpace { page: Page, permissions: Permissions, ) -> Result<(), MemoryError> { + if permissions.user_accessible { + let start = page.start_address().as_u64(); + let end = start + .checked_add(Size4KiB::SIZE) + .ok_or(MemoryError::NonCanonicalAddress)?; + if start < super::USER_MIN + || end > super::USER_MAX + || permissions.writable && permissions.executable + { + return Err(MemoryError::InvalidPermissions); + } + } paging::update_permissions(&mut self.mapper, page, permissions) } @@ -127,9 +1309,80 @@ unsafe fn active_level_4_table(phys_mem_offset: VirtAddr) -> &'static mut PageTa unsafe { &mut *page_table_ptr } } +fn pml4_index(address: u64) -> u16 { + ((address >> 39) & 0x1ff) as u16 +} + +fn read_u16(bytes: &[u8], offset: usize) -> Result { + let data = bytes + .get(offset..offset.checked_add(2).ok_or("ELF field offset overflow")?) + .ok_or("kernel ELF field is truncated")?; + Ok(u16::from_le_bytes([data[0], data[1]])) +} + +fn read_u32(bytes: &[u8], offset: usize) -> Result { + let data = bytes + .get(offset..offset.checked_add(4).ok_or("ELF field offset overflow")?) + .ok_or("kernel ELF field is truncated")?; + Ok(u32::from_le_bytes([data[0], data[1], data[2], data[3]])) +} + +fn read_u64(bytes: &[u8], offset: usize) -> Result { + let data = bytes + .get(offset..offset.checked_add(8).ok_or("ELF field offset overflow")?) + .ok_or("kernel ELF field is truncated")?; + Ok(u64::from_le_bytes([ + data[0], data[1], data[2], data[3], data[4], data[5], data[6], data[7], + ])) +} + +fn kernel_page_is_relro( + bytes: &[u8], + phoff: usize, + phentsize: usize, + phnum: usize, + image_base: u64, + page: u64, +) -> Result { + const PT_GNU_RELRO: u32 = 0x6474_e552; + for index in 0..phnum { + let header = phoff + .checked_add( + index + .checked_mul(phentsize) + .ok_or("program-header offset overflow")?, + ) + .ok_or("program-header address overflow")?; + if read_u32(bytes, header)? != PT_GNU_RELRO { + continue; + } + let start = image_base + .checked_add(read_u64(bytes, header + 16)?) + .ok_or("kernel RELRO start overflow")?; + let len = read_u64(bytes, header + 40)?; + if len == 0 { + continue; + } + let end = start.checked_add(len).ok_or("kernel RELRO end overflow")?; + let rounded_start = start & !(Size4KiB::SIZE - 1); + let rounded_end = end + .checked_add(Size4KiB::SIZE - 1) + .ok_or("kernel RELRO page-rounding overflow")? + & !(Size4KiB::SIZE - 1); + if page >= rounded_start && page < rounded_end { + return Ok(true); + } + } + Ok(false) +} + /// Builds the single kernel address space. Must be called at most once; see /// [`super::init`], which enforces this through [`Once::call_once`]. -pub(super) fn init(phys_mem_offset: VirtAddr, frame_allocator: BootFrameAllocator) { +pub(super) fn init( + phys_mem_offset: VirtAddr, + frame_allocator: BootFrameAllocator, + boot_mappings: BootMappingInventory, +) { KERNEL_ADDRESS_SPACE.call_once(|| { // SAFETY: `phys_mem_offset` is forwarded unchanged from // `BootInfo::physical_memory_offset` by `super::init`, which itself @@ -144,10 +1397,21 @@ pub(super) fn init(phys_mem_offset: VirtAddr, frame_allocator: BootFrameAllocato // requires. let mapper = unsafe { OffsetPageTable::new(level_4_table, phys_mem_offset) }; - Mutex::new(KernelAddressSpace { + let (kernel_root, kernel_root_flags) = Cr3::read(); + let mut space = KernelAddressSpace { mapper, frame_allocator, - }) + physical_memory_offset: phys_mem_offset, + boot_mappings, + kernel_root, + kernel_root_flags, + boot_mapping_audited: false, + }; + space + .verify_boot_mapping_policy() + .expect("runtime boot mapping audit failed; refusing process roots"); + space.boot_mapping_audited = true; + Mutex::new(space) }); } diff --git a/kernel/src/arch/x86_64/memory/mod.rs b/kernel/src/arch/x86_64/memory/mod.rs index 2555004..ceb821f 100644 --- a/kernel/src/arch/x86_64/memory/mod.rs +++ b/kernel/src/arch/x86_64/memory/mod.rs @@ -21,6 +21,7 @@ mod allocator; mod frame_allocator; mod heap; mod paging; +mod user_stack; #[cfg(feature = "qemu-test")] use core::sync::atomic::{AtomicBool, AtomicU64, Ordering}; @@ -32,12 +33,19 @@ use x86_64::structures::idt::PageFaultErrorCode; use x86_64::{ VirtAddr, registers::control::{Cr0, Cr0Flags, Cr4, Cr4Flags}, - registers::model_specific::{Efer, EferFlags}, + registers::model_specific::{Efer, EferFlags, Msr}, structures::paging::PageTableFlags, }; -pub use address_space::with_kernel_address_space; +pub use address_space::KernelAddressSpace; +pub use address_space::{ + AddressSpace, AddressSpaceError, AddressSpaceHandle, ValidatedUserContext, + with_kernel_address_space, +}; pub use paging::{Permissions, Translation}; +#[cfg(feature = "qemu-test")] +pub use user_stack::USER_STACK_PAGES; +pub use user_stack::{USER_STACK_SIZE, USER_STACK_TOP, UserStack}; use frame_allocator::BootFrameAllocator; use paging::MemoryError; @@ -64,6 +72,20 @@ pub const TEMP_MAPPING_BASE: u64 = 0xFFFF_9000_0000_0000; #[cfg_attr(not(feature = "userspace-test"), allow(dead_code))] pub const USERSPACE_RANGE_BASE: u64 = 0x0000_6000_0000_0000; +/// First accepted address for process mappings; the first 64 KiB stays unmapped. +pub const USER_MIN: u64 = 0x0000_0000_0001_0000; +/// Exclusive upper bound for the lower canonical userspace half. +pub const USER_MAX: u64 = 0x0000_8000_0000_0000; + +/// PML4 index 256: first upper-half slot reserved for bootloader dynamics. +pub const BOOTLOADER_DYNAMIC_START: u64 = 0xffff_8000_0000_0000; +/// Exclusive boundary before PML4 index 288, where pvKernel's explicit +/// scratch, heap, and task-stack mappings begin. +pub const BOOTLOADER_DYNAMIC_END: u64 = 0xffff_9000_0000_0000; +/// The bootloader rounds its dynamic-range end to PML4 entries. This page +/// sentinel makes it reserve index 288 onward and leave indices 256..287. +pub const BOOTLOADER_DYNAMIC_END_SENTINEL: u64 = BOOTLOADER_DYNAMIC_END - 4096; + /// Placeholder [`MemoryRegions`] used only to give /// `BootInfo::memory_regions` a valid value to hold once [`init`] has moved /// the real one out of it (see [`init`]). Always empty, and never read @@ -101,7 +123,7 @@ static mut EMPTY_MEMORY_REGIONS: [MemoryRegion; 0] = []; /// misconfiguration, not an ordinary runtime condition; none of this /// module's operations are possible without it, so continuing would be /// unsafe. -pub fn init(boot_info: &'static mut BootInfo) { +pub fn init(boot_info: &'static mut BootInfo, framebuffer_mapping: Option<(u64, u64)>) { // Paging-mode prerequisites must hold before anything below interprets // CR3 as a four-level table root (`address_space::init` does, via // `active_level_4_table`/`OffsetPageTable::new`). @@ -112,6 +134,51 @@ pub fn init(boot_info: &'static mut BootInfo) { .into_option() .expect("bootloader did not provide a physical-memory offset mapping"); let phys_mem_offset = VirtAddr::new(physical_memory_offset); + let max_physical_address = boot_info + .memory_regions + .iter() + .map(|region| region.end) + .max() + .unwrap_or(0) + .max(0x1_0000_0000); + let memory_regions_start = boot_info.memory_regions.as_ptr() as u64; + let memory_regions_len = boot_info + .memory_regions + .len() + .checked_mul(core::mem::size_of::()) + .and_then(|bytes| u64::try_from(bytes).ok()) + .expect("boot memory-map byte length overflow"); + let ramdisk = boot_info + .ramdisk_addr + .into_option() + .map(|start| address_space::MappingRange { + start, + len: boot_info.ramdisk_len, + }); + let boot_mappings = address_space::BootMappingInventory { + kernel_image_base: boot_info.kernel_image_offset, + kernel_elf_physical: boot_info.kernel_addr, + kernel_elf_len: boot_info.kernel_len, + kernel_stack: address_space::MappingRange { + start: boot_info.kernel_stack_bottom, + len: boot_info.kernel_stack_len, + }, + boot_info: address_space::MappingRange { + start: boot_info as *mut BootInfo as u64, + len: core::mem::size_of::() as u64, + }, + memory_regions: address_space::MappingRange { + start: memory_regions_start, + len: memory_regions_len, + }, + physical_memory: address_space::MappingRange { + start: physical_memory_offset, + len: max_physical_address, + }, + framebuffer: framebuffer_mapping + .map(|(start, len)| address_space::MappingRange { start, len }), + ramdisk, + }; // Moves the real `MemoryRegions` out of `boot_info`, leaving the // always-empty placeholder in its place so the field stays valid. This @@ -146,7 +213,7 @@ pub fn init(boot_info: &'static mut BootInfo) { let frame_allocator = unsafe { BootFrameAllocator::new(regions) }; let usable_frames = frame_allocator.usable_frame_count(); - address_space::init(phys_mem_offset, frame_allocator); + address_space::init(phys_mem_offset, frame_allocator, boot_mappings); kprintln!("[memory] physical frame allocator initialized"); kprintln!( @@ -175,6 +242,302 @@ pub fn run_heap_smoke_tests() { heap::run_smoke_tests(); } +/// Exercises inactive-root construction without scheduling userspace. The +/// same lower-half VA must resolve to distinct frames in two roots, while +/// the active kernel mapping is shared supervisor-only in both. +#[cfg(feature = "qemu-test")] +pub fn run_address_space_smoke_test() { + use x86_64::structures::paging::{Page, PageSize, PageTableFlags, Size4KiB}; + + kprintln!("[process-test] checking inactive address-space roots"); + with_kernel_address_space(|kernel| { + let mut a = kernel + .create_address_space() + .expect("failed to create process A address space"); + let mut b = kernel + .create_address_space() + .expect("failed to create process B address space"); + let page = Page::::from_start_address(VirtAddr::new(USER_MIN)) + .expect("USER_MIN must be page aligned"); + let frame_a = a + .map_new_user_page(kernel, page, Permissions::USER_DATA) + .expect("failed to map process A test page"); + let frame_b = b + .map_new_user_page(kernel, page, Permissions::USER_DATA) + .expect("failed to map process B test page"); + assert_ne!(frame_a, frame_b, "process roots aliased one user frame"); + a.write_user_page(kernel, page, 0, &[0xAA]) + .expect("failed to initialize process A page"); + b.write_user_page(kernel, page, 0, &[0xBB]) + .expect("failed to initialize process B page"); + + for (space, frame, expected) in [(&mut a, frame_a, 0xAA), (&mut b, frame_b, 0xBB)] { + match space + .translate(kernel, VirtAddr::new(USER_MIN)) + .expect("inactive root translation failed") + { + Translation::Mapped { phys_addr, flags } => { + assert_eq!(phys_addr, frame.start_address()); + assert!(flags.contains(PageTableFlags::USER_ACCESSIBLE)); + assert!(flags.contains(PageTableFlags::WRITABLE)); + assert!(flags.contains(PageTableFlags::NO_EXECUTE)); + let alias = kernel + .physical_alias(phys_addr) + .expect("user test frame lacks supervisor physical alias"); + // SAFETY: each tested frame was uniquely allocated and + // zeroed, and the corresponding alias was verified as a + // present supervisor mapping while IRQs are masked. + let value = unsafe { alias.as_ptr::().read_volatile() }; + assert_eq!(value, expected); + } + other => panic!("test page missing from inactive root: {other:?}"), + } + } + assert_eq!(a.mapped_user_pages(), 1); + assert_eq!(b.mapped_user_pages(), 1); + + let (stack_low, _) = + crate::arch::x86_64::task::stack_bounds(0).expect("kernel stack slot zero must exist"); + let kernel_translation = kernel.translate(VirtAddr::new(stack_low)); + for space in [&mut a, &mut b] { + match space + .translate(kernel, VirtAddr::new(stack_low)) + .expect("shared kernel stack translation failed") + { + Translation::Mapped { phys_addr, flags } => { + match kernel_translation { + Translation::Mapped { + phys_addr: expected, + .. + } => assert_eq!(phys_addr, expected), + other => panic!("active kernel stack missing: {other:?}"), + } + assert!(!flags.contains(PageTableFlags::USER_ACCESSIBLE)); + } + other => panic!("kernel stack absent in process root: {other:?}"), + } + } + for address in [0x8000, USER_MAX - Size4KiB::SIZE] { + assert!(matches!( + a.translate(kernel, VirtAddr::new(address)).unwrap(), + Translation::NotMapped + )); + } + assert!(paging::canonical_virt_addr(USER_MAX).is_err()); + assert!(matches!( + kernel.translate(VirtAddr::new(USER_MIN)), + Translation::NotMapped + )); + }); + kprintln!("[process-test] inactive roots, isolation, and shared kernel mappings passed"); +} + +/// Loads each process root on the CPU and reads the same user VA through the +/// active translation. The kernel stack/code/globals remain live because the +/// audited upper-half PML4 entries are identical. This test runs with IF=0 +/// inside `with_kernel_address_space`; it does not schedule user contexts. +#[cfg(feature = "qemu-test")] +pub fn run_address_space_switch_smoke_test() { + use x86_64::structures::paging::{Page, Size4KiB}; + + kprintln!("[process-test] checking runtime CR3 address-space switches"); + let (a, b) = with_kernel_address_space(|kernel| { + let mut a = kernel + .create_address_space() + .expect("failed to create CR3 test root A"); + let mut b = kernel + .create_address_space() + .expect("failed to create CR3 test root B"); + let page = Page::::from_start_address(VirtAddr::new(USER_MIN)) + .expect("USER_MIN must be page aligned"); + a.map_new_user_page(kernel, page, Permissions::USER_DATA) + .expect("failed to map CR3 test page A"); + b.map_new_user_page(kernel, page, Permissions::USER_DATA) + .expect("failed to map CR3 test page B"); + a.write_user_page(kernel, page, 0, &[0xAA]) + .expect("failed to initialize CR3 test page A"); + b.write_user_page(kernel, page, 0, &[0xBB]) + .expect("failed to initialize CR3 test page B"); + (a, b) + }); + + for (space, slot, expected) in [(&a, 0, 0xAA), (&b, 1, 0xBB), (&a, 0, 0xAA), (&b, 1, 0xBB)] { + let root_switch = + with_kernel_address_space(|kernel| kernel.prepare_address_space_activation(space)) + .expect("audited process CR3 activation failed"); + root_switch + .activate() + .expect("verified process CR3 activation failed"); + let (_, kernel_stack_top) = crate::arch::x86_64::task::stack_bounds(slot) + .expect("user task kernel stack slot is unavailable"); + crate::arch::x86_64::gdt::set_current_rsp0(kernel_stack_top) + .expect("user task RSP0 rejected its owned kernel stack"); + // SAFETY: the selected root maps USER_MIN to a zeroed frame + // populated above; CPL0 may read a user page, the upper-half + // mapping audit preserves this code and stack, and IF is clear. + let actual = unsafe { VirtAddr::new(USER_MIN).as_ptr::().read_volatile() }; + assert_eq!(actual, expected, "active CR3 exposed another process page"); + } + let kernel_root = with_kernel_address_space(|kernel| kernel.prepare_kernel_root_activation()) + .expect("failed to prepare kernel CR3 restoration"); + kernel_root + .activate() + .expect("failed to restore kernel CR3 after smoke test"); + crate::arch::x86_64::gdt::set_kernel_fallback_rsp0() + .expect("failed to restore bootstrap RSP0 after smoke test"); + assert_eq!( + crate::arch::x86_64::gdt::current_rsp0().unwrap(), + crate::arch::x86_64::gdt::kernel_fallback_rsp0().as_u64() + ); + kprintln!("[process-test] runtime CR3 switching and same-VA isolation passed"); +} + +#[cfg(feature = "qemu-test")] +pub fn run_user_stack_smoke_test() { + use x86_64::structures::paging::{Page, PageSize, PageTableFlags, Size4KiB}; + + kprintln!("[process-test] checking guarded userspace stack and RSP0 policy"); + with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("failed to create user-stack test root"); + let stack = UserStack::map(&mut address_space, kernel) + .expect("failed to map fixed userspace stack"); + assert_eq!(stack.top, USER_STACK_TOP); + assert_eq!(stack.top - stack.bottom, USER_STACK_SIZE); + assert_eq!(stack.top % Size4KiB::SIZE, 0); + assert!(matches!( + address_space + .translate(kernel, VirtAddr::new(stack.guard)) + .unwrap(), + Translation::NotMapped + )); + let mut address = stack.bottom; + while address < stack.top { + match address_space + .translate(kernel, VirtAddr::new(address)) + .expect("user stack translation failed") + { + Translation::Mapped { flags, .. } => assert!(flags.contains( + PageTableFlags::PRESENT + | PageTableFlags::USER_ACCESSIBLE + | PageTableFlags::WRITABLE + | PageTableFlags::NO_EXECUTE + )), + other => panic!("user stack page is absent: {other:?}"), + } + address = address + .checked_add(Size4KiB::SIZE) + .expect("stack walk overflow"); + } + assert_eq!(address_space.mapped_user_pages(), USER_STACK_PAGES); + let (_, kernel_stack_top) = crate::arch::x86_64::task::stack_bounds(0) + .expect("kernel task stack slot zero must exist"); + crate::arch::x86_64::gdt::set_current_rsp0(kernel_stack_top) + .expect("validated task stack RSP0 was rejected"); + assert_eq!( + crate::arch::x86_64::gdt::current_rsp0().expect("TSS is unavailable"), + kernel_stack_top + ); + crate::arch::x86_64::gdt::set_kernel_fallback_rsp0() + .expect("static kernel fallback RSP0 could not be restored"); + assert_eq!( + crate::arch::x86_64::gdt::current_rsp0().expect("TSS is unavailable"), + crate::arch::x86_64::gdt::kernel_fallback_rsp0().as_u64() + ); + // Keep Page imported here as an explicit alignment/type check for the + // accepted stack boundary. + assert!(Page::::from_start_address(VirtAddr::new(stack.bottom)).is_ok()); + }); + kprintln!("[process-test] guarded user stack and RSP0 policy passed"); +} + +/// Exercises page-chunked user-copy validation and transfer against an +/// inactive process root. Invalid ranges must fail before either destination +/// buffer or user memory is partially changed. +#[cfg(feature = "user-copy-test")] +pub fn run_user_copy_test() { + use x86_64::structures::paging::{Page, PageSize, Size4KiB}; + + const FIRST: u64 = USER_MIN; + const SECOND: u64 = USER_MIN + Size4KiB::SIZE; + const READ_ONLY: u64 = USER_MIN + 4 * Size4KiB::SIZE; + const SUPERVISOR: u64 = USER_MIN + 5 * Size4KiB::SIZE; + + kprintln!("[process-test] checking safe cross-page user copies"); + with_kernel_address_space(|kernel| { + let mut space = kernel + .create_address_space() + .expect("failed to create user-copy test root"); + for base in [FIRST, SECOND] { + space + .map_new_user_page( + kernel, + Page::::from_start_address(VirtAddr::new(base)).unwrap(), + Permissions::USER_DATA, + ) + .expect("failed to map writable user-copy page"); + } + space + .map_new_user_page( + kernel, + Page::::from_start_address(VirtAddr::new(READ_ONLY)).unwrap(), + Permissions::USER_READ_ONLY, + ) + .expect("failed to map read-only user-copy page"); + space + .map_test_supervisor_page( + kernel, + Page::::from_start_address(VirtAddr::new(SUPERVISOR)).unwrap(), + ) + .expect("failed to map supervisor-only negative-test page"); + + let first_bytes = [0x10, 0x20, 0x30, 0x40]; + space + .copy_to_user(kernel, FIRST + 8, &first_bytes) + .expect("same-page copy_to_user failed"); + let mut copied = [0; 4]; + space + .copy_from_user(kernel, FIRST + 8, &mut copied) + .expect("same-page copy_from_user failed"); + assert_eq!(copied, first_bytes); + + let crossing = [0xA1, 0xA2, 0xB1, 0xB2]; + let cross_address = FIRST + Size4KiB::SIZE - 2; + space + .copy_to_user(kernel, cross_address, &crossing) + .expect("cross-page copy_to_user failed"); + let mut cross_output = [0; 4]; + space + .copy_from_user(kernel, cross_address, &mut cross_output) + .expect("cross-page copy_from_user failed"); + assert_eq!(cross_output, crossing); + + for (address, length) in [ + (0, 1), + (0xFFFF, 2), + (TEMP_MAPPING_BASE, 1), + (USER_MAX, 1), + (u64::MAX - 1, 8), + (SECOND + Size4KiB::SIZE - 1, 2), + (SUPERVISOR, 1), + ] { + let mut rejected = [0xCC; 16]; + let result = space.copy_from_user(kernel, address, &mut rejected[..length.min(16)]); + assert!(result.is_err(), "copy_from_user accepted {address:#x}"); + assert_eq!( + rejected, [0xCC; 16], + "failed read partially modified output" + ); + } + assert!(space.copy_to_user(kernel, READ_ONLY, &[0x55]).is_err()); + assert!(space.copy_to_user(kernel, SUPERVISOR, &[0x55]).is_err()); + assert!(space.copy_from_user(kernel, 0, &mut []).is_ok()); + assert!(space.copy_to_user(kernel, u64::MAX, &[]).is_ok()); + kprintln!("[process-test] user-copy positive and negative cases passed"); + }); +} + #[cfg(all(feature = "heap-guard-fault-test", feature = "qemu-test"))] pub(crate) fn take_expected_heap_guard_fault( address: u64, @@ -208,6 +571,74 @@ fn validate_paging_prerequisites() { "pvKernel requires execute-disable support with IA32_EFER.NXE enabled" ); + // `int 0x80` is the only supported privilege-entry ABI. UEFI may leave + // the fast-entry MSRs configured, so make SYSCALL unavailable and make + // SYSENTER fault instead of jumping to firmware-owned targets after a + // CPL3 payload executes either instruction. + let efer = Efer::read(); + if efer.contains(EferFlags::SYSTEM_CALL_EXTENSIONS) { + // SAFETY: only EFER.SCE is cleared; the read value preserves NXE and + // every other defined EFER feature bit required by the running kernel. + unsafe { Efer::write(efer & !EferFlags::SYSTEM_CALL_EXTENSIONS) }; + } + assert!( + !Efer::read().contains(EferFlags::SYSTEM_CALL_EXTENSIONS), + "pvKernel requires EFER.SCE clear; int 0x80 is the only syscall ABI" + ); + + // In 64-bit mode SYSENTER requires a non-null IA32_SYSENTER_CS selector. + // Clear all three legacy fast-entry targets so a CPL3 SYSENTER raises + // #GP instead of entering an inherited firmware address/stack. + for register in [0x174, 0x175, 0x176] { + let mut msr = Msr::new(register); + // SAFETY: IA32_SYSENTER_CS/ESP/EIP are architectural MSRs supported + // by x86_64; writing zero disables the entry by making CS null and + // leaves no firmware-selected stack or instruction target active. + unsafe { msr.write(0) }; + } + for register in [0x174, 0x175, 0x176] { + let msr = Msr::new(register); + // SAFETY: these architectural SYSENTER MSRs are supported by x86_64, + // and reading them has no side effects. + assert_eq!( + unsafe { msr.read() }, + 0, + "SYSENTER MSR {register:#x} not disabled" + ); + } + + // CR0.EM blocks legacy SSE, but AVX uses XSAVE-managed YMM/ZMM state and + // is architecturally controlled by CR4.OSXSAVE plus XCR0. pvKernel does + // not own that state, so ensure AVX-family instructions #UD by clearing + // OSXSAVE rather than relying on firmware's XCR0 configuration. + let cr4 = Cr4::read(); + if cr4.contains(Cr4Flags::OSXSAVE) { + // SAFETY: clearing OSXSAVE is valid in long mode and disables use of + // XSAVE-managed user state; no kernel path uses XSAVE/AVX. + unsafe { Cr4::write(cr4 & !Cr4Flags::OSXSAVE) }; + } + assert!( + !Cr4::read().contains(Cr4Flags::OSXSAVE), + "pvKernel requires CR4.OSXSAVE clear until task FP state is saved" + ); + + // This milestone does not save/restore x87, MMX, SSE, or AVX state on a + // task switch. Set CR0.EM before any CPL3 context can run: Intel defines + // MMX/SIMD execution as #UD with EM set, and x87 execution as #NM. The + // corresponding handlers contain user attempts and keep kernel attempts + // fatal. The pinned x86_64-unknown-none target uses soft-float, so kernel + // code does not require these execution units. + let cr0 = Cr0::read(); + if !cr0.contains(Cr0Flags::EMULATE_COPROCESSOR) { + // SAFETY: setting CR0.EM is valid in long mode and prevents the + // kernel/userspace from using unsaved floating-point register state. + unsafe { Cr0::write(cr0 | Cr0Flags::EMULATE_COPROCESSOR) }; + } + assert!( + Cr0::read().contains(Cr0Flags::EMULATE_COPROCESSOR), + "pvKernel requires CR0.EM to prevent unsaved floating-point state" + ); + // Supervisor write protection is required for read-only page-table // permissions to constrain kernel code. Without CR0.WP, supervisor // writes can bypass those mappings and undermine page protections and diff --git a/kernel/src/arch/x86_64/memory/paging.rs b/kernel/src/arch/x86_64/memory/paging.rs index a3481a0..6afeaa7 100644 --- a/kernel/src/arch/x86_64/memory/paging.rs +++ b/kernel/src/arch/x86_64/memory/paging.rs @@ -2,8 +2,8 @@ //! //! Every function here operates on a caller-supplied [`OffsetPageTable`] and //! [`BootFrameAllocator`] rather than reaching into global state directly; -//! [`super::address_space`] owns the single, lock-protected instance of -//! both that pvKernel actually uses. +//! [`super::address_space`] owns the lock-protected kernel instance and +//! creates scoped mappers for inactive process roots. use x86_64::{ PhysAddr, VirtAddr, @@ -81,6 +81,20 @@ impl Permissions { user_accessible: true, }; + /// Read-only, non-executable userspace data. + pub const USER_READ_ONLY: Self = Self { + writable: false, + executable: false, + user_accessible: true, + }; + + /// Read-only executable userspace code. + pub const USER_EXECUTABLE: Self = Self { + writable: false, + executable: true, + user_accessible: true, + }; + fn to_flags(self) -> Result { if self.writable && self.executable { return Err(MemoryError::InvalidPermissions); @@ -183,8 +197,8 @@ pub fn page_at(addr: VirtAddr) -> Result, MemoryError> { /// /// Allocates any missing intermediate page tables from `frame_allocator`. /// Flushes the mapped page's TLB entry before returning on success. -pub fn map_page( - mapper: &mut OffsetPageTable<'static>, +pub fn map_page<'a>( + mapper: &mut OffsetPageTable<'a>, frame_allocator: &mut impl FrameAllocator, page: Page, frame: PhysFrame, @@ -218,8 +232,8 @@ pub fn map_page( /// Note that the returned frame is not returned to any allocator: this /// milestone's [`super::frame_allocator::BootFrameAllocator`] cannot accept /// it back, so callers must not treat the frame as reusable. -pub fn unmap_page( - mapper: &mut OffsetPageTable<'static>, +pub fn unmap_page<'a>( + mapper: &mut OffsetPageTable<'a>, page: Page, ) -> Result, MemoryError> { match mapper.unmap(page) { @@ -235,8 +249,8 @@ pub fn unmap_page( /// Changes the permissions of an existing mapping and flushes its TLB /// entry. -pub fn update_permissions( - mapper: &mut OffsetPageTable<'static>, +pub fn update_permissions<'a>( + mapper: &mut OffsetPageTable<'a>, page: Page, permissions: Permissions, ) -> Result<(), MemoryError> { @@ -260,7 +274,7 @@ pub fn update_permissions( /// Translates `addr` to a physical address, distinguishing an unmapped /// address from a huge-page mapping. Never panics on an ordinary unmapped /// address. -pub fn translate(mapper: &OffsetPageTable<'static>, addr: VirtAddr) -> Translation { +pub fn translate(mapper: &OffsetPageTable<'_>, addr: VirtAddr) -> Translation { match mapper.translate(addr) { TranslateResult::Mapped { frame: MappedFrame::Size4KiB(frame), diff --git a/kernel/src/arch/x86_64/memory/user_stack.rs b/kernel/src/arch/x86_64/memory/user_stack.rs new file mode 100644 index 0000000..e21b8f8 --- /dev/null +++ b/kernel/src/arch/x86_64/memory/user_stack.rs @@ -0,0 +1,98 @@ +//! Fixed, guarded initial userspace stack layout for 0.0.9. + +use x86_64::{ + VirtAddr, + structures::paging::{Page, PageSize, Size4KiB}, +}; + +use super::{AddressSpace, AddressSpaceError, KernelAddressSpace, Permissions, Translation}; + +pub const USER_STACK_TOP: u64 = 0x0000_7fff_ffff_f000; +pub const USER_STACK_PAGES: u64 = 8; +pub const USER_STACK_SIZE: u64 = USER_STACK_PAGES * Size4KiB::SIZE; + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct UserStack { + pub bottom: u64, + pub top: u64, + pub guard: u64, +} + +impl UserStack { + /// Maps and zeroes the fixed 32 KiB USER RW/NX stack. The lower guard is + /// left unmapped. On failure, already installed leaves are removed; the + /// monotonic allocator retains their physical frames by design. + pub fn map( + address_space: &mut AddressSpace, + kernel: &mut KernelAddressSpace, + ) -> Result { + let bottom = USER_STACK_TOP + .checked_sub(USER_STACK_SIZE) + .ok_or(AddressSpaceError::OutsideUserRange)?; + let guard = bottom + .checked_sub(Size4KiB::SIZE) + .ok_or(AddressSpaceError::OutsideUserRange)?; + if guard < super::USER_MIN || USER_STACK_TOP > super::USER_MAX { + return Err(AddressSpaceError::OutsideUserRange); + } + let guard_page = Page::::from_start_address(VirtAddr::new(guard)) + .map_err(|_| AddressSpaceError::OutsideUserRange)?; + if !matches!( + address_space.translate(kernel, guard_page.start_address())?, + Translation::NotMapped + ) { + return Err(AddressSpaceError::StackCollision); + } + + let mut mapped = 0_u64; + while mapped < USER_STACK_PAGES { + let offset = mapped + .checked_mul(Size4KiB::SIZE) + .ok_or(AddressSpaceError::OutsideUserRange)?; + let address = bottom + .checked_add(offset) + .ok_or(AddressSpaceError::OutsideUserRange)?; + let page = Page::::from_start_address(VirtAddr::new(address)) + .map_err(|_| AddressSpaceError::OutsideUserRange)?; + if !matches!( + address_space.translate(kernel, VirtAddr::new(address))?, + Translation::NotMapped + ) { + rollback(address_space, kernel, bottom, mapped)?; + return Err(AddressSpaceError::StackCollision); + } + if let Err(error) = + address_space.map_new_user_page(kernel, page, Permissions::USER_DATA) + { + rollback(address_space, kernel, bottom, mapped)?; + return Err(error); + } + mapped += 1; + } + Ok(Self { + bottom, + top: USER_STACK_TOP, + guard, + }) + } +} + +fn rollback( + address_space: &mut AddressSpace, + kernel: &mut KernelAddressSpace, + bottom: u64, + count: u64, +) -> Result<(), AddressSpaceError> { + for index in 0..count { + let offset = index + .checked_mul(Size4KiB::SIZE) + .ok_or(AddressSpaceError::OutsideUserRange)?; + let address = bottom + .checked_add(offset) + .ok_or(AddressSpaceError::OutsideUserRange)?; + let page = Page::::from_start_address(VirtAddr::new(address)) + .map_err(|_| AddressSpaceError::OutsideUserRange)?; + address_space.unmap_user_page(kernel, page)?; + } + Ok(()) +} diff --git a/kernel/src/arch/x86_64/mod.rs b/kernel/src/arch/x86_64/mod.rs index 1b17df9..7bb7590 100644 --- a/kernel/src/arch/x86_64/mod.rs +++ b/kernel/src/arch/x86_64/mod.rs @@ -38,14 +38,14 @@ const PIT_TARGET_HZ: u32 = 100; /// this function, so that once `boot_info` is moved in here, nothing in the /// kernel retains any other way to reach it. See [`memory::init`] for why /// that matters. -pub fn init(boot_info: &'static mut BootInfo) { +pub fn init(boot_info: &'static mut BootInfo, framebuffer_mapping: Option<(u64, u64)>) { x86_64::instructions::interrupts::disable(); crate::kprintln!("[cpu] maskable hardware interrupts disabled"); gdt::init(); interrupts::init(); - memory::init(boot_info); + memory::init(boot_info, framebuffer_mapping); pic::initialize(); pic::mask_all(); diff --git a/kernel/src/arch/x86_64/syscall/dispatch.rs b/kernel/src/arch/x86_64/syscall/dispatch.rs index d469c83..c6ace81 100644 --- a/kernel/src/arch/x86_64/syscall/dispatch.rs +++ b/kernel/src/arch/x86_64/syscall/dispatch.rs @@ -67,6 +67,10 @@ pub(super) extern "C" fn pvkernel_syscall_dispatch(frame: *const SyscallFrame) - numbers::ABI_VERSION_QUERY => numbers::ABI_VERSION, numbers::DEBUG_WRITE_BYTE => debug_write_byte(arg0), numbers::TIMER_TICKS => crate::arch::x86_64::pit::ticks(), + numbers::GETPID => { + crate::task::current_user_pid().unwrap_or_else(|| SyscallError::InvalidContext.encode()) + } + numbers::EXIT => crate::task::exit_current_user(frame.rdi as u32 as i32), #[cfg(feature = "userspace-test")] n if n >= numbers::TEST_NAMESPACE_BASE => super::test::dispatch(n, frame), _ => SyscallError::Unsupported.encode(), diff --git a/kernel/src/arch/x86_64/syscall/numbers.rs b/kernel/src/arch/x86_64/syscall/numbers.rs index 61c5458..aa37606 100644 --- a/kernel/src/arch/x86_64/syscall/numbers.rs +++ b/kernel/src/arch/x86_64/syscall/numbers.rs @@ -38,6 +38,14 @@ pub const DEBUG_WRITE_BYTE: u64 = 1; /// Arguments: none. Returns: `arch::x86_64::pit::ticks()` in `RAX`. pub const TIMER_TICKS: u64 = 2; +/// Returns the current process's human-visible PID, or `InvalidContext` for +/// bootstrap/kernel execution without a process-owned user Task. +pub const GETPID: u64 = 3; + +/// Terminates the current process with `RDI as i32`. This syscall never +/// returns to its caller; process exit is completed after a timer handoff. +pub const EXIT: u64 = 4; + /// Start of the test-only syscall namespace. /// /// Numbers at or above this value are only ever dispatched when the diff --git a/kernel/src/arch/x86_64/task/mod.rs b/kernel/src/arch/x86_64/task/mod.rs index c5725ce..d4515e5 100644 --- a/kernel/src/arch/x86_64/task/mod.rs +++ b/kernel/src/arch/x86_64/task/mod.rs @@ -3,5 +3,8 @@ mod stack; mod switch; pub use context::Context; -pub use stack::{MAX_KERNEL_TASKS, STACK_SIZE, init_stacks, stack_bounds, verify_stack_mappings}; +pub use stack::{ + MAX_KERNEL_TASKS, STACK_SIZE, init_stacks, is_validated_stack_top, stack_bounds, + verify_stack_mappings, +}; pub use switch::{task_trampoline, timer_irq_entry}; diff --git a/kernel/src/arch/x86_64/task/stack.rs b/kernel/src/arch/x86_64/task/stack.rs index 81185f5..5a446fa 100644 --- a/kernel/src/arch/x86_64/task/stack.rs +++ b/kernel/src/arch/x86_64/task/stack.rs @@ -1,3 +1,4 @@ +use core::sync::atomic::{AtomicBool, Ordering}; use x86_64::{ VirtAddr, structures::paging::{Page, PageTableFlags, Size4KiB}, @@ -15,6 +16,7 @@ const GUARD_PAGES: u64 = 2; const SLOT_PAGES: u64 = STACK_SIZE / PAGE_SIZE + GUARD_PAGES; pub const STACK_REGION_START: u64 = 0xffff_9000_2000_0000; const STACK_REGION_SIZE: u64 = stack_region_size(); +static STACKS_INITIALIZED: AtomicBool = AtomicBool::new(false); const fn stack_region_size() -> u64 { let slot_size = match SLOT_PAGES.checked_mul(PAGE_SIZE) { @@ -102,6 +104,7 @@ pub fn init_stacks() { )); } }); + STACKS_INITIALIZED.store(true, Ordering::Release); kprintln!("[scheduler] task stacks verified supervisor RW/NX with guard pages"); } @@ -109,6 +112,11 @@ pub fn stack_bounds(slot: usize) -> Option<(u64, u64)> { (slot < MAX_KERNEL_TASKS).then(|| bounds_unchecked(slot)) } +pub fn is_validated_stack_top(top: u64) -> bool { + STACKS_INITIALIZED.load(Ordering::Acquire) + && (0..MAX_KERNEL_TASKS).any(|slot| stack_bounds(slot).is_some_and(|(_, high)| high == top)) +} + fn bounds_unchecked(slot: usize) -> (u64, u64) { let stride = SLOT_PAGES .checked_mul(PAGE_SIZE) diff --git a/kernel/src/main.rs b/kernel/src/main.rs index fe4336c..8153175 100644 --- a/kernel/src/main.rs +++ b/kernel/src/main.rs @@ -9,6 +9,7 @@ mod arch; mod drivers; mod graphics; mod halt; +mod process; mod task; use bootloader_api::config::{BootloaderConfig, Mapping}; @@ -33,6 +34,12 @@ use halt::idle_loop; const fn bootloader_config() -> BootloaderConfig { let mut config = BootloaderConfig::new_default(); config.mappings.physical_memory = Some(Mapping::Dynamic); + // Bootloader 0.11.17 chooses Mapping::Dynamic addresses from contiguous + // free PML4 entries. Keep every retained boot mapping in indices 256..287, + // wholly inside the kernel half and immediately below pvKernel's explicit + // index-288 mappings (scratch, heap and scheduler stacks). + config.mappings.dynamic_range_start = Some(arch::x86_64::memory::BOOTLOADER_DYNAMIC_START); + config.mappings.dynamic_range_end = Some(arch::x86_64::memory::BOOTLOADER_DYNAMIC_END_SENTINEL); config } @@ -72,16 +79,33 @@ fn kernel_main(boot_info: &'static mut BootInfo) -> ! { // This is what lets `kernel_main` keep using the framebuffer afterward // without retaining any other access to `boot_info`. let mut framebuffer = boot_info.framebuffer.take(); + let framebuffer_mapping = framebuffer.as_mut().map(|buffer| { + let bytes = buffer.buffer_mut(); + (bytes.as_mut_ptr() as u64, bytes.len() as u64) + }); // `boot_info` is moved into `arch::init` here and not returned: after // this call, `kernel_main` has no further way to reach `BootInfo` (the // compiler enforces this, since `boot_info` is a `&'static mut` // reference and this function no longer has a live binding for it). - arch::init(boot_info); + arch::init(boot_info, framebuffer_mapping); arch::x86_64::memory::run_boot_smoke_test(); arch::x86_64::memory::run_heap_smoke_tests(); task::init(); + process::init(); + #[cfg(feature = "qemu-test")] + arch::x86_64::memory::run_address_space_smoke_test(); + #[cfg(feature = "qemu-test")] + arch::x86_64::memory::run_address_space_switch_smoke_test(); + #[cfg(feature = "qemu-test")] + task::run_prepared_user_task_test(); + #[cfg(feature = "qemu-test")] + arch::x86_64::memory::run_user_stack_smoke_test(); + #[cfg(feature = "user-copy-test")] + arch::x86_64::memory::run_user_copy_test(); + #[cfg(feature = "qemu-test")] + process::elf::run_loader_qemu_tests(); if let Some(framebuffer) = framebuffer.as_mut() { let info = framebuffer.info(); @@ -105,6 +129,16 @@ fn kernel_main(boot_info: &'static mut BootInfo) -> ! { arch::x86_64::interrupts::run_breakpoint_test(); let scheduler_smoke_tasks = task::prepare_smoke_test(); + #[cfg(feature = "process-capacity-test")] + task::run_process_capacity_test(); + #[cfg(not(any(feature = "qemu-test", feature = "userspace-test")))] + task::prepare_release_userspace_smoke(); + #[cfg(feature = "process-preemption-test")] + task::prepare_process_preemption_test(); + #[cfg(feature = "process-smoke-test")] + task::prepare_process_smoke_test(); + #[cfg(feature = "address-space-isolation-test")] + task::prepare_address_space_isolation_test(); let ticks_before = pit::ticks(); kprintln!("[test] waiting for timer interrupts"); @@ -118,6 +152,19 @@ fn kernel_main(boot_info: &'static mut BootInfo) -> ! { pit::wait_for_ticks(TIMER_TEST_TICKS); task::run_smoke_test(scheduler_smoke_tasks.0, scheduler_smoke_tasks.1); + #[cfg(feature = "user-fault-containment-test")] + task::prepare_user_fault_containment_test(); + #[cfg(not(any(feature = "qemu-test", feature = "userspace-test")))] + task::wait_for_release_userspace_smoke(); + + #[cfg(feature = "process-preemption-test")] + task::wait_for_process_preemption_test(); + #[cfg(feature = "process-smoke-test")] + task::wait_for_process_smoke_test(); + #[cfg(feature = "user-fault-containment-test")] + task::wait_for_user_fault_containment_test(); + #[cfg(feature = "address-space-isolation-test")] + task::wait_for_address_space_isolation_test(); #[cfg(feature = "scheduler-preemption-test")] task::run_preemption_test(); diff --git a/kernel/src/process/elf.rs b/kernel/src/process/elf.rs new file mode 100644 index 0000000..5b6a0b7 --- /dev/null +++ b/kernel/src/process/elf.rs @@ -0,0 +1,833 @@ +//! Bounded checked loader for static x86_64 ELF64 `ET_EXEC` images. + +#[cfg(feature = "qemu-test")] +use x86_64::structures::paging::PageTableFlags; +use x86_64::{ + VirtAddr, + structures::paging::{Page, PageSize, Size4KiB}, +}; + +use crate::arch::x86_64::memory::{ + AddressSpace, AddressSpaceError, KernelAddressSpace, Permissions, Translation, +}; + +const ELF_MIN: u64 = 0x0000_0000_0040_0000; +const ELF_MAX: u64 = 0x0000_4000_0000_0000; +const MAX_LOAD_SEGMENTS: usize = 16; +const MAX_IMAGE_PAGES: u64 = 4096; +const ELF_HEADER_SIZE: usize = 64; +const PROGRAM_HEADER_SIZE: usize = 56; + +const PT_LOAD: u32 = 1; +const PT_DYNAMIC: u32 = 2; +const PT_INTERP: u32 = 3; +const PT_TLS: u32 = 7; +const PF_R: u32 = 4; +const PF_W: u32 = 2; +const PF_X: u32 = 1; + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum ElfError { + Truncated, + BadMagic, + UnsupportedClass, + UnsupportedEndian, + UnsupportedVersion, + UnsupportedMachine, + UnsupportedType, + InvalidHeaderSize, + InvalidProgramHeaderSize, + InvalidProgramHeaderTable, + UnsupportedInterpreter, + UnsupportedDynamicImage, + UnsupportedTls, + TooManySegments, + TooManyPages, + InvalidSegment, + FileRangeOverflow, + VirtualRangeOverflow, + OutsideUserRange, + NonCanonicalAddress, + InvalidAlignment, + SegmentOverlap, + StackOverlap, + WritableExecutable, + InvalidEntry, + EntryNotExecutable, + SegmentAlreadyMapped, + AddressSpace(AddressSpaceError), +} + +impl From for ElfError { + fn from(error: AddressSpaceError) -> Self { + Self::AddressSpace(error) + } +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +struct Segment { + file_offset: usize, + file_size: usize, + virtual_start: u64, + memory_end: u64, + page_start: u64, + page_end: u64, + flags: u32, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +struct ElfPlan { + entry: u64, + segments: [Option; MAX_LOAD_SEGMENTS], + segment_count: usize, + page_count: u64, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct LoadedImage { + pub entry: u64, + pub mapped_pages: u64, +} + +/// Validates all headers and load ranges before allocating or mapping any +/// user frame. The image is static ET_EXEC: there is no dynamic linker, +/// relocation, TLS, PIE, or demand paging. +fn validate(bytes: &[u8]) -> Result { + if bytes.len() < ELF_HEADER_SIZE { + return Err(ElfError::Truncated); + } + if bytes.get(0..4) != Some(b"\x7fELF") { + return Err(ElfError::BadMagic); + } + if bytes[4] != 2 { + return Err(ElfError::UnsupportedClass); + } + if bytes[5] != 1 { + return Err(ElfError::UnsupportedEndian); + } + if bytes[6] != 1 { + return Err(ElfError::UnsupportedVersion); + } + if read_u16(bytes, 18)? != 62 { + return Err(ElfError::UnsupportedMachine); + } + if read_u16(bytes, 16)? != 2 { + return Err(ElfError::UnsupportedType); + } + if read_u32(bytes, 20)? != 1 || read_u16(bytes, 52)? as usize != ELF_HEADER_SIZE { + return Err(ElfError::InvalidHeaderSize); + } + let entry = read_u64(bytes, 24)?; + let phoff = + usize::try_from(read_u64(bytes, 32)?).map_err(|_| ElfError::InvalidProgramHeaderTable)?; + let phentsize = read_u16(bytes, 54)? as usize; + let phnum = read_u16(bytes, 56)? as usize; + if phentsize != PROGRAM_HEADER_SIZE { + return Err(ElfError::InvalidProgramHeaderSize); + } + if phnum == 0 { + return Err(ElfError::InvalidProgramHeaderTable); + } + let table_size = phentsize + .checked_mul(phnum) + .ok_or(ElfError::InvalidProgramHeaderTable)?; + let table_end = phoff + .checked_add(table_size) + .ok_or(ElfError::InvalidProgramHeaderTable)?; + if phoff < ELF_HEADER_SIZE || table_end > bytes.len() { + return Err(ElfError::InvalidProgramHeaderTable); + } + + let mut plan = ElfPlan { + entry, + segments: [None; MAX_LOAD_SEGMENTS], + segment_count: 0, + page_count: 0, + }; + let mut entry_executable = false; + for index in 0..phnum { + let offset = phoff + .checked_add( + index + .checked_mul(phentsize) + .ok_or(ElfError::InvalidProgramHeaderTable)?, + ) + .ok_or(ElfError::InvalidProgramHeaderTable)?; + let kind = read_u32(bytes, offset)?; + if kind == PT_INTERP { + return Err(ElfError::UnsupportedInterpreter); + } + if kind == PT_DYNAMIC { + return Err(ElfError::UnsupportedDynamicImage); + } + if kind == PT_TLS { + return Err(ElfError::UnsupportedTls); + } + if kind != PT_LOAD { + continue; + } + if plan.segment_count == MAX_LOAD_SEGMENTS { + return Err(ElfError::TooManySegments); + } + + let flags = read_u32(bytes, offset + 4)?; + if flags & !(PF_R | PF_W | PF_X) != 0 { + return Err(ElfError::InvalidSegment); + } + let file_offset_u64 = read_u64(bytes, offset + 8)?; + let virtual_start = read_u64(bytes, offset + 16)?; + let file_size_u64 = read_u64(bytes, offset + 32)?; + let memory_size = read_u64(bytes, offset + 40)?; + let alignment = read_u64(bytes, offset + 48)?; + if file_size_u64 > memory_size { + return Err(ElfError::InvalidSegment); + } + if flags & PF_W != 0 && flags & PF_X != 0 { + return Err(ElfError::WritableExecutable); + } + if memory_size == 0 { + if file_size_u64 != 0 { + return Err(ElfError::InvalidSegment); + } + continue; + } + let file_offset_end = file_offset_u64 + .checked_add(file_size_u64) + .ok_or(ElfError::FileRangeOverflow)?; + if file_offset_end > bytes.len() as u64 { + return Err(ElfError::InvalidSegment); + } + let file_offset = usize::try_from(file_offset_u64).map_err(|_| ElfError::InvalidSegment)?; + let file_size = usize::try_from(file_size_u64).map_err(|_| ElfError::InvalidSegment)?; + let memory_end = virtual_start + .checked_add(memory_size) + .ok_or(ElfError::VirtualRangeOverflow)?; + if VirtAddr::try_new(virtual_start).is_err() || VirtAddr::try_new(memory_end - 1).is_err() { + return Err(ElfError::NonCanonicalAddress); + } + if virtual_start < crate::arch::x86_64::memory::USER_MIN { + return Err(ElfError::OutsideUserRange); + } + if alignment > 1 + && (!alignment.is_power_of_two() + || virtual_start % alignment != file_offset_u64 % alignment) + { + return Err(ElfError::InvalidAlignment); + } + let page_start = virtual_start & !(Size4KiB::SIZE - 1); + let page_end = memory_end + .checked_add(Size4KiB::SIZE - 1) + .ok_or(ElfError::VirtualRangeOverflow)? + & !(Size4KiB::SIZE - 1); + let pages = page_end + .checked_sub(page_start) + .ok_or(ElfError::VirtualRangeOverflow)? + / Size4KiB::SIZE; + plan.page_count = plan + .page_count + .checked_add(pages) + .ok_or(ElfError::TooManyPages)?; + if plan.page_count > MAX_IMAGE_PAGES { + return Err(ElfError::TooManyPages); + } + for previous in plan.segments[..plan.segment_count].iter().flatten() { + if page_start < previous.page_end && page_end > previous.page_start { + return Err(ElfError::SegmentOverlap); + } + } + let stack_guard = crate::arch::x86_64::memory::USER_STACK_TOP + .checked_sub(crate::arch::x86_64::memory::USER_STACK_SIZE) + .and_then(|bottom| bottom.checked_sub(Size4KiB::SIZE)) + .ok_or(ElfError::VirtualRangeOverflow)?; + if page_start < crate::arch::x86_64::memory::USER_STACK_TOP && page_end > stack_guard { + return Err(ElfError::StackOverlap); + } + if virtual_start < ELF_MIN || memory_end > ELF_MAX { + return Err(ElfError::OutsideUserRange); + } + if flags & PF_X != 0 && entry >= virtual_start && entry < memory_end { + entry_executable = true; + } + plan.segments[plan.segment_count] = Some(Segment { + file_offset, + file_size, + virtual_start, + memory_end, + page_start, + page_end, + flags, + }); + plan.segment_count += 1; + } + if plan.segment_count == 0 { + return Err(ElfError::InvalidProgramHeaderTable); + } + if VirtAddr::try_new(entry).is_err() { + return Err(ElfError::NonCanonicalAddress); + } + if !(ELF_MIN..ELF_MAX).contains(&entry) { + return Err(ElfError::InvalidEntry); + } + if !entry_executable { + return Err(ElfError::EntryNotExecutable); + } + Ok(plan) +} + +/// Loads the validated image into an inactive process root. New user pages +/// are zeroed RW/NX before bytes are copied, then changed to final segment +/// permissions. Any failure removes all image leaves; backing frames remain +/// kernel-lifetime owned by the monotonic allocator. +pub fn load_static_elf( + address_space: &mut AddressSpace, + kernel: &mut KernelAddressSpace, + bytes: &[u8], +) -> Result { + let plan = validate(bytes)?; + for segment in plan.segments[..plan.segment_count].iter().flatten() { + let mut address = segment.page_start; + while address < segment.page_end { + match address_space.translate(kernel, VirtAddr::new(address))? { + Translation::NotMapped => {} + _ => return Err(ElfError::SegmentAlreadyMapped), + } + address = address + .checked_add(Size4KiB::SIZE) + .ok_or(ElfError::VirtualRangeOverflow)?; + } + } + + for segment in plan.segments[..plan.segment_count].iter().flatten() { + let mut address = segment.page_start; + while address < segment.page_end { + let page = Page::::from_start_address(VirtAddr::new(address)) + .map_err(|_| ElfError::VirtualRangeOverflow)?; + if let Err(error) = + address_space.map_new_user_page(kernel, page, Permissions::USER_DATA) + { + cleanup(address_space, kernel, &plan); + return Err(ElfError::AddressSpace(error)); + } + address = address + .checked_add(Size4KiB::SIZE) + .ok_or(ElfError::VirtualRangeOverflow)?; + } + } + + for segment in plan.segments[..plan.segment_count].iter().flatten() { + let mut copied = 0_u64; + while copied < segment.file_size as u64 { + let virtual_address = segment + .virtual_start + .checked_add(copied) + .ok_or(ElfError::VirtualRangeOverflow)?; + let page_start = virtual_address & !(Size4KiB::SIZE - 1); + let page_offset = usize::try_from(virtual_address - page_start) + .map_err(|_| ElfError::VirtualRangeOverflow)?; + let remaining = (segment.file_size as u64) + .checked_sub(copied) + .ok_or(ElfError::InvalidSegment)?; + let chunk = core::cmp::min(remaining, Size4KiB::SIZE - page_offset as u64) as usize; + let source_start = segment + .file_offset + .checked_add(usize::try_from(copied).map_err(|_| ElfError::FileRangeOverflow)?) + .ok_or(ElfError::FileRangeOverflow)?; + let source_end = source_start + .checked_add(chunk) + .ok_or(ElfError::FileRangeOverflow)?; + let source = bytes + .get(source_start..source_end) + .ok_or(ElfError::InvalidSegment)?; + let page = Page::::from_start_address(VirtAddr::new(page_start)) + .map_err(|_| ElfError::VirtualRangeOverflow)?; + if let Err(error) = address_space.write_user_page(kernel, page, page_offset, source) { + cleanup(address_space, kernel, &plan); + return Err(ElfError::AddressSpace(error)); + } + copied = copied + .checked_add(chunk as u64) + .ok_or(ElfError::FileRangeOverflow)?; + } + } + + for segment in plan.segments[..plan.segment_count].iter().flatten() { + let permissions = if segment.flags & PF_X != 0 { + Permissions::USER_EXECUTABLE + } else if segment.flags & PF_W != 0 { + Permissions::USER_DATA + } else { + Permissions::USER_READ_ONLY + }; + // x86-64 does not provide a per-page read-disable bit: PF_R is + // advisory in this loader. PF_W controls WRITABLE and PF_X controls + // execute-disable; W+X was rejected during validation. + let mut address = segment.page_start; + while address < segment.page_end { + let page = Page::::from_start_address(VirtAddr::new(address)) + .map_err(|_| ElfError::VirtualRangeOverflow)?; + if let Err(error) = address_space.update_user_permissions(kernel, page, permissions) { + cleanup(address_space, kernel, &plan); + return Err(ElfError::AddressSpace(error)); + } + address = address + .checked_add(Size4KiB::SIZE) + .ok_or(ElfError::VirtualRangeOverflow)?; + } + } + Ok(LoadedImage { + entry: plan.entry, + mapped_pages: plan.page_count, + }) +} + +fn cleanup(address_space: &mut AddressSpace, kernel: &mut KernelAddressSpace, plan: &ElfPlan) { + for segment in plan.segments[..plan.segment_count].iter().flatten() { + let mut address = segment.page_start; + while address < segment.page_end { + if let Ok(page) = Page::::from_start_address(VirtAddr::new(address)) + && !matches!( + address_space.translate(kernel, VirtAddr::new(address)), + Ok(Translation::NotMapped) + ) + { + let _ = address_space.unmap_user_page(kernel, page); + } + let Some(next) = address.checked_add(Size4KiB::SIZE) else { + break; + }; + address = next; + } + } +} + +fn read_u16(bytes: &[u8], offset: usize) -> Result { + let end = offset.checked_add(2).ok_or(ElfError::Truncated)?; + let value = bytes.get(offset..end).ok_or(ElfError::Truncated)?; + Ok(u16::from_le_bytes([value[0], value[1]])) +} + +fn read_u32(bytes: &[u8], offset: usize) -> Result { + let end = offset.checked_add(4).ok_or(ElfError::Truncated)?; + let value = bytes.get(offset..end).ok_or(ElfError::Truncated)?; + Ok(u32::from_le_bytes([value[0], value[1], value[2], value[3]])) +} + +fn read_u64(bytes: &[u8], offset: usize) -> Result { + let end = offset.checked_add(8).ok_or(ElfError::Truncated)?; + let value = bytes.get(offset..end).ok_or(ElfError::Truncated)?; + Ok(u64::from_le_bytes( + value.try_into().map_err(|_| ElfError::Truncated)?, + )) +} + +#[cfg(feature = "qemu-test")] +pub fn run_loader_qemu_tests() { + use crate::arch::x86_64::memory::with_kernel_address_space; + + crate::kprintln!("[process-test] validating and loading static ELF64 image"); + let image_bytes = test_elf(); + with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("failed to create ELF test root"); + let loaded = load_static_elf(&mut address_space, kernel, &image_bytes) + .expect("valid static test ELF was rejected"); + assert_eq!(loaded.entry, 0x0040_0000); + assert_eq!(loaded.mapped_pages, 3); + assert_eq!(address_space.mapped_user_pages(), 3); + + let code_page = Page::::from_start_address(VirtAddr::new(0x0040_0000)).unwrap(); + match address_space + .translate(kernel, VirtAddr::new(0x0040_0000)) + .unwrap() + { + Translation::Mapped { flags, .. } => { + assert!(flags.contains(PageTableFlags::USER_ACCESSIBLE)); + assert!(!flags.contains(PageTableFlags::WRITABLE)); + assert!(!flags.contains(PageTableFlags::NO_EXECUTE)); + } + other => panic!("ELF code page missing: {other:?}"), + } + let mut code = [0; 8]; + address_space + .read_user_page(kernel, code_page, 0, &mut code) + .unwrap(); + assert_eq!(&code[..4], &[0x90, 0x90, 0x90, 0xCC]); + assert_eq!(&code[4..], &[0; 4], "ELF code page padding leaked data"); + + for address in [0x0040_2000, 0x0040_3000] { + match address_space + .translate(kernel, VirtAddr::new(address)) + .unwrap() + { + Translation::Mapped { flags, .. } => assert!(flags.contains( + PageTableFlags::USER_ACCESSIBLE + | PageTableFlags::WRITABLE + | PageTableFlags::NO_EXECUTE + )), + other => panic!("ELF data/BSS page missing: {other:?}"), + } + } + let data_page = Page::::from_start_address(VirtAddr::new(0x0040_2000)).unwrap(); + let mut data = [0; 8]; + address_space + .read_user_page(kernel, data_page, 0, &mut data) + .unwrap(); + assert_eq!(&data[..4], &[1, 2, 3, 4]); + assert_eq!(&data[4..], &[0; 4], "ELF BSS tail was not zero"); + let bss_page = Page::::from_start_address(VirtAddr::new(0x0040_3000)).unwrap(); + let mut bss = [0xFF; 8]; + address_space + .read_user_page(kernel, bss_page, 0, &mut bss) + .unwrap(); + assert_eq!(bss, [0; 8], "ELF BSS page exposed stale contents"); + }); + + macro_rules! rejects { + ($name:literal, $edit:expr, $expected:expr) => {{ + let mut malformed = test_elf(); + ($edit)(&mut malformed); + assert_eq!(validate(&malformed), Err($expected), $name); + }}; + } + rejects!( + "bad magic", + |b: &mut alloc::vec::Vec| b[0] = 0, + ElfError::BadMagic + ); + rejects!( + "ELF32", + |b: &mut alloc::vec::Vec| b[4] = 1, + ElfError::UnsupportedClass + ); + rejects!( + "wrong endian", + |b: &mut alloc::vec::Vec| b[5] = 2, + ElfError::UnsupportedEndian + ); + rejects!( + "wrong machine", + |b: &mut alloc::vec::Vec| put_u16(b, 18, 3), + ElfError::UnsupportedMachine + ); + rejects!( + "unsupported type", + |b: &mut alloc::vec::Vec| put_u16(b, 16, 3), + ElfError::UnsupportedType + ); + rejects!( + "PT_INTERP", + |b: &mut alloc::vec::Vec| put_u32(b, 64, PT_INTERP), + ElfError::UnsupportedInterpreter + ); + rejects!( + "PT_DYNAMIC", + |b: &mut alloc::vec::Vec| put_u32(b, 64, PT_DYNAMIC), + ElfError::UnsupportedDynamicImage + ); + rejects!( + "PT_TLS", + |b: &mut alloc::vec::Vec| put_u32(b, 64, PT_TLS), + ElfError::UnsupportedTls + ); + rejects!( + "bad ELF header size", + |b: &mut alloc::vec::Vec| put_u16(b, 52, 63), + ElfError::InvalidHeaderSize + ); + rejects!( + "bad program header size", + |b: &mut alloc::vec::Vec| put_u16(b, 54, 55), + ElfError::InvalidProgramHeaderSize + ); + rejects!( + "program header table range", + |b: &mut alloc::vec::Vec| put_u64(b, 32, u64::MAX), + ElfError::InvalidProgramHeaderTable + ); + rejects!( + "filesz > memsz", + |b: &mut alloc::vec::Vec| put_u64(b, 120 + 32, 5), + ElfError::InvalidSegment + ); + rejects!( + "offset overflow", + |b: &mut alloc::vec::Vec| put_u64(b, 120 + 8, u64::MAX - 3), + ElfError::FileRangeOverflow + ); + rejects!( + "VA overflow", + |b: &mut alloc::vec::Vec| { + put_u64(b, 120 + 16, u64::MAX - 3); + put_u64(b, 120 + 40, 8); + }, + ElfError::VirtualRangeOverflow + ); + rejects!( + "segment outside ELF band", + |b: &mut alloc::vec::Vec| put_u64(b, 120 + 16, ELF_MAX), + ElfError::OutsideUserRange + ); + rejects!( + "segment in forbidden low area", + |b: &mut alloc::vec::Vec| put_u64(b, 120 + 16, 0x8000), + ElfError::OutsideUserRange + ); + rejects!( + "segment overlap", + |b: &mut alloc::vec::Vec| put_u64(b, 120 + 16, 0x0040_0000), + ElfError::SegmentOverlap + ); + rejects!( + "stack overlap", + |b: &mut alloc::vec::Vec| { + put_u64( + b, + 120 + 16, + crate::arch::x86_64::memory::USER_STACK_TOP - Size4KiB::SIZE, + ); + put_u64(b, 120 + 40, Size4KiB::SIZE); + }, + ElfError::StackOverlap + ); + rejects!( + "W+X", + |b: &mut alloc::vec::Vec| put_u32(b, 64 + 4, PF_W | PF_X), + ElfError::WritableExecutable + ); + rejects!( + "invalid entry", + |b: &mut alloc::vec::Vec| put_u64(b, 24, 0x0040_1000), + ElfError::EntryNotExecutable + ); + rejects!( + "noncanonical segment", + |b: &mut alloc::vec::Vec| put_u64(b, 120 + 16, crate::arch::x86_64::memory::USER_MAX), + ElfError::NonCanonicalAddress + ); + crate::kprintln!("[process-test] static ELF loader and malformed-header tests passed"); +} + +#[cfg(feature = "qemu-test")] +fn test_elf() -> alloc::vec::Vec { + let mut bytes = alloc::vec![0; 0x2004]; + bytes[..16].copy_from_slice(b"\x7fELF\x02\x01\x01\0\0\0\0\0\0\0\0\0"); + put_u16(&mut bytes, 16, 2); + put_u16(&mut bytes, 18, 62); + put_u32(&mut bytes, 20, 1); + put_u64(&mut bytes, 24, 0x0040_0000); + put_u64(&mut bytes, 32, ELF_HEADER_SIZE as u64); + put_u16(&mut bytes, 52, ELF_HEADER_SIZE as u16); + put_u16(&mut bytes, 54, PROGRAM_HEADER_SIZE as u16); + put_u16(&mut bytes, 56, 2); + test_load_header(&mut bytes, 64, PF_X | PF_R, 0x1000, 0x0040_0000, 4, 4); + test_load_header(&mut bytes, 120, PF_W | PF_R, 0x2000, 0x0040_2000, 4, 0x1004); + bytes[0x1000..0x1004].copy_from_slice(&[0x90, 0x90, 0x90, 0xCC]); + bytes[0x2000..0x2004].copy_from_slice(&[1, 2, 3, 4]); + bytes +} + +/// Builds the embedded process-exit smoke ELF with its expected PID baked +/// into the test code. This is a deterministic QEMU fixture, not a loader +/// shortcut or a production executable provider. +#[cfg(any( + not(any(feature = "qemu-test", feature = "userspace-test")), + feature = "process-smoke-test" +))] +pub(crate) fn process_smoke_elf(expected_pid: u64) -> alloc::vec::Vec { + const ENTRY: u64 = 0x0040_0000; + const FILE_OFFSET: usize = 0x1000; + let mut code = [0u8; 50]; + code[0..5].copy_from_slice(&[0xB8, 3, 0, 0, 0]); // mov eax, GETPID + code[5..7].copy_from_slice(&[0xCD, 0x80]); + code[7..9].copy_from_slice(&[0x48, 0xB9]); // mov rcx, expected PID + code[9..17].copy_from_slice(&expected_pid.to_le_bytes()); + code[17..20].copy_from_slice(&[0x48, 0x39, 0xC8]); // cmp rax, rcx + code[20..22].copy_from_slice(&[0x75, 0x0E]); // jne failure + code[22..27].copy_from_slice(&[0xBF, 0, 0, 0, 0]); // mov edi, 0 + code[27..32].copy_from_slice(&[0xB8, 4, 0, 0, 0]); // mov eax, EXIT + code[32..34].copy_from_slice(&[0xCD, 0x80]); + code[34..36].copy_from_slice(&[0x0F, 0x0B]); // unreachable UD2 sentinel + code[36..41].copy_from_slice(&[0xBF, 1, 0, 0, 0]); // mov edi, 1 + code[41..46].copy_from_slice(&[0xB8, 4, 0, 0, 0]); + code[46..48].copy_from_slice(&[0xCD, 0x80]); + code[48..50].copy_from_slice(&[0x0F, 0x0B]); // exit must never return + + let mut bytes = alloc::vec![0; FILE_OFFSET + code.len()]; + bytes[..16].copy_from_slice(b"\x7fELF\x02\x01\x01\0\0\0\0\0\0\0\0\0"); + put_u16(&mut bytes, 16, 2); + put_u16(&mut bytes, 18, 62); + put_u32(&mut bytes, 20, 1); + put_u64(&mut bytes, 24, ENTRY); + put_u64(&mut bytes, 32, ELF_HEADER_SIZE as u64); + put_u16(&mut bytes, 52, ELF_HEADER_SIZE as u16); + put_u16(&mut bytes, 54, PROGRAM_HEADER_SIZE as u16); + put_u16(&mut bytes, 56, 1); + test_load_header( + &mut bytes, + 64, + PF_R | PF_X, + FILE_OFFSET as u64, + ENTRY, + code.len() as u64, + code.len() as u64, + ); + bytes[FILE_OFFSET..].copy_from_slice(&code); + bytes +} + +/// Builds one small ELF fixture whose entry triggers a requested hardware +/// exception immediately. Used only to validate CPL3 fault containment. +#[cfg(feature = "user-fault-containment-test")] +pub(crate) fn process_fault_elf(vector: u8) -> alloc::vec::Vec { + process_fault_elf_at(vector, 0x3000_0000) +} + +#[cfg(feature = "user-fault-containment-test")] +pub(crate) fn process_fault_elf_at(vector: u8, fault_address: u64) -> alloc::vec::Vec { + const ENTRY: u64 = 0x0040_0000; + const FILE_OFFSET: usize = 0x1000; + let code: &[u8] = match vector { + 6 => &[0x66, 0x0F, 0xEF, 0xC0], // PXOR XMM0, XMM0: SIMD must be disabled + 7 => &[0xD9, 0xE8], // FLD1: x87 must be disabled + 8 => &[0xC5, 0xF9, 0xEF, 0xC0], // VPXOR XMM0, XMM0, XMM0: AVX must be disabled + 13 => &[0x0F, 0x20, 0xC0], // MOV RAX, CR0: privileged at CPL3 + 14 => return process_page_fault_elf(fault_address), + _ => panic!("unsupported process fault fixture vector"), + }; + + let mut bytes = alloc::vec![0; FILE_OFFSET + code.len()]; + bytes[..16].copy_from_slice(b"\x7fELF\x02\x01\x01\0\0\0\0\0\0\0\0\0"); + put_u16(&mut bytes, 16, 2); + put_u16(&mut bytes, 18, 62); + put_u32(&mut bytes, 20, 1); + put_u64(&mut bytes, 24, ENTRY); + put_u64(&mut bytes, 32, ELF_HEADER_SIZE as u64); + put_u16(&mut bytes, 52, ELF_HEADER_SIZE as u16); + put_u16(&mut bytes, 54, PROGRAM_HEADER_SIZE as u16); + put_u16(&mut bytes, 56, 1); + test_load_header( + &mut bytes, + 64, + PF_R | PF_X, + FILE_OFFSET as u64, + ENTRY, + code.len() as u64, + code.len() as u64, + ); + bytes[FILE_OFFSET..].copy_from_slice(code); + bytes +} + +#[cfg(feature = "user-fault-containment-test")] +fn process_page_fault_elf(fault_address: u64) -> alloc::vec::Vec { + const ENTRY: u64 = 0x0040_0000; + const FILE_OFFSET: usize = 0x1000; + let mut code = [0u8; 13]; + code[0..2].copy_from_slice(&[0x48, 0xB8]); // movabs rax, fault_address + code[2..10].copy_from_slice(&fault_address.to_le_bytes()); + code[10..13].copy_from_slice(&[0x48, 0x8B, 0x00]); // mov rax, [rax] + let mut bytes = alloc::vec![0; FILE_OFFSET + code.len()]; + bytes[..16].copy_from_slice(b"\x7fELF\x02\x01\x01\0\0\0\0\0\0\0\0\0"); + put_u16(&mut bytes, 16, 2); + put_u16(&mut bytes, 18, 62); + put_u32(&mut bytes, 20, 1); + put_u64(&mut bytes, 24, ENTRY); + put_u64(&mut bytes, 32, ELF_HEADER_SIZE as u64); + put_u16(&mut bytes, 52, ELF_HEADER_SIZE as u16); + put_u16(&mut bytes, 54, PROGRAM_HEADER_SIZE as u16); + put_u16(&mut bytes, 56, 1); + test_load_header( + &mut bytes, + 64, + PF_R | PF_X, + FILE_OFFSET as u64, + ENTRY, + code.len() as u64, + code.len() as u64, + ); + bytes[FILE_OFFSET..].copy_from_slice(&code); + bytes +} + +/// Builds one ELF for a process-isolation test. Both roots map the same +/// private VA, and the payload checks its expected per-process value while +/// repeated PIT/syscall boundaries permit real scheduler switches. +#[cfg(feature = "address-space-isolation-test")] +pub(crate) fn process_isolation_elf(expected: u64) -> alloc::vec::Vec { + const ENTRY: u64 = 0x0040_0000; + const FILE_OFFSET: usize = 0x1000; + const PRIVATE_VA: u64 = 0x0010_0000; + let mut code = [0u8; 69]; + code[0..2].copy_from_slice(&[0x48, 0xB8]); // movabs rax, PRIVATE_VA + code[2..10].copy_from_slice(&PRIVATE_VA.to_le_bytes()); + code[10..13].copy_from_slice(&[0x48, 0x8B, 0x18]); // mov rbx, [rax] + code[13..15].copy_from_slice(&[0x48, 0xB9]); // movabs rcx, expected + code[15..23].copy_from_slice(&expected.to_le_bytes()); + code[23..26].copy_from_slice(&[0x48, 0x39, 0xCB]); // cmp rbx, rcx + code[26..28].copy_from_slice(&[0x75, 0x1B]); // jne failure + code[28..33].copy_from_slice(&[0xB8, 2, 0, 0, 0]); // mov eax, TIMER_TICKS + code[33..35].copy_from_slice(&[0xCD, 0x80]); + code[35..39].copy_from_slice(&[0x48, 0x83, 0xF8, 30]); // cmp rax, 30 + code[39..41].copy_from_slice(&[0x72, 0xF3]); // jb loop + code[41..46].copy_from_slice(&[0xBF, 0, 0, 0, 0]); // exit(0) + code[46..51].copy_from_slice(&[0xB8, 4, 0, 0, 0]); + code[51..53].copy_from_slice(&[0xCD, 0x80]); + code[53..55].copy_from_slice(&[0x0F, 0x0B]); // unreachable sentinel + code[55..60].copy_from_slice(&[0xBF, 1, 0, 0, 0]); // failure: exit(1) + code[60..65].copy_from_slice(&[0xB8, 4, 0, 0, 0]); + code[65..67].copy_from_slice(&[0xCD, 0x80]); + code[67..69].copy_from_slice(&[0x0F, 0x0B]); + + let mut bytes = alloc::vec![0; FILE_OFFSET + code.len()]; + bytes[..16].copy_from_slice(b"\x7fELF\x02\x01\x01\0\0\0\0\0\0\0\0\0"); + put_u16(&mut bytes, 16, 2); + put_u16(&mut bytes, 18, 62); + put_u32(&mut bytes, 20, 1); + put_u64(&mut bytes, 24, ENTRY); + put_u64(&mut bytes, 32, ELF_HEADER_SIZE as u64); + put_u16(&mut bytes, 52, ELF_HEADER_SIZE as u16); + put_u16(&mut bytes, 54, PROGRAM_HEADER_SIZE as u16); + put_u16(&mut bytes, 56, 1); + test_load_header( + &mut bytes, + 64, + PF_R | PF_X, + FILE_OFFSET as u64, + ENTRY, + code.len() as u64, + code.len() as u64, + ); + bytes[FILE_OFFSET..].copy_from_slice(&code); + bytes +} + +fn test_load_header( + bytes: &mut [u8], + at: usize, + flags: u32, + offset: u64, + va: u64, + filesz: u64, + memsz: u64, +) { + put_u32(bytes, at, PT_LOAD); + put_u32(bytes, at + 4, flags); + put_u64(bytes, at + 8, offset); + put_u64(bytes, at + 16, va); + put_u64(bytes, at + 32, filesz); + put_u64(bytes, at + 40, memsz); + put_u64(bytes, at + 48, Size4KiB::SIZE); +} + +fn put_u16(bytes: &mut [u8], at: usize, value: u16) { + bytes[at..at + 2].copy_from_slice(&value.to_le_bytes()); +} + +fn put_u32(bytes: &mut [u8], at: usize, value: u32) { + bytes[at..at + 4].copy_from_slice(&value.to_le_bytes()); +} + +fn put_u64(bytes: &mut [u8], at: usize, value: u64) { + bytes[at..at + 8].copy_from_slice(&value.to_le_bytes()); +} diff --git a/kernel/src/process/mod.rs b/kernel/src/process/mod.rs new file mode 100644 index 0000000..e1f8201 --- /dev/null +++ b/kernel/src/process/mod.rs @@ -0,0 +1,723 @@ +//! Bounded, generation-safe process identity and lifecycle state. +//! +//! Architecture-specific address-space and task resources are attached in +//! later layers; this module owns only process policy and bounded metadata. + +pub mod elf; +pub mod user_copy; + +use spin::{Mutex, Once}; + +use crate::{ + arch::x86_64::memory::{AddressSpace, AddressSpaceError, AddressSpaceHandle}, + kprintln, + task::TaskId, +}; + +pub const MAX_PROCESSES: usize = 8; +pub const PROCESS_NAME_CAPACITY: usize = 32; + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct ProcessId { + slot: u8, + generation: u64, +} + +impl ProcessId { + pub const fn slot(self) -> usize { + self.slot as usize + } + + pub const fn generation(self) -> u64 { + self.generation + } +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq, Ord, PartialOrd)] +pub struct Pid(u64); + +impl Pid { + pub const fn get(self) -> u64 { + self.0 + } +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct ProcessName { + bytes: [u8; PROCESS_NAME_CAPACITY], + len: u8, +} + +impl ProcessName { + pub const EMPTY: Self = Self { + bytes: [0; PROCESS_NAME_CAPACITY], + len: 0, + }; + + pub fn new(value: &str) -> Result { + if value.is_empty() || value.len() > PROCESS_NAME_CAPACITY { + return Err(ProcessError::InvalidName); + } + let mut bytes = [0; PROCESS_NAME_CAPACITY]; + bytes[..value.len()].copy_from_slice(value.as_bytes()); + Ok(Self { + bytes, + len: u8::try_from(value.len()).map_err(|_| ProcessError::InvalidName)?, + }) + } + + #[allow(dead_code)] // Process names are retained for future in-kernel snapshot consumers. + pub fn as_str(&self) -> &str { + core::str::from_utf8(&self.bytes[..self.len as usize]) + .expect("ProcessName bytes were validated at construction") + } +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum ProcessState { + Vacant, + Creating, + Alive, + Exiting, + Zombie, + Retired, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum ProcessExitReason { + Exited { + status: i32, + }, + Fault { + vector: u8, + error_code: Option, + fault_address: Option, + rip: u64, + }, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum ProcessError { + TableFull, + PidExhausted, + InvalidId, + InvalidTransition, + InvalidName, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct ProcessSnapshot { + pub id: ProcessId, + pub pid: Pid, + pub parent: Option, + pub state: ProcessState, + pub name: ProcessName, + pub created_tick: u64, + pub exit_tick: Option, + pub cpu_ticks: u64, + pub mapped_user_pages: u64, + pub task_count: u8, + pub exit_reason: Option, +} + +struct ProcessSlot { + generation: u64, + state: ProcessState, + pid: Option, + parent: Option, + name: ProcessName, + created_tick: u64, + exit_tick: Option, + cpu_ticks: u64, + mapped_user_pages: u64, + task_count: u8, + exit_reason: Option, + address_space: Option, + primary_task: Option, +} + +impl ProcessSlot { + const EMPTY: Self = Self { + generation: 0, + state: ProcessState::Vacant, + pid: None, + parent: None, + name: ProcessName::EMPTY, + created_tick: 0, + exit_tick: None, + cpu_ticks: 0, + mapped_user_pages: 0, + task_count: 0, + exit_reason: None, + address_space: None, + primary_task: None, + }; + + fn clear_to_vacant(&mut self) { + self.state = ProcessState::Vacant; + self.pid = None; + self.parent = None; + self.name = ProcessName::EMPTY; + self.created_tick = 0; + self.exit_tick = None; + self.cpu_ticks = 0; + self.mapped_user_pages = 0; + self.task_count = 0; + self.exit_reason = None; + self.address_space = None; + self.primary_task = None; + } + + fn snapshot(&self, id: ProcessId) -> ProcessSnapshot { + ProcessSnapshot { + id, + pid: self.pid.expect("live process slot has no PID"), + parent: self.parent, + state: self.state, + name: self.name, + created_tick: self.created_tick, + exit_tick: self.exit_tick, + cpu_ticks: self.cpu_ticks, + mapped_user_pages: self.mapped_user_pages, + task_count: self.task_count, + exit_reason: self.exit_reason, + } + } +} + +struct ProcessTable { + slots: [ProcessSlot; MAX_PROCESSES], + next_pid: Option, +} + +impl ProcessTable { + const fn new() -> Self { + Self { + slots: [const { ProcessSlot::EMPTY }; MAX_PROCESSES], + next_pid: Some(1), + } + } + + fn reserve( + &mut self, + parent: Option, + name: ProcessName, + created_tick: u64, + ) -> Result { + if let Some(parent_id) = parent { + let parent_slot = self.slot_for(parent_id)?; + if parent_slot.state != ProcessState::Alive { + return Err(ProcessError::InvalidTransition); + } + } + let mut available = None; + for (index, slot) in self.slots.iter_mut().enumerate() { + if slot.state != ProcessState::Vacant { + continue; + } + let Some(generation) = slot.generation.checked_add(1) else { + slot.state = ProcessState::Retired; + continue; + }; + available = Some((index, generation)); + break; + } + let (index, generation) = available.ok_or(ProcessError::TableFull)?; + + let Some(raw_pid) = self.next_pid else { + // The generation is consumed for every reservation attempt, even + // when boot-lifetime PID space has already been exhausted. + self.slots[index].generation = generation; + return Err(ProcessError::PidExhausted); + }; + self.next_pid = raw_pid.checked_add(1); + let pid = Pid(raw_pid); + let id = ProcessId { + slot: u8::try_from(index).expect("process capacity exceeds ProcessId slot"), + generation, + }; + self.slots[index] = ProcessSlot { + generation, + state: ProcessState::Creating, + pid: Some(pid), + parent, + name, + created_tick, + ..ProcessSlot::EMPTY + }; + Ok(id) + } + + fn slot_for(&self, id: ProcessId) -> Result<&ProcessSlot, ProcessError> { + let slot = self.slots.get(id.slot()).ok_or(ProcessError::InvalidId)?; + if slot.generation != id.generation + || matches!(slot.state, ProcessState::Vacant | ProcessState::Retired) + { + return Err(ProcessError::InvalidId); + } + Ok(slot) + } + + fn slot_for_mut(&mut self, id: ProcessId) -> Result<&mut ProcessSlot, ProcessError> { + let slot = self + .slots + .get_mut(id.slot()) + .ok_or(ProcessError::InvalidId)?; + if slot.generation != id.generation + || matches!(slot.state, ProcessState::Vacant | ProcessState::Retired) + { + return Err(ProcessError::InvalidId); + } + Ok(slot) + } + + fn transition( + &mut self, + id: ProcessId, + expected: ProcessState, + next: ProcessState, + ) -> Result<(), ProcessError> { + if !matches!( + (expected, next), + (ProcessState::Creating, ProcessState::Alive) + | (ProcessState::Exiting, ProcessState::Zombie) + ) { + return Err(ProcessError::InvalidTransition); + } + let slot = self.slot_for_mut(id)?; + if slot.state != expected { + return Err(ProcessError::InvalidTransition); + } + slot.state = next; + Ok(()) + } + + fn begin_exit( + &mut self, + id: ProcessId, + reason: ProcessExitReason, + tick: u64, + ) -> Result<(), ProcessError> { + let slot = self.slot_for_mut(id)?; + if slot.state != ProcessState::Alive { + return Err(ProcessError::InvalidTransition); + } + slot.state = ProcessState::Exiting; + slot.exit_reason = Some(reason); + slot.exit_tick = Some(tick); + Ok(()) + } + + fn reap(&mut self, id: ProcessId) -> Result<(), ProcessError> { + let slot = self.slot_for_mut(id)?; + if slot.state != ProcessState::Zombie { + return Err(ProcessError::InvalidTransition); + } + // Generation changes only when a later reservation reuses this + // Vacant slot. Vacant itself invalidates this ProcessId immediately. + slot.clear_to_vacant(); + Ok(()) + } + + fn abort_creation(&mut self, id: ProcessId) -> Result<(), ProcessError> { + let slot = self.slot_for_mut(id)?; + if slot.state != ProcessState::Creating { + return Err(ProcessError::InvalidTransition); + } + // The PID and generation remain consumed. Vacant immediately makes + // this Creating identity stale; reuse advances generation later. + slot.clear_to_vacant(); + Ok(()) + } + + fn install_address_space( + &mut self, + id: ProcessId, + address_space: AddressSpace, + ) -> Result<(), ProcessError> { + let slot = self.slot_for_mut(id)?; + if slot.state != ProcessState::Creating || slot.address_space.is_some() { + return Err(ProcessError::InvalidTransition); + } + slot.mapped_user_pages = address_space.mapped_user_pages(); + slot.address_space = Some(address_space); + Ok(()) + } + + fn install_primary_task(&mut self, id: ProcessId, task: TaskId) -> Result<(), ProcessError> { + let slot = self.slot_for_mut(id)?; + if slot.state != ProcessState::Creating + || slot.address_space.is_none() + || slot.primary_task.is_some() + { + return Err(ProcessError::InvalidTransition); + } + slot.primary_task = Some(task); + slot.task_count = 1; + Ok(()) + } + + #[allow(dead_code)] // Used by release process creation; QEMU-only builds omit that entry point. + fn publish_created(&mut self, id: ProcessId, task: TaskId) -> Result<(), ProcessError> { + let slot = self.slot_for_mut(id)?; + if slot.state != ProcessState::Creating + || slot.address_space.is_none() + || slot.primary_task != Some(task) + || slot.task_count != 1 + { + return Err(ProcessError::InvalidTransition); + } + slot.state = ProcessState::Alive; + Ok(()) + } + + fn address_space_root( + &self, + id: ProcessId, + ) -> Result< + x86_64::structures::paging::PhysFrame, + ProcessError, + > { + let slot = self.slot_for(id)?; + slot.address_space + .as_ref() + .map(AddressSpace::root_frame) + .ok_or(ProcessError::InvalidTransition) + } + + fn address_space_handle(&self, id: ProcessId) -> Result { + let slot = self.slot_for(id)?; + if slot.state != ProcessState::Alive { + return Err(ProcessError::InvalidTransition); + } + slot.address_space + .as_ref() + .map(AddressSpace::handle) + .ok_or(ProcessError::InvalidTransition) + } + + fn pid(&self, id: ProcessId) -> Result { + let slot = self.slot_for(id)?; + if slot.state != ProcessState::Alive { + return Err(ProcessError::InvalidTransition); + } + slot.pid.ok_or(ProcessError::InvalidTransition) + } + + fn snapshot(&self, id: ProcessId) -> Result { + let slot = self.slot_for(id)?; + Ok(slot.snapshot(id)) + } + + fn add_cpu_tick(&mut self, id: ProcessId) -> Result<(), ProcessError> { + let slot = self.slot_for_mut(id)?; + if !matches!(slot.state, ProcessState::Alive | ProcessState::Exiting) { + return Err(ProcessError::InvalidTransition); + } + slot.cpu_ticks = slot.cpu_ticks.saturating_add(1); + Ok(()) + } +} + +static PROCESS_TABLE: Once> = Once::new(); + +pub fn init() { + kprintln!("[process] initializing"); + PROCESS_TABLE.call_once(|| Mutex::new(ProcessTable::new())); + run_model_self_tests(); + kprintln!("[process] process capacity: {MAX_PROCESSES}"); + kprintln!("[process] identity and lifecycle initialized"); +} + +fn with_table(f: impl FnOnce(&mut ProcessTable) -> R) -> R { + let table = PROCESS_TABLE + .get() + .expect("process subsystem used before init"); + // Callers execute on the single CPU; mask IRQs so a timer cannot + // re-enter while process metadata is mutably borrowed. + x86_64::instructions::interrupts::without_interrupts(|| f(&mut table.lock())) +} + +pub fn reserve( + parent: Option, + name: &str, + created_tick: u64, +) -> Result { + let name = ProcessName::new(name)?; + with_table(|table| table.reserve(parent, name, created_tick)) +} + +pub(crate) fn install_address_space( + id: ProcessId, + address_space: AddressSpace, +) -> Result<(), ProcessError> { + with_table(|table| table.install_address_space(id, address_space)) +} + +pub(crate) fn install_primary_task(id: ProcessId, task: TaskId) -> Result<(), ProcessError> { + with_table(|table| table.install_primary_task(id, task)) +} + +#[allow(dead_code)] // Used by release process creation; QEMU-only builds omit that entry point. +pub(crate) fn publish_created(id: ProcessId, task: TaskId) -> Result<(), ProcessError> { + with_table(|table| table.publish_created(id, task)) +} + +#[cfg(feature = "qemu-test")] +pub fn is_alive(id: ProcessId) -> bool { + let Some(table) = PROCESS_TABLE.get() else { + return false; + }; + x86_64::instructions::interrupts::without_interrupts(|| { + table + .lock() + .slot_for(id) + .is_ok_and(|slot| slot.state == ProcessState::Alive) + }) +} + +#[allow(dead_code)] // Required by callers that roll back a failed process construction transaction. +pub fn abort_creation(id: ProcessId) -> Result<(), ProcessError> { + with_table(|table| table.abort_creation(id)) +} + +pub fn begin_exit(id: ProcessId, reason: ProcessExitReason, tick: u64) -> Result<(), ProcessError> { + with_table(|table| table.begin_exit(id, reason, tick)) +} + +pub fn mark_zombie(id: ProcessId) -> Result<(), ProcessError> { + with_table(|table| table.transition(id, ProcessState::Exiting, ProcessState::Zombie)) +} + +#[allow(dead_code)] // Kept as the internal Zombie reaper API until a wait/reaper service consumes it. +pub fn reap(id: ProcessId) -> Result<(), ProcessError> { + with_table(|table| table.reap(id)) +} + +pub fn snapshot(id: ProcessId) -> Result { + let table = PROCESS_TABLE + .get() + .expect("process subsystem used before init"); + x86_64::instructions::interrupts::without_interrupts(|| table.lock().snapshot(id)) +} + +/// Returns a bounded point-in-time view of every non-vacant, non-retired +/// process slot. Empty array entries represent Vacant or permanently Retired +/// slots. The fixed result requires no heap allocation and is suitable as an +/// internal source for future process-monitoring services. +#[cfg(feature = "process-smoke-test")] +pub fn snapshot_all() -> [Option; MAX_PROCESSES] { + let table = PROCESS_TABLE + .get() + .expect("process subsystem used before init"); + x86_64::instructions::interrupts::without_interrupts(|| { + let table = table.lock(); + core::array::from_fn(|index| { + let slot = &table.slots[index]; + if matches!(slot.state, ProcessState::Vacant | ProcessState::Retired) { + return None; + } + let id = ProcessId { + slot: index as u8, + generation: slot.generation, + }; + Some(slot.snapshot(id)) + }) + }) +} + +pub(crate) fn account_cpu_tick(id: ProcessId) { + if let Some(table) = PROCESS_TABLE.get() { + x86_64::instructions::interrupts::without_interrupts(|| { + // Accounting saturates so a process running for the entire boot + // lifetime never wraps its historical CPU-time counter. + let _ = table.lock().add_cpu_tick(id); + }); + } +} + +pub(crate) fn pid(id: ProcessId) -> Result { + let table = PROCESS_TABLE.get().ok_or(ProcessError::InvalidId)?; + x86_64::instructions::interrupts::without_interrupts(|| table.lock().pid(id)) +} + +pub(crate) fn address_space_root( + id: ProcessId, +) -> Result, ProcessError> +{ + let table = PROCESS_TABLE.get().ok_or(ProcessError::InvalidId)?; + x86_64::instructions::interrupts::without_interrupts(|| table.lock().address_space_root(id)) +} + +pub(crate) fn address_space_handle(id: ProcessId) -> Result { + let table = PROCESS_TABLE + .get() + .expect("process subsystem used before init"); + x86_64::instructions::interrupts::without_interrupts(|| table.lock().address_space_handle(id)) +} + +#[allow(dead_code)] // 0.0.9 has no buffer-bearing syscall; future handlers must use this checked path. +pub(crate) fn copy_from_user( + id: ProcessId, + address: u64, + output: &mut [u8], +) -> Result<(), AddressSpaceError> { + let table = PROCESS_TABLE + .get() + .ok_or(AddressSpaceError::InvalidCopyRange)?; + x86_64::instructions::interrupts::without_interrupts(|| { + let mut table = table.lock(); + let slot = table + .slot_for_mut(id) + .map_err(|_| AddressSpaceError::InvalidCopyRange)?; + if slot.state != ProcessState::Alive { + return Err(AddressSpaceError::InvalidCopyRange); + } + let address_space = slot + .address_space + .as_mut() + .ok_or(AddressSpaceError::InvalidCopyRange)?; + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + address_space.copy_from_user(kernel, address, output) + }) + }) +} + +#[allow(dead_code)] // 0.0.9 has no buffer-bearing syscall; future handlers must use this checked path. +pub(crate) fn copy_to_user( + id: ProcessId, + address: u64, + input: &[u8], +) -> Result<(), AddressSpaceError> { + let table = PROCESS_TABLE + .get() + .ok_or(AddressSpaceError::InvalidCopyRange)?; + x86_64::instructions::interrupts::without_interrupts(|| { + let mut table = table.lock(); + let slot = table + .slot_for_mut(id) + .map_err(|_| AddressSpaceError::InvalidCopyRange)?; + if slot.state != ProcessState::Alive { + return Err(AddressSpaceError::InvalidCopyRange); + } + let address_space = slot + .address_space + .as_mut() + .ok_or(AddressSpaceError::InvalidCopyRange)?; + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + address_space.copy_to_user(kernel, address, input) + }) + }) +} + +fn run_model_self_tests() { + let mut table = ProcessTable::new(); + let name = ProcessName::new("model-test").unwrap(); + let first = table.reserve(None, name, 10).unwrap(); + assert_eq!(first.generation(), 1); + assert_eq!(table.snapshot(first).unwrap().pid.get(), 1); + assert_eq!( + table.transition(first, ProcessState::Creating, ProcessState::Zombie), + Err(ProcessError::InvalidTransition) + ); + table + .transition(first, ProcessState::Creating, ProcessState::Alive) + .unwrap(); + table + .begin_exit(first, ProcessExitReason::Exited { status: 7 }, 12) + .unwrap(); + assert_eq!(table.reap(first), Err(ProcessError::InvalidTransition)); + table + .transition(first, ProcessState::Exiting, ProcessState::Zombie) + .unwrap(); + table.reap(first).unwrap(); + assert_eq!(table.snapshot(first), Err(ProcessError::InvalidId)); + let second = table.reserve(None, name, 20).unwrap(); + assert_eq!(second.slot(), first.slot()); + assert_eq!(second.generation(), first.generation() + 1); + assert_eq!(table.snapshot(second).unwrap().pid.get(), 2); + assert_eq!(table.snapshot(first), Err(ProcessError::InvalidId)); + assert!(table.add_cpu_tick(second).is_err()); + assert_eq!( + table.snapshot(ProcessId { + slot: u8::MAX, + generation: 1 + }), + Err(ProcessError::InvalidId) + ); + + table + .transition(second, ProcessState::Creating, ProcessState::Alive) + .unwrap(); + table + .begin_exit( + second, + ProcessExitReason::Fault { + vector: 6, + error_code: None, + fault_address: None, + rip: 0x400000, + }, + 22, + ) + .unwrap(); + assert!(matches!( + table.snapshot(second).unwrap().exit_reason, + Some(ProcessExitReason::Fault { vector: 6, .. }) + )); + + table.slots[1].generation = u64::MAX; + let next = table.reserve(None, name, 21).unwrap(); + assert_ne!(next.slot(), 1); + assert_eq!(table.slots[1].state, ProcessState::Retired); + + let mut full = ProcessTable::new(); + for _ in 0..MAX_PROCESSES { + full.reserve(None, name, 0).unwrap(); + } + assert_eq!(full.reserve(None, name, 0), Err(ProcessError::TableFull)); + + let mut pid_exhausted = ProcessTable::new(); + pid_exhausted.next_pid = None; + assert_eq!( + pid_exhausted.reserve(None, name, 0), + Err(ProcessError::PidExhausted) + ); + assert_eq!(pid_exhausted.slots[0].generation, 1); + assert_eq!(pid_exhausted.slots[0].state, ProcessState::Vacant); + + let mut last_pid = ProcessTable::new(); + last_pid.next_pid = Some(u64::MAX); + let final_id = last_pid.reserve(None, name, 0).unwrap(); + assert_eq!(last_pid.snapshot(final_id).unwrap().pid.get(), u64::MAX); + assert_eq!( + last_pid.reserve(None, name, 1), + Err(ProcessError::PidExhausted) + ); + + let mut parent_table = ProcessTable::new(); + let parent = parent_table.reserve(None, name, 0).unwrap(); + assert_eq!( + parent_table.reserve(Some(parent), name, 1), + Err(ProcessError::InvalidTransition), + "Creating process cannot parent another process" + ); + parent_table + .transition(parent, ProcessState::Creating, ProcessState::Alive) + .unwrap(); + let child = parent_table.reserve(Some(parent), name, 1).unwrap(); + assert_eq!(parent_table.snapshot(child).unwrap().parent, Some(parent)); + parent_table.abort_creation(child).unwrap(); + parent_table + .begin_exit(parent, ProcessExitReason::Exited { status: 0 }, 2) + .unwrap(); + parent_table + .transition(parent, ProcessState::Exiting, ProcessState::Zombie) + .unwrap(); + parent_table.reap(parent).unwrap(); + assert_eq!( + parent_table.reserve(Some(parent), name, 3), + Err(ProcessError::InvalidId), + "stale parent ProcessId was accepted" + ); +} diff --git a/kernel/src/process/user_copy.rs b/kernel/src/process/user_copy.rs new file mode 100644 index 0000000..8a6c46d --- /dev/null +++ b/kernel/src/process/user_copy.rs @@ -0,0 +1,22 @@ +//! Checked process-aware user-memory copies. +//! +//! System calls should use these entry points rather than dereferencing a +//! raw userspace pointer. Copying is page-chunked through the audited +//! supervisor physical map, with the entire mapping range checked first. + +use crate::{arch::x86_64::memory::AddressSpaceError, task}; + +/// Copies from the current process into a kernel-owned buffer. +#[allow(dead_code)] // Public-in-module syscall API; getpid/exit do not carry user buffers. +pub fn copy_from_user(address: u64, output: &mut [u8]) -> Result<(), AddressSpaceError> { + let process_id = task::current_user_process_id().ok_or(AddressSpaceError::InvalidCopyRange)?; + super::copy_from_user(process_id, address, output) +} + +/// Copies from a kernel-owned buffer into writable memory in the current +/// process. +#[allow(dead_code)] // Public-in-module syscall API; getpid/exit do not carry user buffers. +pub fn copy_to_user(address: u64, input: &[u8]) -> Result<(), AddressSpaceError> { + let process_id = task::current_user_process_id().ok_or(AddressSpaceError::InvalidCopyRange)?; + super::copy_to_user(process_id, address, input) +} diff --git a/kernel/src/task/mod.rs b/kernel/src/task/mod.rs index 3dad645..b3bcd98 100644 --- a/kernel/src/task/mod.rs +++ b/kernel/src/task/mod.rs @@ -1,17 +1,47 @@ +#[cfg(any(feature = "process-preemption-test", feature = "process-smoke-test"))] +use core::sync::atomic::AtomicBool; use core::sync::atomic::{AtomicU64, Ordering}; use spin::{Mutex, Once}; use x86_64::instructions::{hlt, interrupts}; use crate::{ + arch::x86_64::memory::ValidatedUserContext, arch::x86_64::{ gdt, pic::{self, IrqLine}, pit, task as arch_task, }, kprintln, + process::{self, ProcessExitReason, ProcessId}, }; +#[cfg(all(feature = "process-preemption-test", feature = "process-smoke-test"))] +compile_error!("process-preemption-test and process-smoke-test are separate QEMU scenarios"); +#[cfg(any( + all( + feature = "user-fault-containment-test", + feature = "process-preemption-test" + ), + all( + feature = "user-fault-containment-test", + feature = "process-smoke-test" + ), + all( + feature = "address-space-isolation-test", + feature = "process-preemption-test" + ), + all( + feature = "address-space-isolation-test", + feature = "process-smoke-test" + ), + all( + feature = "address-space-isolation-test", + feature = "user-fault-containment-test" + ) +))] +compile_error!("process QEMU scenarios have separate feature selections"); + pub const MAX_KERNEL_TASKS: usize = arch_task::MAX_KERNEL_TASKS; pub type KernelTaskEntry = fn(); @@ -21,6 +51,16 @@ pub struct TaskId { generation: u64, } +/// Complete target description for the architecture handoff. Scheduler and +/// process locks are released before this plan is resolved to a root and +/// applied; assembly receives only `context_rsp` after CR3/RSP0 are safe. +#[derive(Clone, Copy)] +struct SwitchPlan { + context_rsp: u64, + target: TaskKind, + task_slot: Option, +} + impl TaskId { pub const fn slot(self) -> usize { self.slot as usize @@ -33,6 +73,7 @@ impl TaskId { #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub enum TaskState { Vacant, + Prepared, Ready, Running, Exiting, @@ -40,18 +81,30 @@ pub enum TaskState { Retired, } +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum TaskKind { + Kernel, + User(ProcessId), +} + #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub enum SpawnError { SchedulerNotInitialized, TaskTableFull, NoTaskStackAvailable, QueueFull, + NotPrepared, + #[allow(dead_code)] // The QEMU-only config omits the process publication entry point. + ProcessNotAlive, + ProcessAddressSpaceMismatch, + InvalidTask, } #[derive(Clone, Copy)] struct TaskSlot { generation: u64, state: TaskState, + kind: TaskKind, saved_rsp: u64, queued: bool, } @@ -60,6 +113,7 @@ impl TaskSlot { const EMPTY: Self = Self { generation: 0, state: TaskState::Vacant, + kind: TaskKind::Kernel, saved_rsp: 0, queued: false, }; @@ -106,6 +160,7 @@ struct Scheduler { queue: ReadyQueue, current: Option, bootstrap_rsp: u64, + pending_process_completion: Option, } impl Scheduler { @@ -116,11 +171,32 @@ impl Scheduler { queue: ReadyQueue::new(), current: None, bootstrap_rsp: 0, + pending_process_completion: None, } } } static SCHEDULER: Once> = Once::new(); +#[cfg(feature = "process-preemption-test")] +static PROCESS_PREEMPTION_ACTIVE: AtomicBool = AtomicBool::new(false); +#[cfg(feature = "process-preemption-test")] +static PROCESS_PREEMPTION_PROGRESS_A: AtomicU64 = AtomicU64::new(0); +#[cfg(feature = "process-preemption-test")] +static PROCESS_PREEMPTION_PROGRESS_B: AtomicU64 = AtomicU64::new(0); +#[cfg(feature = "process-preemption-test")] +static PROCESS_PREEMPTION_START_TICK: AtomicU64 = AtomicU64::new(0); +#[cfg(feature = "process-smoke-test")] +static PROCESS_SMOKE_ACTIVE: AtomicBool = AtomicBool::new(false); +#[cfg(feature = "process-smoke-test")] +static PROCESS_SMOKE_START_TICK: AtomicU64 = AtomicU64::new(0); +#[cfg(feature = "process-smoke-test")] +static PROCESS_SMOKE_ID: Once = Once::new(); +#[cfg(feature = "user-fault-containment-test")] +static USER_FAULT_IDS: Once<[ProcessId; 7]> = Once::new(); +#[cfg(feature = "address-space-isolation-test")] +static ADDRESS_ISOLATION_IDS: Once<[ProcessId; 2]> = Once::new(); +#[cfg(not(any(feature = "qemu-test", feature = "userspace-test")))] +static RELEASE_SMOKE_ID: Once = Once::new(); static SMOKE_A: AtomicU64 = AtomicU64::new(0); static SMOKE_B: AtomicU64 = AtomicU64::new(0); static SMOKE_A_SP: AtomicU64 = AtomicU64::new(0); @@ -276,6 +352,7 @@ pub fn spawn_kernel_task(entry: KernelTaskEntry) -> Result { scheduler.slots[index] = TaskSlot { generation, state: TaskState::Ready, + kind: TaskKind::Kernel, saved_rsp: context_addr, queued: true, }; @@ -283,6 +360,207 @@ pub fn spawn_kernel_task(entry: KernelTaskEntry) -> Result { }) } +/// Reserves and initializes a user task context without making it runnable. +/// The caller must have already built the process AddressSpace and user stack; +/// `entry` and `user_rsp` are checked scalar values from that validated image. +pub(crate) fn prepare_user_task( + process_id: ProcessId, + context: ValidatedUserContext, +) -> Result { + let root = process::address_space_root(process_id) + .map_err(|_| SpawnError::ProcessAddressSpaceMismatch)?; + if context.root_frame() != root { + return Err(SpawnError::ProcessAddressSpaceMismatch); + } + let scheduler = SCHEDULER.get().ok_or(SpawnError::SchedulerNotInitialized)?; + interrupts::without_interrupts(|| { + let mut scheduler = scheduler.lock(); + if !scheduler.initialized { + return Err(SpawnError::SchedulerNotInitialized); + } + let index = reserve_slot(&mut scheduler)?; + let generation = scheduler.slots[index] + .generation + .checked_add(1) + .ok_or(SpawnError::TaskTableFull)?; + let id = TaskId { + slot: index as u8, + generation, + }; + let (_, top) = arch_task::stack_bounds(index).ok_or(SpawnError::NoTaskStackAvailable)?; + let context_size = u64::try_from(core::mem::size_of::()) + .map_err(|_| SpawnError::NoTaskStackAvailable)?; + let context_addr = top + .checked_sub(context_size) + .ok_or(SpawnError::NoTaskStackAvailable)?; + if context_addr % 16 != 0 { + return Err(SpawnError::NoTaskStackAvailable); + } + let context = arch_task::Context { + r15: 0, + r14: 0, + r13: 0, + r12: 0, + r11: 0, + r10: 0, + r9: 0, + r8: 0, + rdi: 0, + rsi: 0, + rbp: 0, + rdx: 0, + rcx: 0, + rbx: 0, + rax: 0, + rip: context.entry(), + cs: u64::from(gdt::user_code_selector().0), + rflags: 0x202, + rsp: context.user_rsp(), + ss: u64::from(gdt::user_data_selector().0), + }; + let (stack_low, _) = + arch_task::stack_bounds(index).ok_or(SpawnError::NoTaskStackAvailable)?; + // SAFETY: scheduler lock and IF masking prove this slot is not active; + // its pages were verified supervisor RW/NX during task initialization. + unsafe { + core::ptr::write_bytes(stack_low as *mut u8, 0, arch_task::STACK_SIZE as usize); + (context_addr as *mut arch_task::Context).write(context); + } + scheduler.slots[index] = TaskSlot { + generation, + state: TaskState::Prepared, + kind: TaskKind::User(process_id), + saved_rsp: context_addr, + queued: false, + }; + Ok(id) + }) +} + +/// Atomically publishes the prepared process/task pair. Lock order is +/// scheduler then process table; IF remains clear through Process Alive and +/// ready-queue publication, so neither timer scheduling nor another kernel +/// observer can see an intermediate state on this single CPU. +#[allow(dead_code)] // Used by the release smoke path, omitted in qemu-test-only builds. +pub(crate) fn publish_prepared_user_task(id: TaskId) -> Result<(), SpawnError> { + let scheduler = SCHEDULER.get().ok_or(SpawnError::SchedulerNotInitialized)?; + interrupts::without_interrupts(|| { + let mut scheduler = scheduler.lock(); + let slot = scheduler + .slots + .get(id.slot()) + .ok_or(SpawnError::InvalidTask)?; + if slot.generation != id.generation || slot.state != TaskState::Prepared || slot.queued { + return Err(SpawnError::NotPrepared); + } + let TaskKind::User(process_id) = slot.kind else { + return Err(SpawnError::NotPrepared); + }; + if scheduler.queue.len == MAX_KERNEL_TASKS { + return Err(SpawnError::QueueFull); + } + process::publish_created(process_id, id).map_err(|_| SpawnError::ProcessNotAlive)?; + scheduler + .queue + .push(id) + .expect("ready queue changed under scheduler lock"); + let slot = &mut scheduler.slots[id.slot()]; + slot.state = TaskState::Ready; + slot.queued = true; + Ok(()) + }) +} + +#[cfg(feature = "qemu-test")] +pub fn run_prepared_user_task_test() { + let process_id = process::reserve(None, "prepared-test", pit::ticks()) + .expect("failed to reserve prepared-task test process"); + let (address_space, context) = + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("failed to create prepared-task test root"); + crate::arch::x86_64::memory::UserStack::map(&mut address_space, kernel) + .expect("failed to create prepared-task test stack"); + let page = x86_64::structures::paging::Page::from_start_address(x86_64::VirtAddr::new( + crate::arch::x86_64::memory::USER_MIN, + )) + .expect("USER_MIN is not page aligned"); + address_space + .map_new_user_page( + kernel, + page, + crate::arch::x86_64::memory::Permissions::USER_CODE, + ) + .expect("failed to map prepared-task entry page"); + let context = address_space + .validate_user_context( + kernel, + crate::arch::x86_64::memory::USER_MIN, + crate::arch::x86_64::memory::USER_STACK_TOP, + ) + .expect("test entry/stack pair did not validate"); + (address_space, context) + }); + process::install_address_space(process_id, address_space) + .expect("failed to attach Creating process AddressSpace"); + let id = prepare_user_task(process_id, context).expect("failed to prepare user task"); + process::install_primary_task(process_id, id).expect("failed to attach prepared primary task"); + assert_eq!(task_state(id), Some(TaskState::Prepared)); + assert_eq!(task_kind(id), Some(TaskKind::User(process_id))); + assert_eq!(task_state(id), Some(TaskState::Prepared)); + assert!(!process::is_alive(process_id)); + let snapshot = process::snapshot(process_id).expect("Creating process snapshot missing"); + assert_eq!(snapshot.state, process::ProcessState::Creating); + assert_eq!(snapshot.mapped_user_pages, 9); + cancel_prepared_task(id).expect("could not cancel prepared test task"); + process::abort_creation(process_id).expect("could not roll back test process reservation"); + assert_eq!(task_state(id), None); + kprintln!("[scheduler-test] prepared user task remains non-runnable until process Alive"); +} + +#[cfg(any(feature = "qemu-test", feature = "process-capacity-test"))] +pub fn cancel_prepared_task(id: TaskId) -> Result<(), SpawnError> { + let scheduler = SCHEDULER.get().ok_or(SpawnError::SchedulerNotInitialized)?; + interrupts::without_interrupts(|| { + let mut scheduler = scheduler.lock(); + let slot = scheduler + .slots + .get_mut(id.slot()) + .ok_or(SpawnError::InvalidTask)?; + if slot.generation != id.generation || slot.state != TaskState::Prepared || slot.queued { + return Err(SpawnError::NotPrepared); + } + slot.state = TaskState::Vacant; + slot.saved_rsp = 0; + slot.kind = TaskKind::Kernel; + Ok(()) + }) +} + +#[cfg(feature = "qemu-test")] +pub fn task_kind(id: TaskId) -> Option { + let scheduler = SCHEDULER.get()?; + interrupts::without_interrupts(|| { + let scheduler = scheduler.lock(); + let slot = scheduler.slots.get(id.slot())?; + (slot.generation == id.generation && slot.state != TaskState::Vacant).then_some(slot.kind) + }) +} + +fn reserve_slot(scheduler: &mut Scheduler) -> Result { + for (index, slot) in scheduler.slots.iter_mut().enumerate() { + if !matches!(slot.state, TaskState::Vacant | TaskState::Exited) { + continue; + } + if slot.generation.checked_add(1).is_some() { + return Ok(index); + } + slot.state = TaskState::Retired; + } + Err(SpawnError::TaskTableFull) +} + pub fn task_state(id: TaskId) -> Option { let scheduler = SCHEDULER.get()?; interrupts::without_interrupts(|| { @@ -294,7 +572,8 @@ pub fn task_state(id: TaskId) -> Option { /// Called only by the assembly IRQ0 boundary with IF cleared by the IDT /// interrupt gate. The pointer is the exact register+hardware frame created -/// by `switch.rs`; CPL3 frames are returned untouched after tick/EOI work. +/// by `switch.rs`. Legacy CPL3 tests run from the bootstrap context and retain +/// their old non-scheduling behavior; process-owned user tasks are preempted. pub(crate) fn timer_schedule(context: *mut arch_task::Context) -> *mut arch_task::Context { pit::record_tick(); // SAFETY: assembly passes the address of its just-saved register prefix; @@ -304,44 +583,120 @@ pub(crate) fn timer_schedule(context: *mut arch_task::Context) -> *mut arch_task if from_user { crate::arch::x86_64::syscall::test::TIMER_INTERRUPTED_CPL3.store(true, Ordering::Relaxed); } - if from_user { - pic::notify_end_of_interrupt(IrqLine::Timer); - return context; - } - let scheduler = SCHEDULER .get() .expect("timer IRQ before scheduler initialization"); - let selected = interrupts::without_interrupts(|| { + #[cfg(feature = "process-preemption-test")] + match process_preemption_test_result() { + Some(true) => { + kprintln!("[process-test] user A and user B both made progress"); + kprintln!("[process-test] CPL3 timer preemption passed"); + pic::notify_end_of_interrupt(IrqLine::Timer); + crate::arch::x86_64::qemu::exit_success(); + } + Some(false) => { + kprintln!("[process-test] CPL3 preemption tick watchdog expired"); + pic::notify_end_of_interrupt(IrqLine::Timer); + crate::arch::x86_64::qemu::exit_failure(); + } + None => {} + } + #[cfg(feature = "process-smoke-test")] + if PROCESS_SMOKE_ACTIVE.load(Ordering::Acquire) + && pit::ticks().saturating_sub(PROCESS_SMOKE_START_TICK.load(Ordering::Relaxed)) >= 100 + { + kprintln!("[process-test] process-smoke PIT watchdog expired"); + pic::notify_end_of_interrupt(IrqLine::Timer); + crate::arch::x86_64::qemu::exit_failure(); + } + if from_user { + let process_task_running = interrupts::without_interrupts(|| { + let scheduler = scheduler.lock(); + scheduler.current.is_some() + }); + // Historical CPL3 syscall tests enter directly from bootstrap, not + // through a process Task. Preserve that test context; a real user + // process always has a current User task. + if !process_task_running { + pic::notify_end_of_interrupt(IrqLine::Timer); + return context; + } + } + + let plan = interrupts::without_interrupts(|| { let mut scheduler = scheduler.lock(); assert!( scheduler.initialized, "timer IRQ before scheduler initialization" ); + if let Some(completed_process) = scheduler.pending_process_completion.take() { + // This IRQ is executing on a different context's kernel stack: + // assembly adopted it after the prior handoff. Zombie metadata + // can now be published without retaining the exiting stack. + process::mark_zombie(completed_process) + .expect("deferred process completion was not Exiting"); + } match scheduler.current { Some(id) => { - let slot = scheduler - .slots - .get_mut(id.slot()) - .expect("current task index invalid"); - assert_eq!( - slot.generation, id.generation, - "current task generation stale" - ); - assert!( - matches!(slot.state, TaskState::Running | TaskState::Exiting), - "current task not Running/Exiting" - ); - assert!(!slot.queued, "Running task appeared in queue"); - slot.saved_rsp = context as u64; - assert_task_stack_pointer(id.slot(), slot.saved_rsp); - if slot.state == TaskState::Exiting { - slot.state = TaskState::Exited; + let exiting_process = { + let slot = scheduler + .slots + .get_mut(id.slot()) + .expect("current task index invalid"); + assert_eq!( + slot.generation, id.generation, + "current task generation stale" + ); + assert!( + matches!(slot.state, TaskState::Running | TaskState::Exiting), + "current task not Running/Exiting" + ); + if slot.state == TaskState::Exiting { + assert!( + !from_user, + "terminal user handoff unexpectedly returned to CPL3" + ); + } else { + assert_eq!( + matches!(slot.kind, TaskKind::User(_)), + from_user, + "saved privilege does not match current task kind" + ); + } + assert!(!slot.queued, "Running task appeared in queue"); + if let TaskKind::User(process_id) = slot.kind { + // PIT tick is charged to the process whose task was + // executing when the interrupt arrived, including + // terminal kernel work on its behalf. The process + // table uses saturating accounting. + process::account_cpu_tick(process_id); + } + slot.saved_rsp = context as u64; + assert_task_stack_pointer(id.slot(), slot.saved_rsp); + if slot.state == TaskState::Exiting { + slot.state = TaskState::Exited; + match slot.kind { + TaskKind::User(process_id) => Some(process_id), + TaskKind::Kernel => None, + } + } else { + slot.state = TaskState::Ready; + slot.queued = true; + scheduler.queue.push(id).expect("runnable queue overflow"); + None + } + }; + if let Some(process_id) = exiting_process { + assert!( + scheduler.pending_process_completion.is_none(), + "more than one process completion is pending" + ); + scheduler.pending_process_completion = Some(process_id); + } + if exiting_process.is_some() + || scheduler.slots[id.slot()].state == TaskState::Exited + { scheduler.current = None; - } else { - slot.state = TaskState::Ready; - slot.queued = true; - scheduler.queue.push(id).expect("runnable queue overflow"); } } None => scheduler.bootstrap_rsp = context as u64, @@ -363,19 +718,157 @@ pub(crate) fn timer_schedule(context: *mut arch_task::Context) -> *mut arch_task slot.state = TaskState::Running; let saved_rsp = slot.saved_rsp; assert_task_stack_pointer(id.slot(), saved_rsp); + let target = slot.kind; scheduler.current = Some(id); - saved_rsp + SwitchPlan { + context_rsp: saved_rsp, + target, + task_slot: Some(id.slot()), + } } else { scheduler.current = None; assert_ne!( scheduler.bootstrap_rsp, 0, "bootstrap context was never captured" ); - scheduler.bootstrap_rsp + SwitchPlan { + context_rsp: scheduler.bootstrap_rsp, + target: TaskKind::Kernel, + task_slot: None, + } } }); pic::notify_end_of_interrupt(IrqLine::Timer); - selected as *mut arch_task::Context + apply_switch_plan(plan); + plan.context_rsp as *mut arch_task::Context +} + +/// Returns the current process PID for the process-aware scalar syscall ABI. +/// Bootstrap CPL3 tests have no scheduler-owned user Task and therefore do +/// not receive a fabricated PID. +pub(crate) fn current_user_pid() -> Option { + let scheduler = SCHEDULER.get()?; + let process_id = interrupts::without_interrupts(|| { + let scheduler = scheduler.lock(); + let id = scheduler.current?; + let slot = scheduler.slots.get(id.slot())?; + if slot.generation != id.generation || slot.state != TaskState::Running { + return None; + } + match slot.kind { + TaskKind::User(process_id) => Some(process_id), + TaskKind::Kernel => None, + } + })?; + process::pid(process_id).ok().map(|pid| pid.get()) +} + +/// Returns the generation-safe ProcessId for the currently running user +/// Task, if any. The scheduler lock is released before callers access the +/// process table or address-space subsystem. +pub(crate) fn current_user_process_id() -> Option { + let scheduler = SCHEDULER.get()?; + interrupts::without_interrupts(|| { + let scheduler = scheduler.lock(); + let id = scheduler.current?; + let slot = scheduler.slots.get(id.slot())?; + if slot.generation != id.generation || slot.state != TaskState::Running { + return None; + } + match slot.kind { + TaskKind::User(process_id) => Some(process_id), + TaskKind::Kernel => None, + } + }) +} + +/// Begins terminal process exit from the current user Task. The syscall gate +/// has IF=0; process and scheduler state are made safe under the documented +/// scheduler -> process lock order, then both locks are released before the +/// explicit STI;HLT scheduler handoff. This function never returns to CPL3. +pub(crate) fn exit_current_user(status: i32) -> ! { + terminate_current_user(ProcessExitReason::Exited { status }); +} + +/// Contains a hardware exception raised by a process Task. The handler passes +/// only scalar CPU state, captured before this function is entered. +pub(crate) fn terminate_current_user_fault( + vector: u8, + error_code: Option, + fault_address: Option, + rip: u64, +) -> ! { + terminate_current_user(ProcessExitReason::Fault { + vector, + error_code, + fault_address, + rip, + }); +} + +fn terminate_current_user(reason: ProcessExitReason) -> ! { + let scheduler = SCHEDULER + .get() + .expect("process exit before scheduler initialization"); + interrupts::without_interrupts(|| { + let mut scheduler = scheduler.lock(); + let id = scheduler + .current + .expect("bootstrap context attempted process exit"); + let slot = scheduler + .slots + .get_mut(id.slot()) + .expect("current process Task slot is invalid"); + assert_eq!( + slot.generation, id.generation, + "current task generation stale" + ); + assert_eq!( + slot.state, + TaskState::Running, + "current task is not Running" + ); + let TaskKind::User(process_id) = slot.kind else { + panic!("kernel Task attempted userspace exit syscall") + }; + process::begin_exit(process_id, reason, pit::ticks()) + .expect("current process could not transition to Exiting"); + slot.state = TaskState::Exiting; + }); + loop { + // SAFETY: all mutable syscall/process/scheduler work and locks have + // completed; this deliberately enables PIT so the scheduler can save + // this Exiting frame and adopt a different safe stack. + unsafe { core::arch::asm!("sti; hlt", options(nomem, nostack)) }; + } +} + +/// Resolves the selected process root and performs architecture state changes +/// after scheduler locks are gone. IF is still clear from the interrupt gate; +/// no lock spans EOI, CR3, RSP0, RSP adoption, or iretq. +fn apply_switch_plan(plan: SwitchPlan) { + let user_target = match plan.target { + TaskKind::Kernel => None, + TaskKind::User(process_id) => Some( + process::address_space_handle(process_id) + .expect("selected user task has no Alive process address space"), + ), + }; + let root_switch = + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| match user_target { + Some(address_space) => kernel.prepare_handle(address_space), + None => kernel.prepare_kernel_root_activation(), + }) + .expect("selected root failed the kernel mapping audit"); + root_switch + .activate() + .expect("verified address-space root activation failed"); + if let Some(slot) = plan.task_slot.filter(|_| user_target.is_some()) { + let (_, top) = arch_task::stack_bounds(slot).expect("user task stack slot invalid"); + gdt::set_current_rsp0(top).expect("selected user task has invalid kernel stack"); + } else { + gdt::set_kernel_fallback_rsp0().expect("kernel fallback RSP0 is unavailable"); + } } fn assert_task_stack_pointer(slot: usize, rsp: u64) { @@ -450,6 +943,494 @@ pub fn prepare_smoke_test() -> (TaskId, TaskId) { (task_a, task_b) } +#[cfg(feature = "process-smoke-test")] +pub fn prepare_process_smoke_test() { + let process_id = process::reserve(None, "process-smoke", pit::ticks()) + .expect("failed to reserve process-smoke process"); + let pid = process::snapshot(process_id) + .expect("Creating process has no metadata snapshot") + .pid + .get(); + let image = process::elf::process_smoke_elf(pid); + let (address_space, context) = + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("failed to create process-smoke AddressSpace"); + crate::arch::x86_64::memory::UserStack::map(&mut address_space, kernel) + .expect("failed to map process-smoke user stack"); + let image = process::elf::load_static_elf(&mut address_space, kernel, &image) + .expect("embedded process-smoke ELF was rejected"); + let context = address_space + .validate_user_context( + kernel, + image.entry, + crate::arch::x86_64::memory::USER_STACK_TOP, + ) + .expect("embedded process-smoke context was invalid"); + (address_space, context) + }); + process::install_address_space(process_id, address_space) + .expect("failed to attach process-smoke AddressSpace"); + let task_id = + prepare_user_task(process_id, context).expect("failed to prepare process-smoke task"); + process::install_primary_task(process_id, task_id) + .expect("failed to attach process-smoke task"); + publish_prepared_user_task(task_id).expect("failed to publish process-smoke process"); + PROCESS_SMOKE_ID.call_once(|| process_id); + PROCESS_SMOKE_START_TICK.store(pit::ticks(), Ordering::Relaxed); + PROCESS_SMOKE_ACTIVE.store(true, Ordering::Release); + kprintln!("[process-test] embedded ELF prepared for PID {pid}"); +} + +/// Prepares the small embedded ELF used on ordinary release-safe boots while +/// pvKernel has no VFS/initrd provider. Its bytes still go through the +/// normal checked static ELF loader and process publication path. +#[cfg(not(any(feature = "qemu-test", feature = "userspace-test")))] +pub fn prepare_release_userspace_smoke() { + kprintln!("[process] initializing"); + kprintln!("[process] process capacity: {}", process::MAX_PROCESSES); + kprintln!("[process] address-space isolation ready"); + kprintln!("[process] loading userspace smoke"); + let process_id = process::reserve(None, "userspace-smoke", pit::ticks()) + .expect("could not reserve release userspace smoke process"); + let pid = process::snapshot(process_id) + .expect("release smoke process lacks metadata") + .pid + .get(); + let bytes = process::elf::process_smoke_elf(pid); + let (address_space, context) = + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("could not create release smoke AddressSpace"); + crate::arch::x86_64::memory::UserStack::map(&mut address_space, kernel) + .expect("could not map release smoke user stack"); + let image = process::elf::load_static_elf(&mut address_space, kernel, &bytes) + .expect("embedded release smoke ELF rejected"); + let context = address_space + .validate_user_context( + kernel, + image.entry, + crate::arch::x86_64::memory::USER_STACK_TOP, + ) + .expect("release smoke user context invalid"); + (address_space, context) + }); + process::install_address_space(process_id, address_space) + .expect("could not attach release smoke AddressSpace"); + let task_id = + prepare_user_task(process_id, context).expect("could not prepare release smoke Task"); + process::install_primary_task(process_id, task_id) + .expect("could not attach release smoke primary Task"); + publish_prepared_user_task(task_id).expect("could not publish release smoke process"); + RELEASE_SMOKE_ID.call_once(|| process_id); +} + +#[cfg(not(any(feature = "qemu-test", feature = "userspace-test")))] +pub fn wait_for_release_userspace_smoke() { + let process_id = *RELEASE_SMOKE_ID + .get() + .expect("release smoke process was not prepared"); + loop { + if let Ok(snapshot) = process::snapshot(process_id) + && snapshot.state == process::ProcessState::Zombie + { + match snapshot.exit_reason { + Some(ProcessExitReason::Exited { status: 0 }) => { + kprintln!("[process] userspace smoke exited cleanly"); + process::reap(process_id).expect("could not reap release smoke process"); + kprintln!("[process] userspace smoke passed"); + return; + } + reason => panic!("release userspace smoke failed: {reason:?}"), + } + } + hlt(); + } +} + +#[cfg(feature = "user-fault-containment-test")] +pub fn prepare_user_fault_containment_test() { + let stack_guard = crate::arch::x86_64::memory::USER_STACK_TOP + - crate::arch::x86_64::memory::USER_STACK_SIZE + - 4096; + let scenarios = [ + (14u8, Some(0x3000_0000), "fault-pf-unmapped"), + (14u8, Some(0xffff_8000_0000_0000), "fault-kernel-isolation"), + (14u8, Some(stack_guard), "fault-user-stack-guard"), + (13u8, None, "fault-gp"), + (6u8, None, "fault-simd-disabled"), + (7u8, None, "fault-x87-disabled"), + (8u8, None, "fault-avx-disabled"), + ]; + let mut ids = [None; 7]; + for (index, (vector, fault_address, name)) in scenarios.into_iter().enumerate() { + let process_id = process::reserve(None, name, pit::ticks()) + .expect("failed to reserve fault-containment process"); + let image = match fault_address { + Some(address) => process::elf::process_fault_elf_at(vector, address), + None => process::elf::process_fault_elf(vector), + }; + let (address_space, context) = + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("failed to create fault-containment AddressSpace"); + crate::arch::x86_64::memory::UserStack::map(&mut address_space, kernel) + .expect("failed to map fault-containment user stack"); + let image = process::elf::load_static_elf(&mut address_space, kernel, &image) + .expect("fault-containment ELF fixture rejected"); + let context = address_space + .validate_user_context( + kernel, + image.entry, + crate::arch::x86_64::memory::USER_STACK_TOP, + ) + .expect("fault-containment context invalid"); + (address_space, context) + }); + process::install_address_space(process_id, address_space) + .expect("could not attach fault-containment address space"); + let task_id = prepare_user_task(process_id, context) + .expect("could not prepare fault-containment task"); + process::install_primary_task(process_id, task_id) + .expect("could not attach fault-containment task"); + publish_prepared_user_task(task_id).expect("could not publish fault-containment process"); + ids[index] = Some(process_id); + kprintln!("[process-test] prepared CPL3 fault fixture vector {vector}"); + } + USER_FAULT_IDS.call_once(|| ids.map(|id| id.expect("fault fixture ID missing"))); +} + +#[cfg(feature = "user-fault-containment-test")] +pub fn wait_for_user_fault_containment_test() { + let ids = USER_FAULT_IDS + .get() + .expect("fault-containment process IDs were not initialized"); + let start_tick = pit::ticks(); + loop { + let mut zombies = 0; + let expected = [ + (14u8, Some(0x3000_0000)), + (14u8, Some(0xffff_8000_0000_0000)), + ( + 14u8, + Some( + crate::arch::x86_64::memory::USER_STACK_TOP + - crate::arch::x86_64::memory::USER_STACK_SIZE + - 4096, + ), + ), + (13u8, None), + (6u8, None), + (7u8, None), + (6u8, None), + ]; + for (process_id, (expected_vector, expected_fault_address)) in ids.iter().zip(expected) { + let snapshot = process::snapshot(*process_id) + .expect("fault-containment process identity became stale before reap"); + if snapshot.state == process::ProcessState::Zombie { + let reason = snapshot + .exit_reason + .expect("faulting process Zombie lost fault metadata"); + match reason { + ProcessExitReason::Fault { + vector, + error_code, + fault_address, + rip, + } => { + assert_eq!(vector, expected_vector); + assert!(rip >= 0x0040_0000 && rip < 0x0040_1000); + match expected_vector { + 14 => { + assert_ne!( + error_code.expect("#PF exit metadata lost error code") & 4, + 0, + "user-origin #PF lacks the USER_MODE bit" + ); + assert_eq!(fault_address, expected_fault_address); + } + 13 => { + assert!(error_code.is_some()); + assert_eq!(fault_address, None); + } + 6 => { + assert_eq!(error_code, None); + assert_eq!(fault_address, None); + } + 7 => { + assert_eq!(error_code, None); + assert_eq!(fault_address, None); + } + _ => unreachable!(), + } + zombies += 1; + } + other => panic!("fault fixture exited normally: {other:?}"), + } + } + } + if zombies == ids.len() { + for process_id in ids { + process::reap(*process_id).expect("could not reap faulting Zombie"); + assert!(process::snapshot(*process_id).is_err()); + } + kprintln!( + "[process-test] #PF/#GP/#UD/#NM containment, guards, isolation and SIMD/AVX denial passed" + ); + crate::arch::x86_64::qemu::exit_success(); + } + if pit::ticks().saturating_sub(start_tick) >= 100 { + kprintln!("[process-test] user-fault tick watchdog expired"); + crate::arch::x86_64::qemu::exit_failure(); + } + hlt(); + } +} + +#[cfg(feature = "address-space-isolation-test")] +pub fn prepare_address_space_isolation_test() { + use x86_64::structures::paging::{Page, Size4KiB}; + + const PRIVATE_VA: u64 = 0x0010_0000; + let mut prepared = [None; 2]; + let mut frames = [None; 2]; + for (index, value) in [0xAAu64, 0xBB].into_iter().enumerate() { + let process_id = process::reserve( + None, + if index == 0 { + "isolation-a" + } else { + "isolation-b" + }, + pit::ticks(), + ) + .expect("failed to reserve isolation process"); + let image = process::elf::process_isolation_elf(value); + let (address_space, context, frame) = + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("failed to create isolation address space"); + crate::arch::x86_64::memory::UserStack::map(&mut address_space, kernel) + .expect("failed to map isolation user stack"); + let image = process::elf::load_static_elf(&mut address_space, kernel, &image) + .expect("isolation ELF fixture rejected"); + let page = Page::::from_start_address(x86_64::VirtAddr::new(PRIVATE_VA)) + .expect("private isolation VA is not page aligned"); + let frame = address_space + .map_new_user_page( + kernel, + page, + crate::arch::x86_64::memory::Permissions::USER_DATA, + ) + .expect("failed to map private isolation page"); + address_space + .write_user_page(kernel, page, 0, &value.to_le_bytes()) + .expect("failed to initialize private isolation data"); + let context = address_space + .validate_user_context( + kernel, + image.entry, + crate::arch::x86_64::memory::USER_STACK_TOP, + ) + .expect("isolation context invalid"); + (address_space, context, frame) + }); + process::install_address_space(process_id, address_space) + .expect("could not attach isolation address space"); + let task_id = + prepare_user_task(process_id, context).expect("could not prepare isolation user Task"); + process::install_primary_task(process_id, task_id) + .expect("could not attach isolation user Task"); + publish_prepared_user_task(task_id).expect("could not publish isolation process"); + prepared[index] = Some(process_id); + frames[index] = Some(frame); + } + assert_ne!(frames[0], frames[1], "same user VA aliases physical frame"); + ADDRESS_ISOLATION_IDS + .call_once(|| prepared.map(|id| id.expect("isolation process ID missing"))); + kprintln!("[process-test] same user VA mapped to private A/B frames"); +} + +/// Fills the scheduler with two already-Ready kernel smoke Tasks and six +/// prepared (non-runnable) process Tasks. The seventh process can reserve a +/// ProcessId and AddressSpace but must receive a typed scheduler-capacity +/// error without publishing a partial process. It then verifies the separate +/// eight-slot process-table bound and rolls back all Creating/prepared state. +#[cfg(feature = "process-capacity-test")] +pub fn run_process_capacity_test() { + use x86_64::structures::paging::{Page, PageSize, Size4KiB}; + + let mut processes = [None; 6]; + let mut tasks = [None; 6]; + for index in 0..6 { + let process_id = process::reserve(None, "capacity-test", pit::ticks()) + .expect("process capacity reached before expected slot"); + let (address_space, context) = + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("could not build capacity-test root"); + let stack = crate::arch::x86_64::memory::UserStack::map(&mut address_space, kernel) + .expect("could not map capacity-test stack"); + let entry = 0x0040_0000 + index as u64 * Size4KiB::SIZE; + address_space + .map_new_user_page( + kernel, + Page::::from_start_address(x86_64::VirtAddr::new(entry)).unwrap(), + crate::arch::x86_64::memory::Permissions::USER_EXECUTABLE, + ) + .expect("could not map capacity-test code page"); + let context = address_space + .validate_user_context(kernel, entry, stack.top) + .expect("capacity-test context invalid"); + (address_space, context) + }); + process::install_address_space(process_id, address_space) + .expect("could not attach capacity-test root"); + let task_id = prepare_user_task(process_id, context) + .expect("scheduler filled before six process Tasks"); + process::install_primary_task(process_id, task_id) + .expect("could not attach capacity-test Task"); + processes[index] = Some(process_id); + tasks[index] = Some(task_id); + } + + let seventh = process::reserve(None, "capacity-seventh", pit::ticks()) + .expect("seventh process identity should fit its independent table"); + let (address_space, context) = + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("could not build seventh process root"); + let stack = crate::arch::x86_64::memory::UserStack::map(&mut address_space, kernel) + .expect("could not map seventh process stack"); + let entry = 0x0040_0000; + address_space + .map_new_user_page( + kernel, + Page::::from_start_address(x86_64::VirtAddr::new(entry)).unwrap(), + crate::arch::x86_64::memory::Permissions::USER_EXECUTABLE, + ) + .expect("could not map seventh process code page"); + let context = address_space + .validate_user_context(kernel, entry, stack.top) + .expect("seventh process context invalid"); + (address_space, context) + }); + process::install_address_space(seventh, address_space) + .expect("could not attach seventh process root"); + assert_eq!( + prepare_user_task(seventh, context), + Err(SpawnError::TaskTableFull) + ); + process::abort_creation(seventh).expect("could not roll back seventh process"); + + let eighth = process::reserve(None, "capacity-eighth", pit::ticks()) + .expect("process table should accept slot eight after task rollback"); + let ninth = process::reserve(None, "capacity-ninth", pit::ticks()) + .expect("process table should accept its eighth live process identity"); + assert_eq!( + process::reserve(None, "capacity-overflow", pit::ticks()), + Err(process::ProcessError::TableFull) + ); + + for (process_id, task_id) in processes.into_iter().zip(tasks) { + cancel_prepared_task(task_id.expect("capacity test lost prepared task")) + .expect("could not cancel prepared capacity-test task"); + process::abort_creation(process_id.expect("capacity test lost process ID")) + .expect("could not abort capacity-test process"); + } + process::abort_creation(eighth).expect("could not abort eighth capacity-test process"); + process::abort_creation(ninth).expect("could not abort final capacity-test process"); + kprintln!("[process-test] mixed task/process capacity and rollback passed"); +} + +#[cfg(feature = "address-space-isolation-test")] +pub fn wait_for_address_space_isolation_test() { + let ids = ADDRESS_ISOLATION_IDS + .get() + .expect("isolation process IDs were not initialized"); + let start_tick = pit::ticks(); + loop { + let mut zombies = 0; + for process_id in ids { + let snapshot = process::snapshot(*process_id) + .expect("isolation process identity became stale before reap"); + if snapshot.state == process::ProcessState::Zombie { + assert_eq!( + snapshot.exit_reason, + Some(ProcessExitReason::Exited { status: 0 }), + "user process observed another address space's value" + ); + assert!(snapshot.cpu_ticks > 0); + zombies += 1; + } + } + if zombies == ids.len() { + for process_id in ids { + process::reap(*process_id).expect("could not reap isolation process"); + } + kprintln!("[process-test] same-VA process isolation across timer switches passed"); + crate::arch::x86_64::qemu::exit_success(); + } + if pit::ticks().saturating_sub(start_tick) >= 100 { + kprintln!("[process-test] address-space isolation watchdog expired"); + crate::arch::x86_64::qemu::exit_failure(); + } + hlt(); + } +} + +#[cfg(feature = "process-smoke-test")] +pub fn wait_for_process_smoke_test() { + let process_id = *PROCESS_SMOKE_ID + .get() + .expect("process-smoke identity was not initialized"); + while PROCESS_SMOKE_ACTIVE.load(Ordering::Acquire) { + if let Ok(snapshot) = process::snapshot(process_id) + && snapshot.state == process::ProcessState::Zombie + { + assert!( + snapshot.cpu_ticks > 0, + "process CPU accounting did not advance" + ); + assert!(process::snapshot_all().iter().flatten().any(|entry| { + entry.id == process_id && entry.state == process::ProcessState::Zombie + })); + match snapshot.exit_reason { + Some(ProcessExitReason::Exited { status: 0 }) => { + kprintln!("[process-test] userspace getpid passed"); + kprintln!("[process-test] exit handoff reached Zombie safely"); + let old_pid = snapshot.pid; + process::reap(process_id).expect("could not reap process-smoke Zombie"); + assert!(process::snapshot(process_id).is_err()); + let replacement = process::reserve(None, "reuse-check", pit::ticks()) + .expect("reaped process slot could not be reused"); + let replacement_snapshot = process::snapshot(replacement) + .expect("replacement process snapshot missing"); + assert_eq!(replacement.slot(), process_id.slot()); + assert!(replacement.generation() > process_id.generation()); + assert_ne!(replacement_snapshot.pid, old_pid); + process::abort_creation(replacement) + .expect("could not roll back process ID reuse fixture"); + kprintln!("[process-test] stale ProcessId rejected after reuse"); + PROCESS_SMOKE_ACTIVE.store(false, Ordering::Release); + crate::arch::x86_64::qemu::exit_success(); + } + reason => { + kprintln!("[process-test] process-smoke exit failed: {reason:?}"); + crate::arch::x86_64::qemu::exit_failure(); + } + } + } + hlt(); + } +} + pub fn run_smoke_test(task_a: TaskId, task_b: TaskId) { let start = pit::ticks(); while SMOKE_A.load(Ordering::Relaxed) == 0 || SMOKE_B.load(Ordering::Relaxed) == 0 { @@ -554,6 +1535,151 @@ pub fn run_preemption_test() { kprintln!("[scheduler-test] preemption completed successfully"); } +#[cfg(feature = "process-preemption-test")] +fn prepare_preemption_process( + name: &str, + shared_frame: Option< + x86_64::structures::paging::PhysFrame, + >, + process_b: bool, +) -> ( + ProcessId, + u64, + x86_64::structures::paging::PhysFrame, +) { + use x86_64::structures::paging::{Page, PageSize, Size4KiB}; + + let process_id = process::reserve(None, name, pit::ticks()) + .expect("failed to reserve process-preemption test process"); + let (address_space, context, progress_alias, progress_frame) = + crate::arch::x86_64::memory::with_kernel_address_space(|kernel| { + let mut address_space = kernel + .create_address_space() + .expect("failed to create process-preemption test root"); + crate::arch::x86_64::memory::UserStack::map(&mut address_space, kernel) + .expect("failed to map process-preemption user stack"); + + let code_page = Page::::from_start_address(x86_64::VirtAddr::new( + crate::arch::x86_64::memory::USER_MIN, + )) + .expect("USER_MIN is not page aligned"); + let progress_address = crate::arch::x86_64::memory::USER_MIN + Size4KiB::SIZE; + let progress_page = + Page::::from_start_address(x86_64::VirtAddr::new(progress_address)) + .expect("progress page is not aligned"); + address_space + .map_new_user_page( + kernel, + code_page, + crate::arch::x86_64::memory::Permissions::USER_DATA, + ) + .expect("failed to allocate user loop code page"); + let progress_frame = if let Some(frame) = shared_frame { + address_space + .map_test_shared_user_page(kernel, progress_page, frame) + .expect("failed to map shared preemption-test progress page"); + frame + } else { + address_space + .map_new_user_page( + kernel, + progress_page, + crate::arch::x86_64::memory::Permissions::USER_DATA, + ) + .expect("failed to allocate user progress page") + }; + + // Trusted closed loop: increment this process's counter, test the + // peer's counter, set an observed-peer flag, and repeat. Neither + // payload invokes a syscall or yields voluntarily. + let mut code = [0u8; 30]; + code[0..2].copy_from_slice(&[0x48, 0xB8]); + let own_counter = progress_address + if process_b { 8 } else { 0 }; + code[2..10].copy_from_slice(&own_counter.to_le_bytes()); + code[10..13].copy_from_slice(&[0x48, 0xFF, 0x00]); + let peer_offset = if process_b { 0xF8 } else { 0x08 }; + code[13..20].copy_from_slice(&[0x48, 0x83, 0x78, peer_offset, 0x01, 0x72, 0xF6]); + code[20..28].copy_from_slice(&[0x48, 0xC7, 0x40, 0x10, 0x01, 0, 0, 0]); + code[28..30].copy_from_slice(&[0xEB, 0xEC]); + address_space + .write_user_page(kernel, code_page, 0, &code) + .expect("failed to write process-preemption test payload"); + address_space + .update_user_permissions( + kernel, + code_page, + crate::arch::x86_64::memory::Permissions::USER_CODE, + ) + .expect("failed to make test code RX"); + let progress_alias = address_space + .test_user_page_kernel_alias(kernel, progress_page) + .expect("failed to obtain progress page supervisor alias") + .as_u64(); + let context = address_space + .validate_user_context( + kernel, + crate::arch::x86_64::memory::USER_MIN, + crate::arch::x86_64::memory::USER_STACK_TOP, + ) + .expect("process-preemption user context failed validation"); + (address_space, context, progress_alias, progress_frame) + }); + process::install_address_space(process_id, address_space) + .expect("failed to attach process-preemption address space"); + let task_id = prepare_user_task(process_id, context) + .expect("failed to prepare process-preemption user task"); + process::install_primary_task(process_id, task_id) + .expect("failed to attach process-preemption task"); + publish_prepared_user_task(task_id).expect("failed to publish process-preemption task"); + (process_id, progress_alias, progress_frame) +} + +/// Adds two non-yielding CPL3 loops to the ready queue. The PIT handler +/// reports QEMU success only after both independent user pages advanced. +#[cfg(feature = "process-preemption-test")] +pub fn prepare_process_preemption_test() { + PROCESS_PREEMPTION_ACTIVE.store(false, Ordering::Relaxed); + let (_, alias_a, progress_frame) = prepare_preemption_process("preempt-a", None, false); + let (_, alias_b, _) = prepare_preemption_process("preempt-b", Some(progress_frame), true); + PROCESS_PREEMPTION_PROGRESS_A.store(alias_a, Ordering::Relaxed); + PROCESS_PREEMPTION_PROGRESS_B.store(alias_b, Ordering::Relaxed); + PROCESS_PREEMPTION_START_TICK.store(pit::ticks(), Ordering::Relaxed); + PROCESS_PREEMPTION_ACTIVE.store(true, Ordering::Release); + kprintln!("[process-test] two non-yielding user loops queued"); +} + +#[cfg(feature = "process-preemption-test")] +pub fn wait_for_process_preemption_test() { + while PROCESS_PREEMPTION_ACTIVE.load(Ordering::Acquire) { + hlt(); + } +} + +#[cfg(feature = "process-preemption-test")] +fn process_preemption_test_result() -> Option { + if !PROCESS_PREEMPTION_ACTIVE.load(Ordering::Acquire) { + return None; + } + let alias_a = PROCESS_PREEMPTION_PROGRESS_A.load(Ordering::Relaxed); + let alias_b = PROCESS_PREEMPTION_PROGRESS_B.load(Ordering::Relaxed); + if alias_a == 0 || alias_b == 0 { + return None; + } + // Both processes map this zeroed test page at the same user VA. Their + // closed loops set distinct flags only after reading the peer's counter. + // SAFETY: aliases refer to the allocator-owned page captured during + // Creating; IF is clear and only the trusted test loops write these + // aligned qwords. Each offset remains inside the page. + let process_a_saw_b = unsafe { ((alias_a + 16) as *const u64).read_volatile() }; + // SAFETY: alias_b denotes the same mapped page and +24 is still in-page. + let process_b_saw_a = unsafe { ((alias_b + 24) as *const u64).read_volatile() }; + if process_a_saw_b != 0 && process_b_saw_a != 0 { + return Some(true); + } + (pit::ticks().saturating_sub(PROCESS_PREEMPTION_START_TICK.load(Ordering::Relaxed)) >= 100) + .then_some(false) +} + #[cfg(feature = "task-capacity-test")] fn capacity_task() { CAPACITY_DONE.fetch_add(1, Ordering::Relaxed);