Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ build/
# Scratch idea notes
ideas/

# Windows toolchain paths (scripts/ensure-toolchain.ps1 output; machine-specific)
scripts/.toolchain.json

# macOS
.DS_Store

Expand Down
48 changes: 48 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,46 @@ make toolchain # once: nasm, x86_64-elf-gcc, qemu
make run
```

### On Windows

There is no x86_64-elf toolchain here, so the same steps run through four
scripts that build with Zig's bundled clang instead. The Windows port follows
the approach proven by [NO_OS](https://github.com/coff33ninja/NO_OS) — a
from-scratch x86-64 kernel of a similar nature — whose
[ELF-reframing ADR](https://github.com/coff33ninja/NO_OS/blob/main/docs/ADR/0003-elf-reframe-qemu.md)
is the one trick this build was missing:

```powershell
.\scripts\ensure-toolchain.ps1 # once: zig, nasm, qemu, python, MSYS2 binutils
.\scripts\build.ps1 # 211 objects -> kernel.bin
.\scripts\run.ps1 # headless: .\scripts\run.ps1 -NoGraphics
.\scripts\test-qemu.ps1 # the 6 QEMU boot-assertion cases, plus the formatter lint
```

Build flags and API-key handling mirror the Makefile exactly (same CFLAGS minus
the GCC-only ones, same `.env` two-name rule, same `opt/fableos/apikey` fw_cfg
channel). Windows-only wrinkles, all contained in the scripts:

- `-fno-sanitize=undefined` is added, or zig's compiler_rt drags in the
`__ubsan_handle_*` helpers and the link fails.
- QEMU's multiboot loader accepts ELF32 only, so after linking the kernel is
re-labeled as an ELF32 *container* with `objcopy -O elf32-i386` — the exact
command the Makefile runs. On Windows that objcopy comes from
`mingw-w64-x86_64-binutils` (installed by `ensure-toolchain.ps1` through
MSYS2's pacman), because zig's bundled llvm-objcopy only emits `binary`.
`run.ps1` prints a warning that "multiboot knows VBE. we don't" — that one
is a benign stderr line from QEMU, not the kernel.
- Objects are linked in the Makefile's source order, not directory order: the
self-registering driver table initializes drivers in link order, and the
boot-assertion cases pin that order.
- `-Wl,-e,start` restores the kernel entry point the clang driver would
otherwise zero out.
- The QEMU suite has no `make`, so `test-qemu.ps1` points the harness's
per-case build hook (`FABLEOS_TEST_MAKE`) at `build.ps1` — the one
`build-cflags:` case (the AC'97 reference bring-up) builds and restores the
tree through it — and the interactive REPL case speaks to the VM over a
loopback TCP serial port instead of the POSIX AF_UNIX socket.

That boots with no key: the kernel still completes a real TLS handshake to
`api.anthropic.com` and gets an honest `401` back, which is itself proof the HTTPS
path works. It will tell you it has no key.
Expand Down Expand Up @@ -203,3 +243,11 @@ build, and design notes per subsystem. `AGENTS.md` has the architectural
constraints anyone — or anything — working on this code needs to know first.

Built and run on macOS via QEMU.

---

*This tree is a fork of [robiot/fable-os](https://github.com/robiot/fable-os),
kept on my profile as [coff33ninja/fable-os](https://github.com/coff33ninja/fable-os)
to carry the Windows build. My own from-scratch OS of a similar nature — an
x86-64 kernel whose only interface is a bytecode VM — is
[NO_OS](https://github.com/coff33ninja/NO_OS).*
314 changes: 314 additions & 0 deletions scripts/build.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,314 @@
<#
.SYNOPSIS
Build the fable-os kernel on Windows with zig cc (clang), NASM and QEMU.

.DESCRIPTION
Windows port of `make` for this tree. The macOS Makefile uses a brew
x86_64-elf-gcc toolchain; here the kernel is compiled with Zig's bundled
clang (zig cc) targeting x86_64-freestanding, assembled with NASM, linked
with lld, and converted to an ELF32 container for QEMU's multiboot loader
with GNU objcopy from MSYS2 (`-O elf32-i386`), exactly as the Makefile's
`$(OBJCOPY) -O elf32-i386` does. zig's bundled llvm-objcopy cannot emit ELF.

The object list below MUST stay in lock-step with the Makefile's
KERNEL_SRCS (the comment marks it as such). The script refuses to run if a
listed source is missing, so a rename in one place fails loudly here too.

Traps encoded here, each paid for in an actual Windows build:
* PowerShell 5.1 splatting unwraps a single-element or scalar array, so
`@flags @w` where $w = '-w' silently passes a bare '-' and zig cc fails
with "unrecognized file extension". Every compile passes ONE array and
splats it exactly once.
* zig cc enables UBSan by default in freestanding builds; the kernel must
not call __ubsan_handle_*, so -fno-sanitize=undefined is mandatory or
the link dies on unresolved symbols.
* Linking with -nostdlib drops zig's compiler_rt, leaving __udivti3 /
__divti3 (mbedTLS bignum, expr.c, gui_demo.c 128-bit division)
undefined. Link WITHOUT -nostdlib; a freestanding target pulls no libc.
* The clang driver overrides the linker script's ENTRY(start) with its
own `-e _start`, producing e_entry=0 and a QEMU that jumps to address
zero. -Wl,-e,start restores the real entry.
* -fno-tree-loop-distribute-patterns (a GCC flag in the Makefile) is
rejected by zig's clang; it is unnecessary here because -fno-builtin
already prevents the memset/memcpy loops it targets.
* lwIP and mbedTLS are vendored and get -w, exactly as the Makefile does
(lwip/%.o mbedtls/%.o: CFLAGS += -w).

.PARAMETER Clean
Wipe the object cache and rebuild everything.

.PARAMETER ExtCFlags
Extra compiler flags, e.g. -ExtCFlags '-DFABLEOS_DIAGNOSTIC'. Mirrors the
Makefile's EXTRA_CFLAGS. Changing it discards cached objects (stamped in
build\.build-flags, same guard as the Makefile's .build-flags).

.PARAMETER SkipInstall
Do not offer to install missing tools via winget; fail instead.

.EXAMPLE
.\scripts\build.ps1
.EXAMPLE
.\scripts\build.ps1 -ExtCFlags '-DFABLEOS_FAULT_TEST=FAULT_INJECT_PF_WRITE'
#>
[CmdletBinding()]
param(
[switch]$Clean,
[string]$ExtCFlags = '',
[switch]$SkipInstall
)

$ErrorActionPreference = 'Stop'
$repo = Split-Path -Parent $PSScriptRoot
Set-Location $repo

# ---------------------------------------------------------------------------
# Tools
# ---------------------------------------------------------------------------
$toolchainJson = Join-Path $PSScriptRoot '.toolchain.json'
function Get-Tools {
if (-not (Test-Path -LiteralPath $toolchainJson)) {
if ($SkipInstall) { throw '.toolchain.json missing; run scripts\ensure-toolchain.ps1' }
& (Join-Path $PSScriptRoot 'ensure-toolchain.ps1') -Install | Out-Null
}
$t = Get-Content -LiteralPath $toolchainJson -Raw | ConvertFrom-Json
foreach ($name in 'zig', 'nasm', 'objcopy') {
if (-not $t.$name -or -not (Test-Path -LiteralPath $t.$name)) {
if ($SkipInstall) { throw "toolchain entry '$name' is stale; run scripts\ensure-toolchain.ps1" }
Remove-Item -LiteralPath $toolchainJson -Force
& (Join-Path $PSScriptRoot 'ensure-toolchain.ps1') -Install | Out-Null
$t = Get-Content -LiteralPath $toolchainJson -Raw | ConvertFrom-Json
}
}
$t
}
$T = Get-Tools

# ---------------------------------------------------------------------------
# Build configuration
# ---------------------------------------------------------------------------
$objDir = Join-Path $repo 'build\obj'
$stamp = Join-Path $repo 'build\.build-flags'
$kernelElf = Join-Path $repo 'kernel.elf'
$kernelBin = Join-Path $repo 'kernel.bin'
New-Item -ItemType Directory -Force -Path $objDir | Out-Null

# EXTRA_CFLAGS guard: a build that silently disagrees with its own flags is
# worse than a failing one (the Makefile's whole .build-flags machinery says
# so). Store the flag string; any change wipes the object cache.
$want = "EXTRA_CFLAGS=$ExtCFlags"
$have = ''
if (Test-Path -LiteralPath $stamp) { $have = (Get-Content -LiteralPath $stamp -Raw).Trim() }
if ($Clean -or ($have -ne $want)) {
if ($have -ne $want -and $have) { Write-Host "EXTRA_CFLAGS changed: rebuilding everything" }
Remove-Item -Recurse -Force (Join-Path $repo 'build\obj'), $kernelElf, $kernelBin -ErrorAction SilentlyContinue
New-Item -ItemType Directory -Force -Path $objDir | Out-Null
Set-Content -LiteralPath $stamp -Value $want -NoNewline
}

# ---------------------------------------------------------------------------
# Source tables -- mirror of the Makefile's KERNEL_SRCS, LWIP_SRCS, MBEDTLS_SRCS
# ---------------------------------------------------------------------------
$kernelSources = @(
'kernel/main.c', 'kernel/drivers.c',
'arch/x86_64/idt.c', 'arch/x86_64/fault.c',
'core/kobject.c', 'core/tool.c', 'core/audio.c', 'core/fiber.c',
'core/capability.c', 'core/agenda.c',
'mm/heap.c', 'device/device.c',
'fs/vfs/vfs.c', 'fs/native/ramfs.c', 'fs/fs.c',
'fs/fat/fat_vol.c', 'fs/fat/fat_dir.c', 'fs/fat/fat.c',
'drivers/block/block.c', 'drivers/block/ata.c',
'lib/base.c', 'lib/libc_shim.c', 'lib/kfmt.c', 'lib/trace.c',
'lib/fb.c', 'lib/font.c', 'lib/font_spleen8x16.c',
'gui/wm.c', 'gui/widgets.c', 'gui/gui_demo.c',
'apps/runtime.c', 'apps/expr.c', 'apps/cap.c', 'apps/app_selftest.c',
'apps/app_format_selftest.c', 'apps/app_audio_selftest.c',
'vm/dvm.c',
'compiler/cc.c', 'compiler/cc_lex.c', 'compiler/cc_parse.c',
'compiler/cc_x64.c', 'compiler/cc_sym.c', 'compiler/cc_store.c',
'drivers/serial/serial.c', 'drivers/fwcfg/fwcfg.c', 'drivers/pci/pci.c',
'drivers/acpi/acpi.c', 'drivers/acpi/power.c',
'drivers/acpi/power_selftest.c',
'drivers/input/input.c', 'drivers/input/kbd.c',
'drivers/input/serial_input.c', 'drivers/input/script.c',
'drivers/input/mouse.c',
'drivers/net/e1000.c', 'drivers/rtc/rtc.c',
'net/json.c', 'net/model.c', 'net/chat.c', 'net/sse.c', 'net/net.c',
'net/fetch.c', 'net/faultchat.c', 'net/tls_ca.c'
)
# The Makefile links the AC'97 reference fixture only into the bring-up build
# that tests/qemu/cases/ac97.case asks for with `build-cflags:`:
# ifneq ($(findstring FABLEOS_AC97_REFERENCE,$(EXTRA_CFLAGS)),)
# KERNEL_SRCS += tests/qemu/fixtures/ac97_boot.c
# The fixture #errors without the define, so this condition must stay in
# lock-step with the Makefile's or that build cannot exist here at all.
if ($ExtCFlags -like '*FABLEOS_AC97_REFERENCE*') {
$kernelSources += 'tests/qemu/fixtures/ac97_boot.c'
}
# tools/ is wildcarded on purpose: a tool self-registers through a linker
# section (REGISTER_TOOL), so this must never be a fixed list.
$toolSources = @(Get-ChildItem -LiteralPath (Join-Path $repo 'tools') -Filter '*.c' |
ForEach-Object { "tools/$($_.Name)" } | Sort-Object)

$lwipSources = @(
@(Get-ChildItem -LiteralPath (Join-Path $repo 'lwip\src\core') -Filter '*.c' |
ForEach-Object { $_.FullName.Substring($repo.Length + 1) })
@(Get-ChildItem -LiteralPath (Join-Path $repo 'lwip\src\core\ipv4') -Filter '*.c' |
ForEach-Object { $_.FullName.Substring($repo.Length + 1) })
'lwip/src/netif/ethernet.c'
'lwip/src/apps/altcp_tls/altcp_tls_mbedtls.c'
'lwip/src/apps/altcp_tls/altcp_tls_mbedtls_mem.c'
)
$mbedtlsSources = @(Get-ChildItem -LiteralPath (Join-Path $repo 'mbedtls\library') -Filter '*.c' |
ForEach-Object { $_.FullName.Substring($repo.Length + 1) })

$allSources = @($kernelSources + $toolSources + $lwipSources + $mbedtlsSources)

# Drift check: every listed source must exist, and every tools/*.c must be in
# the list. A source that vanished (renamed, deleted) would otherwise link
# silently into an image missing a driver or a tool.
$missing = @($allSources | Where-Object { -not (Test-Path -LiteralPath (Join-Path $repo $_)) })
if ($missing.Count -gt 0) {
throw ("source table drift: these files do not exist:`n" +
($missing | ForEach-Object { " $_" }) +
"`nSync scripts/build.ps1 with the Makefile's source lists.")
}

# ---------------------------------------------------------------------------
# Compile
# ---------------------------------------------------------------------------
$includeDirs = @('-Iinclude', '-Iport', '-Ilwip/src/include', '-Imbedtls/include')
# The Makefile passes -DMBEDTLS_CONFIG_FILE='"mbedtls_config.h"'. GCC strips the
# single quotes when the macro lands in `#include MBEDTLS_CONFIG_FILE`; clang
# (zig) keeps them and errors with 'expected "FILENAME"'. The backslash-quote
# form makes clang expand it to a proper string literal instead.
$mbedtlsDefine = '-DMBEDTLS_CONFIG_FILE=\"mbedtls_config.h\"'
$commonFlags = @(
'-target', 'x86_64-freestanding',
'-ffreestanding', '-m64', '-mno-red-zone', '-mno-mmx', '-mno-sse', '-mno-sse2',
'-fno-stack-protector', '-fno-pic', '-fno-builtin',
'-std=gnu11', '-fno-sanitize=undefined',
$mbedtlsDefine, '-MMD', '-MP', '-c'
)
if ($ExtCFlags) { $commonFlags += $ExtCFlags }

# Run a native command, treating a non-zero exit as a fatal build error.
# Windows PowerShell 5.1 gotchas encoded here:
# * With $ErrorActionPreference='Stop', ANY native stderr redirection
# (`2>&1` or `2>file`) routes the process's stderr through PowerShell's
# error stream, and the first compiler warning -- which legitimately goes
# to stderr -- becomes a TERMINATING error. So the native call runs with
# EAP suspended; cmdlet errors elsewhere still stop the script.
# * A single-element/scalar array splat is unwrapped, which turned one '-w'
# into a bare '-' (the caller passes full arrays built once, and splats
# exactly once).
function Invoke-Native {
param([string]$File, [string[]]$ArgList, [string]$What)
$errTmp = Join-Path $env:TEMP ([IO.Path]::GetRandomFileName() + '.err')
$oldEAP = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
try {
& $File @ArgList 1>$null 2>$errTmp
$code = $LASTEXITCODE
}
finally {
$ErrorActionPreference = $oldEAP
}
$err = Get-Content -LiteralPath $errTmp -ErrorAction SilentlyContinue
Remove-Item -LiteralPath $errTmp -Force -ErrorAction SilentlyContinue
if ($code -ne 0) {
Write-Host "FAIL: $What"
$err | Select-Object -First 8
exit 1
}
if ($err) { $err }
}

function Invoke-Compile {
param([string]$Source, [string]$Obj, [string[]]$ExtraFlags)
$argv = @('cc')
$argv += $commonFlags
$argv += $ExtraFlags
$argv += $includeDirs
$argv += '-o', $Obj, $Source
# ONE splat. PowerShell 5.1 unwraps a scalar/single-element array on splat,
# which silently turned a single '-w' into a bare '-' (see header comment).
Invoke-Native -File $T.zig -ArgList $argv -What $Source
}

# Rebuild an object when its source or any header its .d file names is newer.
# This is the Windows counterpart of the Makefile's `-include $(OBJS:.o=.d)`.
function Test-Stale {
param([string]$Source, [string]$Obj)
if (-not (Test-Path -LiteralPath $Obj)) { return $true }
$newest = (Get-Item -LiteralPath $Source).LastWriteTimeUtc
$depFile = [IO.Path]::ChangeExtension($Obj, '.d')
if (Test-Path -LiteralPath $depFile) {
$raw = (Get-Content -LiteralPath $depFile -Raw) -replace '\\\r?\n', ' '
$parts = $raw -split ':' , 2
foreach ($dep in ($parts[1] -split '\s+' | Where-Object { $_ -and -not $_.EndsWith(':') })) {
if (Test-Path -LiteralPath $dep) {
$t = (Get-Item -LiteralPath $dep).LastWriteTimeUtc
if ($t -gt $newest) { $newest = $t }
}
}
}
return $newest -gt (Get-Item -LiteralPath $Obj).LastWriteTimeUtc
}

$objOf = { param($s) Join-Path $objDir (($s -replace '[\\/]', '_') + '.o') }

# Assembler objects (NASM, elf64), the same three as the Makefile.
$asmPairs = @(
@{ src = 'boot/boot.asm'; obj = 'boot_boot.asm.o' },
@{ src = 'arch/x86_64/isr.asm'; obj = 'arch_x86_64_isr.asm.o' },
@{ src = 'arch/x86_64/switch.asm'; obj = 'arch_x86_64_switch.asm.o' }
)
foreach ($a in $asmPairs) {
$dst = Join-Path $objDir $a.obj
if (-not (Test-Path -LiteralPath $dst) -or
(Get-Item -LiteralPath (Join-Path $repo $a.src)).LastWriteTimeUtc -gt (Get-Item -LiteralPath $dst).LastWriteTimeUtc) {
Write-Host "nasm $($a.src)"
Invoke-Native -File $T.nasm -ArgList @('-f', 'elf64', '-o', $dst, (Join-Path $repo $a.src)) -What $a.src
}
}

$n = 0
foreach ($src in $allSources) {
$n++
$obj = & $objOf $src
$vendored = $src -like 'lwip\*' -or $src -like 'lwip/*' -or
$src -like 'mbedtls\*' -or $src -like 'mbedtls/*'
if (-not (Test-Stale -Source (Join-Path $repo $src) -Obj $obj)) { continue }
Write-Host ("[{0}/{1}] {2}" -f $n, $allSources.Count, $src)
$extra = if ($vendored) { @('-w') } else { @('-Wall', '-Wextra') }
Invoke-Compile -Source (Join-Path $repo $src) -Obj $obj -ExtraFlags $extra
}
Write-Host "objects: $((Get-ChildItem -LiteralPath $objDir -Filter '*.o').Count) up to date"

# ---------------------------------------------------------------------------
# Link -- no -nostdlib (drops compiler_rt), -Wl,-e,start (restores the entry
# the clang driver would otherwise zero out).
# ---------------------------------------------------------------------------
$linkArgs = @('cc', '-target', 'x86_64-freestanding', '-Wl,-e,start', '-T', 'linker.ld', '-o', $kernelElf)
# Objects on the link line in the Makefile's OBJS order (boot, isr/switch, then
# KERNEL_SRCS order), NOT the alphabet Get-ChildItem returns. The driver_table
# and tool_table sections self-register in the order their objects appear on
# the link line, so boot-time init order would silently follow directory
# sorting otherwise -- and the QEMU boot-assertion suite pins that order.
$linkArgs += ($asmPairs | ForEach-Object { Join-Path $objDir $_.obj })
$linkArgs += ($allSources | ForEach-Object { & $objOf $_ })
Write-Host "link kernel.elf"
Invoke-Native -File $T.zig -ArgList $linkArgs -What 'kernel link'

# ---------------------------------------------------------------------------
# ELF32 container (QEMU's multiboot loader rejects ELFCLASS64). GNU objcopy
# re-labels the ELF64 image as ELF32; the 64-bit code, entry point and program
# headers are untouched. This is the Makefile's own `objcopy -O elf32-i386`
# step -- zig's llvm-objcopy cannot do it (only -O binary), hence the MSYS2
# binutils provisioned by ensure-toolchain.ps1.
# ---------------------------------------------------------------------------
Write-Host "convert kernel.bin (ELF32 container)"
Invoke-Native -File $T.objcopy -ArgList @('-O', 'elf32-i386', $kernelElf, $kernelBin) -What 'elf32 container conversion'

$sz = (Get-Item -LiteralPath $kernelBin).Length
Write-Host ("kernel.bin: {0:N0} bytes - boot it with .\scripts\run.ps1" -f $sz)
Loading