From 98b593a2ae6e1621494521890d9a3afa1f0446fc Mon Sep 17 00:00:00 2001 From: Jon Janzen Date: Thu, 30 Jul 2026 11:33:41 -0700 Subject: [PATCH 1/2] Add reproducible Windows shim build + CI verification Make the Windows shim build byte-for-byte reproducible and add a CI job that rebuilds it and fails if the checked-in binaries are stale. - release.py now links with the toolchain-bundled rust-lld (no MSVC "Rich" header) and passes /Brepro for content-hashed timestamps, so the output depends only on the pinned toolchain. It also accepts an optional target triple so CI can build one architecture per native runner. - Pin rust-toolchain.toml to a dated nightly so the reproducible output is stable over time. - Add the verify-windows-shim workflow: builds x86_64 on windows-latest and aarch64 on windows-11-arm, uploads each rebuilt .exe as an artifact, then fails on `git diff` if the committed binary differs. The artifact-before- check ordering lets contributors without a Windows machine download the correct binaries from the run and commit them. - Update windows_shim/README.md and website/docs/windows.md accordingly. Note: the checked-in .exe files still need to be regenerated (via CI artifacts or `py release.py`) to match the new reproducible build. --- .github/workflows/verify-windows-shim.yml | 71 +++++++++++++++++++++++ website/docs/windows.md | 5 +- windows_shim/README.md | 33 ++++++++++- windows_shim/release.py | 51 +++++++++++++--- windows_shim/rust-toolchain.toml | 6 +- 5 files changed, 154 insertions(+), 12 deletions(-) create mode 100644 .github/workflows/verify-windows-shim.yml diff --git a/.github/workflows/verify-windows-shim.yml b/.github/workflows/verify-windows-shim.yml new file mode 100644 index 0000000..47600c1 --- /dev/null +++ b/.github/workflows/verify-windows-shim.yml @@ -0,0 +1,71 @@ +name: verify windows shim + +# Rebuilds the Windows shim from dotslash_windows_shim.rs and fails if the +# checked-in .exe does not match, so the source cannot change without the +# regenerated binary. The build is byte-for-byte reproducible (rust-lld with +# /Brepro, plus the pinned toolchain in windows_shim/rust-toolchain.toml), so a +# plain `git diff` is a reliable check. Each architecture is built on its own +# native runner to avoid cross-linking. +# +# The freshly built binary is uploaded as an artifact before the diff check, so +# when the check fails (e.g. the committed binaries are out of date) you can +# download the correct binary from the run's "Artifacts" section and commit it +# without needing a local Windows machine. + +on: + push: + branches: [main] + paths: + - windows_shim/** + - .github/workflows/verify-windows-shim.yml + pull_request: + paths: + - windows_shim/** + - .github/workflows/verify-windows-shim.yml + workflow_dispatch: + +permissions: + contents: read + +jobs: + verify-windows-shim: + strategy: + fail-fast: false + matrix: + include: + - os: windows-latest + target: x86_64-pc-windows-msvc + arch: x86_64 + - os: windows-11-arm + target: aarch64-pc-windows-msvc + arch: aarch64 + runs-on: ${{ matrix.os }} + timeout-minutes: 20 + defaults: + run: + shell: bash + working-directory: windows_shim + steps: + - uses: actions/checkout@v6 + - name: Provision the pinned toolchain and target + # `cargo --version` auto-installs the nightly pinned by + # rust-toolchain.toml; the target then provisions its precompiled std. + run: | + cargo --version + rustup target add "${{ matrix.target }}" + - name: Rebuild the shim + run: python release.py "${{ matrix.target }}" + - name: Upload the rebuilt shim + # Runs before the diff check so the binary is downloadable even when the + # committed copy is out of date and the job ultimately fails. + uses: actions/upload-artifact@v4 + with: + name: dotslash_windows_shim-${{ matrix.arch }} + path: windows_shim/dotslash_windows_shim-${{ matrix.arch }}.exe + if-no-files-found: error + - name: Verify the checked-in binary is up to date + run: | + if ! git diff --exit-code -- 'dotslash_windows_shim-*.exe'; then + echo "::error::Checked-in shim binary is out of date. Download the 'dotslash_windows_shim-${{ matrix.arch }}' artifact from this run and commit it (or run 'py release.py' locally)." + exit 1 + fi diff --git a/website/docs/windows.md b/website/docs/windows.md index 6a676db..7c2d48b 100644 --- a/website/docs/windows.md +++ b/website/docs/windows.md @@ -69,8 +69,9 @@ of this documentation. ### DotSlash Windows Shim -**This is the preferred method.** The _DotSlash Windows Shim_ is a tiny 4KB -`.exe` executable that is placed next to the DotSlash file that performs the +**This is the preferred method.** The _DotSlash Windows Shim_ is a tiny +(a few kilobytes) `.exe` executable that is placed next to the DotSlash file +that performs the same function as the [batch script](#sibling-batch-script) above, but is a native executable rather than a batch script. This is the _ideal_ method that allows for easy execution without any of the drawbacks of batch scripts. But diff --git a/windows_shim/README.md b/windows_shim/README.md index 7b502cd..8f9bd13 100644 --- a/windows_shim/README.md +++ b/windows_shim/README.md @@ -25,14 +25,45 @@ The _DotSlash Windows Shim_ does this: ## Binary size _DotSlash Windows Shim_ builds without a standard library and only uses Windows -APIs. Release binaries are around ~5KB. +APIs to stay small. It is meant to be checked into source control next to every +DotSlash file that needs to run on Windows, so the release binaries are only a +few kilobytes. ## Release +The checked-in `dotslash_windows_shim-x86_64.exe` and +`dotslash_windows_shim-aarch64.exe` are built from `dotslash_windows_shim.rs`. +Regenerate them on Windows with: + ```shell py release.py ``` +Building both architectures requires their targets to be installed +(`rustup target add x86_64-pc-windows-msvc aarch64-pc-windows-msvc`). Pass a +single target triple to build just one architecture: + +```shell +py release.py aarch64-pc-windows-msvc +``` + +The build is byte-for-byte reproducible. It links with the Rust-bundled +`rust-lld` — which, unlike MSVC's `link.exe`, embeds no toolchain-specific +"Rich" header — and passes `/Brepro` so timestamps are content hashes rather +than wall-clock time. The output therefore depends only on the toolchain pinned +in `rust-toolchain.toml`. The `verify windows shim` GitHub Actions workflow +rebuilds the shim whenever anything under `windows_shim/` changes and fails if +the committed binaries are stale, so regenerate and commit them in the same +change as any edit to the source or a bump of the pinned toolchain. + +If you do not have a Windows machine, let CI build the binaries for you: push +your change (or trigger the workflow manually), then download the +`dotslash_windows_shim-x86_64` and `dotslash_windows_shim-aarch64` artifacts +from the workflow run — each contains the freshly built `.exe`. Because the +build is reproducible, those artifacts are exactly what a local `py release.py` +would produce; commit them into `windows_shim/` and re-run the workflow to +confirm it passes. + ## Testing ```shell diff --git a/windows_shim/release.py b/windows_shim/release.py index a5663b0..c4c4d56 100644 --- a/windows_shim/release.py +++ b/windows_shim/release.py @@ -11,6 +11,7 @@ import os import shutil import subprocess +import sys from pathlib import Path IS_WINDOWS: bool = os.name == "nt" @@ -18,10 +19,19 @@ target_triplets: list[str] = ["x86_64-pc-windows-msvc", "aarch64-pc-windows-msvc"] -def main() -> None: +def main(targets: list[str] | None = None) -> None: if not IS_WINDOWS: raise Exception("Only Windows is supported.") + # Default to all targets; a caller (e.g. CI) may pass a subset to build just + # the target that matches the current runner's architecture. + if targets: + unknown = [t for t in targets if t not in target_triplets] + if unknown: + raise SystemExit(f"Unknown target(s): {', '.join(unknown)}") + else: + targets = target_triplets + dotslash_windows_shim_root = Path(os.path.realpath(__file__)).parent target_dir = ( @@ -30,7 +40,36 @@ def main() -> None: else None ) - for triplet in target_triplets: + # Link with the linker bundled in the active Rust toolchain (rust-lld) + # instead of the MSVC link.exe. lld-link emits no "Rich" header, so the + # output depends only on the pinned toolchain (see rust-toolchain.toml) and + # not on whichever Visual Studio version happens to be installed. Together + # with /Brepro below this keeps the checked-in binaries byte-for-byte + # reproducible, which the verify-windows-shim CI job relies on. + target_libdir = Path( + subprocess.check_output(["rustc", "--print", "target-libdir"], text=True).strip() + ) + rust_lld = target_libdir.parent / "bin" / "rust-lld.exe" + if not rust_lld.is_file(): + raise FileNotFoundError(f"Rust's bundled linker was not found: {rust_lld}") + + rustflags = [ + f"-Clinker={rust_lld}", + "-Clinker-flavor=lld-link", + "-Clink-arg=/DEBUG:NONE", # Avoid an embedded PDB path. + "-Clink-arg=/NODEFAULTLIB:msvcrt", # The shim does not use the CRT. + "-Clink-arg=/Brepro", # Hash-based timestamps instead of wall-clock time. + ] + + # Ambient RUSTFLAGS could change the measured release layout and break + # reproducibility. Encoded flags also preserve the linker path as a single + # argument when the workspace path contains spaces. + build_env = {**os.environ} + build_env.pop("RUSTFLAGS", None) + build_env["RUSTC_BOOTSTRAP"] = "1" # Required by no_std language items. + build_env["CARGO_ENCODED_RUSTFLAGS"] = "\x1f".join(rustflags) + + for triplet in targets: subprocess.run( [ "cargo", @@ -43,11 +82,7 @@ def main() -> None: f"--target={triplet}", ], check=True, - env={ - **os.environ, - "RUSTC_BOOTSTRAP": "1", - "RUSTFLAGS": "-Clink-arg=/DEBUG:NONE", # Avoid embedded pdb path - }, + env=build_env, ) src = ( @@ -64,4 +99,4 @@ def main() -> None: if __name__ == "__main__": - main() + main(sys.argv[1:]) diff --git a/windows_shim/rust-toolchain.toml b/windows_shim/rust-toolchain.toml index 5d56faf..bcd545e 100644 --- a/windows_shim/rust-toolchain.toml +++ b/windows_shim/rust-toolchain.toml @@ -1,2 +1,6 @@ [toolchain] -channel = "nightly" +# Pinned so the checked-in shim binaries stay byte-for-byte reproducible: the +# CI job in .github/workflows/verify-windows-shim.yml rebuilds the shim and +# fails if the result differs from what is committed. Bump this and regenerate +# the binaries (`py release.py`) in the same change. +channel = "nightly-2026-07-28" From 49afc2dd05f5b0840950ca7949e43f6cc73806c6 Mon Sep 17 00:00:00 2001 From: Jon Janzen Date: Thu, 30 Jul 2026 12:39:35 -0700 Subject: [PATCH 2/2] Update exe files --- .../dotslash_windows_shim-aarch64.exe | Bin 4096 -> 4096 bytes windows_shim/dotslash_windows_shim-x86_64.exe | Bin 4096 -> 4096 bytes 2 files changed, 0 insertions(+), 0 deletions(-) diff --git a/windows_shim/dotslash_windows_shim-aarch64.exe b/windows_shim/dotslash_windows_shim-aarch64.exe index 4f33d81fe595e8fe2c29603d41bacd1436c755a7..09d294662f43838a1118574c25e8f5537af04dfe 100644 GIT binary patch literal 4096 zcmeHKZ-^9S7=LDGJw~roJnyccx;ec<4|dU1`a{@`>)93Ozuk+o6x?y=z1ykTopWZk z+wg;HLP!cRAQTaJQiQuC9AS`A-Dw{ZLbxOGOCQ?1z0AX)VAULA+n)EG8Fy2wPkpF- zVfOd@d!P6BKJ&cu-fey{2OIzd@o5~9M&(cB9NHIle7X=$%|GAy3DWHX|g5jSp)i)mdU;Vyt}Jpf9Pha;0;4fyGQhGozkSm=@wT$CH_UBx{oVVqhQyfFjZRwr?IWZuJTY6B*{r|`6U&eD+2fQ zMQSgv0ehNXS`@ZR&J5quJ6>|507S|ZBA!;(YS7D2pFiGNnOwv#LT(gIHP*Fd5A9&$by zm}(Edpmg=36OC zj)<`xLD-vG0R1!v^$Tg23(L8(K~{Y^@@g*!YiXSr*D;stPUK2CQ%*Z5@8*aDv!Dz2 z=3`iMXMF!tg#~Plj^a4~;EfsFOBtW<`Ht>Oh(F*{;<=M^2WgHLlmjQbu&yPZKY4jM zoIb&GQ_pou&fIb+M36VG-|g3N8gt+~E(*`O9FH}@UODLQX_Vh}gM<7MVYBNS4Z^zd zcDHw^PY6lQ@i^GDZnPH1aUOok$EeSI2>S}()#$-pHz?AYjJ8KkH@<7u4}H8X@%b2j z@|H6FLcl(Qx{Ccl`+?PRwYz?Y0DCzO`dQp0XJ4sJ(BkesXDIY>1{oYphq;S;}Y{pJt{AJe!* zAmXrd@x03iFRN48CYnpaoDruI|B{^x#z_p?)7Yao_jQVA9M^dJs_q*vRC%yBdOFVe z$A+R?rY4)I(1@B*^bs>;rquM4;sOP9u)qWjWc*FBD@Q~YNjn69Y65b|Tsc144BWe8=(v!1D##Hr;pZ*zQZPt~nMmFJUTjVqu@I9twARXkFNy<1$ zw(7GsWCsU{A!0sbP}4_Pz5TyU=(XH94t3VG?k)g&$1ldmACIAJ0(G&$Hh{Lt2HlVK z@v`T;?xC>U^|X1o!o&3*Zt+m@Fz4YL9=_w@2OfUp;W-a4cv$k#=ken$dK2=dqs>EC z1NgV`1&C_8NqVtKHG-Rn8U$Nlo1rI(X%5g`^p( zwf4!T)ngdCL9^W6e1EXI>$CMR(#z9JD^V75&8-5vi>1@Pmh;Mv}RXd!-xm`A(^FY6<;YO*&Uo63Pn@b z>PI6@>hz_ndRjoxQv7<+H#8fE#Bf18@l;4T-IfUq(@!U;ITrTwFHhxx_V&d0N7exY(s@ zow^(qgK{jU8DhI4>fJFh5)=0}1;nTpQha%NIVHC1lW$c1@@;X;nT7e{nF*U7LzsKy zrT%LO{gGgpbe4*!xfUQa$ivl!kHU`5ZP*C8uKW!wIl)3I9|k^L`5bO!5VT zzah?K>v_`y3IDH1(_SLt!e%K=d(Q>FO5V+d^!x*iH3%lj=g?;geGAK^^c_3KaJG~# zydw4w{N429TO52&dTlN#&0@~8eBs5v*qu$js?n#Eo(p`#VrDVC&6|^4!hhX~N#sqL zPx!w_dM@x~^6o2K@?HVix{~vsV$Dvu#@FB;6C0)Uh|8)GhxS)9=lnvUwO{H4m*h~# zGs>GOtkW3zI$WKk*o1XbI)ybfvWxmD&(X{~eu}k_EBcPfH&C4m%Sa~WLd7i}%7$IS zAE_K@nRN;MTo#yA$JynOUWWd*a8g=C-C{Q!tw$a5L2_ZA6$|s}U4$QSaOw4W4l3xJ zSgdQ^EY?pjR*IQoI!AG1y33=C85gRvS8*e`f3b;ON_ekY>bHIUe@*L}DSW zC+>}hBhgARCvNDGSf?1&VnNwZVzLp@Vq%9JQI*i1?v4&c7q!GaZo|V zHOzkeMD!liy48?KFBYX&33eNb=I^lIu+wz#^^O- zpeKC(w8{~qPSXPzu&Ok*A5nsa9jdQcHo`T*uwRK{xV62QNqiv@-wTSdPYZRcN*(64 zUydrRjvcFx^Y+O@Y6Rzu2Pb|M*ub{{y$PK60i{jv2w#ML-%ECq4t{WH z@-4cgudgrE-ih?Jjn)souh)7T&|9a9^ekWHh=PQ?wF|r*#5;(*MSE5dkNV{}?P=NW zA|Bv7qZEPj_F%lnENh@`-IQxj$qa{cdU|>_zI+&sxCf*ag^++LzUKOy&&*7}N3yKi zarceV87zk|d2(htjBz=m0sC-;XvOOA4pMI-@Ae|SUgUj%yrssIcCVr?Y_5P2s>eQ8 z6@YbFt$M^@jRvc@TI}6sT?;DlI8OTh70mNb_akm2{7hac=^DJRPU5f^YE&(**d=;EI*oGNsMU2% z2Y6M{V~V;4==hvrGzF*aHl)B-No6N&Fn1g zgGR~nH`ev%2}5E+jKK$eFop*xK9rxNl8B0$0^tE3eAomYK#i#q7{5C^OP3$@$p->A zy>rg_&d)vf+;eBPYhD>4iHPbkYe&3R_y>Ljc*}yZ7CP|Y$+m-1;$+)~bSAIlEOW?G zvr0-e4AWKyxMCFyC1WVv>-vpt?K0a`i|2C+_Vb(({0E}fYlKtT3O|5=segxS|k4# zyJ8{Hm;YiEvH6G%eRBX0u}`swTMk1ZN=7VAwN;{hLI=PrfL|j>-iQ;$Be{y5mpk@A zs1>24wIdF@TKPVI?^a-Ae;+GeiZW-CIp4F=rqC0`MRe$Q7eh`yM8t|(NFnANVa{J2 z&gIq4SyxU2|9};bM_+wy^RbQn@Lyc@pY=tMXlyR(S^fkenDZxdel712Y-!eKLMI8` zP5_4l=gP-s69t<(Hi|^HU7GT+^T0X|UU#v)b7V|(xhtOo#!6p@-iE}S8LT+e2wC2M zm{_HJ3eni;^2F$HfB9Brclnk~@ ztf;nP!9<8Lr#lo6&h8FHaV(2zqC2D@F2zou%00319Dalm^3?idGF+7Q}QT!;jrbLF_OostTcC$rIIK>+TgkkalXo|{~` z8BnkZhx#7&JT@OmTx^CkAf(43m4CVBy1^z_zKR1XPSq9WLK}eg;sr40XTj_d;dC0( zOyRUhNGBle6}omI9j-K|!kp90`HnfKT=|+f?$j9;*xbmrU3GD|!;i7i7F5`k&kEg+ zaeF>-dvIu`o!8ZTIy{^)G;=r~&Zjfk z=aiWmnPgL*bW-t5lY;t*nKBEyrikzQ5Vw`8w~{imS=G=KK9aE^;2YnsX6#j_)t51b zbiQt23s2d~pqkOS7P;ZYM|i4Ws{=asA`Y4bLyJtWP*L;SR*CegcDgH-UeB}UR^BsW zb0eQIjcWK8xV^^I3OZkf)z+$6-e1+^Z42ojKO?2WNwufFItzn?+)@zF8r02U?38Z& zHbJZ3G8=wB{(+fVWtVVR{7P$d=i!sNi6$$tIF3XpwOE>eJp-FX) zO>|itEzoP3DW1>wi(U1gb$uBUny9PN(}|41g#i#@(uL-pHWO$<&D%YeWm+Q2?ag-+ zT3O#)50-u$#?_o})n{urQY3+Q=(?GLQt+!RE>fpETrGiG8QW`l@H#pjE$*!&c%2&_ zf9H!MVhRm1>ad0?)e!Q_1=HqzITa;>_v_p;xc=1Ah^FJEkjsR#dGXQ+r%a27b1)C< zv2ZLL_4N6)J`5X=Cuv4K%~&6Z*5#z6)DWPCPQejL%?;EnEuaUY^|?T@?t*8x)VHgb zsJ>HVc7Ob@6~N1_15?BO7IYV84!ocr<9FU0-~|=`hrJ74&}Ph!z~3Kte+B*q6Yv8k literal 4096 zcmeHKYitx%6h7M*1!=cfR}pzk%bK*tZnov25Muhofhg-N4J4>6! zgpvhxXqxm75#k?WjKBDS2_{heQ65Pl0gaE8Vq#*{giWXhksmyi@jEkjTM`Q~;fETz z<<2?xJLi7)+%xCSUbgHQrF`hv5$Vi%oX4q4{t^&S?i=Mfb- z(5!S4z)3;Nr>?!ZNn$Qgmn0|HPav#Pa3xWzqlrh5Xv6(up`WiHS~>^*Ku97w{y-y9 zz*GlJ*vt8)azMNXu@7iYlZd(kMpQ8sqGN@C05}N{Il{-W$Sa<`qK89faS=PoWHAB-h0>HL)5=a9xg=`wy%qA`^U{?vIT4A zoWyZ}znuADqinCc;8hI9o@1Z`z{Q3RUT5$0Eob3I7#mLB!#5AR8{^WqeNQv<$Y=f` zjAUz`oarsb*mr?p>vncK(+p8?FV9%brXWN@RlOgMb|6N|qK7$o(pN5H<^+8s%WCwFY%kJ zy?{5{Qaj$lSy+ystO?uqONkR@?N^(u>$a~9XvX%WJLO@Jjv;v{s(gU^$AOKs`#Uq0 z=~u9$s?|fIZ2hm_3PSr|mn;>?0*pU>IAD5VAvmFyj$tYF>H?Y&&4P z_5+l|?Bu-qop1uMi~)RpBYIkkN_=Bc2h>Q~RHfWnU`-^HWHc}XxE@NkraBXP^fauq1ih;Hm&ss_n_$BA zC#=NHjk?i>fHie%_nT_O6rl!M6*JZpiEUF82)AVbJ4wxMB=D+gZqcJ@P2Grny{078 zaBd#rr z0lP1j)r@g{z0I+>@vUWyf`U)nRUGUg56`>;nzt=KnAhj&@)YG$Q6pm&`&{U#l3&Td-bPAH5^t7PHTMa#;rc$(_P{X0r)MS{b&e3r)p>Jgd;Uud{AJLC_ zEE_aUk5D;V-6D6Fzzpv!HdFXOY3{>9H0W4fPwS@I)~6sCaP0Abx5+)np?FdaQ^@fG z+FIZiEIW>(&G6V7#RdI7)mGI7qMAme^u=ouDgK((M07)~>B9%0Mysu