Add reproducible builds support in OHCL-Linux-Kernel#115
Draft
namancse wants to merge 5 commits intoproduct/hcl-main/6.12from
Draft
Add reproducible builds support in OHCL-Linux-Kernel#115namancse wants to merge 5 commits intoproduct/hcl-main/6.12from
namancse wants to merge 5 commits intoproduct/hcl-main/6.12from
Conversation
Contributor
saurabh-sengar
left a comment
saurabh-sengar
left a comment
There was a problem hiding this comment.
Can we upstream Linux kernel changes in this PR ?
Is there no way reproducibitly builds are supported by Linux kernel today ?
Ref: https://docs.kernel.org/kbuild/reproducible-builds.html
…rted Returning -EOPNOTSUPP is a fatal error. It means that The driver doesn't know the feature. Not that the feature is not supported due to the runtime platform reason. Return 0 for MSHV_CAP_LOWER_VTL_TIMER_VIRT on non-TDX platform, which is safer. Fixes: 3528fd7 ("drivers: hv: mshv_vtl: Advertise TDX timer service extension") Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Add NixOS flake configuration and helper scripts for reproducible kernel builds. Files added: - flake.nix: Nix environment with pinned toolchain (GCC 13.2.0, binutils, etc.) - flake.lock: Locked package versions for reproducibility - Microsoft/nix-setup.sh: One-time Nix installation helper - Microsoft/nix-clean.sh: Build artifact cleanup - .gitignore: Add Nix-related entries This establishes the foundation for bit-reproducible kernel builds across different machines by providing a hermetic build environment with pinned dependencies. Signed-off-by: Naman Jain <namjain@linux.microsoft.com>
Add nix-build.sh that orchestrates reproducible kernel builds using the Nix environment established in the previous commit. Features: - Pure Nix environment with --ignore-environment flag - Fixed build paths for reproducible absolute path embeddings - Reproducible environment variables: - SOURCE_DATE_EPOCH= timestamp of top git commit embedded - KBUILD_BUILD_USER=builder - KBUILD_BUILD_HOST=nixos - KBUILD_BUILD_VERSION=1 - Copies source to fixed path to ensure identical embedded paths - Invokes build-hcl-kernel.sh within the controlled environment - Copies artifacts back to original location - Cleanup on exit Usage: ./Microsoft/nix-build.sh x64 # Build x64 kernel ./Microsoft/nix-build.sh arm64 # Build arm64 kernel ./Microsoft/nix-build.sh x64 cvm # Build x64 cvm kernel ./Microsoft/nix-build.sh arm64 cvm # Build arm64 cvm kernel Signed-off-by: Naman Jain <namjain@linux.microsoft.com>
Enhance build-hcl-kernel.sh to support reproducible builds when invoked from nix-build.sh or other reproducible environments. Changes: - Detect host architecture to avoid unnecessary cross-compilation - Set CC explicitly to gcc/cross-compiler for Nix toolchain - Add LOCALVERSION= to prevent '+' suffix in version string - Add KCFLAGS=-fdebug-prefix-map to normalize debug paths - Add SHA256 checksum output of vmlinux for verification - Remove KBUILD_BUILD_ID=none (not needed) When REPRODUCIBLE_BUILD=1: - Uses Nix's gcc instead of system gcc for native builds - Only uses cross-compiler when actually cross-compiling - Ensures consistent compiler identification in kernel binary Otherwise, let users continue using this script for dev work as before. Signed-off-by: Naman Jain <namjain@linux.microsoft.com>
Add build-hcl-kernel-pipeline.sh for Azure DevOps CI integration with reproducible build support. Features: - Supports amd64 and arm64 architectures - CVM config merge support via merge_cvm_config() - Optional reproducible build mode (--reproducible flag) - Generates kernel, headers, modules, and debug symbols - Progress indicators for build stages [1/5] through [5/5] - SHA256 checksum output for reproducibility verification Key differences from build-hcl-kernel.sh: - Standalone script that doesn't depend on nix-build.sh wrapper - Implements complete build workflow in one script - Uses KBUILD_OUTPUT=$BUILD_DIR/linux subdirectory structure - Handles CVM config merging inline - Moves artifacts from /linux subdirectory to BUILD_DIR root for pipeline - When --reproducible: sets up Nix environment and reproducible variables Build directory structure: - $BUILD_DIR/linux/ # KBUILD_OUTPUT during build - $BUILD_DIR/vmlinux # Final artifacts at root - $BUILD_DIR/linux-headers/ - $BUILD_DIR/debug_symbols/ Usage: ./build-hcl-kernel-pipeline.sh -s <source> -b <build> -c <config> -a <arch> ./build-hcl-kernel-pipeline.sh ... --reproducible ./build-hcl-kernel-pipeline.sh ... --cvm-config <config> Signed-off-by: Naman Jain <namjain@linux.microsoft.com>
namancse
force-pushed
the
user/namjain/reprobuild-pipeline-golden-working
branch
from
2ac6268 to
6d4613e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OHCL-Linux-Kernel has Microsoft/build-hcl-kernel.sh script which is used to build kernel. However, in build pipelines, that script is not used and similar code in pipeline code itself is used.
To implement reproducible builds, add this support in both local build script (Microsoft/build-hcl-kernel.sh) and the pipeline code. Instead of adding the support in pipeline directly, move the kernel build code from pipeline to a new script "Microsoft/build-hcl-kernel-pipeline.sh" and ad reproducible builds changes in it. With that, buddy/official pipeline would then call this script to build kernel.