From 5c9c50f10591f181cc5871feae74adba910837d2 Mon Sep 17 00:00:00 2001 From: mmelnich Date: Wed, 12 Aug 2026 17:33:26 -0700 Subject: [PATCH 1/6] Add install/install.sh for Linux and macOS RandBLAS has had no installer. RandLAPACK's declines to cover it ("RandBLAS is intentionally not covered: it stays a git submodule"), so anyone wanting RandBLAS alone had to hand-build BLAS++ and Random123 and copy a cmake line out of INSTALL.md. Release-plan item B1. The script builds RandBLAS and its dependencies into a RandNLA-project directory laid out exactly as RandLAPACK's installer lays one out, and honours RANDNLA_PROJECT_DIR, so running both installers on one machine shares a single dependency tree instead of building BLAS++ twice. Design points worth calling out: * Pins, not branches. BLAS++, Random123, GoogleTest and LAPACK++ are fetched at an immutable ref by shallow fetch of one commit. Each install and each source tree carries a provenance stamp, and is reused only when the stamp matches what we would build now. Reuse keyed on mere presence makes changing a pin a silent no-op for anyone who already ran the script. * The integer width is read back, not assumed. BLAS++ probes int32 before int64 and blas_int only filters library *names*, so a successful blas_int=int64 configure proves ILP64 only for MKL, where mkl_intel_ilp64 is a distinct library. For OpenBLAS there is just -lopenblas, and an LP64 build passes the int32 probe and is accepted. The resolved width therefore comes from BLAS++'s generated blas/defines.h after the build. Trusting the request would stamp an LP64 install as ILP64 and every later run would reuse it believing otherwise. * ILP64 is preferred wherever it is real, and the fallback is loud. Accelerate is refused outright for ILP64: Apple has shipped one since macOS 13.3 but BLAS++ implements only the legacy LP64 interface (icl-utk-edu/lapackpp#43). * Verification runs, it does not just link. A conftest sketches a matrix, multiplies through BLAS++ and checks the result is symmetric positive semidefinite. It goes through BLAS++ because that is how RandBLAS reaches the BLAS. * GoogleTest is provisioned rather than assumed, because BUILD_TESTS defaults to ON while find_package(GTest) is not REQUIRED. * Examples are a post-install offer, not a default. They need two dependencies RandBLAS does not (LAPACK++, fast_matrix_market) and require OpenMP, which stock Apple Clang cannot supply. * The clone is never moved. RandLAPACK's installer relocates its own repository into lib/, which breaks git worktrees; a symlink gives the same layout without touching anything. .gitignore matched "**install/", which was meant for build-output trees but also swallowed a source directory named install/. Narrowed to "**/*-install/", the pattern RandLAPACK already uses. The "!**install/.gitkeep" exception it carried was vestigial: no .gitkeep is tracked. The new install-script workflow runs the installer per OS and asserts what the core workflows cannot: idempotent re-runs, dependency discovery, the LP64 fallback warning firing for stock OpenBLAS, ILP64 for MKL, Accelerate refusing ILP64, and piped output containing no terminal escape sequences. A separate packager lane deliberately does not run the installer at all -- it configures with plain CMake against hand-installed dependencies inside a network namespace with no interfaces, which is the contract conda-forge and Spack actually depend on. Verified on Linux: fresh MKL build, idempotent re-run, dependency discovery, the examples path (all seven binaries, fast_matrix_market at the pinned v1.7.6), a clean actionable failure when no OpenBLAS exists, and redirected output free of escape sequences. --- .github/workflows/install-script.yml | 236 +++++++ .gitignore | 7 +- install/install.sh | 950 +++++++++++++++++++++++++++ 3 files changed, 1191 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/install-script.yml create mode 100755 install/install.sh diff --git a/.github/workflows/install-script.yml b/.github/workflows/install-script.yml new file mode 100644 index 00000000..e3260a81 --- /dev/null +++ b/.github/workflows/install-script.yml @@ -0,0 +1,236 @@ +# Exercises install/install.sh itself -- the one artifact the core workflows +# never run, since they hand-replicate the build recipe instead. What each lane +# proves: +# 1. a fresh checkout run non-interactively builds, and its tests pass; +# 2. re-running in place succeeds, because idempotent re-runs are part of the +# script's contract and are how people recover from a failed run; +# 3. the discovery path reuses dependencies pointed at by *_INSTALL_DIR +# rather than rebuilding them; +# 4. piped output stays free of escape sequences, so CI logs and redirected +# transcripts remain readable. +# +# The packager lane is separate and deliberately does NOT run the installer: +# conda-forge and Spack never do. It configures with plain CMake against +# hand-installed dependencies, with the network off, which is the contract a +# recipe actually depends on. +name: install-script + +on: + pull_request: + workflow_dispatch: + # Only main. A branch with an open pull request is already covered by the + # pull_request event, and firing on both runs every job twice per commit. + push: + branches: + - main + +# One run per ref. Superseding is only safe for pull requests: on main every +# commit should be validated, not just the newest. +concurrency: + group: ${{ github.workflow }}-${{ github.head_ref || github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + install-linux: + name: linux-${{ matrix.backend }} + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + # openblas exercises the LP64 path and, with it, the warning the + # installer emits when ILP64 was preferred but is unavailable. + # mkl exercises the ILP64 path, which is the default wherever the + # backend can actually provide it. + backend: [openblas, mkl] + steps: + - uses: actions/checkout@v4 + with: + path: RandBLAS + + - name: install a compiler, CMake and a BLAS + run: | + export DEBIAN_FRONTEND=noninteractive + sudo apt-get update -qq + sudo apt-get install -qq -y g++ gfortran cmake git + if [ "${{ matrix.backend }}" = "openblas" ]; then + sudo apt-get install -qq -y libopenblas-dev + else + sudo apt-get install -qq -y intel-oneapi-mkl-devel || \ + { wget -qO- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \ + | sudo gpg --dearmor -o /usr/share/keyrings/oneapi.gpg + echo "deb [signed-by=/usr/share/keyrings/oneapi.gpg] https://apt.repos.intel.com/oneapi all main" \ + | sudo tee /etc/apt/sources.list.d/oneAPI.list + sudo apt-get update -qq + sudo apt-get install -qq -y intel-oneapi-mkl-devel; } + echo "MKLROOT=/opt/intel/oneapi/mkl/latest" >> "$GITHUB_ENV" + fi + + - name: keep a pristine checkout for the discovery test + run: cp -a RandBLAS RandBLAS-discovery + + - name: run the installer, capturing output for the escape-sequence check + run: | + bash RandBLAS/install/install.sh --yes \ + --blas=${{ matrix.backend }} \ + --project-dir "$GITHUB_WORKSPACE/RandNLA-project" \ + 2>&1 | tee installer.out + + # Redirected output must contain no ANSI escapes and no carriage + # returns. Without this, adding a progress bar later silently fills + # every CI log and every user's redirected install.log with control + # characters, and nobody notices until the log is unreadable. + - name: piped output is free of terminal control sequences + run: | + if LC_ALL=C grep -qP '\x1b\[|\r' installer.out; then + echo "Found terminal control sequences in non-TTY output:" + LC_ALL=C grep -nP '\x1b\[|\r' installer.out | head -20 + exit 1 + fi + echo "OK: no escape sequences in redirected output." + + - name: the reported integer width matches the backend + run: | + grep -E '^ Backend ' installer.out + if [ "${{ matrix.backend }}" = "mkl" ]; then + grep -qE '^ Backend .*ILP64' installer.out \ + || { echo "MKL should have produced an ILP64 build."; exit 1; } + else + # OpenBLAS from apt is LP64, so the installer must both fall back + # and say that it did. + grep -qE '^ Backend .*LP64' installer.out \ + || { echo "Expected an LP64 build for stock OpenBLAS."; exit 1; } + grep -q 'No ILP64 openblas was available' installer.out \ + || { echo "The LP64 fallback happened without warning about it."; exit 1; } + fi + + - name: run the test suite + run: ctest --test-dir RandNLA-project/build/RandBLAS-build --output-on-failure + + - name: re-run the installer in place (idempotency) + run: | + bash RandBLAS/install/install.sh --yes \ + --blas=${{ matrix.backend }} \ + --project-dir "$GITHUB_WORKSPACE/RandNLA-project" | tee rerun.out + grep -q 'already built' rerun.out \ + || { echo "The second run rebuilt dependencies instead of reusing them."; exit 1; } + + - name: install a second project through dependency discovery + run: | + BLASPP_INSTALL_DIR="$GITHUB_WORKSPACE/RandNLA-project/install/blaspp-${{ matrix.backend }}-install" \ + RANDOM123_INSTALL_DIR="$GITHUB_WORKSPACE/RandNLA-project/install/Random123-install" \ + GTEST_ROOT="$GITHUB_WORKSPACE/RandNLA-project/install/googletest-install" \ + bash RandBLAS-discovery/install/install.sh --yes \ + --blas=${{ matrix.backend }} \ + --project-dir "$GITHUB_WORKSPACE/RandNLA-project-discovery" | tee discovery.out + grep -q 'external install' discovery.out \ + || { echo "Discovery did not reuse the pre-installed dependencies."; exit 1; } + test -d RandNLA-project-discovery/build/RandBLAS-build + + install-macos: + name: macos-accelerate + runs-on: macos-latest + steps: + - uses: actions/checkout@v4 + with: + path: RandBLAS + + - name: install libomp + run: brew install libomp + + # Accelerate is the default on macOS and is LP64-only: BLAS++ implements + # only Apple's legacy interface, so there is no ILP64 lane to run here. + - name: run the installer + run: | + bash RandBLAS/install/install.sh --yes \ + --project-dir "$GITHUB_WORKSPACE/RandNLA-project" | tee installer.out + grep -qE '^ Backend *accelerate' installer.out + + - name: run the test suite + run: ctest --test-dir RandNLA-project/build/RandBLAS-build --output-on-failure + + - name: re-run the installer in place (idempotency) + run: | + bash RandBLAS/install/install.sh --yes \ + --project-dir "$GITHUB_WORKSPACE/RandNLA-project" + + - name: asking for ILP64 on Accelerate is refused, not silently downgraded + run: | + if bash RandBLAS/install/install.sh --yes --blas=accelerate --blas-int=ilp64 \ + --project-dir "$GITHUB_WORKSPACE/RandNLA-project-ilp64" > refuse.out 2>&1; then + echo "The installer accepted an impossible configuration."; cat refuse.out; exit 1 + fi + grep -q 'not available with Accelerate' refuse.out + + packager: + # The contract conda-forge and Spack rely on: dependencies already + # installed, plain CMake, no install script, and no network during + # configure. If this lane stays green, a recipe is possible; if it goes + # red, packaging is broken no matter how well the installer works. + name: linux-plain-cmake-offline + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: install dependencies the way a package manager would + run: | + export DEBIAN_FRONTEND=noninteractive + sudo apt-get update -qq + sudo apt-get install -qq -y g++ gfortran cmake git libopenblas-dev libgtest-dev + + - name: build BLAS++ and Random123 into a prefix + run: | + PREFIX="$GITHUB_WORKSPACE/deps" + git clone --quiet --depth 1 https://github.com/icl-utk-edu/blaspp.git blaspp-src + cmake -S blaspp-src -B blaspp-build -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX="$PREFIX" -Dblas=openblas -Dbuild_tests=OFF + cmake --build blaspp-build -j"$(nproc)" --target install + git clone --quiet --depth 1 --branch v1.14.0 \ + https://github.com/DEShawResearch/Random123.git random123-src + mkdir -p "$PREFIX/include" + cp -r random123-src/include/Random123 "$PREFIX/include/Random123" + + # Configure inside a network namespace with no interfaces, so any + # FetchContent or git call added to the build later fails here rather + # than in a packager's sandbox months from now. "unshare -rn" rather + # than "sudo unshare -n": the -r user mapping keeps the generated build + # tree owned by the runner user, where sudo would leave it root-owned + # and break the non-root build step that follows. + - name: configure with plain CMake and no network + run: | + unshare -rn \ + cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_PREFIX_PATH="$GITHUB_WORKSPACE/deps" \ + -DRandom123_DIR="$GITHUB_WORKSPACE/deps/include" \ + -DCMAKE_INSTALL_PREFIX="$GITHUB_WORKSPACE/pkg" + + - name: build, install and test + run: | + cmake --build build -j"$(nproc)" --target install + ctest --test-dir build --output-on-failure + + - name: a downstream project finds the installed package + run: | + mkdir -p consumer && cd consumer + cat > CMakeLists.txt <<'EOF' + cmake_minimum_required(VERSION 3.21) + project(consumer CXX) + find_package(RandBLAS REQUIRED) + add_executable(use main.cc) + target_link_libraries(use RandBLAS) + EOF + cat > main.cc <<'EOF' + #include + #include + int main() { + std::vector M(16); + RandBLAS::DenseDist D(4, 4); + RandBLAS::RNGState state(0); + RandBLAS::fill_dense(D, M.data(), state); + return 0; + } + EOF + cmake -S . -B build \ + -DCMAKE_PREFIX_PATH="$GITHUB_WORKSPACE/pkg;$GITHUB_WORKSPACE/deps" \ + -DRandom123_DIR="$GITHUB_WORKSPACE/deps/include" + cmake --build build -j"$(nproc)" + ./build/use diff --git a/.gitignore b/.gitignore index e70ab24e..a33ff4e3 100644 --- a/.gitignore +++ b/.gitignore @@ -48,8 +48,11 @@ breathe/* **/compile_commands.json **build/ !**build/.gitkeep -**install/ -!**install/.gitkeep +# Install trees produced by a build: blaspp-install, RandBLAS-install, +# googletest-install and friends. Matched as *-install rather than a bare +# "install" so that the tracked install/ source directory, which holds the +# installer scripts, is not swallowed too. This mirrors RandLAPACK's pattern. +**/*-install/ ## Local diff --git a/install/install.sh b/install/install.sh new file mode 100755 index 00000000..86771a4c --- /dev/null +++ b/install/install.sh @@ -0,0 +1,950 @@ +#!/bin/bash +# RandBLAS autoinstaller for Linux and macOS. +# +# Builds RandBLAS and the dependencies it needs, into a self-contained +# "RandNLA-project" directory laid out as: +# lib: blaspp (and lapackpp, only if examples are requested) sources +# install: RandBLAS-install, blaspp-install, Random123, googletest-install +# build: one build directory per project above +# +# Nothing is installed system-wide and your shell configuration is not touched. +# +# You bring a C++20 compiler, CMake 3.21+, Git and a BLAS. This script does not +# install compilers or package managers; when something is missing it says so +# and tells you the usual way to get it. +# +# Prerequisites and supported configurations are listed in INSTALL.md. + +set -euo pipefail + +usage() { + # A heredoc rather than a line-range sed over this file's own comments: + # the latter silently starts printing unrelated code the moment anyone + # adds a line above it. + cat <<'USAGE' +Usage: bash install/install.sh [options] + +Backend selection: + --blas=BACKEND auto | openblas | mkl | accelerate | custom + (default: auto -- Accelerate on macOS, MKL on Linux + when MKLROOT is set, otherwise OpenBLAS) + --blas-int=WIDTH ilp64 | lp64. Default is ilp64 wherever the backend + can provide it, falling back to lp64 with a warning. + Accelerate is lp64-only and rejects ilp64. + --blas-libraries=L Link line for --blas=custom, e.g. + "/opt/aocl/lib/libblis.so;/opt/aocl/lib/libflame.so" + +Locations: + --project-dir=DIR Where dependencies, builds and installs go. + Default: $RANDNLA_PROJECT_DIR if set, otherwise + ../RandNLA-project next to this clone. + --prefix=DIR Install RandBLAS itself here instead of + /install/RandBLAS-install. Dependencies + still go in the project directory. + +Build: + -j, --jobs N Parallel build jobs (default: number of cores) + --fresh Clear build directories and rebuild dependencies + --no-tests Do not provision GoogleTest, and configure with + -DBUILD_TESTS=OFF + --no-openmp Configure without OpenMP + --examples Build the examples too, instead of offering them + after the install finishes + +Output: + -y, --yes Assume "yes" at every prompt. This is also the + behavior when stdin is not a terminal (CI, pipes). + --no-progress Plain one-line-per-step output, no redrawing + -h, --help Show this help and exit + +Every option has an environment-variable equivalent (flags win): + RANDBLAS_INSTALL_BLAS, RANDBLAS_INSTALL_BLAS_INT, + RANDBLAS_INSTALL_BLAS_LIBRARIES, RANDBLAS_INSTALL_PROJECT_DIR, + RANDBLAS_INSTALL_PREFIX, RANDBLAS_INSTALL_JOBS, RANDBLAS_INSTALL_FRESH, + RANDBLAS_INSTALL_TESTS, RANDBLAS_INSTALL_OPENMP, + RANDBLAS_INSTALL_EXAMPLES, RANDBLAS_INSTALL_YES, + RANDBLAS_INSTALL_PROGRESS + +Already-installed dependencies are reused when pointed at by: + BLASPP_INSTALL_DIR, RANDOM123_INSTALL_DIR, LAPACKPP_INSTALL_DIR, GTEST_ROOT + +All compiler output goes to /install.log; the console shows one +line per step. On failure the log path is printed. +USAGE +} + +#============================================================================== +# Option parsing. Environment variables provide defaults; flags override. +#============================================================================== +BLAS_BACKEND="${RANDBLAS_INSTALL_BLAS:-auto}" +BLAS_INT_CHOICE="${RANDBLAS_INSTALL_BLAS_INT:-auto}" # auto | ilp64 | lp64 +BLAS_LIBRARIES_ARG="${RANDBLAS_INSTALL_BLAS_LIBRARIES:-}" +PROJECT_DIR_OVERRIDE="${RANDBLAS_INSTALL_PROJECT_DIR:-}" +PREFIX_OVERRIDE="${RANDBLAS_INSTALL_PREFIX:-}" +JOBS="${RANDBLAS_INSTALL_JOBS:-}" +FRESH="${RANDBLAS_INSTALL_FRESH:-0}" +WANT_TESTS="${RANDBLAS_INSTALL_TESTS:-1}" +WANT_OPENMP="${RANDBLAS_INSTALL_OPENMP:-1}" +WANT_EXAMPLES="${RANDBLAS_INSTALL_EXAMPLES:-0}" +ASSUME_YES="${RANDBLAS_INSTALL_YES:-0}" +WANT_PROGRESS="${RANDBLAS_INSTALL_PROGRESS:-1}" + +die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; } + +while [[ $# -gt 0 ]]; do + case "$1" in + --blas) BLAS_BACKEND="${2:?--blas requires a backend}"; shift ;; + --blas=*) BLAS_BACKEND="${1#*=}" ;; + --blas-int) BLAS_INT_CHOICE="${2:?--blas-int requires a width}"; shift ;; + --blas-int=*) BLAS_INT_CHOICE="${1#*=}" ;; + --blas-libraries) BLAS_LIBRARIES_ARG="${2:?--blas-libraries requires a value}"; shift ;; + --blas-libraries=*) BLAS_LIBRARIES_ARG="${1#*=}" ;; + --project-dir) PROJECT_DIR_OVERRIDE="${2:?--project-dir requires a path}"; shift ;; + --project-dir=*) PROJECT_DIR_OVERRIDE="${1#*=}" ;; + --prefix) PREFIX_OVERRIDE="${2:?--prefix requires a path}"; shift ;; + --prefix=*) PREFIX_OVERRIDE="${1#*=}" ;; + -j|--jobs) JOBS="${2:?--jobs requires a number}"; shift ;; + --jobs=*) JOBS="${1#*=}" ;; + -j*) JOBS="${1#-j}" ;; # attached form, as in -j8 + --fresh) FRESH=1 ;; + --no-tests) WANT_TESTS=0 ;; + --no-openmp) WANT_OPENMP=0 ;; + --examples) WANT_EXAMPLES=1 ;; + -y|--yes) ASSUME_YES=1 ;; + --no-progress) WANT_PROGRESS=0 ;; + -h|--help) usage; exit 0 ;; + *) printf 'Unknown option: %s (see --help)\n' "$1" >&2; exit 2 ;; + esac + shift +done + +case "$BLAS_BACKEND" in + auto|openblas|mkl|accelerate|custom) ;; + *) die "--blas must be auto, openblas, mkl, accelerate or custom (got '$BLAS_BACKEND')" ;; +esac +case "$BLAS_INT_CHOICE" in + auto|ilp64|lp64) ;; + *) die "--blas-int must be ilp64 or lp64 (got '$BLAS_INT_CHOICE')" ;; +esac +if [[ "$BLAS_BACKEND" == "custom" && -z "$BLAS_LIBRARIES_ARG" ]]; then + die "--blas=custom needs --blas-libraries=" +fi +if [[ -n "$BLAS_LIBRARIES_ARG" && "$BLAS_BACKEND" != "custom" ]]; then + die "--blas-libraries only applies to --blas=custom (backend is '$BLAS_BACKEND')" +fi + +if [[ -z "$JOBS" ]]; then + JOBS=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 8) +fi + +#============================================================================== +# Interactivity and output style. +# +# Prompts happen only on a terminal and only without --yes. When stdin is not a +# terminal (piped, CI) every prompt silently takes its default, so this script +# can never hang waiting for input nobody is there to give. +#============================================================================== +INTERACTIVE=0 +if [[ -t 0 && "$ASSUME_YES" != "1" ]]; then + INTERACTIVE=1 +fi + +# ask -> returns 0 for yes. +ask() { + local question="$1" default="$2" reply + if [[ "$INTERACTIVE" != "1" ]]; then + [[ "$default" == "y" ]] + return + fi + read -r -p "$question [$( [[ $default == y ]] && echo Y/n || echo y/N )]: " reply + reply="${reply:-$default}" + [[ "$reply" == "y" || "$reply" == "Y" || "$reply" == "yes" ]] +} + +# Plain output when stdout is not a terminal, or when NO_COLOR / TERM=dumb ask +# for it. Piped output must stay free of escape sequences so that build logs +# and CI transcripts remain readable. +if [[ -t 1 && -z "${NO_COLOR:-}" && "${TERM:-}" != "dumb" && "$WANT_PROGRESS" == "1" ]]; then + C_OK=$'\033[32m'; C_ERR=$'\033[31m'; C_WARN=$'\033[33m'; C_BOLD=$'\033[1m'; C_OFF=$'\033[0m' +else + C_OK=""; C_ERR=""; C_WARN=""; C_BOLD=""; C_OFF="" +fi + +note() { printf '%s\n' "$*"; } +warn() { printf '%swarning:%s %s\n' "$C_WARN" "$C_OFF" "$*" >&2; } + +# Collected and reprinted in the final summary. A warning emitted twenty +# minutes and several thousand log lines before the summary is a warning +# nobody reads. +WARNINGS=() +record_warning() { WARNINGS+=("$1"); warn "$1"; } + +#============================================================================== +# Toolchain preflight. Report everything missing at once rather than failing on +# the first one, so a bare machine takes one round trip instead of three. +#============================================================================== +UNAME_S="$(uname -s)" +MISSING=() +command -v cmake >/dev/null 2>&1 || MISSING+=("cmake") +command -v git >/dev/null 2>&1 || MISSING+=("git") +if ! command -v c++ >/dev/null 2>&1 && ! command -v g++ >/dev/null 2>&1 && \ + ! command -v clang++ >/dev/null 2>&1; then + MISSING+=("a C++ compiler") +fi +if (( ${#MISSING[@]} )); then + printf 'ERROR: missing prerequisites: %s\n\n' "${MISSING[*]}" >&2 + if [[ "$UNAME_S" == "Darwin" ]]; then + printf ' xcode-select --install # Apple Clang and git\n' >&2 + printf ' brew install cmake\n\n' >&2 + else + printf ' sudo apt install g++ cmake git # Debian, Ubuntu\n' >&2 + printf ' sudo dnf install gcc-c++ cmake git # Fedora, RHEL\n\n' >&2 + fi + printf 'See INSTALL.md for the full prerequisite list.\n' >&2 + exit 1 +fi + +CMAKE_VERSION="$(cmake --version | head -n1 | awk '{print $3}')" +if [[ "$(printf '%s\n3.21\n' "$CMAKE_VERSION" | sort -V | head -n1)" != "3.21" ]]; then + die "CMake 3.21 or later is required (found $CMAKE_VERSION). See INSTALL.md." +fi + +# RandBLAS uses C++20 concepts, which GCC only implements completely from 13. +if command -v g++ >/dev/null 2>&1 && [[ "${CXX:-g++}" != *clang* ]]; then + GXX_MAJOR="$(g++ -dumpversion 2>/dev/null | cut -d. -f1)" + if [[ -n "$GXX_MAJOR" && "$GXX_MAJOR" -lt 13 ]]; then + record_warning "g++ $GXX_MAJOR is older than the supported minimum of 13; RandBLAS uses C++20 concepts and may not compile." + fi +fi + +#============================================================================== +# Project layout. +# +# Precedence for the layout root: --project-dir, then RANDNLA_PROJECT_DIR, then +# a sibling of this clone. Honouring RANDNLA_PROJECT_DIR is what lets this +# installer and RandLAPACK's share one dependency tree: whichever runs second +# finds the first one's BLAS++ and reuses it. +# +# Unlike RandLAPACK's installer, this one never moves your clone. Relocating a +# repository out from under the user breaks git worktrees and is surprising; +# the layout below is created by mkdir regardless of where the clone lives, and +# lib/RandBLAS is a symlink so the tree still reads as complete. +#============================================================================== +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_DIR="$(dirname "$SCRIPT_DIR")" + +if [[ -n "$PROJECT_DIR_OVERRIDE" ]]; then + PROJECT_DIR="$PROJECT_DIR_OVERRIDE" +elif [[ -n "${RANDNLA_PROJECT_DIR:-}" ]]; then + PROJECT_DIR="$RANDNLA_PROJECT_DIR" +else + PROJECT_DIR="$(dirname "$REPO_DIR")/RandNLA-project" +fi +mkdir -p "$PROJECT_DIR" +PROJECT_DIR="$(cd "$PROJECT_DIR" && pwd)" + +mkdir -p "$PROJECT_DIR"/{install,lib,build} + +# A symlink, not a move: the clone stays where the user put it. +if [[ ! -e "$PROJECT_DIR/lib/RandBLAS" ]]; then + ln -s "$REPO_DIR" "$PROJECT_DIR/lib/RandBLAS" +fi + +RANDBLAS_INSTALL_DIR="${PREFIX_OVERRIDE:-$PROJECT_DIR/install/RandBLAS-install}" + +LOG="$PROJECT_DIR/install.log" +# Appended, not truncated: the previous run's output is exactly what you want +# when the current run fails the same way. +{ + printf '\n===============================================================\n' + printf 'RandBLAS install started %s\n' "$(date)" + printf 'command: %s\n' "$0 $*" + printf '===============================================================\n' +} >> "$LOG" + +STEP=0 +TOTAL_STEPS=0 +run_step() { + local label="$1"; shift + STEP=$((STEP + 1)) + printf '%s[%d/%d]%s %s ... ' "$C_BOLD" "$STEP" "$TOTAL_STEPS" "$C_OFF" "$label" + local t0 t1 + t0=$(date +%s) + { + printf '\n===== [%d/%d] %s =====\n' "$STEP" "$TOTAL_STEPS" "$label" + printf '$ %s\n' "$*" + } >> "$LOG" + if "$@" >> "$LOG" 2>&1; then + t1=$(date +%s) + printf '%sdone%s (%ds)\n' "$C_OK" "$C_OFF" "$((t1 - t0))" + else + printf '%sFAILED%s\n' "$C_ERR" "$C_OFF" >&2 + printf '\nStep "%s" failed. Full output: %s\n' "$label" "$LOG" >&2 + printf 'The last 20 log lines:\n' >&2 + tail -20 "$LOG" >&2 + exit 1 + fi +} + +skip_step() { + STEP=$((STEP + 1)) + printf '%s[%d/%d]%s %s\n' "$C_BOLD" "$STEP" "$TOTAL_STEPS" "$C_OFF" "$1" +} + +#============================================================================== +# Provenance stamps. +# +# A dependency is reused only when it was built from the source we would build +# from now, and in the configuration we want now. Reuse keyed on mere presence +# means that changing a pin, or switching BLAS backend, is a silent no-op for +# anyone who already ran this script -- they keep the old artifact and the new +# setting never takes effect. +#============================================================================== +stamp_file() { printf '%s/.randblas-provenance' "$1"; } + +stamp_matches() { # + local dir="$1" expected="$2" + [[ -f "$(stamp_file "$dir")" ]] || return 1 + [[ "$(cat "$(stamp_file "$dir")")" == "$expected" ]] +} + +write_stamp() { # + printf '%s\n' "$2" > "$(stamp_file "$1")" +} + +# Shallow-fetch exactly one commit or tag, so the pin cannot drift the way a +# branch name would and we do not pay for the full history. The source tree +# gets its own stamp: a shallow checkout of a tag does not keep the tag ref +# locally, so "is this clone at the pinned ref?" cannot be answered by asking +# git afterwards, and without the stamp a source tree left over from an older +# pin would be reused as if it were current. +clone_pinned() { # + local url="$1" dest="$2" ref="$3" + rm -rf "$dest" + mkdir -p "$dest" + git -C "$dest" init --quiet + git -C "$dest" remote add origin "$url" + git -C "$dest" fetch --quiet --depth 1 origin "$ref" + git -C "$dest" checkout --quiet FETCH_HEAD + write_stamp "$dest" "$url@$ref" +} + +source_is_current() { # + stamp_matches "$1" "$2@$3" +} + +#============================================================================== +# Dependency pins. Immutable refs only: a tag or a full commit hash, never a +# branch name. These match the refs RandLAPACK's Windows provisioner validated. +#============================================================================== +BLASPP_URL="https://github.com/icl-utk-edu/blaspp.git" +# The commit that merged the MSVC portability fix (blaspp PR #132, 2026-08-06). +# Not in a release yet -- the latest tag, v2025.05.28, predates it. Move to a +# tag once one includes it. +BLASPP_REF="30571853f980d3a2a1737124ea4789e025a5e045" + +LAPACKPP_URL="https://github.com/icl-utk-edu/lapackpp.git" +LAPACKPP_REF="40b9d0daf29b6f1f3fa58bc3f22bd6cfb2c67fe4" + +RANDOM123_URL="https://github.com/DEShawResearch/Random123.git" +RANDOM123_REF="v1.14.0" + +GTEST_URL="https://github.com/google/googletest.git" +GTEST_REF="v1.18.0" + +#============================================================================== +# Backend resolution. +# +# "auto" prefers what the platform actually ships: Accelerate on macOS, MKL on +# Linux when MKLROOT says it is installed, OpenBLAS otherwise. The choice is +# always printed, because a silent default is the thing people later cannot +# explain. +#============================================================================== +BREW_PREFIX="" +if [[ "$UNAME_S" == "Darwin" ]] && command -v brew >/dev/null 2>&1; then + # Never hardcode /opt/homebrew: that is Apple Silicon only, and breaks both + # Intel Macs (/usr/local) and any custom HOMEBREW_PREFIX. + BREW_PREFIX="$(brew --prefix)" +fi + +if [[ "$BLAS_BACKEND" == "auto" ]]; then + if [[ "$UNAME_S" == "Darwin" ]]; then + BLAS_BACKEND="accelerate" + elif [[ -n "${MKLROOT:-}" && -d "${MKLROOT:-}" ]]; then + BLAS_BACKEND="mkl" + else + BLAS_BACKEND="openblas" + fi + note "Selected BLAS backend: $BLAS_BACKEND (from --blas=auto)" +fi + +# Integer width policy: prefer ILP64 wherever the backend can provide it, and +# fall back to LP64 only where it cannot. +# +# ILP64 matters because LP64 caps every individual BLAS dimension at 2^31, and +# because RandBLAS's MKL sparse path requires MKL_INT to match its int64_t +# sparse indices. RandBLAS's own API is int64_t either way -- BLAS++ presents +# int64_t regardless of the underlying width -- so this choice is about what +# the BLAS underneath can represent, not about RandBLAS's interface. +# +# Accelerate is the one backend with no ILP64 route at all. Apple has shipped +# an ILP64 interface since macOS 13.3, behind ACCELERATE_NEW_LAPACK and +# ACCELERATE_LAPACK_ILP64, but BLAS++ does not implement it: BLASFinder.cmake +# emits only "-framework Accelerate", the legacy LP64 path. Tracked upstream as +# icl-utk-edu/lapackpp#43. +WIDTH_ORDER=() +case "$BLAS_BACKEND" in + accelerate) + if [[ "$BLAS_INT_CHOICE" == "ilp64" ]]; then + die "--blas-int=ilp64 is not available with Accelerate: BLAS++ implements only Apple's legacy LP64 interface (see icl-utk-edu/lapackpp#43). Use --blas=openblas or --blas=mkl for ILP64." + fi + WIDTH_ORDER=(int32) + ;; + *) + case "$BLAS_INT_CHOICE" in + ilp64) WIDTH_ORDER=(int64) ;; + lp64) WIDTH_ORDER=(int32) ;; + auto) WIDTH_ORDER=(int64 int32) ;; # ILP64 first, LP64 as fallback + esac + ;; +esac + +# blaspp's own backend selector. Its matcher accepts "apple" or "accelerate". +BLASPP_BACKEND_FLAGS=() +case "$BLAS_BACKEND" in + openblas) BLASPP_BACKEND_FLAGS=(-Dblas=openblas) ;; + mkl) BLASPP_BACKEND_FLAGS=(-Dblas=mkl) ;; + accelerate) BLASPP_BACKEND_FLAGS=(-Dblas=apple) ;; + custom) BLASPP_BACKEND_FLAGS=(-DBLAS_LIBRARIES="$BLAS_LIBRARIES_ARG") ;; +esac + +#============================================================================== +# OpenMP. +# +# Apple Clang ships no OpenMP runtime. Homebrew's libomp supplies one, but only +# via -Xpreprocessor -fopenmp plus explicit library paths, so it has to be wired +# up by hand rather than found by FindOpenMP. +#============================================================================== +OPENMP_FLAGS=() +if [[ "$WANT_OPENMP" != "1" ]]; then + OPENMP_FLAGS=(-DCMAKE_DISABLE_FIND_PACKAGE_OpenMP=TRUE) +elif [[ "$UNAME_S" == "Darwin" ]]; then + LIBOMP="" + if [[ -n "$BREW_PREFIX" && -f "$BREW_PREFIX/opt/libomp/lib/libomp.dylib" ]]; then + LIBOMP="$BREW_PREFIX/opt/libomp" + fi + if [[ -n "$LIBOMP" ]]; then + export CXXFLAGS="${CXXFLAGS:-} -Xpreprocessor -fopenmp -I$LIBOMP/include" + export LDFLAGS="${LDFLAGS:-} -L$LIBOMP/lib" + OPENMP_FLAGS=( + "-DOpenMP_CXX_LIB_NAMES=omp" + "-DOpenMP_omp_LIBRARY=$LIBOMP/lib/libomp.dylib" + "-DOpenMP_CXX_FLAGS=-Xpreprocessor;-fopenmp" + ) + else + OPENMP_FLAGS=(-DCMAKE_DISABLE_FIND_PACKAGE_OpenMP=TRUE) + record_warning "OpenMP is unavailable, so RandBLAS will be single-threaded. Apple Clang has no OpenMP runtime; install one with 'brew install libomp' and re-run." + WANT_OPENMP=0 + fi +fi + +#============================================================================== +# Dependency discovery. An install pointed at by an environment variable is +# taken as given: the user knows something we do not, and rebuilding over it +# would be both slow and rude. +#============================================================================== +find_cmake_config() { # -> prints the config dir, or nothing + local root="$1" pkg="$2" libdir + for libdir in lib lib64 lib/x86_64-linux-gnu lib/aarch64-linux-gnu; do + if [[ -f "$root/$libdir/cmake/$pkg/${pkg}Config.cmake" ]]; then + printf '%s/%s/cmake/%s' "$root" "$libdir" "$pkg" + return 0 + fi + done + return 0 +} + +# blaspp installs are per-configuration: an ILP64 MKL build and an LP64 +# OpenBLAS build cannot share a directory, and silently reusing one for the +# other is exactly the mismatch this installer exists to prevent. +BLASPP_INSTALL="$PROJECT_DIR/install/blaspp-$BLAS_BACKEND-install" +RANDOM123_INSTALL="$PROJECT_DIR/install/Random123-install" +GTEST_INSTALL="$PROJECT_DIR/install/googletest-install" +LAPACKPP_INSTALL="$PROJECT_DIR/install/lapackpp-install" + +EXTERNAL_BLASPP=0 +EXTERNAL_RANDOM123=0 +EXTERNAL_GTEST=0 + +BLASPP_CMAKE_DIR="" +RANDOM123_DIR="" +GTEST_ROOT_DIR="" + +note "" +note "Dependency discovery:" + +if [[ -n "${BLASPP_INSTALL_DIR:-}" ]]; then + BLASPP_CMAKE_DIR="$(find_cmake_config "$BLASPP_INSTALL_DIR" blaspp)" + if [[ -n "$BLASPP_CMAKE_DIR" ]]; then + EXTERNAL_BLASPP=1 + note " [blaspp] external install: $BLASPP_INSTALL_DIR" + else + note " [blaspp] BLASPP_INSTALL_DIR is set but holds no blasppConfig.cmake; building from source." + fi +fi +if [[ -n "${RANDOM123_INSTALL_DIR:-}" && -f "$RANDOM123_INSTALL_DIR/include/Random123/philox.h" ]]; then + EXTERNAL_RANDOM123=1 + RANDOM123_DIR="$RANDOM123_INSTALL_DIR/include" + note " [Random123] external install: $RANDOM123_INSTALL_DIR" +fi +if [[ -n "${GTEST_ROOT:-}" && -f "$GTEST_ROOT/include/gtest/gtest.h" ]]; then + EXTERNAL_GTEST=1 + GTEST_ROOT_DIR="$GTEST_ROOT" + note " [GoogleTest] external install: $GTEST_ROOT" +fi + +# What we would build, and therefore what a prior install must match to be +# reused. Backend and integer width are part of the stamp precisely because +# the directory name alone cannot distinguish an ILP64 build from an LP64 one. +BLASPP_STAMP_BASE="$BLASPP_URL@$BLASPP_REF backend=$BLAS_BACKEND libs=$BLAS_LIBRARIES_ARG" +RANDOM123_STAMP="$RANDOM123_URL@$RANDOM123_REF" +GTEST_STAMP="$GTEST_URL@$GTEST_REF" + +#============================================================================== +# Step accounting. Computed up front so "[3/7]" means something. +#============================================================================== +BUILD_BLASPP=0 +BUILD_RANDOM123=0 +BUILD_GTEST=0 + +if (( ! EXTERNAL_BLASPP )); then BUILD_BLASPP=1; fi +if (( ! EXTERNAL_RANDOM123 )); then BUILD_RANDOM123=1; fi +if (( WANT_TESTS && ! EXTERNAL_GTEST )); then BUILD_GTEST=1; fi + +if (( FRESH )); then + rm -rf "$PROJECT_DIR/build" + mkdir -p "$PROJECT_DIR/build" +fi + +# Per dependency: BLAS++ spends three steps (source, configure, build), which +# the reuse path also spends as skips so the counter agrees; Random123 one; +# GoogleTest two. Then RandBLAS configure, RandBLAS build, and verification. +# Examples add lapackpp source, lapackpp build, examples configure, examples +# build. Keep these in step with the run_step calls below -- a counter that +# reads "[9/7]" is worse than no counter. +TOTAL_STEPS=$(( BUILD_BLASPP * 3 + BUILD_RANDOM123 + BUILD_GTEST * 2 + 3 )) +if (( WANT_EXAMPLES )); then TOTAL_STEPS=$(( TOTAL_STEPS + 4 )); fi +note "" +#============================================================================== +# BLAS++. +# +# The integer width is settled here, by trying to configure BLAS++ at each +# width in WIDTH_ORDER until one succeeds. Letting BLAS++ do the searching is +# deliberate: it already knows the library names, symbol suffixes and probe +# programs for every vendor, and a second implementation here would be a worse +# copy that drifts. +# +# Each attempt gets a clean build directory. BLAS++ caches its detection +# results, and re-running cmake over a directory where detection previously +# failed regenerates blas/defines.h without the Fortran-mangling and backend +# defines, which then breaks every downstream compile in a way that looks +# nothing like the original failure. +#============================================================================== +BLAS_INT_RESOLVED="" +BLASPP_SRC="$PROJECT_DIR/lib/blaspp" + +configure_blaspp_at_width() { # -> 0 on success + local width="$1" + local build="$PROJECT_DIR/build/blaspp-build-$width" + rm -rf "$build" + mkdir -p "$build" + cmake -S "$BLASPP_SRC" -B "$build" \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX="$BLASPP_INSTALL" \ + -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON \ + -Dblas_int="$width" \ + -Dbuild_tests=OFF \ + "${BLASPP_BACKEND_FLAGS[@]}" \ + "${OPENMP_FLAGS[@]}" >> "$LOG" 2>&1 +} + +if (( BUILD_BLASPP )); then + # Reuse only when the recorded provenance matches what we would build now, + # including the resolved width, which is stored beside the stamp. + PRIOR_WIDTH="$(cat "$BLASPP_INSTALL/.randblas-width" 2>/dev/null || true)" + REUSABLE=0 + if (( ! FRESH )) && [[ -n "$PRIOR_WIDTH" ]] && \ + stamp_matches "$BLASPP_INSTALL" "$BLASPP_STAMP_BASE width=$PRIOR_WIDTH"; then + # A prior build is only reusable if its width is one we would accept. + for w in "${WIDTH_ORDER[@]}"; do + if [[ "$w" == "$PRIOR_WIDTH" ]]; then REUSABLE=1; break; fi + done + if (( ! REUSABLE )); then + note " [blaspp] existing install is $PRIOR_WIDTH but this run wants ${WIDTH_ORDER[0]}; rebuilding." + fi + fi + + if (( REUSABLE )); then + BLAS_INT_RESOLVED="$PRIOR_WIDTH" + BLASPP_CMAKE_DIR="$(find_cmake_config "$BLASPP_INSTALL" blaspp)" + # Three skips, matching the three steps the build path below spends, so + # the step counter reads the same either way. + skip_step "BLAS++ source ... already present" + skip_step "BLAS++ ... reusing the $BLAS_INT_RESOLVED install" + skip_step "BLAS++ ... already built" + else + if source_is_current "$BLASPP_SRC" "$BLASPP_URL" "$BLASPP_REF"; then + skip_step "BLAS++ source ... already at $BLASPP_REF" + else + run_step "Fetching BLAS++ ($BLASPP_REF)" \ + clone_pinned "$BLASPP_URL" "$BLASPP_SRC" "$BLASPP_REF" + fi + + STEP=$((STEP + 1)) + printf '%s[%d/%d]%s Configuring BLAS++ ... ' "$C_BOLD" "$STEP" "$TOTAL_STEPS" "$C_OFF" + for width in "${WIDTH_ORDER[@]}"; do + if configure_blaspp_at_width "$width"; then + BLAS_INT_RESOLVED="$width" + break + fi + done + if [[ -z "$BLAS_INT_RESOLVED" ]]; then + printf '%sFAILED%s\n' "$C_ERR" "$C_OFF" >&2 + printf '\nBLAS++ could not find a usable %s BLAS at any of: %s\n' \ + "$BLAS_BACKEND" "${WIDTH_ORDER[*]}" >&2 + printf 'Full output: %s\n' "$LOG" >&2 + case "$BLAS_BACKEND" in + openblas) printf '\n sudo apt install libopenblas64-dev # ILP64, Debian/Ubuntu\n sudo apt install libopenblas-dev # LP64\n' >&2 ;; + mkl) printf '\n Set MKLROOT, or source the oneAPI setvars script.\n' >&2 ;; + esac + exit 1 + fi + printf '%sdone%s (requested %s)\n' "$C_OK" "$C_OFF" "$BLAS_INT_RESOLVED" + + run_step "Building and installing BLAS++" \ + cmake --build "$PROJECT_DIR/build/blaspp-build-$BLAS_INT_RESOLVED" \ + -j "$JOBS" --target install + + # The width actually built, read back from BLAS++'s own generated + # header rather than inferred from which configure attempt succeeded. + # + # Those two can differ. BLAS++ probes int32 first and int64 second + # (BLASFinder.cmake), while blas_int only filters which *library names* + # to consider. For MKL that is enough, because mkl_intel_lp64 and + # mkl_intel_ilp64 are different libraries. For OpenBLAS there is only + # -lopenblas, so asking for int64 and getting a configure success tells + # us nothing -- an LP64 OpenBLAS passes the int32 probe and is accepted. + # Trusting the request here would stamp an LP64 install as ILP64, and + # every later run would reuse it believing it had ILP64. + if grep -q '^#define BLAS_ILP64' "$BLASPP_INSTALL/include/blas/defines.h" 2>/dev/null; then + BLAS_INT_BUILT="int64" + else + BLAS_INT_BUILT="int32" + fi + if [[ "$BLAS_INT_BUILT" != "$BLAS_INT_RESOLVED" ]]; then + note " [blaspp] requested $BLAS_INT_RESOLVED, BLAS++ selected $BLAS_INT_BUILT" + fi + BLAS_INT_RESOLVED="$BLAS_INT_BUILT" + + if [[ "$BLAS_INT_RESOLVED" == "int32" && "${WIDTH_ORDER[0]}" == "int64" ]]; then + record_warning "No ILP64 $BLAS_BACKEND was available, so BLAS++ was built LP64 (32-bit BLAS integers). RandBLAS works either way, but individual BLAS dimensions are then capped at 2^31 and the MKL sparse path is unavailable. For an ILP64 OpenBLAS, install one (on Debian or Ubuntu: libopenblas64-dev) and pass its library explicitly with --blas=custom --blas-libraries=... --blas-int=ilp64, because BLAS++ only ever looks for plain -lopenblas." + fi + + write_stamp "$BLASPP_INSTALL" "$BLASPP_STAMP_BASE width=$BLAS_INT_RESOLVED" + printf '%s\n' "$BLAS_INT_RESOLVED" > "$BLASPP_INSTALL/.randblas-width" + BLASPP_CMAKE_DIR="$(find_cmake_config "$BLASPP_INSTALL" blaspp)" + fi +fi + +[[ -n "$BLASPP_CMAKE_DIR" ]] || die "BLAS++ was installed but blasppConfig.cmake could not be located under $BLASPP_INSTALL" + +#============================================================================== +# Random123. Header-only: fetch and copy, nothing to build. +#============================================================================== +if (( BUILD_RANDOM123 )); then + if (( ! FRESH )) && stamp_matches "$RANDOM123_INSTALL" "$RANDOM123_STAMP"; then + RANDOM123_DIR="$RANDOM123_INSTALL/include" + skip_step "Random123 ... reusing existing install" + else + install_random123() { + local src="$PROJECT_DIR/lib/Random123" + clone_pinned "$RANDOM123_URL" "$src" "$RANDOM123_REF" + rm -rf "$RANDOM123_INSTALL/include/Random123" + mkdir -p "$RANDOM123_INSTALL/include" + cp -r "$src/include/Random123" "$RANDOM123_INSTALL/include/Random123" + } + run_step "Fetching Random123 ($RANDOM123_REF)" install_random123 + write_stamp "$RANDOM123_INSTALL" "$RANDOM123_STAMP" + RANDOM123_DIR="$RANDOM123_INSTALL/include" + fi +fi + +#============================================================================== +# GoogleTest. +# +# Provisioned rather than assumed: RandBLAS defaults BUILD_TESTS to ON while +# its find_package(GTest) is not REQUIRED, so a machine without GoogleTest +# produces a build with zero tests that otherwise looks like a success. +#============================================================================== +if (( BUILD_GTEST )); then + if (( ! FRESH )) && stamp_matches "$GTEST_INSTALL" "$GTEST_STAMP"; then + GTEST_ROOT_DIR="$GTEST_INSTALL" + skip_step "GoogleTest ... reusing existing install" + skip_step "GoogleTest ... already built" + else + GTEST_SRC="$PROJECT_DIR/lib/googletest" + run_step "Fetching GoogleTest ($GTEST_REF)" \ + clone_pinned "$GTEST_URL" "$GTEST_SRC" "$GTEST_REF" + run_step "Building and installing GoogleTest" \ + bash -c 'cmake -S "$1" -B "$2" -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX="$3" -DBUILD_GMOCK=OFF -DINSTALL_GTEST=ON \ + && cmake --build "$2" -j "$4" --target install' _ \ + "$GTEST_SRC" "$PROJECT_DIR/build/googletest-build" "$GTEST_INSTALL" "$JOBS" + write_stamp "$GTEST_INSTALL" "$GTEST_STAMP" + GTEST_ROOT_DIR="$GTEST_INSTALL" + fi +fi + +#============================================================================== +# RandBLAS itself. +#============================================================================== +RANDBLAS_BUILD="$PROJECT_DIR/build/RandBLAS-build" +mkdir -p "$RANDBLAS_BUILD" + +RB_ARGS=( + -S "$REPO_DIR" -B "$RANDBLAS_BUILD" + -DCMAKE_BUILD_TYPE=Release + -Dblaspp_DIR="$BLASPP_CMAKE_DIR" + -DRandom123_DIR="$RANDOM123_DIR" + -DCMAKE_INSTALL_PREFIX="$RANDBLAS_INSTALL_DIR" + -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON +) +if (( WANT_TESTS )); then + RB_ARGS+=(-DBUILD_TESTS=ON) + if [[ -n "$GTEST_ROOT_DIR" ]]; then + RB_ARGS+=(-DGTest_ROOT="$GTEST_ROOT_DIR") + fi +else + RB_ARGS+=(-DBUILD_TESTS=OFF) +fi +RB_ARGS+=("${OPENMP_FLAGS[@]}") + +run_step "Configuring RandBLAS" cmake "${RB_ARGS[@]}" +run_step "Building and installing RandBLAS" \ + cmake --build "$RANDBLAS_BUILD" -j "$JOBS" --target install + +#============================================================================== +# Verification. +# +# Compile, link and *run* a program against the freshly installed RandBLAS. +# Configuring successfully is not the same as producing something that works: +# this is what catches a BLAS that resolves at configure time but fails to +# link, a missing runtime library, and a width that is not what was asked for. +# It tests through BLAS++ rather than against raw dgemm_ because that is how +# RandBLAS actually reaches the BLAS. +#============================================================================== +CONFTEST_DIR="$PROJECT_DIR/build/conftest" +rm -rf "$CONFTEST_DIR" +mkdir -p "$CONFTEST_DIR/src" + +cat > "$CONFTEST_DIR/src/CMakeLists.txt" <<'CONFTEST_CMAKE' +cmake_minimum_required(VERSION 3.21) +project(randblas_conftest CXX) +find_package(RandBLAS REQUIRED) +add_executable(conftest conftest.cc) +target_link_libraries(conftest RandBLAS) +CONFTEST_CMAKE + +cat > "$CONFTEST_DIR/src/conftest.cc" <<'CONFTEST_CC' +// Sketch a small matrix and multiply through BLAS++, then check the numbers. +// A wrong-width or half-linked BLAS shows up here rather than in a user's +// first real run. +#include +#include +#include +#include +#include +#include + +int main() { + // BLAS++ bakes BLAS_ILP64 into its installed blas/defines.h, so this + // reports the width the headers were configured for. The blas_int typedef + // itself is not reachable from , and sizeof() on it would report + // the same header-level fact anyway -- the gemm check below is what + // actually exercises the linked library. +#if defined(BLAS_ILP64) + std::printf("blas_ilp64=1\n"); +#else + std::printf("blas_ilp64=0\n"); +#endif + + const int64_t m = 8, n = 4; + std::vector S(m * n); + RandBLAS::DenseDist D(m, n); + RandBLAS::RNGState state(0); + RandBLAS::fill_dense(D, S.data(), state); + + // C = S^T S, which must be symmetric positive semidefinite. + std::vector C(n * n, 0.0); + blas::gemm(blas::Layout::ColMajor, blas::Op::Trans, blas::Op::NoTrans, + n, n, m, 1.0, S.data(), m, S.data(), m, 0.0, C.data(), n); + + for (int64_t i = 0; i < n; ++i) { + if (!(C[i + i * n] > 0.0) || !std::isfinite(C[i + i * n])) { + std::printf("FAIL: diagonal entry %lld is %f\n", (long long)i, C[i + i * n]); + return 1; + } + for (int64_t j = 0; j < n; ++j) { + if (std::fabs(C[i + j * n] - C[j + i * n]) > 1e-12) { + std::printf("FAIL: not symmetric at (%lld,%lld)\n", (long long)i, (long long)j); + return 1; + } + } + } + std::printf("OK\n"); + return 0; +} +CONFTEST_CC + +verify_install() { + cmake -S "$CONFTEST_DIR/src" -B "$CONFTEST_DIR/build" \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_PREFIX_PATH="$RANDBLAS_INSTALL_DIR" \ + -Dblaspp_DIR="$BLASPP_CMAKE_DIR" \ + -DRandom123_DIR="$RANDOM123_DIR" \ + "${OPENMP_FLAGS[@]}" >> "$LOG" 2>&1 + cmake --build "$CONFTEST_DIR/build" -j "$JOBS" >> "$LOG" 2>&1 + "$CONFTEST_DIR/build/conftest" > "$CONFTEST_DIR/output.txt" 2>&1 + grep -q '^OK$' "$CONFTEST_DIR/output.txt" +} +run_step "Verifying the install links and runs" verify_install +cat "$CONFTEST_DIR/output.txt" >> "$LOG" + +# Read the width back from what was actually compiled, rather than trusting +# the value this script asked for. They differ whenever BLAS++ came from +# somewhere else -- BLASPP_INSTALL_DIR, a system package, a previous run -- +# and that is exactly the case worth reporting accurately. +case "$(sed -n 's/^blas_ilp64=//p' "$CONFTEST_DIR/output.txt" | head -n1)" in + 1) OBSERVED_WIDTH="ILP64 (64-bit BLAS integers)" ;; + 0) OBSERVED_WIDTH="LP64 (32-bit BLAS integers)" ;; + *) OBSERVED_WIDTH="unknown" ;; +esac + +#============================================================================== +# Examples. +# +# Not part of the core install, and not merely because they take time: they +# need two dependencies RandBLAS itself does not (LAPACK++ and +# fast_matrix_market), and examples/CMakeLists.txt requires OpenMP outright, +# which stock Apple Clang cannot provide. Making them a separate opt-in keeps +# a plain install from failing over something the library does not need. +#============================================================================== +build_examples() { + if [[ -n "${LAPACKPP_INSTALL_DIR:-}" ]] && \ + [[ -n "$(find_cmake_config "$LAPACKPP_INSTALL_DIR" lapackpp)" ]]; then + LAPACKPP_CMAKE_DIR="$(find_cmake_config "$LAPACKPP_INSTALL_DIR" lapackpp)" + skip_step "LAPACK++ source ... using $LAPACKPP_INSTALL_DIR" + skip_step "LAPACK++ ... reusing external install" + else + local src="$PROJECT_DIR/lib/lapackpp" + local stamp="$LAPACKPP_URL@$LAPACKPP_REF blaspp=$BLASPP_CMAKE_DIR" + if (( ! FRESH )) && stamp_matches "$LAPACKPP_INSTALL" "$stamp"; then + skip_step "LAPACK++ source ... already present" + skip_step "LAPACK++ ... reusing existing install" + else + run_step "Fetching LAPACK++ ($LAPACKPP_REF)" \ + clone_pinned "$LAPACKPP_URL" "$src" "$LAPACKPP_REF" + run_step "Building and installing LAPACK++" \ + bash -c 'cmake -S "$1" -B "$2" -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX="$3" -Dblaspp_DIR="$4" \ + -Dbuild_tests=OFF -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON \ + && cmake --build "$2" -j "$5" --target install' _ \ + "$src" "$PROJECT_DIR/build/lapackpp-build" "$LAPACKPP_INSTALL" \ + "$BLASPP_CMAKE_DIR" "$JOBS" + write_stamp "$LAPACKPP_INSTALL" "$stamp" + fi + LAPACKPP_CMAKE_DIR="$(find_cmake_config "$LAPACKPP_INSTALL" lapackpp)" + fi + + run_step "Configuring examples" \ + cmake -S "$REPO_DIR/examples" -B "$PROJECT_DIR/build/examples-build" \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_PREFIX_PATH="$RANDBLAS_INSTALL_DIR" \ + -Dblaspp_DIR="$BLASPP_CMAKE_DIR" \ + -Dlapackpp_DIR="$LAPACKPP_CMAKE_DIR" \ + -DRandom123_DIR="$RANDOM123_DIR" \ + -DFETCHCONTENT_BASE_DIR="$PROJECT_DIR/build/fetchcontent-cache" \ + "${OPENMP_FLAGS[@]}" + run_step "Building examples" \ + cmake --build "$PROJECT_DIR/build/examples-build" -j "$JOBS" +} + +# Reproduce this invocation with --examples added, for the offer below. Every +# option that changed the result has to be carried across, or the printed +# command silently builds something different from what was just installed -- +# and for --blas=custom it would not run at all. +# +# Written as if-blocks rather than "[[ test ]] && append": under set -e a +# false test at the end of a list exits the script. +EXAMPLES_COMMAND="bash $SCRIPT_DIR/install.sh --examples --blas=$BLAS_BACKEND --project-dir=$PROJECT_DIR" +if [[ -n "$PREFIX_OVERRIDE" ]]; then + EXAMPLES_COMMAND+=" --prefix=$PREFIX_OVERRIDE" +fi +if [[ -n "$BLAS_LIBRARIES_ARG" ]]; then + EXAMPLES_COMMAND+=" --blas-libraries='$BLAS_LIBRARIES_ARG'" +fi +if [[ "$BLAS_INT_CHOICE" != "auto" ]]; then + EXAMPLES_COMMAND+=" --blas-int=$BLAS_INT_CHOICE" +fi +if (( ! WANT_TESTS )); then EXAMPLES_COMMAND+=" --no-tests"; fi +if (( ! WANT_OPENMP )); then EXAMPLES_COMMAND+=" --no-openmp"; fi + +if (( WANT_EXAMPLES )); then + build_examples +fi + +#============================================================================== +# Summary. +#============================================================================== +printf '\n%s%sRandBLAS installed successfully.%s\n\n' "$C_OK" "$C_BOLD" "$C_OFF" +printf ' Backend %s, %s\n' "$BLAS_BACKEND" "$OBSERVED_WIDTH" +printf ' OpenMP %s\n' "$( ((WANT_OPENMP)) && echo enabled || echo disabled )" +printf ' Project layout %s\n' "$PROJECT_DIR" +printf ' Installed library %s\n' "$RANDBLAS_INSTALL_DIR" +if (( WANT_EXAMPLES )); then + printf ' Examples %s\n' "$PROJECT_DIR/build/examples-build" +fi +printf ' Full build log %s\n' "$LOG" + +if (( ${#WARNINGS[@]} )); then + printf '\n%s%d warning(s) from this run:%s\n' "$C_WARN" "${#WARNINGS[@]}" "$C_OFF" + for w in "${WARNINGS[@]}"; do + printf ' - %s\n' "$w" + done +fi + +if (( WANT_TESTS )); then + printf '\n Run the test suite:\n ctest --test-dir %s\n' "$RANDBLAS_BUILD" +fi + +printf '\n Consume from CMake with:\n -DRandBLAS_DIR=%s\n' \ + "$(find_cmake_config "$RANDBLAS_INSTALL_DIR" RandBLAS)" + +# The examples offer. Interactive users get asked; everyone else gets the +# command, so the option is discoverable either way rather than living only in +# --help where nobody looks after a successful install. +if (( ! WANT_EXAMPLES )); then + printf '\n The examples are not built by default: they additionally need\n' + printf ' LAPACK++ and fast_matrix_market, and they require OpenMP.\n' + if (( INTERACTIVE )) && ask " Build them now?" n; then + TOTAL_STEPS=$(( STEP + 4 )) + printf '\n' + build_examples + printf '\n%s%sExamples built.%s\n %s\n' "$C_OK" "$C_BOLD" "$C_OFF" \ + "$PROJECT_DIR/build/examples-build" + else + printf ' To build them later, re-run with --examples:\n %s\n' "$EXAMPLES_COMMAND" + fi +fi + +printf '\n' From 58c49d57561cd6c84cc4caa181caf5185790df7c Mon Sep 17 00:00:00 2001 From: mmelnich Date: Wed, 12 Aug 2026 17:41:34 -0700 Subject: [PATCH 2/6] Fix the three failures the first CI run exposed macOS: the installer set only the OpenMP_CXX_* variables for Homebrew libomp. BLAS++'s installed config calls find_dependency(OpenMP) without restricting components, so configuring RandBLAS against it also resolves OpenMP_C and failed with "Could NOT find OpenMP_C" -- after BLAS++ itself had built cleanly, which made it look like a RandBLAS problem. Set the C variables and export CFLAGS alongside CXXFLAGS. CI, MKL lane: oneAPI's apt packages do not put MKL on the linker's search path, so BLAS++ found the headers, failed to link, and reported "BLAS library not found". Source setvars.sh and propagate the variables, which is what the installer's own error message tells a user to do. Also dropped the "try the package, else add the repository" fallback: the first attempt always failed with "Unable to locate package", so it was noise pretending to be resilience. CI, everywhere: steps piping the installer into tee reported tee's exit status rather than the installer's, so the MKL lane's genuine failure was recorded as a passing step and only surfaced two steps later as a confusing assertion failure. Added set -euo pipefail to every step that pipes. Also fetch tags on checkout. rb_version.cmake runs `git describe --tags`, and without them the version degrades to 0.0.0-0-gunknown -- which the macOS log showed being baked into the install and reported by the configuration summary. --- .github/workflows/install-script.yml | 47 +++++++++++++++++++++++----- install/install.sh | 9 ++++++ 2 files changed, 48 insertions(+), 8 deletions(-) diff --git a/.github/workflows/install-script.yml b/.github/workflows/install-script.yml index e3260a81..299b1e66 100644 --- a/.github/workflows/install-script.yml +++ b/.github/workflows/install-script.yml @@ -46,30 +46,52 @@ jobs: - uses: actions/checkout@v4 with: path: RandBLAS + # rb_version.cmake runs `git describe --tags`. Without the tags the + # version silently degrades to 0.0.0-0-gunknown, and the installed + # package and the configuration summary both report it. + fetch-depth: 0 - name: install a compiler, CMake and a BLAS run: | + set -euo pipefail export DEBIAN_FRONTEND=noninteractive sudo apt-get update -qq sudo apt-get install -qq -y g++ gfortran cmake git if [ "${{ matrix.backend }}" = "openblas" ]; then sudo apt-get install -qq -y libopenblas-dev else - sudo apt-get install -qq -y intel-oneapi-mkl-devel || \ - { wget -qO- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \ - | sudo gpg --dearmor -o /usr/share/keyrings/oneapi.gpg - echo "deb [signed-by=/usr/share/keyrings/oneapi.gpg] https://apt.repos.intel.com/oneapi all main" \ - | sudo tee /etc/apt/sources.list.d/oneAPI.list - sudo apt-get update -qq - sudo apt-get install -qq -y intel-oneapi-mkl-devel; } - echo "MKLROOT=/opt/intel/oneapi/mkl/latest" >> "$GITHUB_ENV" + wget -qO- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \ + | sudo gpg --dearmor -o /usr/share/keyrings/oneapi.gpg + echo "deb [signed-by=/usr/share/keyrings/oneapi.gpg] https://apt.repos.intel.com/oneapi all main" \ + | sudo tee /etc/apt/sources.list.d/oneAPI.list + sudo apt-get update -qq + sudo apt-get install -qq -y intel-oneapi-mkl-devel fi + # oneAPI's apt packages do not put MKL on the linker's search path; + # setvars.sh is what exports LIBRARY_PATH, LD_LIBRARY_PATH and MKLROOT. + # Without it BLAS++ finds the headers, fails to link, and reports "BLAS + # library not found" -- which is exactly the diagnosis the installer's + # error message points at, so propagate the variables into GITHUB_ENV + # rather than working around it. + - name: put oneAPI on the search path + if: matrix.backend == 'mkl' + run: | + set -euo pipefail + source /opt/intel/oneapi/setvars.sh > /dev/null + for v in MKLROOT LIBRARY_PATH LD_LIBRARY_PATH CPATH NLSPATH PKG_CONFIG_PATH; do + if [ -n "${!v:-}" ]; then echo "$v=${!v}" >> "$GITHUB_ENV"; fi + done + - name: keep a pristine checkout for the discovery test run: cp -a RandBLAS RandBLAS-discovery + # pipefail matters here: without it the pipeline reports tee's exit + # status, so a failing installer looks like a passing step and the + # assertions below are the first thing to notice. - name: run the installer, capturing output for the escape-sequence check run: | + set -euo pipefail bash RandBLAS/install/install.sh --yes \ --blas=${{ matrix.backend }} \ --project-dir "$GITHUB_WORKSPACE/RandNLA-project" \ @@ -108,6 +130,7 @@ jobs: - name: re-run the installer in place (idempotency) run: | + set -euo pipefail bash RandBLAS/install/install.sh --yes \ --blas=${{ matrix.backend }} \ --project-dir "$GITHUB_WORKSPACE/RandNLA-project" | tee rerun.out @@ -116,6 +139,7 @@ jobs: - name: install a second project through dependency discovery run: | + set -euo pipefail BLASPP_INSTALL_DIR="$GITHUB_WORKSPACE/RandNLA-project/install/blaspp-${{ matrix.backend }}-install" \ RANDOM123_INSTALL_DIR="$GITHUB_WORKSPACE/RandNLA-project/install/Random123-install" \ GTEST_ROOT="$GITHUB_WORKSPACE/RandNLA-project/install/googletest-install" \ @@ -133,6 +157,10 @@ jobs: - uses: actions/checkout@v4 with: path: RandBLAS + # rb_version.cmake runs `git describe --tags`. Without the tags the + # version silently degrades to 0.0.0-0-gunknown, and the installed + # package and the configuration summary both report it. + fetch-depth: 0 - name: install libomp run: brew install libomp @@ -141,6 +169,7 @@ jobs: # only Apple's legacy interface, so there is no ILP64 lane to run here. - name: run the installer run: | + set -euo pipefail bash RandBLAS/install/install.sh --yes \ --project-dir "$GITHUB_WORKSPACE/RandNLA-project" | tee installer.out grep -qE '^ Backend *accelerate' installer.out @@ -170,6 +199,8 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 + with: + fetch-depth: 0 - name: install dependencies the way a package manager would run: | diff --git a/install/install.sh b/install/install.sh index 86771a4c..d6030c3a 100755 --- a/install/install.sh +++ b/install/install.sh @@ -434,11 +434,20 @@ elif [[ "$UNAME_S" == "Darwin" ]]; then LIBOMP="$BREW_PREFIX/opt/libomp" fi if [[ -n "$LIBOMP" ]]; then + export CFLAGS="${CFLAGS:-} -Xpreprocessor -fopenmp -I$LIBOMP/include" export CXXFLAGS="${CXXFLAGS:-} -Xpreprocessor -fopenmp -I$LIBOMP/include" export LDFLAGS="${LDFLAGS:-} -L$LIBOMP/lib" + # Both the C and CXX components must be described. BLAS++'s installed + # blasppConfig.cmake does find_dependency(OpenMP) without restricting + # components, so a consumer configuring against it resolves OpenMP_C + # as well -- and with only the CXX variables set, that fails with + # "Could NOT find OpenMP_C" while configuring RandBLAS, long after + # BLAS++ itself built cleanly. OPENMP_FLAGS=( + "-DOpenMP_C_LIB_NAMES=omp" "-DOpenMP_CXX_LIB_NAMES=omp" "-DOpenMP_omp_LIBRARY=$LIBOMP/lib/libomp.dylib" + "-DOpenMP_C_FLAGS=-Xpreprocessor;-fopenmp" "-DOpenMP_CXX_FLAGS=-Xpreprocessor;-fopenmp" ) else From 8598d1e39f3f5a21f787402d7312240cb62ef8e5 Mon Sep 17 00:00:00 2001 From: mmelnich Date: Wed, 12 Aug 2026 17:46:52 -0700 Subject: [PATCH 3/6] CI: fix the two environment failures in the installer lanes linux-openblas and macos-accelerate now pass, which confirms the OpenMP_C fix and the LP64-fallback assertions. The two remaining failures were both in the workflow rather than the installer. MKL lane: "set -u" and oneAPI's setvars.sh are incompatible. Its vars.sh reads OCL_ICD_FILENAMES and other variables without a default, so nounset made sourcing it fail with "unbound variable" before it exported anything. Enable nounset after the source instead. Packager lane: GitHub's runners restrict unprivileged user namespaces, so "unshare -rn" fails with "write failed /proc/self/uid_map: Operation not permitted" -- it works locally, which is how it got written that way. Use "sudo unshare -n" and hand the resulting build tree back to the runner user, since otherwise the root-owned tree breaks the non-root build step after it. --- .github/workflows/install-script.yml | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/.github/workflows/install-script.yml b/.github/workflows/install-script.yml index 299b1e66..67c61623 100644 --- a/.github/workflows/install-script.yml +++ b/.github/workflows/install-script.yml @@ -77,8 +77,12 @@ jobs: - name: put oneAPI on the search path if: matrix.backend == 'mkl' run: | - set -euo pipefail + set -eo pipefail + # Deliberately no "set -u" around the source: oneAPI's own vars.sh + # reads OCL_ICD_FILENAMES and other variables without a default, so + # nounset makes sourcing it fail outright. source /opt/intel/oneapi/setvars.sh > /dev/null + set -u for v in MKLROOT LIBRARY_PATH LD_LIBRARY_PATH CPATH NLSPATH PKG_CONFIG_PATH; do if [ -n "${!v:-}" ]; then echo "$v=${!v}" >> "$GITHUB_ENV"; fi done @@ -222,17 +226,22 @@ jobs: # Configure inside a network namespace with no interfaces, so any # FetchContent or git call added to the build later fails here rather - # than in a packager's sandbox months from now. "unshare -rn" rather - # than "sudo unshare -n": the -r user mapping keeps the generated build - # tree owned by the runner user, where sudo would leave it root-owned - # and break the non-root build step that follows. + # than in a packager's sandbox months from now. + # + # "sudo unshare -n" rather than the tidier unprivileged "unshare -rn": + # GitHub's runners restrict unprivileged user namespaces, so -r fails + # with "write failed /proc/self/uid_map: Operation not permitted". + # Running under sudo leaves the generated tree root-owned, so hand it + # back before the non-root build step that follows. - name: configure with plain CMake and no network run: | - unshare -rn \ + set -euo pipefail + sudo unshare -n \ cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_PREFIX_PATH="$GITHUB_WORKSPACE/deps" \ -DRandom123_DIR="$GITHUB_WORKSPACE/deps/include" \ -DCMAKE_INSTALL_PREFIX="$GITHUB_WORKSPACE/pkg" + sudo chown -R "$(id -u):$(id -g)" build - name: build, install and test run: | From 35d8e5ba2f2a97510af5f59bce148851a6e56c8e Mon Sep 17 00:00:00 2001 From: mmelnich Date: Wed, 12 Aug 2026 17:53:19 -0700 Subject: [PATCH 4/6] Add install/install.ps1, and pin the Windows dependency provisioner Completes the installer pair. install.ps1 produces the same RandNLA-project layout as install.sh, honours RANDNLA_PROJECT_DIR with the same precedence, and delegates dependency provisioning to the setup script CI already uses so there is one implementation rather than two that drift. The x64 toolchain guard is the reason this exists in the form it does. "Developer PowerShell for VS" and "Developer Command Prompt for VS" both default to an *x86* toolchain, and an x86 linker cannot use the x64 import libraries every BLAS backend ships. Left unchecked the failure surfaces three layers down as BLAS++ reporting "BLAS library not found", blaming the libraries when the compiler is at fault -- which is exactly how this was diagnosed in RandLAPACK. toolchain-arch.ps1 reads VSCMD_ARG_TGT_ARCH, then the bin\Host\\ convention, then cl.exe's banner, and refuses x86 and arm64 with different messages because they need different answers. Provisioner changes: * Off personal forks. BLAS++ and LAPACK++ came from RaphaelArkadyMeyerNYU/*; both MSVC fixes merged upstream on 2026-08-06 (icl-utk-edu/blaspp#132, icl-utk-edu/lapackpp#87), so both now come from icl-utk-edu pinned to the merge commits. * Pinned and provenance-stamped. Clone-Head took a branch name and returned early whenever the destination merely existed, so a branch tip could move between runs and changing a ref was a silent no-op for anyone who already had the directory. Random123 in particular was fetched at the default branch, unpinned. Clone-Pinned fetches one ref and records it. Also exports randblas_stage_runtime_dlls() from the installed package. Windows searches an executable's own directory first and PATH last, so a downstream project linking installed RandBLAS could not find the BLAS DLLs at run time -- the function existed only in the build tree. Found because the installer's own verification step is such a consumer and could not configure without it. Verified on Windows 11 with Windows PowerShell 5.1 and VS 2022 Build Tools: missing-prerequisite path, x86 toolchain refused at preflight under a real vcvars32 environment, and a full x64 install from scratch -- oneMKL through vcpkg, BLAS++, GoogleTest, RandBLAS -- ending with the verification program compiling, linking, staging its DLLs and running, reporting ILP64. --- .../setup-randblas-deps-windows/setup.ps1 | 66 +++- .github/scripts/windows/toolchain-arch.ps1 | 75 ++++ CMake/RandBLASConfig.cmake.in | 7 + CMake/rb_config.cmake | 9 +- install/install.ps1 | 364 ++++++++++++++++++ 5 files changed, 500 insertions(+), 21 deletions(-) create mode 100644 .github/scripts/windows/toolchain-arch.ps1 create mode 100644 install/install.ps1 diff --git a/.github/actions/setup-randblas-deps-windows/setup.ps1 b/.github/actions/setup-randblas-deps-windows/setup.ps1 index 63454e33..44c7cfbe 100644 --- a/.github/actions/setup-randblas-deps-windows/setup.ps1 +++ b/.github/actions/setup-randblas-deps-windows/setup.ps1 @@ -44,25 +44,59 @@ function Find-PackageConfigDirectory { return $config.Directory.FullName } -function Clone-Head { +# Fetch exactly one commit or tag, and record where it came from. +# +# This replaces a clone that took a branch name and returned early whenever the +# destination merely existed. Two problems with that: a branch tip moves, so +# two runs of the same script could build different source; and reuse keyed on +# presence means changing a ref is a silent no-op for anyone who already has +# the directory, so the new pin never takes effect. The stamp is needed +# because a shallow fetch of a tag does not keep the tag ref locally, so git +# cannot be asked afterwards whether a tree is at the pin. +function Clone-Pinned { param( [Parameter(Mandatory = $true)][string] $Url, [Parameter(Mandatory = $true)][string] $Destination, - [string] $Branch = "" + [Parameter(Mandatory = $true)][string] $Ref ) - if (Test-Path -LiteralPath $Destination) { + $stampPath = Join-Path $Destination ".randblas-provenance" + $stamp = "$Url@$Ref" + if ((Test-Path -LiteralPath $stampPath) -and + ((Get-Content -LiteralPath $stampPath -Raw).Trim() -eq $stamp)) { + Write-Host "Reusing $Destination (already at $Ref)" return } - $arguments = @("clone", "--depth", "1") - if ($Branch) { - $arguments += @("--branch", $Branch) + if (Test-Path -LiteralPath $Destination) { + Remove-Item -Recurse -Force -LiteralPath $Destination } - $arguments += @($Url, $Destination) - Invoke-Checked -Program "git" -Arguments $arguments + New-Item -ItemType Directory -Force -Path $Destination | Out-Null + Invoke-Checked -Program "git" -Arguments @("-C", $Destination, "init", "--quiet") + Invoke-Checked -Program "git" -Arguments @("-C", $Destination, "remote", "add", "origin", $Url) + Invoke-Checked -Program "git" -Arguments @("-C", $Destination, "fetch", "--quiet", "--depth", "1", "origin", $Ref) + Invoke-Checked -Program "git" -Arguments @("-C", $Destination, "checkout", "--quiet", "FETCH_HEAD") + Set-Content -LiteralPath $stampPath -Value $stamp -Encoding ascii } +#------------------------------------------------------------------ pins ------ +# Immutable refs only: a tag or a full commit hash, never a branch. These match +# install/install.sh and the refs RandLAPACK validated, so the two installers +# and CI cannot disagree about what they built. +# +# BLAS++ and LAPACK++ previously came from personal forks carrying one-line +# MSVC fixes. Both merged upstream on 2026-08-06 (icl-utk-edu/blaspp#132, +# icl-utk-edu/lapackpp#87), so both now come from icl-utk-edu, pinned to the +# merge commits: the latest release of each, v2025.05.28, predates the fixes. +$BlasppUrl = "https://github.com/icl-utk-edu/blaspp.git" +$BlasppRef = "30571853f980d3a2a1737124ea4789e025a5e045" +$LapackppUrl = "https://github.com/icl-utk-edu/lapackpp.git" +$LapackppRef = "40b9d0daf29b6f1f3fa58bc3f22bd6cfb2c67fe4" +$Random123Url = "https://github.com/DEShawResearch/Random123.git" +$Random123Ref = "v1.14.0" +$GTestUrl = "https://github.com/google/googletest.git" +$GTestRef = "v1.18.0" + function Export-GitHubValue { param( [Parameter(Mandatory = $true)][string] $Name, @@ -188,8 +222,7 @@ $gtestVariant = if ($SanitizeAddress) { "googletest-asan" } else { "googletest" $gtestBuild = Join-Path $DependencyRoot "$gtestVariant-build" $gtestInstall = Join-Path $DependencyRoot "$gtestVariant-install" if (-not (Test-Path -LiteralPath (Join-Path $gtestInstall "lib\cmake\GTest\GTestConfig.cmake"))) { - Clone-Head -Url "https://github.com/google/googletest.git" ` - -Destination $gtestSource -Branch "v1.17.0" + Clone-Pinned -Url $GTestUrl -Destination $gtestSource -Ref $GTestRef $gtestArguments = @( "-S", $gtestSource, "-B", $gtestBuild, @@ -215,8 +248,7 @@ $random123Source = Join-Path $DependencyRoot "Random123" $random123Install = Join-Path $DependencyRoot "Random123-install" $random123Include = Join-Path $random123Install "include" if (-not (Test-Path -LiteralPath (Join-Path $random123Include "Random123\philox.h"))) { - Clone-Head -Url "https://github.com/DEShawResearch/Random123.git" ` - -Destination $random123Source + Clone-Pinned -Url $Random123Url -Destination $random123Source -Ref $Random123Ref New-Item -ItemType Directory -Force -Path $random123Include | Out-Null Copy-Item -LiteralPath (Join-Path $random123Source "include\Random123") ` -Destination $random123Include -Recurse @@ -229,10 +261,7 @@ $blasppConfig = Get-ChildItem -LiteralPath $blasppInstall -Recurse -File ` -Filter "blasppConfig.cmake" -ErrorAction SilentlyContinue | Select-Object -First 1 if (-not $blasppConfig) { - Clone-Head ` - -Url "https://github.com/RaphaelArkadyMeyerNYU/blaspp.git" ` - -Destination $blasppSource ` - -Branch "windows-portability" + Clone-Pinned -Url $BlasppUrl -Destination $blasppSource -Ref $BlasppRef $blasLibraryArgument = ($mklLibraries | ForEach-Object { Convert-ToCMakePath $_ }) -join ";" @@ -267,10 +296,7 @@ if ($InstallLapackpp) { -Filter "lapackppConfig.cmake" -ErrorAction SilentlyContinue | Select-Object -First 1 if (-not $lapackppConfig) { - Clone-Head ` - -Url "https://github.com/RaphaelArkadyMeyerNYU/lapackpp.git" ` - -Destination $lapackppSource ` - -Branch "msvc-compatibility" + Clone-Pinned -Url $LapackppUrl -Destination $lapackppSource -Ref $LapackppRef Invoke-Checked -Program "cmake" -Arguments @( "-S", $lapackppSource, diff --git a/.github/scripts/windows/toolchain-arch.ps1 b/.github/scripts/windows/toolchain-arch.ps1 new file mode 100644 index 00000000..906be200 --- /dev/null +++ b/.github/scripts/windows/toolchain-arch.ps1 @@ -0,0 +1,75 @@ +# Toolchain architecture detection, shared by install/install.ps1 (user-facing +# preflight) and .github/actions/setup-randlapack-deps-windows/setup.ps1 (which +# also runs standalone in CI). Dot-source it; it defines functions only. +# +# Why this check exists: RandBLAS and every BLAS backend the installer +# provisions are 64-bit, but the "Developer PowerShell for VS" and "Developer +# Command Prompt for VS" Start-menu entries both default to an *x86* toolchain. +# An x86 linker cannot use an x64 import library, and the failure surfaces +# three layers down as BLAS++ reporting "BLAS library not found" -- which +# blames the libraries when the compiler is at fault. Note that the shell's own +# bitness is not a usable signal: the Developer Command Prompt is a 64-bit +# process that still selects x86 tools. + +function Get-ClTargetArchitecture { + # Returns the compiler's TARGET architecture, lowercased ("x64", "x86", + # "arm64", "arm"), or "" if it genuinely cannot be determined. + # + # Three independent signals, most reliable first -- the same + # probe-several-things approach Find-OneMklLayout uses, and for the same + # reason: a missed detection here fails *open*, which defeats the check. + # 1. VSCMD_ARG_TGT_ARCH, exported by vcvarsall.bat / VsDevCmd (and so + # by ilammy/msvc-dev-cmd in CI). Never localized. + # 2. The toolset path: MSVC lays cl.exe out as + # ...\bin\Host\\cl.exe, a stable convention. + # 3. The banner, last, for anything matching neither of the above. + # On its own this would be wrong on a localized Visual Studio, where + # the words around the architecture are translated. + if ($env:VSCMD_ARG_TGT_ARCH) { return $env:VSCMD_ARG_TGT_ARCH.ToLowerInvariant() } + $cl = Get-Command "cl.exe" -ErrorAction SilentlyContinue + if (-not $cl) { return "" } + if ($cl.Source -match '\\bin\\Host[^\\]+\\([^\\]+)\\cl\.exe$') { + return $Matches[1].ToLowerInvariant() + } + # Native stderr merged via 2>&1 becomes ErrorRecords, which would throw + # under $ErrorActionPreference = "Stop"; relax it for this one call. + $previous = $ErrorActionPreference + $ErrorActionPreference = "Continue" + try { + $banner = (& $cl.Source 2>&1 | Out-String) + } finally { + $ErrorActionPreference = $previous + } + if ($banner -match '\bfor\s+(x64|x86|ARM64|ARM)\b') { return $Matches[1].ToLowerInvariant() } + return "" +} + +function Get-ToolchainArchitectureProblem { + # Returns a description of why $Arch is unusable, or "" if it is fine. + # x86 and ARM64 fail for completely different reasons and deserve + # different advice: x86 means the wrong shell was opened and is a + # one-command fix, ARM64 means the platform is genuinely unsupported. + param([string]$Arch) + if ($Arch -eq "" -or $Arch -eq "x64" -or $Arch -eq "amd64") { return "" } + if ($Arch -eq "x86") { + # Single-quoted: the cmd one-liner contains both double quotes and + # backticks, which are literal here but would need escaping in a + # double-quoted PowerShell string. + $vcvarsHint = 'for /f "usebackq delims=" %i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -property installationPath`) do call "%i\VC\Auxiliary\Build\vcvars64.bat"' + return ("cl.exe targets x86, but RandBLAS and its BLAS backends are 64-bit " + + "(x64).`n" + + " You are in a 32-bit developer shell. 'Developer PowerShell for VS 2022' and " + + "'Developer Command Prompt for VS 2022' both default to x86.`n" + + " Fix: open 'x64 Native Tools Command Prompt for VS 2022' from the Start menu, " + + "or run this in any Command Prompt (any edition or version):`n" + + " $vcvarsHint`n" + + " Then delete the RandNLA-project directory before retrying: dependencies already " + + "configured by the x86 compiler are reused as-is and would keep failing.") + } + return ("cl.exe targets $Arch, which this installer does not support: the Windows build " + + "is x64-only.`n" + + " Intel oneMKL publishes no $Arch build, and the OpenBLAS binaries pinned here are " + + "x64. Supplying an $Arch BLAS/LAPACK through -Backend custom is the only route, and " + + "it is untested.`n" + + " If you meant to build x64, open 'x64 Native Tools Command Prompt for VS 2022'.") +} diff --git a/CMake/RandBLASConfig.cmake.in b/CMake/RandBLASConfig.cmake.in index 3ac3f89a..4b5ee52c 100644 --- a/CMake/RandBLASConfig.cmake.in +++ b/CMake/RandBLASConfig.cmake.in @@ -50,4 +50,11 @@ endif() # MKL sparse BLAS set(RandBLAS_HAS_MKL @RandBLAS_HAS_MKL@) +# Provides randblas_stage_runtime_dlls(), which copies a target's +# imported DLL dependencies next to the executable. Windows searches the +# executable's own directory first and PATH last, so a consumer that links +# installed RandBLAS otherwise cannot find the BLAS DLLs at run time. Exported +# rather than kept build-tree-only so consumers do not each reinvent it. +include("${CMAKE_CURRENT_LIST_DIR}/RuntimeDLLs.cmake") + include(RandBLAS) diff --git a/CMake/rb_config.cmake b/CMake/rb_config.cmake index 83e2c903..ec8ae331 100644 --- a/CMake/rb_config.cmake +++ b/CMake/rb_config.cmake @@ -11,7 +11,14 @@ configure_file(CMake/RandBLASConfigVersion.cmake.in ${CMAKE_INSTALL_LIBDIR}/cmake/RandBLAS/RandBLASConfigVersion.cmake @ONLY) if (PROJECT_NAME STREQUAL "RandBLAS") - install(FILES CMake/FindRandom123.cmake + # RuntimeDLLs.cmake ships with the package because downstream Windows + # consumers need randblas_stage_runtime_dlls() as much as this project + # does: on Windows the loader searches the executable's own directory + # first and PATH last, so an executable linking installed RandBLAS has no + # way to find the BLAS DLLs unless they are staged beside it. Without + # this, the function exists only in the build tree and every consumer has + # to reinvent it. + install(FILES CMake/FindRandom123.cmake CMake/RuntimeDLLs.cmake DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/RandBLAS") endif() diff --git a/install/install.ps1 b/install/install.ps1 new file mode 100644 index 00000000..fd2a43e3 --- /dev/null +++ b/install/install.ps1 @@ -0,0 +1,364 @@ +# RandBLAS autoinstaller for native Windows (MSVC). +# +# Builds RandBLAS and the dependencies it needs into a self-contained +# "RandNLA-project" directory, the same layout install.sh produces on Linux and +# macOS: +# lib: dependency sources +# install: RandBLAS-install and the dependency installs +# build: one build directory per project above +# +# Nothing is installed system-wide, no PATH entry is created, and your +# environment is not modified unless you pass -ModifyEnvironment. +# +# You bring Visual Studio (or the Build Tools), CMake and Git, in an x64 +# developer shell. This script does not install a toolchain; when one is +# missing or wrong it says so and tells you how to fix it. +# +# Prerequisites and supported configurations are in INSTALL.md. + +[CmdletBinding()] +param( + # Where dependencies, builds and installs go. Defaults to + # $env:RANDNLA_PROJECT_DIR when set -- which is what lets this installer + # and RandLAPACK's share one dependency tree -- and otherwise to a + # RandNLA-project directory beside this clone. + [string] $ProjectDir = "", + + # Where the dependency stack lives. Defaults to \install. CI + # points this at a cache shared with the core workflow. + [string] $DependencyRoot = "", + + # Install RandBLAS itself here instead of \install\RandBLAS-install. + # Dependencies still go in the project directory. + [string] $Prefix = "", + + [int] $Jobs = 0, + [switch] $Fresh, + [switch] $SkipTests, + [switch] $Examples, + [switch] $ModifyEnvironment, + [switch] $Yes +) + +$ErrorActionPreference = "Stop" +Set-StrictMode -Version Latest + +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$repoDir = Split-Path -Parent $scriptDir + +#============================================================================== +# Toolchain preflight. +# +# The architecture guard is the important one. "Developer PowerShell for VS" +# and "Developer Command Prompt for VS" both default to an *x86* toolchain, and +# an x86 linker cannot use the x64 import libraries every BLAS backend here +# ships. Without this check the failure surfaces three layers down as BLAS++ +# reporting "BLAS library not found", which blames the libraries when the +# compiler is at fault. +#============================================================================== +. (Join-Path $repoDir ".github\scripts\windows\toolchain-arch.ps1") + +$missing = @() +foreach ($tool in @("cl.exe", "cmake.exe", "git.exe")) { + if (-not (Get-Command $tool -ErrorAction SilentlyContinue)) { $missing += $tool } +} +if ($missing.Count -gt 0) { + Write-Host "" + Write-Host "PREREQUISITE MISSING: $($missing -join ', ') not found on PATH." -ForegroundColor Red + Write-Host "" + Write-Host " RandBLAS needs Visual Studio (or the Build Tools) with the C++ workload," + Write-Host " plus CMake and Git, in an x64 developer shell." + Write-Host "" + Write-Host " Open 'x64 Native Tools Command Prompt for VS 2022' from the Start menu," + Write-Host " or run this in any Command Prompt to configure one:" + Write-Host "" + Write-Host ' for /f "usebackq delims=" %i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -property installationPath`) do call "%i\VC\Auxiliary\Build\vcvars64.bat"' + Write-Host "" + Write-Host " See INSTALL.md for the full prerequisite list." + exit 1 +} + +$arch = Get-ClTargetArchitecture +$archProblem = Get-ToolchainArchitectureProblem $arch +if ($archProblem) { + Write-Host "" + Write-Host "WRONG TOOLCHAIN ARCHITECTURE" -ForegroundColor Red + Write-Host "" + Write-Host " $archProblem" + Write-Host "" + exit 1 +} + +#============================================================================== +# Interactivity. +# +# Prompts happen only when someone is there to answer: not with -Yes, and not +# when stdin is redirected. Every question has a defensible unattended default +# so an automated run cannot hang. +#============================================================================== +$script:Interactive = -not $Yes -and -not [Console]::IsInputRedirected ` + -and [Environment]::UserInteractive + +function Read-YesNo { + param([string] $Question, [bool] $Default) + if (-not $script:Interactive) { return $Default } + $suffix = if ($Default) { "[Y/n]" } else { "[y/N]" } + while ($true) { + $reply = (Read-Host "$Question $suffix").Trim() + if ($reply -eq "") { return $Default } + if ($reply -match '^(y|yes)$') { return $true } + if ($reply -match '^(n|no)$') { return $false } + } +} + +if ($Jobs -le 0) { + $Jobs = [Environment]::ProcessorCount +} + +#============================================================================== +# Project layout. +# +# Precedence matches install.sh exactly: the flag, then RANDNLA_PROJECT_DIR, +# then a sibling of this clone. Honouring the environment variable is what +# lets a machine that has already run RandLAPACK's installer reuse its BLAS++ +# rather than building a second copy. +#============================================================================== +if (-not $ProjectDir) { + if ($env:RANDNLA_PROJECT_DIR) { + $ProjectDir = $env:RANDNLA_PROJECT_DIR + } else { + $ProjectDir = Join-Path (Split-Path $repoDir -Parent) "RandNLA-project" + } +} +$ProjectDir = [IO.Path]::GetFullPath($ProjectDir) + +# Deep dependency build trees plus MSVC's own path limits make long project +# paths fail in ways that are hard to attribute, so warn before the build +# rather than after. +if ($ProjectDir.Length -gt 150) { + Write-Warning ("ProjectDir is $($ProjectDir.Length) characters long. Deep dependency " + + "build trees may exceed Windows path limits; consider something shorter, such as C:\RandNLA.") +} + +if (-not $DependencyRoot) { $DependencyRoot = Join-Path $ProjectDir "install" } +$DependencyRoot = [IO.Path]::GetFullPath($DependencyRoot) + +$installDir = if ($Prefix) { + [IO.Path]::GetFullPath($Prefix) +} else { + Join-Path $ProjectDir "install\RandBLAS-install" +} +$buildDir = Join-Path $ProjectDir "build\RandBLAS-build" + +foreach ($d in @($ProjectDir, $DependencyRoot, (Join-Path $ProjectDir "lib"), (Join-Path $ProjectDir "build"))) { + New-Item -ItemType Directory -Force -Path $d | Out-Null +} +if ($Fresh -and (Test-Path -LiteralPath $buildDir)) { + Remove-Item -Recurse -Force -LiteralPath $buildDir +} +New-Item -ItemType Directory -Force -Path $buildDir | Out-Null + +Write-Host "" +Write-Host "RandBLAS installer" -ForegroundColor Cyan +Write-Host " toolchain x64 ($arch)" +Write-Host " project dir $ProjectDir" +Write-Host " dependencies $DependencyRoot" +Write-Host " install to $installDir" +Write-Host "" + +#============================================================================== +# Dependencies. +# +# Delegated to the same provisioner CI uses, so there is one implementation of +# "fetch oneMKL, build BLAS++, build GoogleTest" rather than two that drift. +# It pins every source to an immutable ref and records provenance, so a +# dependency is reused only when it came from what we would fetch now. +#============================================================================== +$setup = Join-Path $repoDir ".github\actions\setup-randblas-deps-windows\setup.ps1" +$setupArgs = @{ DependencyRoot = $DependencyRoot } +if ($Examples) { $setupArgs["InstallLapackpp"] = $true } + +Write-Host "[1/4] Provisioning dependencies (oneMKL, BLAS++, Random123, GoogleTest) ..." +# No $LASTEXITCODE check: setup.ps1 is a PowerShell script that sets +# $ErrorActionPreference = "Stop" and throws, so a failure propagates on its +# own. Reading $LASTEXITCODE here would be worse than redundant -- it is unset +# until some native command runs, and Set-StrictMode turns reading an unset +# variable into an error. That made the whole installer fail on exactly the +# runs where every dependency was already cached and no native command had run. +& $setup @setupArgs + +#============================================================================== +# RandBLAS. +#============================================================================== +$cmakeArgs = @( + "-S", $repoDir, + "-B", $buildDir, + "-G", "Ninja", + "-DCMAKE_BUILD_TYPE=Release", + "-DCMAKE_INSTALL_PREFIX=$($installDir.Replace('\','/'))", + "-Dblaspp_DIR=$($env:blaspp_DIR)", + "-DRandom123_DIR=$($env:Random123_DIR)" +) +if ($SkipTests) { + $cmakeArgs += "-DBUILD_TESTS=OFF" +} else { + $cmakeArgs += @("-DBUILD_TESTS=ON", "-DGTest_ROOT=$($env:googletest_PREFIX)") +} + +# Ninja is not guaranteed present outside a full Visual Studio install; fall +# back to the NMake generator the CI provisioner already uses. +if (-not (Get-Command "ninja.exe" -ErrorAction SilentlyContinue)) { + $cmakeArgs[5] = "NMake Makefiles" +} + +Write-Host "[2/4] Configuring RandBLAS ..." +& cmake @cmakeArgs +if ($LASTEXITCODE -ne 0) { throw "CMake configure failed." } + +Write-Host "[3/4] Building and installing RandBLAS ..." +& cmake --build $buildDir -j $Jobs --target install +if ($LASTEXITCODE -ne 0) { throw "Build failed." } + +#============================================================================== +# Verification. +# +# Compile, link and run a program against the finished install. Configuring is +# not the same as producing something that works: this catches a BLAS that +# resolves at configure time but fails to link, and a runtime DLL that was +# never staged beside the executable. +#============================================================================== +Write-Host "[4/4] Verifying the install links and runs ..." +$conftest = Join-Path $ProjectDir "build\conftest" +if (Test-Path -LiteralPath $conftest) { Remove-Item -Recurse -Force -LiteralPath $conftest } +New-Item -ItemType Directory -Force -Path (Join-Path $conftest "src") | Out-Null + +Set-Content -Path (Join-Path $conftest "src\CMakeLists.txt") -Encoding ascii -Value @( + 'cmake_minimum_required(VERSION 3.21)', + 'project(randblas_conftest CXX)', + 'find_package(RandBLAS REQUIRED)', + 'add_executable(conftest conftest.cc)', + 'target_link_libraries(conftest RandBLAS)', + 'randblas_stage_runtime_dlls(conftest)') + +Set-Content -Path (Join-Path $conftest "src\conftest.cc") -Encoding ascii -Value @( + '#include ', + '#include ', + '#include ', + '#include ', + '#include ', + '#include ', + 'int main() {', + '#if defined(BLAS_ILP64)', + ' std::printf("blas_ilp64=1\n");', + '#else', + ' std::printf("blas_ilp64=0\n");', + '#endif', + ' const int64_t m = 8, n = 4;', + ' std::vector S(m * n);', + ' RandBLAS::DenseDist D(m, n);', + ' RandBLAS::RNGState state(0);', + ' RandBLAS::fill_dense(D, S.data(), state);', + ' std::vector C(n * n, 0.0);', + ' blas::gemm(blas::Layout::ColMajor, blas::Op::Trans, blas::Op::NoTrans,', + ' n, n, m, 1.0, S.data(), m, S.data(), m, 0.0, C.data(), n);', + ' for (int64_t i = 0; i < n; ++i) {', + ' if (!(C[i + i * n] > 0.0) || !std::isfinite(C[i + i * n])) return 1;', + ' }', + ' std::printf("OK\n");', + ' return 0;', + '}') + +$conftestGenerator = if (Get-Command "ninja.exe" -ErrorAction SilentlyContinue) { "Ninja" } else { "NMake Makefiles" } +& cmake -S (Join-Path $conftest "src") -B (Join-Path $conftest "build") -G $conftestGenerator ` + "-DCMAKE_BUILD_TYPE=Release" ` + "-DCMAKE_PREFIX_PATH=$($installDir.Replace('\','/'))" ` + "-Dblaspp_DIR=$($env:blaspp_DIR)" ` + "-DRandom123_DIR=$($env:Random123_DIR)" | Out-Host +if ($LASTEXITCODE -ne 0) { throw "The verification program failed to configure." } +& cmake --build (Join-Path $conftest "build") | Out-Host +if ($LASTEXITCODE -ne 0) { throw "The verification program failed to build." } + +$conftestExe = Get-ChildItem -LiteralPath (Join-Path $conftest "build") -Recurse -Filter "conftest.exe" | + Select-Object -First 1 +if (-not $conftestExe) { throw "The verification program built but produced no executable." } +$conftestOutput = & $conftestExe.FullName +if ($LASTEXITCODE -ne 0 -or ($conftestOutput -notcontains "OK")) { + throw "The verification program ran but did not produce a correct result:`n$($conftestOutput -join "`n")" +} +$observedWidth = if ($conftestOutput -contains "blas_ilp64=1") { + "ILP64 (64-bit BLAS integers)" +} else { + "LP64 (32-bit BLAS integers)" +} + +#============================================================================== +# Optional: persist RANDNLA_PROJECT_DIR. +# +# Opt-in, mirroring install.sh's --modify-rc. SetEnvironmentVariable at User +# scope is the Windows equivalent of appending to a shell profile, and the only +# mechanism that survives a new shell. +#============================================================================== +if ($ModifyEnvironment) { + [Environment]::SetEnvironmentVariable("RANDNLA_PROJECT_DIR", $ProjectDir, "User") + Write-Host "" + Write-Host "Set RANDNLA_PROJECT_DIR=$ProjectDir for your user account (open a new shell to pick it up)." +} + +#============================================================================== +# Summary. +#============================================================================== +Write-Host "" +Write-Host "RandBLAS installed successfully." -ForegroundColor Green +Write-Host "" +Write-Host " Backend oneMKL, $observedWidth" +Write-Host " Project layout $ProjectDir" +Write-Host " Installed library $installDir" +Write-Host "" +if (-not $SkipTests) { + Write-Host " Run the test suite:" + Write-Host " ctest --test-dir $buildDir" + Write-Host "" +} +Write-Host " Consume from CMake with:" +Write-Host " -DRandBLAS_DIR=$($installDir.Replace('\','/'))/lib/cmake/RandBLAS" +if (-not $ModifyEnvironment) { + Write-Host "" + Write-Host " To have other RandNLA installers reuse these dependencies, set:" + Write-Host " setx RANDNLA_PROJECT_DIR `"$ProjectDir`"" + Write-Host " (or re-run with -ModifyEnvironment)" +} + +if (-not $Examples) { + Write-Host "" + Write-Host " The examples are not built by default: they additionally need" + Write-Host " LAPACK++ and fast_matrix_market, and they require OpenMP." + $buildNow = Read-YesNo " Build them now?" $false + if (-not $buildNow) { + Write-Host " To build them later, re-run with -Examples:" + Write-Host " powershell -ExecutionPolicy Bypass -File $scriptDir\install.ps1 -Examples -ProjectDir `"$ProjectDir`"" + Write-Host "" + exit 0 + } + $Examples = $true + & $setup -DependencyRoot $DependencyRoot -InstallLapackpp +} + +if ($Examples) { + $examplesBuild = Join-Path $ProjectDir "build\examples-build" + Write-Host "" + Write-Host "Configuring and building examples ..." + & cmake -S (Join-Path $repoDir "examples") -B $examplesBuild -G $conftestGenerator ` + "-DCMAKE_BUILD_TYPE=Release" ` + "-DCMAKE_PREFIX_PATH=$($installDir.Replace('\','/'))" ` + "-Dblaspp_DIR=$($env:blaspp_DIR)" ` + "-Dlapackpp_DIR=$($env:lapackpp_DIR)" ` + "-DRandom123_DIR=$($env:Random123_DIR)" ` + "-DFETCHCONTENT_BASE_DIR=$($ProjectDir.Replace('\','/'))/build/fetchcontent-cache" | Out-Host + if ($LASTEXITCODE -ne 0) { throw "Examples failed to configure." } + & cmake --build $examplesBuild -j $Jobs | Out-Host + if ($LASTEXITCODE -ne 0) { throw "Examples failed to build." } + Write-Host "" + Write-Host "Examples built: $examplesBuild" -ForegroundColor Green +} + +Write-Host "" From 95d6c47398f4c4a8bbee98781dd863c85361bff1 Mon Sep 17 00:00:00 2001 From: mmelnich Date: Wed, 12 Aug 2026 17:54:03 -0700 Subject: [PATCH 5/6] INSTALL.md: document the installers, the tested configurations, and packaging Three additions and one correction. A quick-start section at the top, because the installers are now the shortest path to a working RandBLAS and the guide opened by describing the manual one. It states the toolchain contract plainly -- you bring a C++20 compiler, CMake 3.21 and Git, the script supplies everything above that -- and covers sharing a dependency tree with RandLAPACK through RANDNLA_PROJECT_DIR, and why examples are opt-in. Appendix B, a tested-configuration table. Every row is a CI lane, so it says what is actually exercised rather than what we believe should work. It also explains the integer-width policy and, in particular, why OpenBLAS is the awkward case: BLAS++ probes int32 before int64 and blas_int only filters library names, so for MKL the choice is real and verifiable while for OpenBLAS an LP64 build passes the int32 probe and is accepted. Anyone with an ILP64 OpenBLAS has to point at it explicitly, and now the documentation says so. Appendix B also carries the packaging contract for conda-forge and Spack: which dependencies are actually required, that nothing is downloaded during configure, that LP64 is the default and therefore agrees with conda-forge's libblas metapackage, that RandBLAS never selects a BLAS itself, and that the installed package relocates correctly. Correction: the guide said RandBLAS selects /openmp:experimental under MSVC. It selects /openmp:llvm, and has since #184. --- INSTALL.md | 164 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 161 insertions(+), 3 deletions(-) diff --git a/INSTALL.md b/INSTALL.md index 44636ed0..636a48ec 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,7 +1,73 @@ # Installing and using RandBLAS -This guide has four main sections and a native-Windows appendix. +## Quick start: the installer scripts + +If you just want a working RandBLAS, run the installer for your platform. It +builds RandBLAS and every dependency it needs into a self-contained +`RandNLA-project` directory beside your clone, installs nothing system-wide, +and does not touch your shell configuration. + +```bash +bash install/install.sh # Linux and macOS +``` + +```powershell +powershell -ExecutionPolicy Bypass -File install\install.ps1 # Windows +``` + +**You supply the toolchain; the installer supplies everything above it.** You +need a C++20 compiler, CMake 3.21 or later, and Git — on Windows, in an *x64* +developer shell. The script does not install compilers or package managers. +When something is missing it says so and points at the usual way to get it. + +Useful options, common to both scripts: + +| Option | Effect | +|---|---| +| `--blas=` / `-Backend` | `auto`, `openblas`, `mkl`, `accelerate`, `custom` | +| `--project-dir=` / `-ProjectDir` | where dependencies, builds and installs go | +| `--prefix=` / `-Prefix` | install RandBLAS itself somewhere else | +| `--examples` / `-Examples` | also build `examples/` (see below) | +| `--fresh`, `--no-tests`, `-j N` | rebuild from scratch, skip GoogleTest, set parallelism | +| `--yes` / `-Yes` | never prompt; also the behavior when stdin is redirected | + +Run with `--help` for the full list. Every option has an environment-variable +equivalent, and already-installed dependencies are reused when you point at +them with `BLASPP_INSTALL_DIR`, `RANDOM123_INSTALL_DIR` or `GTEST_ROOT`. + +### Sharing dependencies with RandLAPACK + +Both installers use the same `RandNLA-project` layout and both honour +`RANDNLA_PROJECT_DIR`. Set it once and whichever installer runs second reuses +the first one's BLAS++ instead of building a second copy: + +```bash +export RANDNLA_PROJECT_DIR=$HOME/RandNLA-project # Linux, macOS +setx RANDNLA_PROJECT_DIR C:\RandNLA-project # Windows +``` + +A dependency is reused only when it was built from the same source *and* in a +compatible configuration; a BLAS++ built for a different backend or integer +width is rebuilt rather than silently reused. + +### Examples are opt-in + +`examples/` is not built by default. It needs two dependencies RandBLAS itself +does not — LAPACK++ and `fast_matrix_market` — and it requires OpenMP, which +stock Apple Clang does not provide. The installer offers to build them when it +finishes, and prints the exact command to do it later. + +### Building without the installer + +The installer is a convenience, never a requirement. Everything it does is +reproducible with plain CMake and pre-installed dependencies, which is what +sections 1 through 3 describe and what packagers should follow. See +**Appendix B** for the packaging contract. + +--- + +The rest of this guide has four main sections and two appendices. Sections 1 through 3 describe how to build and install RandBLAS using CMake. @@ -9,6 +75,9 @@ Section 4 explains how to use RandBLAS in other CMake projects. Appendix A follows the same general flow for a native Windows build with MSVC. +Appendix B lists the configurations we test, and what a conda-forge or Spack +recipe needs to know. + If you want a TL;DR version of this guide, refer to one of the following. * Our GitHub Actions to [workflow files](https://github.com/BallisticLA/RandBLAS/tree/main/.github/workflows). * The [examples folder](https://github.com/BallisticLA/RandBLAS/tree/main/examples). @@ -291,8 +360,11 @@ cmake --build C:/randblas-work/build/googletest --target install ``` MSVC supplies OpenMP support. RandBLAS's CMake configuration automatically -selects `/openmp:experimental` under MSVC because its sparse kernels use -`#pragma omp simd`. No OpenMP flag needs to be added manually. +selects `/openmp:llvm` under MSVC. The classic `/openmp` mode implements only +OpenMP 2.0, which rejects the `omp simd` directive the sparse kernels use, as +well as 64-bit loop indices and the `collapse` clause downstream consumers +such as RandLAPACK rely on. No OpenMP flag needs to be added manually; to +choose a different mode, set `-DOpenMP_CXX_FLAGS=...` at configure time. OpenMP is optional. To request a serial build explicitly, add `-DCMAKE_DISABLE_FIND_PACKAGE_OpenMP=TRUE` to the RandBLAS configuration @@ -385,3 +457,89 @@ set "PATH=C:\randblas-work\vcpkg-installed\x64-windows\bin;C:\randblas-work\inst C:\path\to\my_randblas_project-build\myexec.exe ``` + + +## Appendix B. Tested configurations, integer width, and packaging + +### B.1. What we test + +Every row below is a lane in CI, so this table is a statement about what is +actually exercised on every commit rather than what we believe should work. +Anything not listed may well work; it is simply untested. + +| OS | Compiler | BLAS backend | Integer width | OpenMP | Notes | +|---|---|---|---|---|---| +| Ubuntu (latest) | gcc | OpenBLAS | LP64 | yes | release, debug+ASan, release+UBSan | +| Ubuntu (latest) | gcc | oneMKL | ILP64 | yes | enables the MKL sparse path | +| Ubuntu (latest) | clang | OpenBLAS | LP64 | yes | release, ASan, TSan | +| macOS 14 | Apple Clang | Accelerate | LP64 | **no** | Apple Clang ships no OpenMP runtime | +| macOS 15 | Homebrew LLVM | Accelerate | LP64 | yes | via Homebrew `libomp` | +| Windows | MSVC | oneMKL | ILP64 | yes (`/openmp:llvm`) | x64 only | + +Compiler floor: RandBLAS uses C++20 [concepts](https://en.cppreference.com/w/cpp/language/constraints), +which in practice means **gcc ≥ 13**. Older gcc will not compile it. CMake +3.21 or later is required on every platform. + +The installer lanes additionally cover a fresh install, an idempotent re-run, +dependency discovery, and a build performed with plain CMake and no network. + +### B.2. Integer width: which BLAS you get, and why + +RandBLAS's own API is `int64_t` regardless of the BLAS underneath, because +BLAS++ presents `int64_t` either way. The width of the *underlying* BLAS still +matters in two places: an LP64 BLAS caps each individual matrix dimension at +2³¹, and the MKL sparse path requires `MKL_INT` to match RandBLAS's `int64_t` +sparse indices. + +The installer therefore **prefers ILP64 wherever the backend can genuinely +provide it, and falls back to LP64 with a warning where it cannot**: + +| Backend | Width | Why | +|---|---|---| +| oneMKL | ILP64 | `mkl_intel_ilp64` is a distinct library, so the choice is real and verifiable | +| OpenBLAS | LP64 | see below | +| Accelerate | LP64 | BLAS++ implements only Apple's legacy interface ([lapackpp#43](https://github.com/icl-utk-edu/lapackpp/issues/43)) | + +**OpenBLAS is the subtle one.** BLAS++ probes `int32` before `int64` and uses +`blas_int` only to filter library *names*. For MKL that is enough. For +OpenBLAS there is only ever `-lopenblas`, so an LP64 build passes the `int32` +probe and is accepted — a successful `blas_int=int64` configure proves +nothing. If you have an ILP64 OpenBLAS (on Debian or Ubuntu, +`libopenblas64-dev`), point at it explicitly rather than hoping it is found: + +```bash +bash install/install.sh --blas=custom --blas-int=ilp64 \ + --blas-libraries=/usr/lib/x86_64-linux-gnu/libopenblas64.so +``` + +The installer reports the width it actually built, read back from BLAS++'s +generated `blas/defines.h` rather than from what was requested, and the CMake +configuration summary reports the same. + +### B.3. Packaging with conda-forge or Spack + +Neither ecosystem runs install scripts. Both configure with CMake against +dependencies they installed themselves, often with no network available. That +path is tested on every commit by the `linux-plain-cmake-offline` lane, which +configures inside a network namespace with no interfaces. + +What a recipe needs to know: + +- **Dependencies are `blaspp` and `Random123`.** LAPACK++ is needed only for + `examples/`, GoogleTest only for the test suite. +- **Nothing is downloaded during configure.** `examples/` is a standalone + `project()` and is the only place using `FetchContent`, so a packager never + reaches it. +- **Default to LP64.** conda-forge's `libblas` metapackage — the mechanism + that lets a user swap BLAS implementations at runtime — is LP64, and this is + RandBLAS's default for every backend except MKL, so the two agree. +- **RandBLAS never selects a BLAS itself.** It reaches the BLAS only through + BLAS++ and never calls `find_package(BLAS)`, so `BLA_VENDOR` and the choice + of implementation stay entirely with the packager. +- **Pass `-DBUILD_TESTS=OFF`** unless you are running the suite; it defaults + to ON, and without GoogleTest that silently produces a build with zero tests. + The configuration summary warns when this happens. +- **The installed package is relocatable.** It records the dependency paths + used at build time, but CMake falls back to a normal `CMAKE_PREFIX_PATH` + search when those paths do not exist, so a package built in one prefix and + consumed from another resolves correctly. From b97044c7897940df9ab678da30e94775e22926aa Mon Sep 17 00:00:00 2001 From: mmelnich Date: Wed, 12 Aug 2026 18:09:36 -0700 Subject: [PATCH 6/6] install.sh: draw a real progress bar during the build steps Three rendering tiers, chosen by probing the terminal: 2 a terminal with colour and UTF-8: a bar redrawn in place, block glyphs 1 a terminal without one of those: the same bar in ASCII 0 not a terminal: one line per step, no escapes, no carriage returns Tier 0 is a requirement rather than a fallback. Redirected output becomes install.log, CI transcripts and bug reports, and control characters make all three unreadable, so run_build_step falls straight through to run_step there and the bytes are identical to before this change. CI already asserts that redirected output contains no escape sequence and no carriage return. The bar is determinate. Both build tools already report progress on the stream being captured anyway -- Ninja writes "[12/34]" and Make writes "[ 42%]" -- so it tracks real work rather than elapsed time. A spinner would have conveyed nothing. Steps with no parseable progress keep their existing one-line form rather than growing a fake bar. Implementation note: the parsing loop necessarily runs in a subshell, so the build command's exit status is passed back through a file rather than a variable, and checked explicitly. Verified: redirected output byte-identical and escape-free; under a pty the bar advances through 5%, 20%, 49% during the RandBLAS build and each step resolves to a "done" line. --- install/install.sh | 106 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 103 insertions(+), 3 deletions(-) diff --git a/install/install.sh b/install/install.sh index d6030c3a..4cf4c869 100755 --- a/install/install.sh +++ b/install/install.sh @@ -170,6 +170,44 @@ else C_OK=""; C_ERR=""; C_WARN=""; C_BOLD=""; C_OFF="" fi +# Progress rendering tier. +# 2 a terminal that can draw: redraw a bar in place, with block characters +# 1 a terminal without colour or UTF-8: same bar, ASCII, still redrawn +# 0 not a terminal: one line per step, no escapes, no carriage returns +# +# Tier 0 is not a fallback, it is a requirement. Redirected output ends up in +# install.log, in CI transcripts and in bug reports, and control characters +# make all three unreadable. CI asserts that redirected output contains no +# escape sequence and no carriage return. +PROGRESS_TIER=0 +if [[ -t 1 && "$WANT_PROGRESS" == "1" && "${TERM:-}" != "dumb" ]]; then + if [[ -z "${NO_COLOR:-}" && "${LC_ALL:-${LC_CTYPE:-${LANG:-}}}" == *[Uu][Tt][Ff]* ]]; then + PROGRESS_TIER=2 + else + PROGRESS_TIER=1 + fi +fi + +if (( PROGRESS_TIER >= 2 )); then + BAR_FULL="━"; BAR_EMPTY="─" +else + BAR_FULL="#"; BAR_EMPTY="-" +fi + +# draw_bar