From db5e26b5983256e0723842b756e9c63a9ed60e31 Mon Sep 17 00:00:00 2001 From: mmelnich Date: Thu, 13 Aug 2026 12:34:24 -0700 Subject: [PATCH 1/4] install.sh: pins, provenance, a run-not-just-link BLAS check, and backend selection Brings the Unix installer up to the standard install.ps1 reached in #156, and in line with the RandBLAS installer. The Windows script already pinned every dependency, recorded provenance and validated the BLAS before building; this one did none of that. Eleven defects, all verified against main before changing anything: * Three `git clone` calls with no ref (:336,339,342) -- blaspp, lapackpp and random123 all tracked upstream default branches, so two runs of the same script could build different source. Now pinned to the refs this repository's own Windows provisioner already validated, fetched one commit deep, with a provenance stamp on each install and each source tree. * No BLAS validation at all. The installer now compiles, links and *runs* a program against the finished install, through BLAS++ and LAPACK++ rather than raw dgemm_ (that is how RandLAPACK reaches them), checking a gemm result and a gesdd factorization numerically. The gesdd call is deliberate. It is the routine Apple's legacy Accelerate computes incorrectly, so a broken SVD surfaces at install time rather than in someone's RSVD output. And the run-it-not-just-link-it part matters for a reason peculiar to integer width: BLAS++/LAPACK++ guard the int64_t downcast and throw rather than truncate, but that guard keys off sizeof(blas_int) as declared by the *header*. If the headers say 64-bit while the loaded library is LP64, the guard compiles out and 64-bit values reach routines reading 32 bits -- which shows up not as wrong numbers but as an absurd workspace size and a run that dies in allocation or never finishes. Nothing catches that by inspection. * /opt/homebrew hardcoded in eight places, which hard-fails on Intel macOS and any custom HOMEBREW_PREFIX. Now `brew --prefix`. * The script moved the user's own clone into /lib/RandLAPACK, breaking git worktrees. It now works in place and puts a symlink there instead, so the layout still reads as complete and the path CI invokes still resolves. * The log was truncated on every run (:211), destroying the previous run's output at exactly the moment you want it. Now appended with a run header. * The libdir search omitted lib/aarch64-linux-gnu. * No --prefix; extras and benchmarks had no way to be skipped. Both added, with extras and benchmarks still built by DEFAULT (--no-extras / --no-benchmarks to opt out): unlike RandBLAS's examples they need nothing this script has not already built, so default-on costs only time. Backend selection is new: --blas=auto|openblas|mkl|accelerate|custom, --blas-int, --blas-libraries. Integer width is requested per backend and then READ BACK from BLAS++'s generated blas/defines.h rather than assumed, because BLAS++ probes int32 before int64 while blas_int only filters library *names* -- so for MKL the request genuinely selects mkl_intel_ilp64, but for OpenBLAS there is only -lopenblas and an LP64 build passes the int32 probe and is accepted. Trusting the request would stamp LP64 installs as ILP64. macOS deliberately keeps Homebrew OpenBLAS as the default rather than moving to Accelerate. Apple's legacy Accelerate has a broken divide-and-conquer gesdd and RandLAPACK calls gesdd in rl_rsvd, rl_abrik, rl_revd2, rl_preconditioners and rl_util, so Accelerate would quietly return wrong singular values across most of the SVD-based drivers -- which is what #157's quarantine of TestQB.Polynomial_Decay_general1 is about. --blas=accelerate is allowed and warns, citing #159. The provenance stamp includes the GPU setting, and the dependency install directories now carry the backend and GPU configuration in their names (blaspp-mkl-cpu-install). Both matter, and the second one for a reason outside this project: RandBLAS's installer uses the same RandNLA-project layout and named its BLAS++ install identically. Sharing that name meant this script would rebuild over RandBLAS's BLAS++ (their stamp is a different file, so it never matched ours), and RandBLAS's next run would reuse an artifact we had replaced underneath it while its own stamp still described the original. Distinct names make that impossible; cross-project sharing is instead explicit, via BLASPP_INSTALL_DIR, with the conftest confirming the result works. Also adds the three progress tiers from the RandBLAS installer -- determinate, parsed from Ninja's "[12/34]" and Make's "[ 42%]" -- with tier 0 (redirected output) byte-identical to the plain step list, and a CI assertion that redirected output contains no escape sequence or carriage return. CI: the install-script lanes globbed rather than hardcoded for the new dependency directory names, so they are not coupled to that naming, and the discovery assertion updated to the text the new script prints. Verified on Linux (gcc 15.2, oneAPI MKL): fresh MKL build; idempotent re-run; full default run with extras and benchmarks (13/13 steps); switching --no-gpu to --gpu correctly invalidating the BLAS++ provenance and rebuilding; dependency discovery reducing to 3 steps; running from a git worktree with the clone left in place; redirected output escape-free; and RandBLAS-then-RandLAPACK into one shared RANDNLA_PROJECT_DIR leaving both projects' dependencies intact, plus RandLAPACK reusing RandBLAS's BLAS++ when pointed at it explicitly. --- .github/workflows/install-script.yaml | 40 +- install/install.sh | 1228 +++++++++++++++++++------ 2 files changed, 965 insertions(+), 303 deletions(-) diff --git a/.github/workflows/install-script.yaml b/.github/workflows/install-script.yaml index e69d7c2e..9347fde8 100644 --- a/.github/workflows/install-script.yaml +++ b/.github/workflows/install-script.yaml @@ -56,26 +56,47 @@ jobs: path: deps-install key: installer-deps-${{ runner.os }}-v1 + # Globbed rather than spelled out: dependency install directories carry + # the backend and GPU configuration in their names (blaspp-openblas-cpu-install), + # so hardcoding them here would couple CI to that naming and break the + # moment it changes. - name: seed discovery variables from cache if: steps.deps-cache.outputs.cache-hit == 'true' run: | - echo "BLASPP_INSTALL_DIR=$GITHUB_WORKSPACE/deps-install/blaspp-install" >> "$GITHUB_ENV" - echo "LAPACKPP_INSTALL_DIR=$GITHUB_WORKSPACE/deps-install/lapackpp-install" >> "$GITHUB_ENV" + set -euo pipefail + echo "BLASPP_INSTALL_DIR=$(echo "$GITHUB_WORKSPACE"/deps-install/blaspp-*-install)" >> "$GITHUB_ENV" + echo "LAPACKPP_INSTALL_DIR=$(echo "$GITHUB_WORKSPACE"/deps-install/lapackpp-*-install)" >> "$GITHUB_ENV" echo "RANDOM123_INSTALL_DIR=$GITHUB_WORKSPACE/deps-install/random123" >> "$GITHUB_ENV" - name: keep a pristine clone for the discovery test run: cp -a RandLAPACK RandLAPACK-discovery - name: run the installer (non-interactive) - run: bash RandLAPACK/install.sh --yes --no-gpu + run: | + set -euo pipefail + bash RandLAPACK/install.sh --yes --no-gpu 2>&1 | tee installer.out + + # Redirected output must carry no ANSI escapes and no carriage returns. + # The installer draws a progress bar on a terminal; without this check, + # that bar could start filling install.log and every CI transcript with + # control characters and nobody would notice until a log was unreadable. + - name: piped output is free of terminal control sequences + run: | + if LC_ALL=C grep -qP '\x1b\[|\r' installer.out; then + echo "Found terminal control sequences in non-TTY output:" + LC_ALL=C grep -nP '\x1b\[|\r' installer.out | head -20 + exit 1 + fi + echo "OK: no escape sequences in redirected output." - name: populate the dependency cache if: steps.deps-cache.outputs.cache-hit != 'true' run: | + set -euo pipefail mkdir -p deps-install - cp -a RandNLA-project/install/blaspp-install deps-install/ - cp -a RandNLA-project/install/lapackpp-install deps-install/ - cp -a RandNLA-project/install/random123 deps-install/ + cp -a RandNLA-project/install/blaspp-*-install deps-install/ + cp -a RandNLA-project/install/lapackpp-*-install deps-install/ + cp -a RandNLA-project/install/random123 deps-install/ - name: test the installed library run: | @@ -87,13 +108,14 @@ jobs: - name: install a second project via dependency discovery run: | - BLASPP_INSTALL_DIR="$GITHUB_WORKSPACE/deps-install/blaspp-install" \ - LAPACKPP_INSTALL_DIR="$GITHUB_WORKSPACE/deps-install/lapackpp-install" \ + set -euo pipefail + BLASPP_INSTALL_DIR="$(echo "$GITHUB_WORKSPACE"/deps-install/blaspp-*-install)" \ + LAPACKPP_INSTALL_DIR="$(echo "$GITHUB_WORKSPACE"/deps-install/lapackpp-*-install)" \ RANDOM123_INSTALL_DIR="$GITHUB_WORKSPACE/deps-install/random123" \ bash RandLAPACK-discovery/install.sh --yes --no-gpu \ --project-dir "$GITHUB_WORKSPACE/RandNLA-project-discovery" \ | tee discovery.out - grep -q "reused external install" discovery.out + grep -q "external install" discovery.out test -d RandNLA-project-discovery/build/RandLAPACK-build install-macos: diff --git a/install/install.sh b/install/install.sh index de471235..e5dd988e 100755 --- a/install/install.sh +++ b/install/install.sh @@ -1,77 +1,176 @@ #!/bin/bash -# RandLAPACK autoinstaller. +# RandLAPACK autoinstaller for Linux and macOS. # -# Installs RandLAPACK with all of its dependencies and builds the extras and -# benchmark projects. The directory that contains the RandLAPACK clone ends up -# with a top-level "RandNLA-project" directory: +# Builds RandLAPACK, its dependencies, the extras and the benchmark projects +# into a self-contained "RandNLA-project" directory laid out as: # lib: RandLAPACK, blaspp, lapackpp sources -# install: RandLAPACK-install, blaspp-install, lapackpp-install, random123 +# install: RandLAPACK-install, blaspp-*-install, lapackpp-*-install, random123 # build: one build directory per project above # -# Usage: bash install.sh [options] +# Nothing is installed system-wide, and your shell configuration is untouched +# unless you pass --modify-rc. # -# -y, --yes Assume "yes" for every prompt (also the behavior when -# stdin is not a terminal, e.g. curl | bash or CI). -# --gpu Build with CUDA support without asking. -# --no-gpu Build without GPU support without asking. -# -j, --jobs Parallel build jobs (default: number of cores). -# --fresh Clear all build directories first. The default reuses -# them, so re-running after a failure or a source -# update is an incremental rebuild. -# --modify-rc Append RANDNLA_PROJECT_DIR / RANDNLA_PROJECT_GPU_AVAIL -# exports to your shell config. The default never -# touches your shell config; the final summary prints -# the export lines to add yourself if you want them. -# --project-dir Place/locate RandNLA-project at D instead of next to -# this clone. -# -h, --help Show this help and exit. +# You bring a C++20 compiler, CMake 3.21+, Git and a BLAS/LAPACK. This script +# does not install compilers or package managers; when something is missing it +# says so and tells you the usual way to get it. # -# Every option has an environment-variable equivalent (flags win): -# RANDLAPACK_INSTALL_YES=1, RANDLAPACK_INSTALL_GPU=on|off, -# RANDLAPACK_INSTALL_JOBS=N, RANDLAPACK_INSTALL_FRESH=1, -# RANDLAPACK_INSTALL_MODIFY_RC=1, RANDLAPACK_INSTALL_PROJECT_DIR=D +# RandBLAS is intentionally not covered here: RandLAPACK vendors it as a git +# submodule pinned to an exact commit, and that pinned copy stays authoritative. +# RandBLAS's own installer is for people who want RandBLAS on its own. # -# Already-installed dependencies are discovered through: -# BLASPP_INSTALL_DIR, LAPACKPP_INSTALL_DIR, RANDOM123_INSTALL_DIR -# (RandBLAS is intentionally not covered: it stays a git submodule.) -# -# All compiler output goes to /install.log; the console shows one -# line per step. On failure the log path is printed. -# -# Prerequisites are listed in INSTALL.md. +# Prerequisites and tested configurations are listed in INSTALL_SCRIPT.md. + set -euo pipefail +usage() { + # A heredoc rather than a line-range sed over this file's own comment block: + # the latter starts printing unrelated code the moment anyone adds a line + # above it. + cat <<'USAGE' +Usage: bash install.sh [options] + +Backend selection: + --blas=BACKEND auto | openblas | mkl | accelerate | custom + (default: auto -- OpenBLAS on macOS, MKL on Linux when + MKLROOT is set, otherwise OpenBLAS) + --blas-int=WIDTH ilp64 | lp64. Defaults to ilp64 wherever the backend + can actually provide it, falling back to lp64 with a + warning. Accelerate is lp64-only and rejects ilp64. + --blas-libraries=L Link line for --blas=custom, used for both BLAS and + LAPACK, e.g. "/opt/aocl/lib/libflame.so;/opt/aocl/lib/libblis.so" + +GPU: + --gpu Build with CUDA support without asking + --no-gpu Build without GPU support without asking + +Locations: + --project-dir=DIR Where dependencies, builds and installs go. + Default: $RANDNLA_PROJECT_DIR if set, otherwise + ../RandNLA-project next to this clone. + --prefix=DIR Install RandLAPACK itself here instead of + /install/RandLAPACK-install. Dependencies + still go in the project directory. + +Build: + -j, --jobs N Parallel build jobs (default: number of cores) + --fresh Clear build directories and rebuild dependencies + --no-extras Skip the extras project + --no-benchmarks Skip the benchmark project + --no-openmp Configure without OpenMP + +Output: + -y, --yes Assume "yes" at every prompt. Also the behavior when + stdin is not a terminal (CI, pipes). + --modify-rc Append RANDNLA_PROJECT_DIR / RANDNLA_PROJECT_GPU_AVAIL + exports to your shell config. The default touches + nothing and prints the lines to add yourself. + --no-progress Plain one-line-per-step output, no redrawing + -h, --help Show this help and exit + +Every option has an environment-variable equivalent (flags win): + RANDLAPACK_INSTALL_BLAS, RANDLAPACK_INSTALL_BLAS_INT, + RANDLAPACK_INSTALL_BLAS_LIBRARIES, RANDLAPACK_INSTALL_GPU, + RANDLAPACK_INSTALL_PROJECT_DIR, RANDLAPACK_INSTALL_PREFIX, + RANDLAPACK_INSTALL_JOBS, RANDLAPACK_INSTALL_FRESH, + RANDLAPACK_INSTALL_EXTRAS, RANDLAPACK_INSTALL_BENCHMARKS, + RANDLAPACK_INSTALL_OPENMP, RANDLAPACK_INSTALL_YES, + RANDLAPACK_INSTALL_MODIFY_RC, RANDLAPACK_INSTALL_PROGRESS + +Already-installed dependencies are reused when pointed at by: + BLASPP_INSTALL_DIR, LAPACKPP_INSTALL_DIR, RANDOM123_INSTALL_DIR + +All compiler output goes to /install.log; the console shows one +line per step. On failure the log path is printed. +USAGE +} + #============================================================================== # Option parsing. Environment variables provide defaults; flags override. #============================================================================== -ASSUME_YES="${RANDLAPACK_INSTALL_YES:-0}" -GPU_CHOICE="${RANDLAPACK_INSTALL_GPU:-ask}" # ask | on | off +BLAS_BACKEND="${RANDLAPACK_INSTALL_BLAS:-auto}" +BLAS_INT_CHOICE="${RANDLAPACK_INSTALL_BLAS_INT:-auto}" # auto | ilp64 | lp64 +BLAS_LIBRARIES_ARG="${RANDLAPACK_INSTALL_BLAS_LIBRARIES:-}" +GPU_CHOICE="${RANDLAPACK_INSTALL_GPU:-ask}" # ask | on | off +PROJECT_DIR_OVERRIDE="${RANDLAPACK_INSTALL_PROJECT_DIR:-}" +PREFIX_OVERRIDE="${RANDLAPACK_INSTALL_PREFIX:-}" JOBS="${RANDLAPACK_INSTALL_JOBS:-}" FRESH="${RANDLAPACK_INSTALL_FRESH:-0}" +WANT_EXTRAS="${RANDLAPACK_INSTALL_EXTRAS:-1}" +WANT_BENCHMARKS="${RANDLAPACK_INSTALL_BENCHMARKS:-1}" +WANT_OPENMP="${RANDLAPACK_INSTALL_OPENMP:-1}" +ASSUME_YES="${RANDLAPACK_INSTALL_YES:-0}" MODIFY_RC="${RANDLAPACK_INSTALL_MODIFY_RC:-0}" -PROJECT_DIR_OVERRIDE="${RANDLAPACK_INSTALL_PROJECT_DIR:-}" +WANT_PROGRESS="${RANDLAPACK_INSTALL_PROGRESS:-1}" -usage() { sed -n '2,45p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; } +die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; } while [[ $# -gt 0 ]]; do case "$1" in - -y|--yes) ASSUME_YES=1 ;; - --gpu) GPU_CHOICE="on" ;; - --no-gpu) GPU_CHOICE="off" ;; - -j|--jobs) JOBS="${2:?--jobs requires a number}"; shift ;; - --jobs=*) JOBS="${1#*=}" ;; - --fresh) FRESH=1 ;; - --modify-rc) MODIFY_RC=1 ;; - --project-dir) PROJECT_DIR_OVERRIDE="${2:?--project-dir requires a path}"; shift ;; - --project-dir=*) PROJECT_DIR_OVERRIDE="${1#*=}" ;; - -h|--help) usage; exit 0 ;; - *) echo "Unknown option: $1 (see --help)" >&2; exit 2 ;; + --blas) BLAS_BACKEND="${2:?--blas requires a backend}"; shift ;; + --blas=*) BLAS_BACKEND="${1#*=}" ;; + --blas-int) BLAS_INT_CHOICE="${2:?--blas-int requires a width}"; shift ;; + --blas-int=*) BLAS_INT_CHOICE="${1#*=}" ;; + --blas-libraries) BLAS_LIBRARIES_ARG="${2:?--blas-libraries requires a value}"; shift ;; + --blas-libraries=*) BLAS_LIBRARIES_ARG="${1#*=}" ;; + --gpu) GPU_CHOICE="on" ;; + --no-gpu) GPU_CHOICE="off" ;; + --project-dir) PROJECT_DIR_OVERRIDE="${2:?--project-dir requires a path}"; shift ;; + --project-dir=*) PROJECT_DIR_OVERRIDE="${1#*=}" ;; + --prefix) PREFIX_OVERRIDE="${2:?--prefix requires a path}"; shift ;; + --prefix=*) PREFIX_OVERRIDE="${1#*=}" ;; + -j|--jobs) JOBS="${2:?--jobs requires a number}"; shift ;; + --jobs=*) JOBS="${1#*=}" ;; + -j*) JOBS="${1#-j}" ;; # attached form, as in -j8 + --fresh) FRESH=1 ;; + --no-extras) WANT_EXTRAS=0 ;; + --no-benchmarks) WANT_BENCHMARKS=0 ;; + --no-openmp) WANT_OPENMP=0 ;; + -y|--yes) ASSUME_YES=1 ;; + --modify-rc) MODIFY_RC=1 ;; + --no-progress) WANT_PROGRESS=0 ;; + -h|--help) usage; exit 0 ;; + *) printf 'Unknown option: %s (see --help)\n' "$1" >&2; exit 2 ;; esac shift done -# Prompts happen only on a terminal and only without --yes. When stdin is not -# a terminal (piped/CI), every prompt silently takes its default. +case "$BLAS_BACKEND" in + auto|openblas|mkl|accelerate|custom) ;; + *) die "--blas must be auto, openblas, mkl, accelerate or custom (got '$BLAS_BACKEND')" ;; +esac +case "$BLAS_INT_CHOICE" in + auto|ilp64|lp64) ;; + *) die "--blas-int must be ilp64 or lp64 (got '$BLAS_INT_CHOICE')" ;; +esac +case "$GPU_CHOICE" in + ask|on|off) ;; + *) die "RANDLAPACK_INSTALL_GPU must be 'on' or 'off' (got '$GPU_CHOICE')" ;; +esac +if [[ "$BLAS_BACKEND" == "custom" && -z "$BLAS_LIBRARIES_ARG" ]]; then + die "--blas=custom needs --blas-libraries=" +fi +if [[ -n "$BLAS_LIBRARIES_ARG" && "$BLAS_BACKEND" != "custom" ]]; then + die "--blas-libraries only applies to --blas=custom (backend is '$BLAS_BACKEND')" +fi +# Checked here rather than during backend resolution, which happens after GPU +# detection: a contradiction between two flags should be reported before the +# user is asked anything. "auto" never resolves to accelerate, so testing the +# literal value is sufficient. +if [[ "$BLAS_BACKEND" == "accelerate" && "$BLAS_INT_CHOICE" == "ilp64" ]]; then + die "--blas-int=ilp64 is not available with Accelerate: BLAS++ implements only Apple's legacy LP64 interface (upstream icl-utk-edu/lapackpp#43). Use --blas=openblas or --blas=mkl for ILP64." +fi + +if [[ -z "$JOBS" ]]; then + JOBS=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 8) +fi + +#============================================================================== +# Interactivity and output style. +# +# Prompts happen only on a terminal and only without --yes. When stdin is not a +# terminal (piped, CI) every prompt silently takes its default, so this script +# can never hang waiting for input nobody is there to give. +#============================================================================== INTERACTIVE=0 if [[ -t 0 && "$ASSUME_YES" != "1" ]]; then INTERACTIVE=1 @@ -89,35 +188,91 @@ ask() { [[ "$reply" == "y" || "$reply" == "Y" || "$reply" == "yes" ]] } -if [[ -z "$JOBS" ]]; then - JOBS=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 8) +if [[ -t 1 && -z "${NO_COLOR:-}" && "${TERM:-}" != "dumb" && "$WANT_PROGRESS" == "1" ]]; then + C_OK=$'\033[32m'; C_ERR=$'\033[31m'; C_WARN=$'\033[33m'; C_BOLD=$'\033[1m'; C_OFF=$'\033[0m' +else + C_OK=""; C_ERR=""; C_WARN=""; C_BOLD=""; C_OFF="" fi -# Plain output when not on a terminal or when NO_COLOR/TERM=dumb ask for it. -if [[ -t 1 && -z "${NO_COLOR:-}" && "${TERM:-}" != "dumb" ]]; then - C_OK=$'\033[32m'; C_ERR=$'\033[31m'; C_BOLD=$'\033[1m'; C_OFF=$'\033[0m' +# Progress rendering tier. +# 2 a terminal that can draw: redraw a bar in place, with block characters +# 1 a terminal without colour or UTF-8: same bar, ASCII, still redrawn +# 0 not a terminal: one line per step, no escapes, no carriage returns +# +# Tier 0 is a requirement, not a fallback. Redirected output ends up in +# install.log, in CI transcripts and in bug reports, and control characters make +# all three unreadable. +PROGRESS_TIER=0 +if [[ -t 1 && "$WANT_PROGRESS" == "1" && "${TERM:-}" != "dumb" ]]; then + if [[ -z "${NO_COLOR:-}" && "${LC_ALL:-${LC_CTYPE:-${LANG:-}}}" == *[Uu][Tt][Ff]* ]]; then + PROGRESS_TIER=2 + else + PROGRESS_TIER=1 + fi +fi +if (( PROGRESS_TIER >= 2 )); then + BAR_FULL="━"; BAR_EMPTY="─" else - C_OK=""; C_ERR=""; C_BOLD=""; C_OFF="" + BAR_FULL="#"; BAR_EMPTY="-" fi +note() { printf '%s\n' "$*"; } +warn() { printf '%swarning:%s %s\n' "$C_WARN" "$C_OFF" "$*" >&2; } + +# Collected and reprinted in the final summary. A warning emitted twenty minutes +# and several thousand log lines before the summary is a warning nobody reads. +WARNINGS=() +record_warning() { WARNINGS+=("$1"); warn "$1"; } + #============================================================================== -# Toolchain checks. Warn always; abort only if the user says so at a prompt. +# Toolchain preflight. Report everything missing at once rather than failing on +# the first one, so a bare machine takes one round trip instead of three. #============================================================================== +UNAME_S="$(uname -s)" +MISSING=() +command -v cmake >/dev/null 2>&1 || MISSING+=("cmake") +command -v git >/dev/null 2>&1 || MISSING+=("git") +if ! command -v c++ >/dev/null 2>&1 && ! command -v g++ >/dev/null 2>&1 && \ + ! command -v clang++ >/dev/null 2>&1; then + MISSING+=("a C++ compiler") +fi +if (( ${#MISSING[@]} )); then + printf 'ERROR: missing prerequisites: %s\n\n' "${MISSING[*]}" >&2 + if [[ "$UNAME_S" == "Darwin" ]]; then + printf ' xcode-select --install # Apple Clang and git\n' >&2 + printf ' brew install cmake\n\n' >&2 + else + printf ' sudo apt install g++ gfortran cmake git # Debian, Ubuntu\n' >&2 + printf ' sudo dnf install gcc-c++ gcc-gfortran cmake git # Fedora, RHEL\n\n' >&2 + fi + printf 'See INSTALL_SCRIPT.md for the full prerequisite list.\n' >&2 + exit 1 +fi + +CMAKE_VERSION="$(cmake --version | head -n1 | awk '{print $3}')" +if [[ "$(printf '%s\n3.21\n' "$CMAKE_VERSION" | sort -V | head -n1)" != "3.21" ]]; then + die "CMake 3.21 or later is required (found $CMAKE_VERSION). See INSTALL_SCRIPT.md." +fi + +# GCC 13.3.0 is the reference version. Warn rather than block: newer usually +# works, and the C++20 concepts RandBLAS uses need at least 13. PREFERRED_GCC_VERSION="13.3.0" CURRENT_GCC_VERSION=$(gcc --version 2>/dev/null | head -n 1 | awk '{print $NF}') -if [[ "$CURRENT_GCC_VERSION" != "$PREFERRED_GCC_VERSION" ]]; then - echo "Note: GCC $PREFERRED_GCC_VERSION is the reference version; found ${CURRENT_GCC_VERSION:-none}." - if ! ask "Continue with the current GCC?" y; then - echo "Stopping at your request. Install GCC $PREFERRED_GCC_VERSION and re-run." - exit 1 +if [[ -n "$CURRENT_GCC_VERSION" && "$CURRENT_GCC_VERSION" != "$PREFERRED_GCC_VERSION" ]]; then + GCC_MAJOR="${CURRENT_GCC_VERSION%%.*}" + if [[ "$GCC_MAJOR" =~ ^[0-9]+$ ]] && (( GCC_MAJOR < 13 )); then + record_warning "gcc $CURRENT_GCC_VERSION is older than 13; RandBLAS uses C++20 concepts and may not compile." + else + note "Note: gcc $PREFERRED_GCC_VERSION is the reference version; found $CURRENT_GCC_VERSION." fi fi #============================================================================== # GPU decision. --gpu/--no-gpu (or RANDLAPACK_INSTALL_GPU) decide outright; -# otherwise detection + prompt. Non-interactive defaults: NVIDIA detected -> -# GPU on; AMD or nothing detected -> GPU off (the CUDA-only build cannot -# succeed on AMD, so saying yes for the user would guarantee a failure). +# otherwise detection plus a prompt. Non-interactive defaults: NVIDIA detected +# means GPU on; AMD or nothing detected means GPU off, because the CUDA-only +# build cannot succeed on AMD and saying yes for the user would guarantee a +# failure. #============================================================================== RANDLAPACK_CUDA="OFF" RANDNLA_PROJECT_GPU_AVAIL="none" @@ -130,302 +285,772 @@ case "$GPU_CHOICE" in RANDLAPACK_CUDA="ON"; RANDNLA_PROJECT_GPU_AVAIL="auto" fi elif { command -v lspci &>/dev/null && lspci | grep -i "VGA" | grep -qi "AMD"; } || \ - { [[ "$(uname)" == "Darwin" ]] && system_profiler SPDisplaysDataType 2>/dev/null | grep -qi "AMD"; }; then + { [[ "$UNAME_S" == "Darwin" ]] && system_profiler SPDisplaysDataType 2>/dev/null | grep -qi "AMD"; }; then if ask "AMD GPU detected, but only a CUDA build is available for now. Attempt a CUDA build anyway?" n; then RANDLAPACK_CUDA="ON"; RANDNLA_PROJECT_GPU_AVAIL="auto" fi else - echo "No GPU detected; building without GPU support." + note "No GPU detected; building without GPU support." fi ;; - *) echo "RANDLAPACK_INSTALL_GPU must be 'on' or 'off' (got '$GPU_CHOICE')" >&2; exit 2 ;; esac if [[ "$RANDNLA_PROJECT_GPU_AVAIL" == "auto" ]]; then PREFERRED_NVCC_VERSION="12.9" CURRENT_NVCC_VERSION=$(nvcc --version 2>/dev/null | grep "release" | awk '{print $5}' | cut -d',' -f1) if [[ "$CURRENT_NVCC_VERSION" != "$PREFERRED_NVCC_VERSION" ]]; then - echo "Note: NVCC $PREFERRED_NVCC_VERSION is the reference version; found ${CURRENT_NVCC_VERSION:-none}." + note "Note: NVCC $PREFERRED_NVCC_VERSION is the reference version; found ${CURRENT_NVCC_VERSION:-none}." if ! ask "Continue with the current NVCC?" y; then - echo "Stopping at your request. Install NVCC $PREFERRED_NVCC_VERSION and re-run." - exit 1 + die "Stopping at your request. Install NVCC $PREFERRED_NVCC_VERSION and re-run." fi fi fi #============================================================================== -# macOS preflight: Homebrew OpenBLAS + libomp, SDK C++ headers, OpenMP hints. +# Project layout. +# +# Precedence: --project-dir, then RANDNLA_PROJECT_DIR, then a sibling of this +# clone. Honouring the environment variable is what lets this installer and +# RandBLAS's share one dependency tree -- whichever runs second finds the +# first one's BLAS++ and reuses it. +# +# This script no longer moves your clone. The previous version relocated the +# repository into /lib/RandLAPACK on first run, which breaks git +# worktrees and surprises anyone who cloned deliberately. The layout below is +# created regardless of where the clone lives, and lib/RandLAPACK is a symlink +# so the tree still reads as complete. #============================================================================== -BLAS_INT="int64" -MACOS_BLAS_FLAGS="" -MACOS_LAPACK_FLAGS="" -MACOS_OPENMP_FLAGS="" -if [[ "$(uname)" == "Darwin" ]]; then - if [[ ! -f /opt/homebrew/opt/openblas/lib/libopenblas.dylib ]]; then - echo "ERROR: OpenBLAS not found. Install it first: brew install openblas" >&2 - exit 1 - fi - if [[ ! -f /opt/homebrew/opt/libomp/lib/libomp.dylib ]]; then - echo "ERROR: libomp not found. Install it first: brew install libomp" >&2 - exit 1 - fi - BLAS_INT="int32" - MACOS_SDK_PATH=$(xcrun --show-sdk-path) - # SDK C++ headers + Apple Clang OpenMP flags (no native OpenMP; Homebrew - # libomp). Appending to CXXFLAGS/CFLAGS lets cmake pick them up via - # CMAKE__FLAGS_INIT for all try_compile tests, including FindOpenMP. - export CXXFLAGS="-isystem ${MACOS_SDK_PATH}/usr/include/c++/v1 -Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include" - export CFLAGS="-Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include" - export LDFLAGS="-L/opt/homebrew/opt/libomp/lib" - MACOS_BLAS_FLAGS="-DBLAS_LIBRARIES=/opt/homebrew/opt/openblas/lib/libopenblas.dylib -Dblas_fortran=add" - MACOS_LAPACK_FLAGS="-DLAPACK_LIBRARIES=/opt/homebrew/opt/openblas/lib/libopenblas.dylib" - MACOS_OPENMP_FLAGS="-DOpenMP_C_LIB_NAMES=omp -DOpenMP_CXX_LIB_NAMES=omp -DOpenMP_omp_LIBRARY=/opt/homebrew/opt/libomp/lib/libomp.dylib -DOpenMP_C_FLAGS=-Xpreprocessor;-fopenmp -DOpenMP_CXX_FLAGS=-Xpreprocessor;-fopenmp" -fi - -#============================================================================== -# Project layout. The clone moves itself into /RandNLA-project/lib/ -# on first run; on re-runs (script already under lib/) the layout is detected. -#============================================================================== -# This script lives in /install/; REPO_DIR is the RandLAPACK clone. -SCRIPT_DIR=$(dirname "$(realpath "${BASH_SOURCE[0]}")") -REPO_DIR=$(dirname "$SCRIPT_DIR") -PARENT_DIR=$(dirname "$REPO_DIR") -PARENT_BASE=$(basename "$PARENT_DIR") +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_DIR="$(dirname "$SCRIPT_DIR")" + if [[ -n "$PROJECT_DIR_OVERRIDE" ]]; then RANDNLA_PROJECT_DIR="$PROJECT_DIR_OVERRIDE" -elif [[ "$PARENT_BASE" == "lib" ]]; then - RANDNLA_PROJECT_DIR=$(dirname "$PARENT_DIR") +elif [[ -n "${RANDNLA_PROJECT_DIR:-}" ]]; then + : # already set in the environment; use it as-is else - RANDNLA_PROJECT_DIR="$PARENT_DIR/RandNLA-project" + RANDNLA_PROJECT_DIR="$(dirname "$REPO_DIR")/RandNLA-project" fi - +mkdir -p "$RANDNLA_PROJECT_DIR" +RANDNLA_PROJECT_DIR="$(cd "$RANDNLA_PROJECT_DIR" && pwd)" mkdir -p "$RANDNLA_PROJECT_DIR"/{install,lib,build} -for d in blaspp-build lapackpp-build RandLAPACK-build extras-build benchmark-build; do - if [[ "$FRESH" == "1" ]]; then - rm -rf "$RANDNLA_PROJECT_DIR/build/$d" - fi - mkdir -p "$RANDNLA_PROJECT_DIR/build/$d" -done + +# A symlink, not a move: the clone stays where the user put it. +if [[ ! -e "$RANDNLA_PROJECT_DIR/lib/RandLAPACK" ]]; then + ln -s "$REPO_DIR" "$RANDNLA_PROJECT_DIR/lib/RandLAPACK" +fi +RL_SRC="$REPO_DIR" + +RANDLAPACK_INSTALL_DIR="${PREFIX_OVERRIDE:-$RANDNLA_PROJECT_DIR/install/RandLAPACK-install}" LOG="$RANDNLA_PROJECT_DIR/install.log" -: > "$LOG" -echo "RandLAPACK install started $(date)" >> "$LOG" +# Appended, not truncated: the previous run's output is exactly what you want +# when the current run fails the same way. +{ + printf '\n===============================================================\n' + printf 'RandLAPACK install started %s\n' "$(date)" + printf '===============================================================\n' +} >> "$LOG" -# run_step