Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions packages/kumo-engine/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ error while other tasks remain available. On ARM64 Linux,
use `pip install "kumo-engine[cpu]"` for CPU retrieval with a compatible backend
wheel for your platform and Python version. For CUDA retrieval, install a
compatible GPU Faiss build directly, building from source on ARM64 Linux if
needed. The CPU extra retains the required platform backend on Linux x86_64
needed; see [GPU Faiss on DGX Spark](https://github.com/NVIDIA/kumo-engine/blob/main/packages/kumo-engine/dgx-spark.md#optional-build-gpu-faiss-for-cuda-link-prediction).
The CPU extra retains the required platform backend on Linux x86_64
and macOS; it does not configure PyTorch or select the execution device.
Choose one backend per environment.

Expand All @@ -62,7 +63,9 @@ pip install "kumo-engine[text]"

Extras combine, for example `pip install "kumo-engine[s3,text]"`.

The wheels on PyPI are for Linux x86_64 and for macOS on Apple silicon. To build and install on Linux ARM64 (DGX Spark), follow the [DGX Spark guide](https://github.com/NVIDIA/kumo-engine/blob/main/packages/kumo-engine/dgx-spark.md).
Prebuilt wheels are available for Linux x86_64 and ARM64, and macOS on Apple
silicon. For Linux ARM64 installation and optional GPU
Comment thread
danielmawhirter marked this conversation as resolved.
Resolved
Faiss setup, follow the [DGX Spark guide](https://github.com/NVIDIA/kumo-engine/blob/main/packages/kumo-engine/dgx-spark.md).

### Optional: faster training on NVIDIA GPUs

Expand Down
128 changes: 84 additions & 44 deletions packages/kumo-engine/dgx-spark.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,56 @@
# Build and install on DGX Spark
# Install on DGX Spark

Build GPU Faiss and `kumo-engine` from source, then install them in a Python
virtual environment.
Use the Linux ARM64 Kumo Engine wheel with CPython **3.11–3.14**. The host
needs glibc 2.34 or newer and CPU AES and NEON instructions.

## Supported versions
The installed-wheel qualification covers CPU ingestion, training, prediction
and model reload on Python 3.11–3.14. GPU execution and Faiss retrieval have
separate dependency requirements below.

Use Python **3.11–3.13** and **Faiss GPU 1.14.3 built with CUDA 13** with one
of these PyTorch versions:
## Install the engine wheel

Create a virtual environment; these commands use Python 3.12:

```bash
python3.12 -m venv "$HOME/kumo-spark-venv"
source "$HOME/kumo-spark-venv/bin/activate"
python -m pip install --upgrade pip
python -m pip install --only-binary=kumo-engine kumo-engine
python -m pip check
```

Follow the [quickstart](README.md#quickstart) to ingest data, train and predict.
See the [installation and extras table](README.md#installation) for data
connectors and text encoding, including combinations such as
`kumo-engine[snowflake,text]`.

The base ARM64 install does not require Faiss. Install a Faiss backend only
for link-prediction retrieval. Other prediction tasks can run without it.

## Optional: CPU link prediction

```bash
python -m pip install 'kumo-engine[cpu]'
```

The `cpu` extra installs CPU Faiss; it does not choose the PyTorch build or
the training device. With CPU-only Faiss, explicitly select CPU execution for
link prediction, including when a GPU is available:

```python
from kumo.engine import Trainer

trainer = Trainer(model_plan, accelerator="cpu")
```

Use `Trainer.load(bundle_dir, accelerator="cpu")` when reloading such a model.
Choose one Faiss backend per environment.

## Optional: CUDA training

A compatible NVIDIA driver and CUDA-enabled PyTorch build are required for
GPU execution. Check the driver with `nvidia-smi`. These PyTorch versions
provide the following ARM64 CUDA wheel variants:

| PyTorch | CUDA wheel variants |
| --- | --- |
Expand All @@ -16,61 +60,57 @@ of these PyTorch versions:
| 2.11.0 | cu128, cu130 |
| 2.12.0 | cu130 |

The commands below use Python 3.12 and PyTorch 2.12.0 with CUDA 13.
Python wheel availability varies by PyTorch version and CUDA index; the
table does not imply every combination supports every Python version.
For example, PyTorch 2.7.0 ARM64 wheels do not cover Python 3.14. See
[PyTorch installation](https://pytorch.org/get-started/locally/) for available
builds and driver requirements.

## Prerequisites
The optional GPU Faiss build below uses **Faiss GPU 1.14.3 built with CUDA 13**.

On your DGX Spark, install:
In the activated Python 3.12 environment, install PyTorch 2.12.0 with CUDA 13:

- A working NVIDIA driver (`nvidia-smi`) and the CUDA 13 toolkit (`nvcc`).
```bash
python -m pip install 'torch==2.12.0' \
--index-url https://download.pytorch.org/whl/cu130
```

This does not require rebuilding Kumo Engine. Tasks without link-prediction
retrieval do not need GPU Faiss. CUDA link prediction also needs the optional
GPU Faiss build below.

## Optional: build GPU Faiss for CUDA link prediction

For this optional build, use a source checkout and install:

- The CUDA 13 toolkit (`nvcc`) and a compatible NVIDIA driver.
- A C/C++ build toolchain and OpenBLAS development files (`libopenblas-dev`
on Ubuntu).
- A current stable Rust toolchain, including `cargo`.
- Git and `uv`.

The build downloads source and dependencies from GitHub and package indexes.
The Faiss builder uses CUDA from `/usr/local/cuda`. If CUDA 13 is installed
elsewhere, set `CUDAToolkit_ROOT` to that directory before building.

## Set up the source and Python environment
Use the activated Python 3.12 environment and CUDA-enabled PyTorch from above.
If CPU Faiss is already installed, uninstall it before installing GPU Faiss:

```bash
git clone https://github.com/NVIDIA/kumo-engine.git
cd kumo-engine
KUMO_CHECKOUT="$PWD"
SPARK_WORK="$HOME/kumo-spark"
uv venv --seed --python 3.12 "$SPARK_WORK/venv"
source "$SPARK_WORK/venv/bin/activate"
python -m pip install 'torch==2.12.0' \
--index-url https://download.pytorch.org/whl/cu130
python -m pip uninstall faiss-cpu
```

## Build and install GPU Faiss

Expect the Faiss build to take 3–5 minutes on DGX Spark.
The build downloads source and dependencies from GitHub and package indexes.
The builder uses CUDA from `/usr/local/cuda`. If CUDA 13 is installed
elsewhere, set `CUDAToolkit_ROOT` to that directory before building.

```bash
SPARK_WORK="$HOME/kumo-spark"
git clone https://github.com/NVIDIA/kumo-engine.git "$SPARK_WORK/kumo-engine"
KUMO_CHECKOUT="$SPARK_WORK/kumo-engine"
FAISS_WHEELS="$(CMAKE_BUILD_PARALLEL_LEVEL="$(nproc)" \
FAISS_CUDA_ARCHES=12.1 \
FAISS_GPU_CACHE_DIR="$SPARK_WORK/faiss" \
"$KUMO_CHECKOUT/scripts/build_faiss_gpu_arm64_wheel.sh" \
--python "$SPARK_WORK/venv/bin/python" --cuda cu130)"
--python "$(command -v python)" --cuda cu130)"
python -m pip install "$FAISS_WHEELS"/faiss_gpu-1.14.3-*.whl
python -m pip check
```

## Build and install Kumo Engine

```bash
cd "$KUMO_CHECKOUT"
python -m pip install ziglang
KUMO_WHEEL="$(PYTHON="$SPARK_WORK/venv/bin/python" \
./packages/kumo-engine/scripts/build_wheel.sh \
--platform-tag manylinux_2_34_aarch64 \
--output-dir "$SPARK_WORK/kumo-wheel")"
python -m pip install "$KUMO_WHEEL"
```

Pip installs the remaining dependencies. To include optional dependencies,
append `[snowflake]` or `[text]` to the full wheel filename when running
`pip install`, and quote the resulting path. These extras install Snowflake
dependencies or GloVe word vectors, respectively.
The engine wheel remains installed; no Rust toolchain or Kumo Engine source
build is needed for this GPU Faiss setup.
Loading