Using an AI coding agent? Paste this into Claude Code, Codex, Cursor or any other coding agent. It installs sflow, checks that it runs, and installs the sflow skills for that agent:
Set up sflow for me by following
https://raw.githubusercontent.com/NVIDIA/nv-sflow/main/docs-site/agents-src/setup.md
step by step, then tell me what you installed and where.
By hand: sflow is on PyPI as nv-sflow; the command is sflow.
uv tool install nv-sflow # sflow on PATH everywhere; or `uv pip install nv-sflow` inside a venv
sflow --versionAlready installed? Upgrade in place -- no incantation to remember:
sflow upgrade # latest PyPI release; --branch <ref> for a git branch, --dry-run to previewComing from v0.3.1 or earlier? The distribution is now nv-sflow on PyPI. Either
pip install nv-sflow, or run sflow upgrade twice (the first lands on the current
GitHub build, the second moves it to PyPI). The next sflow command tidies up the old
sflow install record by itself. Never pip install sflow -- that name on pypi.org is an
unrelated project -- and don't pip uninstall sflow.
No uv? pip install uv (or curl -LsSf https://astral.sh/uv/install.sh | sh).
Optional extras: nv-sflow[s3] (S3 uploads), nv-sflow[monitor] (PNG hardware charts).
Contributing? Skip to Development Setup for the editable install.
Quickstart: https://nvidia.github.io/nv-sflow/docs/user/quickstart
A declarative workflow descriptor for massive GPU clusters that separates what to deploy from where to deploy it, see our project page.
One semantic across every platform -- backend agnostic by design. A deployment's logic never changes: start etcd and NATS, launch a frontend, spin up workers, run the benchmark. Only the infrastructure glue does -- and today that glue is rewritten from scratch for every platform. sflow consolidates it into a single portable YAML: tasks, dependencies, resources, and launch methods. The same sflow.yaml runs on Docker, Slurm, and Kubernetes -- swap the backend block, rewrite nothing. Backends delegate to each platform's native ecosystem rather than reimplementing it.
Cluster-level orchestration at scale. Topology-aware node and GPU placement, multi-node replicas and sweeps, readiness/failure probes, and batch submission -- so one descriptor drives hundreds of GPUs instead of a pile of hand-written bash.
All four backends ship today: local, docker, slurm, and kubernetes.
Click for the full-quality video.
| Feature | Description |
|---|---|
| Backend Agnostic | One recipe on local, docker, slurm, kubernetes (k8s / k8s_mpi) -- swap the backend fragment, keep the DAG |
| Modular Composition | Reusable YAML fragments deep-merged by name at runtime (sflow compose, multi-file sflow run -f), with required_by reverse edges |
| Topology-aware GPU Allocation | Automatic node/GPU placement and CUDA_VISIBLE_DEVICES slicing; pin devices with gpus.indices, UUID-based re-selection inside Slurm steps, and machine-local GPU reservation on Docker |
| Probes | Readiness and failure gates -- TCP port, HTTP, log watch with pattern matching |
| Replicas & Sweeps | Parallel/sequential replicas with Cartesian product variable sweeps |
| Batch Mode | Generate sbatch scripts, CSV-driven bulk sweeps, parallel preflight validation |
| Expressions | Jinja2 ${{ }} syntax for variables, backend info, and task metadata |
| Artifacts | Named URIs (fs://, file://, http://) with inline content generation |
| Live TUI | Rich terminal interface with task status, log tailing, and allocation maps |
| Results & Uploads | Declarative result: parsing into result.json, plus uploads: / upload_all: to S3 |
| Hardware Monitoring | monitor: GPU/CPU/memory/network telemetry with per-device charts and log-marker window: reports (Local, Docker, Slurm) |
| Run Diagnostics | Self-explaining sflow_summary.log: GPU assignment, node topology, external command health, event-loop stall dumps |
| AI Agent Skills | Built-in skills that teach coding assistants (Cursor, Copilot) to write and debug sflow YAML |
| Preflight Validation | Container image checks, GPU oversubscription detection, dependency cycle analysis |
Modular workflow samples for LLM inference serving with NVIDIA Dynamo:
| Framework | Aggregated | Disaggregated (P/D) | Multi-Node |
|---|---|---|---|
| SGLang | Yes | Yes | Yes |
| vLLM | Yes | Yes | Yes |
| TRT-LLM | Yes | Yes | Yes |
All frameworks share a common infrastructure layer (etcd, NATS, frontend, nginx) -- only the server task files differ.
The same workload, three platforms. examples/modular/backend_agnostic/ composes a backend fragment + a workload + a benchmark. Moving platforms means swapping one -f:
# Kubernetes
sflow run -f backends/k8s.yaml -f workloads/dynamo_common.yaml -f workloads/dynamo_trtllm.yaml -f benchmark.yaml
# Slurm -- same workload, same benchmark
sflow run -f backends/slurm.yaml -f workloads/dynamo_common.yaml -f workloads/dynamo_trtllm.yaml -f benchmark.yaml
# One local box, containers
sflow run -f backends/docker.yaml -f workloads/sglang_serve.yaml -f benchmark.yaml
| Command | Purpose | Key Flags |
|---|---|---|
sflow run |
Execute a workflow | --dry-run --tui --set -f (multi-file) |
sflow batch |
Generate sbatch scripts | --submit --bulk-input --row |
sflow compose |
Merge multiple YAMLs | --resolve --missable-tasks -o |
sflow visualize |
Render DAG graph | --format png/svg/mermaid |
sflow sample |
List / copy examples | --list -o |
sflow skill |
Install AI agent skills (Claude Code, Cursor, Codex) | --target claude|cursor|codex|all --global --list |
sflow upgrade |
Upgrade to the latest PyPI release in place (alias: sflow update) |
--sflow-version --branch --dry-run --force |
Full user documentation: https://nvidia.github.io/nv-sflow/
- Introduction -- concepts and architecture
- Quickstart -- local and Slurm setup
- Configuration -- full YAML schema
- Modular Workflows -- multi-file composition
- Quick Reference -- all fields at a glance
- Backends -- local, Docker, Slurm, Kubernetes
- Monitor -- hardware telemetry and reports
- Understanding Run Output -- output tree and failure triage
- CLI Reference -- commands and flags
- Sample Workflows -- production examples
Release notes live in docs/release_notes/ -- start with v0.3.1 for the current breaking changes and upgrade checklist.
Validate the workflow engine locally (no Slurm required):
sflow run --file examples/local_hello_world.yaml --tuiMinimal workflow:
version: "0.1"
variables:
WHO:
description: "who to greet"
value: Nvidia
workflow:
name: hello_local
tasks:
- name: hello
script:
- echo "Hello ${WHO}"Run a modular multi-file workflow on Slurm:
sflow run \
-f slurm_config.yaml -f common_workflow.yaml \
-f sglang/prefill.yaml -f sglang/decode.yaml -f benchmark_aiperf.yaml \
--missable-tasks agg_server --tuiInstall the AI agent skills into your coding agent:
sflow skill --target claude # or cursor, codex, all; --global for the user-level dir-
Python 3.10 or higher
-
uv (Python package installer and resolver)
curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/NVIDIA/nv-sflow.git
cd nv-sflow
uv venv
source .venv/bin/activate
uv pip install -e ".[dev]"
pytestPlease see CONTRIBUTING.md for details on how to contribute to this project.
This project is licensed under the Apache License 2.0.