diff --git a/docs/integrator/kubernetes-deployment.md b/docs/integrator/kubernetes-deployment.md index feef946fc..ba1bd7612 100644 --- a/docs/integrator/kubernetes-deployment.md +++ b/docs/integrator/kubernetes-deployment.md @@ -609,7 +609,7 @@ as HTTP 429 responses with the `X-RateLimit-*` headers. ```shell # Update image kubectl set image deployment/aicrd \ - api-server=ghcr.io/nvidia/aicrd:v0.8.0 \ + api-server=ghcr.io/nvidia/aicrd:v0.19.0 \ -n aicr # Watch rollout diff --git a/docs/user/cli-reference.md b/docs/user/cli-reference.md index 0c1d6ef87..0611b7b3c 100644 --- a/docs/user/cli-reference.md +++ b/docs/user/cli-reference.md @@ -1526,7 +1526,7 @@ The `--accelerated-node-selector` and `--accelerated-node-toleration` flags cont NFD (Node Feature Discovery) workers must run on **all nodes** (GPU, CPU, and system) to detect hardware features. This matches the gpu-operator default behavior where NFD workers also run on control-plane nodes. The `--accelerated-node-selector` is intentionally not applied to NFD workers so they are not restricted to GPU nodes. -> **Note:** When no `--accelerated-node-toleration` is specified, a default toleration (`operator: Exists`) is applied to both GPU daemonsets and NFD workers, allowing them to run on nodes with any taint. +> **Note:** When no `--accelerated-node-toleration` is specified, a default toleration (`operator: Exists`) is applied to both GPU DaemonSets and NFD workers, allowing them to run on nodes with any taint. **Example:** @@ -1544,7 +1544,7 @@ aicr bundle --recipe recipe.yaml \ > **Cluster node requirements:** This example assumes the cluster has nodes labeled `nodeGroup=system-worker` with taints `dedicated=system-workload:NoSchedule,NoExecute` for system infrastructure, and GPU nodes labeled `nodeGroup=gpu-worker` with taints `dedicated=worker-workload:NoSchedule,NoExecute`. This results in: -- **GPU daemonsets** (driver, device-plugin, toolkit, dcgm): `nodeSelector=nodeGroup=gpu-worker` + tolerations for `dedicated=worker-workload` with both `NoSchedule` and `NoExecute` +- **GPU DaemonSets** (driver, device-plugin, toolkit, dcgm): `nodeSelector=nodeGroup=gpu-worker` + tolerations for `dedicated=worker-workload` with both `NoSchedule` and `NoExecute` - **NFD workers**: no nodeSelector (runs on all nodes) + tolerations for `dedicated=worker-workload` with both `NoSchedule` and `NoExecute` - **System components** (gpu-operator controller, NFD gc/master, dynamo grove, agentgateway proxy): `nodeSelector=nodeGroup=system-worker` + tolerations for `dedicated=system-workload` with both `NoSchedule` and `NoExecute` @@ -2629,9 +2629,9 @@ Components that use operator patterns with custom resources that reconcile async ##### DRA kubelet plugin registration -After installing `nvidia-dra-driver-gpu`, the script automatically restarts the DRA kubelet plugin daemonset. This is a best-effort mitigation for a known issue: after uninstall/reinstall, the kubelet's plugin watcher (`fsnotify`) may not detect new registration sockets, causing `DRA driver gpu.nvidia.com is not registered` errors. +After installing `nvidia-dra-driver-gpu`, the script automatically restarts the DRA kubelet plugin DaemonSet. This is a best-effort mitigation for a known issue: after uninstall/reinstall, the kubelet's plugin watcher (`fsnotify`) may not detect new registration sockets, causing `DRA driver gpu.nvidia.com is not registered` errors. -If DRA pods fail with this error after redeployment, the daemonset restart alone may not be sufficient — a **node reboot** is required to reset the kubelet's plugin registration state. To reboot GPU nodes: +If DRA pods fail with this error after redeployment, the DaemonSet restart alone may not be sufficient — a **node reboot** is required to reset the kubelet's plugin registration state. To reboot GPU nodes: ```bash # Cordon, drain, and reboot the affected node