diff --git a/docs/v0.6.1-rc/api.md b/docs/v0.6.1-rc/api.md
new file mode 100644
index 000000000..b45a8a464
--- /dev/null
+++ b/docs/v0.6.1-rc/api.md
@@ -0,0 +1,205 @@
+# API
+
+This page provides a brief overview of the NVCF API. All API endpoints are served through your gateway. See [gateway-routing](./gateway-routing.md) for details on configuring your gateway domain and DNS.
+
+## OpenAPI Specification
+
+This page does not cover all endpoints.
+
+Please refer to the [OpenAPI Spec](https://api.nvcf.nvidia.com/v3/openapi) for the latest API information.
+
+
+The OpenAPI spec linked above documents the full NVCF API surface. Replace the hosted domain with your own gateway domain when making requests. See [gateway-routing](./gateway-routing.md) for your deployment's base URL.
+
+
+
+The NVCF API is divided into the following sets of APIs:
+
+| APIs | Usage |
+| --------------------- | ------------------------------------------------------------------------------------- |
+| Function Invocation | Execution of a function that runs on a worker node. Usually an inference call. |
+| Cluster Groups & GPUs | Defines endpoints to list Cluster Groups and GPUs as targets for function deployment. |
+| Function Management | The creation, modification and deletion of functions |
+| Function Deployment | Endpoints for creating and managing function deployments. |
+| Task Management | GPU-backed batch jobs via NVIDIA Cloud Tasks (NVCT). |
+
+**API Versioning**
+
+All API endpoints include versioning in the path prefix.
+
+```bash
+/v2/nvcf
+```
+
+## Authorization
+
+Self-hosted NVCF uses three bearer credential types:
+
+| Credential | CLI source | Typical use |
+| --- | --- | --- |
+| `NVCF_TOKEN` | `nvcf-cli init` | Default CLI credential for management operations and self-hosted cluster management |
+| `NVCF_API_KEY` | `nvcf-cli api-key generate` | Default CLI credential for function invocation, function discovery, and queue status |
+| `NVCF_NVCT_API_KEY` | `nvcf-cli api-key generate` | Credential for task commands (`task create`, `task list`, etc.) |
+
+All three credential types are sent as bearer credentials:
+
+```bash
+Authorization: Bearer
+```
+
+Use the CLI for normal credential management:
+
+```bash
+nvcf-cli init
+nvcf-cli api-key generate
+```
+
+`nvcf-cli init` calls the API Keys admin endpoint and mints the JWT used as
+`NVCF_TOKEN`. The CLI saves the token in `~/.nvcf-cli.state` for the default
+configuration and reads it automatically on later commands. Export it only when
+you are making direct API calls or using tooling that reads environment
+variables:
+
+```bash
+export NVCF_TOKEN=$(jq -r .token ~/.nvcf-cli.state)
+```
+
+The default self-hosted admin issuer role gives this token the following
+scopes:
+
+- `register_function`
+- `list_functions`
+- `list_functions_details`
+- `deploy_function`
+- `update_function`
+- `update_secrets`
+- `delete_function`
+- `manage_telemetries`
+- `manage_registry_credentials`
+- `cluster-management`
+
+The function API key (`NVCF_API_KEY`) is created with these default scopes:
+
+- `invoke_function`
+- `list_functions`
+- `queue_details`
+- `list_functions_details`
+
+The task API key (`NVCF_NVCT_API_KEY`) is created with these default scopes:
+
+- `launch_task`
+- `list_tasks`
+- `task_details`
+- `cancel_task`
+- `delete_task`
+- `list_events`
+- `list_results`
+- `update_secrets`
+
+NVCF API authorization is scope-based. For the NVCF API endpoints below, either
+`NVCF_TOKEN` or `NVCF_API_KEY` can be used if that bearer includes the required
+scope. The CLI prefers `NVCF_API_KEY` for read, invoke, and queue commands. It
+prefers `NVCF_TOKEN` for management commands when both credentials are
+configured.
+
+Pass `--scopes` when you need to narrow a key or create a key with management
+scopes for testing.
+
+### Scope reference
+
+The OpenAPI spec describes the permission checked for each endpoint. "Accepted
+bearer" means which saved CLI credential type can be sent as
+`Authorization: Bearer ` when it includes the listed scope.
+
+| Category | Method and endpoint | Accepted bearer | Scope |
+| --- | --- | --- | --- |
+| Function invocation | `POST /v2/nvcf/pexec/functions/{functionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `invoke_function` |
+| Function invocation | `POST /v2/nvcf/pexec/functions/{functionId}/versions/{versionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `invoke_function` |
+| Function invocation | `GET /v2/nvcf/pexec/status/{requestId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `invoke_function` |
+| Queue details | `GET /v2/nvcf/queues/{requestId}/position` | `NVCF_TOKEN` or `NVCF_API_KEY` | `queue_details` |
+| Queue details | `GET /v2/nvcf/queues/functions/{functionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `queue_details` |
+| Queue details | `GET /v2/nvcf/queues/functions/{functionId}/versions/{versionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `queue_details` |
+| Function management | `GET /v2/nvcf/functions` | `NVCF_TOKEN` or `NVCF_API_KEY` | `list_functions` or `list_functions_details` |
+| Function management | `GET /v2/nvcf/functions/ids` | `NVCF_TOKEN` or `NVCF_API_KEY` | `list_functions` or `list_functions_details` |
+| Function management | `POST /v2/nvcf/functions` | `NVCF_TOKEN` or `NVCF_API_KEY` | `register_function` |
+| Function management | `GET /v2/nvcf/functions/{functionId}/versions` | `NVCF_TOKEN` or `NVCF_API_KEY` | `list_functions` or `list_functions_details` |
+| Function management | `POST /v2/nvcf/functions/{functionId}/versions` | `NVCF_TOKEN` or `NVCF_API_KEY` | `register_function` |
+| Function management | `GET /v2/nvcf/functions/{functionId}/versions/{functionVersionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `list_functions` or `list_functions_details` |
+| Function management | `PUT /v2/nvcf/functions/{functionId}/versions/{functionVersionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `update_function` |
+| Function management | `DELETE /v2/nvcf/functions/{functionId}/versions/{functionVersionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `delete_function` |
+| Function management | `PUT /v2/nvcf/metadata/functions/{functionId}/versions/{functionVersionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `update_function` |
+| Function deployment | `GET /v2/nvcf/deployments/functions/{functionId}/versions/{functionVersionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` |
+| Function deployment | `POST /v2/nvcf/deployments/functions/{functionId}/versions/{functionVersionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` |
+| Function deployment | `PUT /v2/nvcf/deployments/functions/{functionId}/versions/{functionVersionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` |
+| Function deployment | `DELETE /v2/nvcf/deployments/functions/{functionId}/versions/{functionVersionId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` |
+| Function deployment | `GET /v2/nvcf/deployments/{deploymentId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` |
+| Function deployment | `PATCH /v2/nvcf/deployments/{deploymentId}/gpu-specifications/{gpuSpecId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` |
+| Registry credential management | `GET /v2/nvcf/registry-credentials` | `NVCF_TOKEN` or `NVCF_API_KEY` | `manage_registry_credentials` |
+| Registry credential management | `POST /v2/nvcf/registry-credentials` | `NVCF_TOKEN` or `NVCF_API_KEY` | `manage_registry_credentials` |
+| Registry credential management | `GET /v2/nvcf/registry-credentials/{registryCredentialId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `manage_registry_credentials` |
+| Registry credential management | `PATCH /v2/nvcf/registry-credentials/{registryCredentialId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `manage_registry_credentials` |
+| Registry credential management | `DELETE /v2/nvcf/registry-credentials/{registryCredentialId}` | `NVCF_TOKEN` or `NVCF_API_KEY` | `manage_registry_credentials` |
+| Registry credential management | `GET /v2/nvcf/recognized-registries` | `NVCF_TOKEN` or `NVCF_API_KEY` | `manage_registry_credentials` |
+
+Self-hosted cluster management uses SIS endpoints from the Spot API. In
+self-hosted CLI workflows, use `NVCF_TOKEN` with the `cluster-management` scope
+for these endpoints.
+
+| Method and endpoint | Accepted bearer | Scope |
+| --- | --- | --- |
+| `GET /v1/accounts/{ncaId}/clusters` | `NVCF_TOKEN` | `cluster-management` |
+| `POST /v1/accounts/{ncaId}/clusters` | `NVCF_TOKEN` | `cluster-management` |
+| `GET /v1/accounts/{ncaId}/clusters/{clusterId}` | `NVCF_TOKEN` | `cluster-management` |
+| `PUT /v1/accounts/{ncaId}/clusters/{clusterId}` | `NVCF_TOKEN` | `cluster-management` |
+| `DELETE /v1/accounts/{ncaId}/clusters/{clusterId}` | `NVCF_TOKEN` | `cluster-management` |
+| `GET /v1/accounts/{ncaId}/clusterVersions` | `NVCF_TOKEN` | `cluster-management` |
+
+Task management uses NVCT endpoints served through your gateway. All task
+endpoints require `NVCF_NVCT_API_KEY` with the listed scope.
+
+| Method and endpoint | Accepted bearer | Scope |
+| --- | --- | --- |
+| `POST /v1/nvct/tasks` | `NVCF_NVCT_API_KEY` | `launch_task` |
+| `GET /v1/nvct/tasks` | `NVCF_NVCT_API_KEY` | `list_tasks` |
+| `POST /v1/nvct/tasks/bulk` | `NVCF_NVCT_API_KEY` | `list_tasks` |
+| `GET /v1/nvct/tasks/{taskId}` | `NVCF_NVCT_API_KEY` | `task_details` |
+| `DELETE /v1/nvct/tasks/{taskId}` | `NVCF_NVCT_API_KEY` | `delete_task` |
+| `POST /v1/nvct/tasks/{taskId}/cancel` | `NVCF_NVCT_API_KEY` | `cancel_task` |
+| `GET /v1/nvct/tasks/{taskId}/events` | `NVCF_NVCT_API_KEY` | `list_events` |
+| `GET /v1/nvct/tasks/{taskId}/results` | `NVCF_NVCT_API_KEY` | `list_results` |
+| `PUT /v1/nvct/secrets/tasks/{taskId}` | `NVCF_NVCT_API_KEY` | `update_secrets` |
+
+### Direct API key creation
+
+The CLI calls the API Keys service for you. If you need to call it directly,
+set `expires_at` and the `scopes` array explicitly.
+
+```bash
+GATEWAY_ADDR=
+EXPIRES_AT=$(date -u -v+1d '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || date -u -d '+1 day' '+%Y-%m-%dT%H:%M:%SZ')
+SERVICE_ID="nvidia-cloud-functions-ncp-service-id-aketm"
+
+curl -s -X POST "http://${GATEWAY_ADDR}/v1/keys" \
+ -H "Host: api-keys.${GATEWAY_ADDR}" \
+ -H "Content-Type: application/json" \
+ -H "Key-Issuer-Service: nvcf-api" \
+ -H "Key-Issuer-Id: ${SERVICE_ID}" \
+ -H "Key-Owner-Id: test@nvcf-api.local" \
+ -d '{
+ "description": "invocation key",
+ "expires_at": "'"${EXPIRES_AT}"'",
+ "authorizations": {
+ "policies": [{
+ "aud": "'"${SERVICE_ID}"'",
+ "auds": ["'"${SERVICE_ID}"'"],
+ "product": "nv-cloud-functions",
+ "resources": [
+ {"id": "*", "type": "account-functions"},
+ {"id": "*", "type": "authorized-functions"}
+ ],
+ "scopes": ["invoke_function", "list_functions", "queue_details", "list_functions_details"]
+ }]
+ },
+ "audience_service_ids": ["'"${SERVICE_ID}"'"]
+ }' | jq .
+```
diff --git a/docs/v0.6.1-rc/autoscaling/architecture.md b/docs/v0.6.1-rc/autoscaling/architecture.md
new file mode 100644
index 000000000..532aa36ba
--- /dev/null
+++ b/docs/v0.6.1-rc/autoscaling/architecture.md
@@ -0,0 +1,64 @@
+# Function Autoscaler Architecture
+
+The function autoscaler is a Rust service deployed as a horizontally scaled Kubernetes Deployment. It reads utilization and request metrics from a Prometheus-compatible timeseries database, stores discovered functions and coordination state in Cassandra, and writes desired instance counts to the NVCF API. Cassandra lightweight transactions handle leader election and short-lived per-function locks.
+
+The work is split into two loops. A leader-elected discovery loop scans the timeseries database for active function versions and upserts them into Cassandra. A scaling loop runs on every replica, but each replica only handles the functions whose IDs hash into its assigned buckets, so the active set is sharded across replicas.
+
+## Sequence Diagram
+
+```mermaid
+sequenceDiagram
+ participant Workers as Workers / Invocation Services
+ participant TSDB as Time Series DB
+ participant Autoscaler as Function Autoscaler
+ participant Cassandra as Cassandra
+ participant NVCF as NVCF Service
+
+ Workers->>TSDB: Emit utilization and instance metrics
+
+ Note over Autoscaler,Cassandra: Discovery loop (~15s, leader-elected)
+ Autoscaler->>TSDB: Query active functions
+ TSDB-->>Autoscaler: Function set
+ Autoscaler->>Cassandra: Upsert newly discovered functions
+
+ Note over Autoscaler,NVCF: Scaling loop (~30s, per-bucket)
+ Autoscaler->>Cassandra: Read active functions for this node's buckets
+ Autoscaler->>TSDB: Query current instances and utilization history
+ TSDB-->>Autoscaler: Metrics
+ Autoscaler->>TSDB: Check recent invocations (scale-to-zero guard, PromQL)
+ Note over Autoscaler: Compute desired instance count
+ Autoscaler->>NVCF: PUT requiredNumberOfInstances
+ NVCF-->>Autoscaler: OK or error
+ Autoscaler->>Cassandra: Write predicted count, refresh function TTL
+```
+
+The discovery loop runs on one leader-elected replica. The scaling loop runs on every replica, but each replica only processes the function buckets assigned to it.
+
+## Timeseries Database
+
+The function autoscaler is a read-only client of a Prometheus-compatible timeseries store. It calls the `/api/v1/query_range` HTTP endpoint and uses PromQL for every metric query, so any backend that implements that interface works: upstream Prometheus, Thanos, Grafana Mimir, or VictoriaMetrics. The reference NVCF deployments point at VictoriaMetrics via the `timeseries_db_url` setting.
+
+The function autoscaler does not run a scrape config of its own and does not write samples. Before it can do anything useful, the rest of the data plane has to be feeding the same store:
+
+- Worker pods export utilization and instance count metrics (`nvcf_worker_service_worker_thread_busy_seconds_total`, `nvcf_worker_service_worker_thread_count_total`, instance gauges).
+- Invocation services and the gRPC proxy export request counters (`function_request`, `function_request_total`) labeled by `function_id`, `function_version_id`, and `nca_id`. These labels are how the discovery loop finds active function versions.
+
+For a self-hosted control plane, you need three things in place before bringing the function autoscaler online:
+
+1. A Prometheus-compatible store reachable from the function autoscaler pod.
+2. A scrape configuration (or remote-write feed) covering the worker pods and the invocation-plane services.
+3. The resulting query endpoint passed in as `timeseries_db_url`. The function autoscaler reports `not ready` on its readiness probe until that endpoint responds.
+
+## Coordination and Self-Healing
+
+Coordination relies on Cassandra TTLs to recover from failures without operator intervention:
+
+- The discovery lock self-expires if the leader replica crashes or is partitioned, so a new leader takes over on the next loop iteration.
+- Bucket ownership is recomputed when replicas join or leave. During a reshuffle a function may be skipped for a single scaling cycle or briefly picked up by a different replica, and a short-lived per-function lock prevents two replicas from racing on the same function in that window.
+- Each active function row carries a TTL refreshed by every scaling cycle, so functions that stop emitting metrics age out of the active set automatically.
+
+## See Also
+
+- [Configure Autoscaling](../configure-autoscaling.md) for setting per-function scaling bounds, factors, thresholds, and stickiness via the NVCF API.
+- [Function Autoscaler Operations](./operations.md) for health endpoints and common issues.
+- [Function Autoscaler Observability](./observability.md) for emitted metrics, traces, and logs.
diff --git a/docs/v0.6.1-rc/autoscaling/index.md b/docs/v0.6.1-rc/autoscaling/index.md
new file mode 100644
index 000000000..525d7bf7b
--- /dev/null
+++ b/docs/v0.6.1-rc/autoscaling/index.md
@@ -0,0 +1,37 @@
+# Function Autoscaling
+
+The NVCF Function Autoscaler is a distributed Rust service that monitors function utilization and uses it to determine the ideal instance count per function on the NVCF control plane. It runs as a horizontally scaled deployment on the same Kubernetes cluster as the rest of the control plane.
+
+On an interval, the function autoscaler reads metrics from the timeseries database, decides how many instances each function should have, and calls the NVCF API to apply that decision.
+
+The function autoscaler depends on a Prometheus-compatible timeseries database fed by the worker pods and invocation-plane services. Without it, the service reports `not ready` and makes no scaling decisions. See [Timeseries database](./architecture.md#timeseries-database) for the required metrics and endpoints.
+
+## Function Autoscaler vs Horizontal Pod Autoscaler
+
+Function autoscaling is distinct from Kubernetes horizontal pod autoscaling (HPA). HPA scales pods within a single cluster, so it cannot reach NVCF worker pods that are spread across multiple clusters. Function autoscaling orchestrates scaling across clusters using global load patterns.
+
+## Key Functionality
+
+- Discovers active functions from invocation and worker metrics in the timeseries database and persists the active set in Cassandra.
+- Periodically computes a desired instance count per function from recent utilization and the function's scaling policy.
+- Applies the desired count by calling the NVCF API's predictions endpoint.
+- Coordinates work across replicas using hash-based bucket assignment and Cassandra Lightweight Transaction (LWT) distributed locks.
+
+## Architecture Overview
+
+```mermaid
+flowchart LR
+ Workers[Workers / Invocation Services] --> TSDB[(Time Series DB)]
+ TSDB --> Autoscaler[Function Autoscaler]
+ Autoscaler <--> Cassandra[(Cassandra)]
+ Autoscaler --> NVCF[NVCF API]
+```
+
+See [Architecture](./architecture.md#sequence-diagram) for the end-to-end sequence diagram and the bucket model.
+
+## See Also
+
+- [Architecture](./architecture.md) for components, data flow, and the Cassandra LWT lock behavior that elects the discovery leader.
+- [Configure Autoscaling](../configure-autoscaling.md) for setting per-function scaling bounds, factors, thresholds, and stickiness via the NVCF API.
+- [Function Autoscaler Operations](./operations.md) for health endpoints and operational guidance.
+- [Function Autoscaler Observability](./observability.md) for the metrics, traces, and logs emitted by the service.
diff --git a/docs/v0.6.1-rc/autoscaling/observability.md b/docs/v0.6.1-rc/autoscaling/observability.md
new file mode 100644
index 000000000..131a30129
--- /dev/null
+++ b/docs/v0.6.1-rc/autoscaling/observability.md
@@ -0,0 +1,67 @@
+# Function Autoscaler Observability
+
+The function autoscaler emits structured logs, Prometheus metrics that explain dependency health statuses and scaling decisions, and OpenTelemetry spans for outbound calls to its dependencies. The Prometheus exporter serves metrics on the address configured in `server.metrics.exporters`. The local settings file at `crates/server/resources/settings-local.yaml` uses `0.0.0.0:41338`.
+
+Job and namespace labels follow the standard NVCF naming convention for the cluster that runs the function autoscaler.
+
+## Metric reference
+
+| Metric name | Metric type | Description |
+|-------------|-------------|-------------|
+| `nvcf_autoscaler.autoscaling.status` | Gauge | Scaling status per function, encoded as a reason code. |
+| `nvcf_autoscaler.scaling.current_instances` | Gauge | Current instance count per function as read from the timeseries database. |
+| `nvcf_autoscaler.scaling.desired_instances` | Gauge | Desired instance count computed by the scaling decision. |
+| `nvcf_autoscaler.scaling.utilization` | Gauge | Utilization percentage per function used in the scaling decision. |
+| `nvcf_autoscaler.requests.queued_total` | Counter | Scaling requests queued for processing. |
+| `nvcf_autoscaler.requests.processed_total` | Counter | Scaling requests processed. |
+| `nvcf_autoscaler.requests.rejected_total` | Counter | Scaling requests rejected by the policy or guard rails. |
+| `nvcf_autoscaler.requests.rate_limited_total` | Counter | Scaling requests rate-limited downstream. |
+| `nvcf_autoscaler.queue.size` | Gauge | Current depth of the scaling work queue. |
+| `nvcf_autoscaler.queue.capacity` | Gauge | Configured capacity of the scaling work queue. |
+| `nvcf_autoscaler.function_table_state` | Gauge | State of the active function table entry per function. |
+| `nvcf_autoscaler.function_discovery_duration_seconds` | Histogram | Duration of each discovery loop run. |
+| `nvcf_autoscaler.timeseries_db.requests_total` | Counter | Timeseries database requests, labeled by status. |
+| `nvcf_autoscaler.timeseries_db.request_duration_milliseconds` | Histogram | Timeseries database request latency. |
+| `nvcf_autoscaler.timeseries_db.auth_failure_total` | Counter | Timeseries database authentication failures. |
+| `nvcf_autoscaler.timeseries_db.server_side_failure_total` | Counter | Timeseries database server-side query failures. |
+| `nvcf_autoscaler.nvcf_api.request_duration_milliseconds` | Histogram | NVCF API request latency. |
+| `nvcf_autoscaler.oauth2_api.request_duration_milliseconds` | Histogram | OAuth2 token endpoint request latency. |
+| `nvcf_autoscaler.oauth2_client.token_refresh_failure_total` | Counter | OAuth2 client token refresh failures. |
+| `nvcf_autoscaler.cassandra.health_status` | Gauge | Cassandra client health. 1 indicates healthy, 0 indicates unhealthy. |
+| `nvcf_autoscaler.health.overall_status` | Gauge | Overall service health status. |
+| `nvcf_autoscaler.health.component_status` | Gauge | Per-component health status. |
+| `nvcf_autoscaler.distributed_lock` | Gauge | State of the discovery distributed lock for this replica. |
+| `nvcf_autoscaler.distributed_lock.acquisition_failures_total` | Counter | Discovery lock acquisition failures. |
+| `nvcf_autoscaler.processing.utilization_data_age_milliseconds` | Histogram | Age of the utilization data used in each scaling decision. |
+
+## Tracing
+
+The function autoscaler emits OpenTelemetry spans for outbound calls to the timeseries database and the NVCF API, with the OTLP endpoint and span filter configurable under `server.tracing`.
+
+## Logging
+
+The function autoscaler writes structured logs to stdout. Set log filter directives in the `server.envfilter_directive` configuration field. The format follows the `tracing_subscriber` env filter syntax (Rust ecosystem standard):
+
+```yaml
+server:
+ envfilter_directive: "server=info,rs_autoscaler=debug,rs_autoscaler::cassandra=warn,info"
+```
+
+The same syntax applies to `server.tracing.logging_envfilter_directive` if you separate logging and tracing filters.
+
+Useful target prefixes:
+
+| Target | Covers |
+|--------|--------|
+| `server` | Binary entry point: startup, server lifecycle. |
+| `rs_autoscaler` | Top-level function autoscaler library crate. |
+| `rs_autoscaler::work` | Scaling loop, discovery loop, bucket reshuffles. |
+| `rs_autoscaler::cassandra` | Cassandra client, LWT lock operations. |
+| `rs_autoscaler::nvcf_api` | OAuth2, NVCF API calls. |
+| `rs_autoscaler::timeseries_db` | Timeseries database query traces. |
+
+## See also
+
+- [Function Autoscaler Operations](./operations.md) for common symptoms tied to these metrics and log lines.
+- [Architecture](./architecture.md) for the components that emit each signal.
+- [Configure Autoscaling](../configure-autoscaling.md) for setting per-function scaling bounds and policy via the NVCF API.
diff --git a/docs/v0.6.1-rc/autoscaling/operations.md b/docs/v0.6.1-rc/autoscaling/operations.md
new file mode 100644
index 000000000..08e74d5b5
--- /dev/null
+++ b/docs/v0.6.1-rc/autoscaling/operations.md
@@ -0,0 +1,65 @@
+# Function Autoscaler Operations
+
+This page covers operating the function autoscaler after deployment, including health probes, common operational issues, and pointers to the Helm chart values. For log filter syntax, metrics, and traces, see [Function Autoscaler Observability](./observability.md).
+
+## Health endpoints
+
+The function autoscaler exposes three HTTP health endpoints. Their exact paths differ from the rest of the NVCF control plane: liveness and readiness are namespaced under `/admin/health/`.
+
+| Endpoint | Purpose | Use as |
+|----------|---------|--------|
+| `GET /admin/health/liveness` | Always returns 200. Indicates the process is alive. | Kubernetes liveness probe. |
+| `GET /admin/health/readiness` | Returns 200 when all components are healthy, 503 otherwise. | Kubernetes readiness probe. |
+| `GET /health` | Returns per-component health for `cassandra_client` and `timeseries_db_client`. | Operator-facing detail and dashboards. |
+
+The liveness probe deliberately does not check Cassandra or the timeseries database. Restarting the pod when those are unreachable does not help, so the function autoscaler stays running and lets readiness flip instead.
+
+## Common operational issues
+
+### Cassandra connection failures
+
+Symptoms: readiness flips to 503, `/health` reports the `cassandra_client` component as unhealthy, log lines from `rs_autoscaler::cassandra` show connection errors.
+
+Checks:
+
+- SSL certificates are mounted at the path expected by `cassandra.ssl`. The function autoscaler container expects the cert directory to exist; create `/etc/app/config` if it is missing.
+- Credentials in the secrets file are valid for the configured keyspace.
+- The contact points resolve from the pod's network namespace.
+
+### Timeseries database query failures
+
+Symptoms: `nvcf_autoscaler.timeseries_db.requests_total` shows a rising error count, `auth_failure_total` or `server_side_failure_total` is non-zero, log lines from `rs_autoscaler::timeseries_db` show 4xx or 5xx responses.
+
+Checks:
+
+- `timeseries_db.timeseries_db_url` is reachable from the pod.
+- The bearer token in the secrets file is current. Token rotation is the most common cause of `auth_failure_total` spikes.
+- Query time ranges fit the retention window of the backing store.
+
+### NVCF API errors
+
+Symptoms: `nvcf_autoscaler.nvcf_api.request_duration_milliseconds` shows a sustained rise in 4xx or 5xx, scaling decisions stop applying.
+
+Checks:
+
+- The OAuth2 token endpoint is reachable and the client credentials in the secrets file are valid.
+- The functions being scaled are still in a deployable status. Functions in unexpected states are skipped, not retried.
+- `nvcf_api.disable_auth` is set as intended for the deployment. Leave it `false` whenever the NVCF API enforces authentication.
+
+### Discovery is stalled
+
+Symptoms: the active function set in Cassandra stops growing despite traffic to new functions, `nvcf_autoscaler.distributed_lock.acquisition_failures_total` is rising across all replicas.
+
+Checks:
+
+- Inspect the `locks` table for the discovery lock row and its TTL. If the row never expires, the previous leader may have stopped refreshing without releasing it.
+- Confirm at least one replica's `nvcf_autoscaler.distributed_lock` gauge reports the leader state.
+- Restart the holding replica if the cluster is otherwise healthy. The lock expires within `discovery_lock_duration_seconds`.
+
+See [Architecture](./architecture.md#cassandra-lightweight-transactions-lwts) for the lock state machine.
+
+## See also
+
+- [Function Autoscaler Observability](./observability.md) for the metrics and traces referenced in the symptoms above.
+- [Configure Autoscaling](../configure-autoscaling.md) for setting per-function scaling bounds and policy via the NVCF API.
+- [Architecture](./architecture.md) for the component layout these symptoms map to.
diff --git a/docs/v0.6.1-rc/caches.md b/docs/v0.6.1-rc/caches.md
new file mode 100644
index 000000000..487327733
--- /dev/null
+++ b/docs/v0.6.1-rc/caches.md
@@ -0,0 +1,47 @@
+# Simulation Cluster Caches
+
+This section covers cache components for self-hosted NVCF deployments. Caches improve performance by storing frequently accessed content locally, reducing network bandwidth usage and accelerating scene loading.
+
+## Overview
+
+Self-hosted NVCF supports several cache components:
+
+- **Derived Data Cache Service (DDCS)** - Caches derived content to reduce scene load time and improve rendering performance
+- **USD Content Cache (UCC)** - Caches USD content from object storage to accelerate scene loading
+
+## When to Use Caches
+
+See the individual cache component guides for detailed information on when to use each cache:
+
+- [Derived Data Cache Service](https://docs.omniverse.nvidia.com/ovcaches/ddcs/5.0/) - Derived Data Cache Service
+- [USD Content Cache](https://docs.omniverse.nvidia.com/ovcaches/ucc/3.0/) - USD Content Cache
+
+## Documentation
+
+Each cache component has comprehensive documentation covering configuration, deployment, and advanced features:
+
+## Configuration
+
+Cache components are configured using Helm values files. Each cache guide includes:
+
+- Base configuration examples
+- Configuration options and parameters
+- Performance tuning recommendations
+- Best practices
+
+## Monitoring
+
+Cache components include Prometheus metrics for monitoring:
+
+- Cache hit ratios
+- Storage utilization
+- Request throughput
+- Performance metrics
+
+## Next Steps
+
+1. **Review cache guides** - Read the detailed guides for caches you want to deploy
+2. **Plan your deployment** - Determine which caches fit your use case
+3. **Configure caches** - Set up Helm values files based on the examples
+4. **Deploy caches** - Install caches using Helm or Helmfile
+5. **Monitor performance** - Set up monitoring dashboards and alerts
diff --git a/docs/v0.6.1-rc/cli.md b/docs/v0.6.1-rc/cli.md
new file mode 100644
index 000000000..a84be034c
--- /dev/null
+++ b/docs/v0.6.1-rc/cli.md
@@ -0,0 +1,1078 @@
+# Self-hosted CLI
+
+This page provides documentation for the NVCF Self-hosted CLI, a command-line interface for managing NVIDIA Cloud Functions in self-hosted deployments.
+
+## Overview
+
+The NVCF Self-hosted CLI provides:
+
+- Automatic Token Generation: Generate `NVCF_TOKEN` and API keys via direct API calls
+- Smart State Management: Persistent workflow context eliminates manual ID copying
+- Multi-Environment Support: Separate configurations for dev/staging/production
+- gRPC Invocation: Native support for gRPC function invocation
+- Shell Completion: Autocompletion for bash, zsh, fish, and PowerShell
+
+## Prerequisites
+
+- Network access to NVCF API endpoints
+- A source checkout with Bazel/Bazelisk when you build from the repository
+- [NGC CLI installed](https://org.ngc.nvidia.com/setup/installers/cli) when you download the CLI release from NGC
+
+## Installation
+
+You can build `nvcf-cli` from this repository or download a packaged CLI release
+from NGC. Use the source build when you are validating local changes or running
+the local k3d quickstart from a repository checkout.
+
+### Build from the repository
+
+Run the build from the repository root:
+
+```bash
+bazel build //src/clis/nvcf-cli:nvcf-cli
+```
+
+The binary is written to:
+
+```text
+bazel-bin/src/clis/nvcf-cli/nvcf-cli_/nvcf-cli
+```
+
+Install it on your `PATH`:
+
+```bash
+install -m 0755 \
+ bazel-bin/src/clis/nvcf-cli/nvcf-cli_/nvcf-cli \
+ /usr/local/bin/nvcf-cli
+```
+
+If your environment cannot reach the configured Bazel remote cache, disable the
+remote cache for this build:
+
+```bash
+bazel build --remote_cache= //src/clis/nvcf-cli:nvcf-cli
+```
+
+### Download from NGC
+
+The CLI is available as a resource from NGC. See
+[download-nvcf-cli](./image-mirroring.md) for detailed download and extraction
+instructions.
+
+The downloaded package includes:
+
+- `nvcf-cli` - The CLI binary
+- `.nvcf-cli.yaml.template` - Configuration template
+- `examples/` - Sample configuration files
+- `USAGE-GUIDE.md` - Detailed usage documentation
+
+## Configuration
+
+The CLI uses YAML configuration files. If you downloaded the packaged CLI, copy
+the included template:
+
+```bash
+cp .nvcf-cli.yaml.template .nvcf-cli.yaml
+```
+
+If you built the CLI from source, create `.nvcf-cli.yaml` from the examples
+below or from `src/clis/nvcf-cli/examples/config-dev.yaml`.
+
+Configuration files are searched in this order:
+
+1. Explicit path via `--config` flag (highest priority)
+2. Current directory: `./.nvcf-cli.yaml`
+3. Home directory: `~/.nvcf-cli.yaml`
+
+
+Place your `.nvcf-cli.yaml` in the directory where you run the CLI for project-specific configuration, or in your home directory for global configuration.
+
+
+
+### Self-Hosted Configuration
+
+For self-hosted deployments, the CLI must be configured to communicate with
+your gateway. The gateway uses hostname-based routing for HTTP services.
+
+
+For Gateway routing details, including architecture diagrams, verification commands, and production DNS/HTTPS setup, see [gateway-routing](./gateway-routing.md).
+
+
+
+#### Prepare Gateway API ingress
+
+For remote Helmfile deployments, set up Gateway API ingress before you configure
+the CLI. The CLI calls the configured API, API Keys,
+invocation, and gRPC endpoints during token minting, cluster registration,
+health checks, and function operations.
+
+Complete [Gateway quickstart](./gateway-routing.md#gateway-quickstart) before you
+configure the CLI. That procedure installs the Gateway API CRDs, creates and
+labels the required namespaces, installs Envoy Gateway, creates the GatewayClass
+and Gateway, waits for the Gateway to be programmed, and exports:
+
+```bash
+echo "$HTTP_GATEWAY_NAMESPACE/$HTTP_GATEWAY_NAME"
+echo "$GRPC_GATEWAY_NAMESPACE/$GRPC_GATEWAY_NAME"
+echo "$GATEWAY_ADDR"
+echo "$GRPC_GATEWAY_ADDR"
+```
+
+These are the same Gateway setup steps used by the Helmfile and standalone
+install paths. Keep the exported values in your shell, then configure the CLI.
+The local k3d quickstart uses local route hostnames instead.
+
+For test environments without production DNS, use the Gateway load balancer
+address as the stack domain:
+
+```bash
+export STACK_DOMAIN="$GATEWAY_ADDR"
+```
+
+For production environments, set `STACK_DOMAIN` to the DNS name that your
+HTTPRoute hostnames use.
+
+#### Configuring the CLI
+
+Create your configuration file:
+
+```bash
+# Copy the template
+cp .nvcf-cli.yaml.template .nvcf-cli.yaml
+```
+
+Complete self-hosted configuration:
+
+```yaml
+# ==============================================================================
+# API Endpoints - Point to your gateway load balancer
+# ==============================================================================
+
+# Main API endpoint (use http:// for non-TLS setups)
+base_http_url: "http://"
+
+# Invocation endpoint (same as base_http_url for self-hosted)
+invoke_url: "http://"
+
+# gRPC endpoint - uses dedicated TCP port (no Host header needed)
+base_grpc_url: ":10081"
+
+# API Keys service endpoint
+api_keys_service_url: "http://"
+
+# ==============================================================================
+# Host Header Overrides (Required for Hostname-Based Routing)
+# ==============================================================================
+#
+# Because the gateway routes HTTP requests based on the Host header,
+# you must specify the correct Host header for each service.
+# These values must match the HTTPRoute hostnames rendered from STACK_DOMAIN.
+#
+# Without these, the gateway returns 404 because it can't match the route.
+
+# Host header for API Keys service
+api_keys_host: "api-keys."
+
+# Host header for NVCF API (function management)
+api_host: "api."
+
+# Host header for Invocation service
+invoke_host: "invocation."
+
+# ==============================================================================
+# API Keys Service Configuration
+# ==============================================================================
+
+api_keys_service_id: "nvidia-cloud-functions-ncp-service-id-aketm"
+api_keys_issuer_service: "nvcf-api"
+api_keys_owner_id: "svc@nvcf-api.local"
+
+# ==============================================================================
+# Account Configuration
+# ==============================================================================
+
+client_id: "nvcf-default"
+
+# ==============================================================================
+# Debugging (optional - set to true for verbose output)
+# ==============================================================================
+
+debug: false
+```
+
+For test environments without production DNS, the URL fields and host fields
+can use the Gateway load balancer address:
+
+```yaml
+# Gateway load balancer address
+base_http_url: "http://a1b2c3d4e5f6.us-west-2.elb.amazonaws.com"
+invoke_url: "http://a1b2c3d4e5f6.us-west-2.elb.amazonaws.com"
+base_grpc_url: "a1b2c3d4e5f6.us-west-2.elb.amazonaws.com:10081"
+api_keys_service_url: "http://a1b2c3d4e5f6.us-west-2.elb.amazonaws.com"
+
+# Host headers matching HTTPRoute configuration
+api_keys_host: "api-keys.a1b2c3d4e5f6.us-west-2.elb.amazonaws.com"
+api_host: "api.a1b2c3d4e5f6.us-west-2.elb.amazonaws.com"
+invoke_host: "invocation.a1b2c3d4e5f6.us-west-2.elb.amazonaws.com"
+
+# API Keys service
+api_keys_service_id: "nvidia-cloud-functions-ncp-service-id-aketm"
+api_keys_issuer_service: "nvcf-api"
+api_keys_owner_id: "svc@nvcf-api.local"
+
+client_id: "nvcf-default"
+debug: true
+```
+
+#### Verifying Your Configuration
+
+After configuring the CLI, verify connectivity:
+
+```bash
+# Test token generation (uses api_keys_host)
+./nvcf-cli init
+
+# Expected output:
+# [INFO] Starting fresh session...
+# [INFO] Generating admin token from API Keys service...
+# [DEBUG] Using Host header override: api-keys.
+# [SUCCESS] Admin token generated and saved
+```
+
+If you see a 404 error, verify:
+
+1. The `api_keys_host` value matches your HTTPRoute hostname
+2. The gateway load balancer is accessible
+3. The API Keys service is running: `kubectl get pods -n api-keys`
+
+
+Why Host headers are needed: the Envoy Gateway uses hostname-based routing to
+direct traffic to different backend services through a single load balancer.
+Without the correct `Host` header, the gateway cannot match the request to a
+route and returns 404.
+
+
+
+
+gRPC does not need Host headers because it uses a dedicated TCP listener on
+port 10081. The gateway routes all traffic on that port directly to the gRPC
+service without hostname matching.
+
+
+
+#### Production Setup: DNS and HTTPS
+
+The Host header configuration above is designed for testing and development. For production deployments, configure proper DNS and TLS to eliminate the need for Host header overrides.
+
+With proper DNS and HTTPS configured:
+
+- DNS records resolve service hostnames directly to your Gateway's load balancer
+- TLS certificates secure all traffic
+- The CLI uses simple URLs without Host header overrides
+- Browsers and other clients can access services directly
+
+```yaml
+# Simple URLs using your domain - no Host header overrides needed!
+base_http_url: "https://api.nvcf.example.com"
+invoke_url: "https://invocation.nvcf.example.com"
+base_grpc_url: "grpc.nvcf.example.com:443"
+api_keys_service_url: "https://api-keys.nvcf.example.com"
+
+# No host header overrides required - DNS handles routing
+# api_keys_host: "" # Not needed
+```
+
+
+For complete instructions on setting up DNS records and TLS certificates, see [production-dns-https](./gateway-routing.md) in the Gateway Routing guide.
+
+
+
+#### Multi-Environment Setup
+
+Use the `--config` flag to manage multiple environments with separate configuration files:
+
+```bash
+# Development
+./nvcf-cli --config dev.yaml init
+./nvcf-cli --config dev.yaml function create --input-file function.json
+
+# Production
+./nvcf-cli --config prod.yaml init
+./nvcf-cli --config prod.yaml function list
+```
+
+Each configuration maintains separate state files (e.g., `~/.nvcf-cli.dev.state` for `dev.yaml`).
+
+#### Debug Mode
+
+Enable debug mode for detailed logging by adding to your configuration file:
+
+```yaml
+debug: true
+```
+
+Or use the `--debug` flag or `NVCF_DEBUG=true` environment variable per-command.
+
+## Quick Start
+
+```bash
+# 1. Initialize - generate admin token
+./nvcf-cli init
+
+# 2. Generate API key for invocations
+./nvcf-cli api-key generate
+
+# 3. Create a function, update example file with your image
+./nvcf-cli function create --input-file examples/create-function.json
+
+# 4. Deploy the function (uses saved context automatically)
+./nvcf-cli function deploy create
+
+# 5. Invoke the function
+./nvcf-cli function invoke --request-body '{"message": "hello world"}'
+
+# 6. Clean up
+./nvcf-cli function deploy remove
+./nvcf-cli function delete
+```
+
+
+For immediate testing, you can use `load_tester_supreme` from `nvcf-onprem` (see [self-hosted-artifact-manifest](./manifest.md)), which supports the `{"message": "hello world"}` request body above. For more function samples, see the [NVCF examples](https://github.com/NVIDIA/nvcf/tree/main/examples) repository and [function-creation](./function-creation.md) for function creation documentation.
+
+
+
+## Authentication
+
+The CLI stores three bearer credential types:
+
+- `NVCF_TOKEN`: Generated by `nvcf-cli init`. The default CLI credential for
+ management operations and self-hosted cluster management.
+- `NVCF_API_KEY`: Generated by `nvcf-cli api-key generate`. The default CLI
+ credential for function invocation, function discovery, and queue status.
+- `NVCF_NVCT_API_KEY`: Generated by `nvcf-cli api-key generate`. Used
+ automatically for all `task` subcommands.
+
+For NVCF API endpoints, either bearer type can be used when it includes the
+required scope. The CLI prefers `NVCF_API_KEY` for read, invoke, and queue
+commands. It prefers `NVCF_TOKEN` for management commands when both credentials
+are configured. Self-hosted SIS cluster management uses `NVCF_TOKEN`. Task
+commands always use `NVCF_NVCT_API_KEY`.
+
+See [Scope reference](./api.md#scope-reference) for the self-hosted scope
+matrix used by CLI commands and API endpoints.
+
+### Generate NVCF_TOKEN
+
+```bash
+# Generate fresh NVCF_TOKEN (clears existing state)
+./nvcf-cli init
+
+# With debug output
+./nvcf-cli init --debug
+
+# Example output:
+# [INFO] Starting fresh session...
+# [INFO] Generating admin token from API Keys service...
+# [SUCCESS] Admin token generated and saved
+# Token:
+# Expires: 2025-11-19 06:08:15
+```
+
+### Refresh NVCF_TOKEN
+
+Refresh your token while preserving function context:
+
+```bash
+# Refresh NVCF_TOKEN (keeps current function state)
+./nvcf-cli refresh
+
+# Example output:
+# [SUCCESS] Admin token refreshed
+# Function ID: func-abc123 (preserved)
+```
+
+### Generate API Keys
+
+`api-key generate` mints both a function key (`NVCF_API_KEY`) and a task key
+(`NVCF_NVCT_API_KEY`) in a single command. Use `--for` to generate only one.
+
+```bash
+# Generate both keys with defaults (24h expiration)
+./nvcf-cli api-key generate
+
+# Generate only the function key
+./nvcf-cli api-key generate --for function
+
+# Generate only the task key
+./nvcf-cli api-key generate --for task
+
+# Custom expiration and description
+./nvcf-cli api-key generate --expires-in 48h --description "Production key"
+
+# Generate with custom scopes (requires --for)
+./nvcf-cli api-key generate --for function --scopes invoke_function,list_functions
+
+# Generate and validate
+./nvcf-cli api-key generate --validate
+```
+
+Default function key scopes:
+
+| Scope | Description |
+| --- | --- |
+| `invoke_function` | Execute deployed functions |
+| `list_functions` | View available functions |
+| `list_functions_details` | View detailed function metadata |
+| `queue_details` | Monitor function execution queues |
+
+Default task key scopes:
+
+| Scope | Description |
+| --- | --- |
+| `launch_task` | Submit new tasks |
+| `list_tasks` | List tasks |
+| `task_details` | Get task status and details |
+| `cancel_task` | Cancel a running task |
+| `delete_task` | Delete a task |
+| `list_events` | List task events |
+| `list_results` | Retrieve task results |
+| `update_secrets` | Update secrets for a task |
+
+## Command Reference
+
+### Self-hosted Deployment Commands
+
+Use these commands to install and inspect self-hosted NVCF deployments. For the local k3d installation flow, see [Quickstart](./quickstart.md).
+
+| Command | Description |
+| --- | --- |
+| `self-hosted check --pre` | Check local tools and Kubernetes access before installation. |
+| `self-hosted check --all` | Run all currently available self-hosted checks. Use this with pod, route, and function smoke validation. |
+| `self-hosted up --cluster-name --nca-id --region ` | Run the local k3d fresh-install flow. |
+| `self-hosted status` | Show a deployment health summary. |
+| `self-hosted install --control-plane` | Run the control-plane installation primitive. |
+| `self-hosted install --compute-plane --cluster-name ` | Run the compute-plane installation primitive for a registered GPU cluster. |
+| `self-hosted uninstall --compute-plane --cluster-name ` | Remove compute-plane components for the GPU cluster. |
+| `self-hosted uninstall --control-plane` | Remove control-plane components. |
+
+Bundle source overrides:
+
+- `--control-plane-stack` selects the control-plane stack bundle.
+- `--compute-plane-stack` selects the compute-plane stack bundle.
+- Both flags accept local paths, git URLs, and `oci://` references.
+
+`self-hosted up` supports only a single local k3d cluster. It requires
+`--env local`, a current `k3d-*` kube context, and no split-context flags. For
+separate control-plane and GPU clusters, use the explicit control-plane and
+compute-plane install primitives with [Self-Managed Clusters](./cluster-management/self-managed.md).
+
+### Cluster Registration
+
+Self-managed GPU clusters must be registered with the control plane before the NVCA
+operator can start an agent. Registration records the GPU cluster's OIDC issuer and public
+JWKS with the control plane (ICMS) so the agent's projected service account tokens (PSAT)
+validate at runtime. The `cluster register` command performs this registration and prints
+the Helm values the operator install needs.
+
+
+`init` does double duty: it mints the admin token and discovers the control-plane issuer.
+Run `init` before `cluster register`. The one-click `self-hosted up` flow runs both
+internally.
+
+
+
+```bash
+# 1. Mint the admin token and discover the control-plane issuer
+./nvcf-cli init
+
+# 2. Register the GPU cluster (prints a summary and a Helm values block)
+./nvcf-cli cluster register \
+ --name \
+ --nca-id \
+ --region \
+ --icms-url "http://" \
+ --ignore-existing
+```
+
+`cluster register` flags:
+
+| Flag | Description |
+| --- | --- |
+| `--name` | Cluster name (required) |
+| `--nca-id` | NCA/tenant ID (required) |
+| `--region` | Cluster region (default: `us-west-1`) |
+| `--icms-url` | SIS/ICMS endpoint URL the agent uses to reach the control plane |
+| `--nats-url` | NATS endpoint URL for the agent (optional) |
+| `--kubeconfig` | Path to the target GPU cluster kubeconfig (defaults to the current context) |
+| `--oidc-issuer-url` | OIDC issuer URL. Overrides auto-detection and skips SPIRE and Kubernetes discovery |
+| `--ignore-existing` | Return existing IDs instead of failing if the cluster is already registered |
+
+Issuer and JWKS discovery: `cluster register` detects the GPU cluster's OIDC issuer and
+fetches its public JWKS, then sends them to ICMS. Detection precedence:
+
+1. `--oidc-issuer-url` if provided (manual override).
+2. A SPIRE OIDC discovery service in the cluster, if present.
+3. The Kubernetes API server OIDC endpoint (default).
+
+The detected source is recorded as `identitySource` in the output: `psat` for the
+Kubernetes API server, `spire` for SPIRE, or `custom` for a manual issuer.
+
+Output: the command prints a summary (cluster group ID, cluster ID, OIDC issuer, region)
+followed by a `--- Helm values for nvca-operator ---` block. Copy that YAML block into a
+`-register-values.yaml` file and pass it to the operator install. The values
+schema:
+
+```yaml
+clusterID:
+clusterGroupID:
+ncaID:
+region:
+selfManaged:
+ identitySource: psat
+ icmsServiceURL: "http://"
+ revalServiceURL: "http://"
+ natsURL: "nats://:4222"
+```
+
+For load-balancer-fronted gateways that route by hostname, add the matching host-header
+overrides (`selfManaged.icmsServiceHostHeaderOverride`,
+`selfManaged.revalServiceHostHeaderOverride`, `selfManaged.natsHostOverride`) to these
+values. See [self-managed-clusters](./cluster-management/self-managed.md) for how the
+register values feed the operator install and when host-header overrides are required.
+
+### General Commands
+
+| Command | Description |
+| --- | --- |
+| `init` | Generate admin token and start fresh session |
+| `refresh` | Refresh admin token while preserving function context |
+| `status` | Display CLI state and configuration (use `--show-tokens` for full token output) |
+| `version` | Show CLI version information |
+| `completion` | Generate shell autocompletion scripts (supports `bash`, `zsh`, `fish`, `powershell`) |
+
+### API Key Commands
+
+| Command | Description |
+| --- | --- |
+| `api-key generate` | Generate function and task API keys (both by default; use `--for function` or `--for task` for one) |
+| `api-key list` | List all API keys |
+| `api-key show` | Show the current saved API key |
+| `api-key delete` | Delete a specific API key (supports `--force`) |
+| `api-key revoke` | Revoke an API key (same as delete, supports `--force`) |
+| `api-key clear` | Clear saved API key from state (supports `--force`) |
+| `api-key clear-all` | Delete all API keys for an owner (supports `--force`) |
+
+### Task Commands
+
+Task commands manage NVCT (NVIDIA Cloud Tasks) workloads. They require a task
+API key, which `api-key generate` mints automatically alongside the function key.
+
+| Command | Description |
+| --- | --- |
+| `task create` | Submit a new task (saves task ID to state) |
+| `task list` | List tasks, optionally filtered by status |
+| `task get` | Get details for a task by ID |
+| `task cancel` | Cancel a running task |
+| `task delete` | Delete a task |
+| `task events` | List events for a task |
+| `task results` | Retrieve results for a completed task |
+| `task update-secrets` | Update secrets for a task |
+| `task bulk` | Retrieve details for multiple tasks by ID |
+
+```bash
+# Generate both keys (required before task commands)
+./nvcf-cli api-key generate
+
+# Submit a container task
+./nvcf-cli task create \
+ --name my-training-job \
+ --gpu H100 \
+ --instance-type GPU.H100_1x \
+ --image my-registry/training:latest
+
+# Check task details
+./nvcf-cli task get
+
+# Stream lifecycle events
+./nvcf-cli task events
+
+# Cancel a running task
+./nvcf-cli task cancel
+
+# List recent tasks
+./nvcf-cli task list
+```
+
+### Function Management Commands
+
+#### Create Function
+
+```bash
+# Create from JSON file
+./nvcf-cli function create --input-file examples/create-function.json
+
+# Create with CLI flags
+./nvcf-cli function create \
+ --name "my-function" \
+ --image "nvcr.io/your-org/your-image:tag" \
+ --inference-url "/predict" \
+ --inference-port 8000 \
+ --health-uri "/health" \
+ --health-port 8000
+
+# Create with additional options
+./nvcf-cli function create \
+ --name "my-function" \
+ --image "nvcr.io/your-org/your-image:tag" \
+ --inference-url "/predict" \
+ --inference-port 8000 \
+ --function-type STREAMING \
+ --container-env "KEY1=value1" \
+ --secrets "API_KEY=secret123" \
+ --tags "production,v2" \
+ --rate-limit "100-S"
+
+# Create an LLM function with model routing metadata
+./nvcf-cli function create \
+ --name "my-llm-function" \
+ --image "nvcr.io/example/openai-compatible:latest" \
+ --inference-url "/" \
+ --inference-port 8000 \
+ --function-type LLM \
+ --llm-model "name=dummy-model,uris=/v1/chat/completions|/v1/responses|/v1/embeddings,routingMethod=round_robin,tokenRateLimit=1000-S"
+```
+
+All `function create` flags:
+
+| Flag | Description |
+| --- | --- |
+| `--name` | Function name (required) |
+| `--image` | Container image (required) |
+| `--inference-url` | Inference endpoint URL (required) |
+| `--inference-port` | Inference endpoint port (required) |
+| `--input-file` | JSON file with function configuration |
+| `--description` | Function description |
+| `--function-type` | `DEFAULT`, `STREAMING`, or `LLM` (default: `DEFAULT`) |
+| `--api-body-format` | API body format (default: `CUSTOM`) |
+| `--health-uri` | Health check endpoint URI |
+| `--health-port` | Health check endpoint port |
+| `--health-protocol` | Health protocol (`HTTP` or `gRPC`) |
+| `--health-timeout` | Health check timeout (ISO 8601 duration, e.g., `PT30S`) |
+| `--health-expected-status` | Expected health check status code (default: 200) |
+| `--container-args` | Arguments for container launch |
+| `--container-env` | Environment variables in `key=value` format (repeatable) |
+| `--secrets` | Secrets in `name=value` format (repeatable) |
+| `--tags` | Comma-separated tags |
+| `--models` | Model artifacts in `name:version:uri` format (repeatable) |
+| `--llm-model` | LLM model config in `name=MODEL,uris=URI\|URI,routingMethod=round_robin\|power_of_two\|groq_multiregion\|pulsar\|random,tokenRateLimit=LIMIT` format (repeatable). Token limits use `-` with `S`, `M`, `H`, `D`, or `W`, for example `1000-S`. Use JSON input for combined token limits because inline model specs use commas as field separators. |
+| `--resources` | Resource artifacts in `name:version:uri` format (repeatable) |
+| `--helm-chart` | Helm chart specification |
+| `--helm-chart-service` | Helm chart service name |
+| `--rate-limit` | Rate limit pattern (e.g., `100-S`, `50-M`, `10-H`, `5-D`) |
+| `--rate-limit-exempted` | NCA IDs exempted from rate limiting (repeatable) |
+| `--rate-limit-sync` | Enable synchronous rate limit checking |
+
+Example function JSON:
+
+```json
+{
+ "name": "my-inference-function",
+ "containerImage": "nvcr.io/your-org/your-image:tag",
+ "inferenceUrl": "/predict",
+ "inferencePort": 8000,
+ "health": {
+ "protocol": "HTTP",
+ "uri": "/health",
+ "port": 8000,
+ "timeout": "PT30S",
+ "expectedStatusCode": 200
+ }
+}
+```
+
+LLM functions use `functionType: "LLM"` and define model routing metadata under `models[].llmConfig`:
+
+```json
+{
+ "name": "sample-llm-function",
+ "containerImage": "nvcr.io/example/openai-compatible:latest",
+ "inferenceUrl": "/",
+ "inferencePort": 8000,
+ "functionType": "LLM",
+ "models": [
+ {
+ "name": "dummy-model",
+ "llmConfig": {
+ "uris": ["/v1/chat/completions", "/v1/responses", "/v1/embeddings"],
+ "routingMethod": "round_robin",
+ "tokenRateLimit": "1000-S"
+ }
+ }
+ ]
+}
+```
+
+For LLM models, `llmConfig.routingMethod` accepts `round_robin`, `power_of_two`, `groq_multiregion`, `pulsar`, or `random`.
+Supported LLM paths are `/v1/chat/completions`, `/v1/responses`, and `/v1/embeddings`.
+`llmConfig.tokenRateLimit` accepts one or more comma-separated positive integer token limits in `-` format. Supported units are `S` (seconds), `M` (minutes), `H` (hours), `D` (days), and `W` (weeks). Use `1000-S` for a single limit, or `1000-S,5000-M,100000-H,500000-D,1000000-W` for a combined limit with distinct units. Use JSON input for combined limits because inline CLI model specs use commas as field separators.
+
+#### Deploy Function
+
+The `function deploy` command group manages deployments with the following subcommands:
+
+| Command | Description |
+| --- | --- |
+| `function deploy create` | Create a new deployment for a function |
+| `function deploy update` | Update an existing deployment |
+| `function deploy get` | Get deployment details (supports `--json` for raw output) |
+| `function deploy remove` | Remove a function deployment |
+
+```bash
+# Deploy using saved context (from create)
+./nvcf-cli function deploy create
+
+# Deploy with explicit IDs and configuration
+./nvcf-cli function deploy create \
+ --function-id \
+ --version-id \
+ --instance-type "NCP.GPU.A10G_1x" \
+ --gpu "A10G" \
+ --min-instances 1 \
+ --max-instances 1
+
+# Deploy from JSON file
+./nvcf-cli function deploy create --input-file examples/deploy-function.json
+
+# View deployment details
+./nvcf-cli function deploy get \
+ --function-id \
+ --version-id
+
+# Update an existing deployment
+./nvcf-cli function deploy update \
+ --function-id \
+ --version-id \
+ --gpu "A10G" \
+ --instance-type "NCP.GPU.A10G_1x" \
+ --min-instances 2 \
+ --max-instances 4
+
+# Remove a deployment
+./nvcf-cli function deploy remove \
+ --function-id \
+ --version-id
+```
+
+Key `function deploy create` flags:
+
+| Flag | Description |
+| --- | --- |
+| `--function-id` | Function ID (uses state if not specified) |
+| `--version-id` | Version ID (uses state if not specified) |
+| `--gpu` | GPU name (default: `H100`) |
+| `--instance-type` | Instance type (default: `NCP.GPU.H100_1x`) |
+| `--min-instances` | Minimum instances (default: 1) |
+| `--max-instances` | Maximum instances (default: 1) |
+| `--max-request-concurrency` | Max request concurrency (1-1024) |
+| `--input-file` | JSON file with deployment configuration |
+| `--backend` | Backend/CSP for the GPU instance |
+| `--regions` | Allowed deployment regions (repeatable) |
+| `--clusters` | Specific clusters (repeatable) |
+| `--availability-zones` | Availability zones (repeatable) |
+| `--timeout` | Deployment timeout in seconds (default: 900) |
+| `--storage` | Available storage (e.g., `80G`) |
+| `--system-memory` | Amount of RAM |
+| `--gpu-memory` | Amount of GPU memory |
+
+Example deployment JSON:
+
+```json
+{
+ "deploymentSpecifications": [
+ {
+ "gpu": "A10G",
+ "instanceType": "NCP.GPU.A10G_1x",
+ "minInstances": 1,
+ "maxInstances": 1
+ }
+ ]
+}
+```
+
+#### List and Get Functions
+
+```bash
+# List all functions
+./nvcf-cli function list
+
+# List function IDs only
+./nvcf-cli function list-ids
+
+# List versions of a specific function
+./nvcf-cli function list-versions
+
+# Get details of a specific function version
+./nvcf-cli function get \
+ --function-id \
+ --version-id
+
+# Get details as raw JSON
+./nvcf-cli function get \
+ --function-id \
+ --version-id \
+ --json
+```
+
+#### Update Function
+
+```bash
+# Update function tags
+./nvcf-cli function update \
+ --function-id \
+ --version-id \
+ --tags "production,v2"
+
+# Update LLM model routing config
+./nvcf-cli function update \
+ --function-id \
+ --version-id \
+ --llm-model-update "name=dummy-model,routingMethod=round_robin,tokenRateLimit=1000-S"
+
+# Update from JSON file
+./nvcf-cli function update \
+ --function-id \
+ --version-id \
+ --input-file metadata-update.json
+```
+
+LLM model updates can also be provided in the input file:
+
+```json
+{
+ "functionId": "",
+ "versionId": "",
+ "modelUpdates": [
+ {
+ "modelName": "dummy-model",
+ "llmConfig": {
+ "routingMethod": "round_robin",
+ "tokenRateLimit": "1000-S,5000-M,100000-H,500000-D,1000000-W"
+ }
+ }
+ ]
+}
+```
+
+#### Invoke Function
+
+```bash
+# Invoke using saved context
+./nvcf-cli function invoke --request-body '{"input": "Hello, World!"}'
+
+# gRPC invocation
+./nvcf-cli function invoke --grpc --request-body '{"input": "test"}'
+
+# gRPC with custom service and method
+./nvcf-cli function invoke --grpc \
+ --grpc-service "MyService" \
+ --grpc-method "Predict" \
+ --request-body '{"input": "test"}'
+
+# Invoke with explicit IDs and timeout
+./nvcf-cli function invoke \
+ --function-id \
+ --version-id \
+ --request-body '{"input": "Hello!"}' \
+ --timeout 120
+
+# Invoke an LLM function with the chat completions path
+./nvcf-cli function invoke \
+ --function-id \
+ --version-id \
+ --model-name dummy-model \
+ --inference-url /v1/chat/completions \
+ --request-body '{"messages":[{"role":"user","content":"Hello"}],"stream":true}'
+
+# Invoke another OpenAI-compatible LLM path
+./nvcf-cli function invoke \
+ --function-id \
+ --version-id \
+ --model-name dummy-model \
+ --inference-url /v1/embeddings \
+ --request-body '{"input":"NVCF embeddings check"}'
+```
+
+Note: The CLI `function invoke` command detects LLM functions automatically.
+For LLM functions, `--model-name` and `--inference-url` are required. The CLI uses the LLM invocation route and sets the OpenAI `model` value to `/`.
+
+For LLM Gateway endpoint behavior, routing, and session stickiness details, see [LLM Gateway](./llm-gateway.md).
+
+For raw HTTP invocation, HTTP streaming, gRPC metadata, and invocation error
+behavior, see [Generic HTTP Function Invocation](./generic-http-function-invocation.md)
+and [gRPC Function Invocation](./grpc-function-invocation.md).
+
+Additional `function invoke` flags:
+
+| Flag | Description |
+| --- | --- |
+| `--grpc` | Use gRPC invocation |
+| `--grpc-service` | gRPC service name |
+| `--grpc-method` | gRPC method name |
+| `--grpc-plaintext` | Use plaintext (insecure) gRPC |
+| `--inference-url` | Function path, or OpenAI-compatible path for LLM functions (required for LLM) |
+| `--model-name` | OpenAI model name for LLM functions |
+| `--timeout` | Request timeout in seconds (default: 60) |
+| `--poll-duration` | Invocation hold-open duration in seconds (default: 5) |
+| `--input-file` | JSON file with invocation configuration |
+
+#### Queue Management
+
+```bash
+# Get queue status for a function
+./nvcf-cli function queue status
+
+# Get position for a specific request
+./nvcf-cli function queue position
+```
+
+#### Delete Function
+
+```bash
+# Delete current function from state
+./nvcf-cli function delete
+
+# Delete specific function
+./nvcf-cli function delete --function-id --version-id
+
+# Delete deployment only (keep function definition)
+./nvcf-cli function delete --deployment-only --graceful
+```
+
+### Registry Credentials Commands
+
+Manage container registry credentials for function images and Helm charts. For comprehensive setup instructions including IAM configuration for AWS ECR, see [third-party-registries-self-hosted](./third-party-registries.md).
+
+| Command | Description |
+| --- | --- |
+| `registry-credential add` | Add a new registry credential |
+| `registry-credential list` | List all registry credentials |
+| `registry-credential get` | Get details of a specific credential |
+| `registry-credential update` | Update an existing credential |
+| `registry-credential delete` | Delete a registry credential |
+| `registry-credential list-recognized` | List all recognized registries |
+
+```bash
+# Add registry credentials using base64 secret
+./nvcf-cli registry-credential add \
+ --hostname "nvcr.io" \
+ --secret "" \
+ --artifact-type CONTAINER \
+ --description "NGC Container Registry"
+
+# Add registry credentials using username/password
+./nvcf-cli registry-credential add \
+ --hostname "nvcr.io" \
+ --username "" \
+ --password "" \
+ --artifact-type CONTAINER
+
+# List registry credentials (with optional filters)
+./nvcf-cli registry-credential list
+./nvcf-cli registry-credential list --artifact-type CONTAINER
+./nvcf-cli registry-credential list --provisioned-by USER
+
+# Get details for a specific credential
+./nvcf-cli registry-credential get
+
+# Update a credential
+./nvcf-cli registry-credential update \
+ --username "" \
+ --password ""
+
+# Delete registry credentials
+./nvcf-cli registry-credential delete
+./nvcf-cli registry-credential delete --force
+
+# List recognized registries
+./nvcf-cli registry-credential list-recognized
+```
+
+
+Registry credential changes take up to about 5 minutes to take effect for task creation. `nvcf-cli registry-credential list` and `get` show the new value immediately, but task processing caches account credentials for about 5 minutes (`nvct.nvcf.cache-ttl`), so a task can keep using the previous value until the cache refreshes. After rotating or deleting a credential, allow up to about 5 minutes, or restart the task service to apply it immediately. See [Credential Propagation Delay](./third-party-registries.md).
+
+
+## Troubleshooting
+
+### Authentication Errors
+
+401 Unauthorized on function creation:
+
+```bash
+# Regenerate admin token
+./nvcf-cli init --debug
+
+# Verify token is being used
+./nvcf-cli function create --debug --input-file function.json
+# Look for: "Using FUNCTION TOKEN for POST"
+```
+
+403 Forbidden on invocation:
+
+```bash
+# Regenerate API key
+./nvcf-cli api-key generate --validate
+
+# Verify API key is being used
+./nvcf-cli function invoke --debug --request-body '{"input": "test"}'
+# Look for: "Using API KEY for POST"
+```
+
+### Token Expiration
+
+```bash
+# Check token status
+./nvcf-cli status
+
+# Refresh admin token
+./nvcf-cli refresh
+
+# Regenerate API key
+./nvcf-cli api-key generate
+```
+
+### Connection Issues
+
+```bash
+# Enable debug mode to see request details
+./nvcf-cli function list --debug
+
+# Verify CLI state and configuration
+./nvcf-cli status
+```
+
+### Token Usage Summary
+
+| Operation | Accepted bearer | Scope | CLI preference |
+| --- | --- | --- | --- |
+| `function create` | `NVCF_TOKEN` or `NVCF_API_KEY` | `register_function` | `NVCF_TOKEN` |
+| `function deploy create` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` | `NVCF_TOKEN` |
+| `function deploy get` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` | `NVCF_TOKEN` |
+| `function deploy update` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` | `NVCF_TOKEN` |
+| `function deploy remove` | `NVCF_TOKEN` or `NVCF_API_KEY` | `deploy_function` | `NVCF_TOKEN` |
+| `function delete` | `NVCF_TOKEN` or `NVCF_API_KEY` | `delete_function` | `NVCF_TOKEN` |
+| `function update` | `NVCF_TOKEN` or `NVCF_API_KEY` | `update_function` | `NVCF_TOKEN` |
+| `function invoke` | `NVCF_TOKEN` or `NVCF_API_KEY` | `invoke_function` | `NVCF_API_KEY` |
+| `function list`, `function list-ids`, `function list-versions`, `function get` | `NVCF_TOKEN` or `NVCF_API_KEY` | `list_functions` or `list_functions_details` | `NVCF_API_KEY` |
+| `function queue status`, `function queue position`, `function queue details` | `NVCF_TOKEN` or `NVCF_API_KEY` | `queue_details` | `NVCF_API_KEY` |
+| `registry-credential` commands | `NVCF_TOKEN` or `NVCF_API_KEY` | `manage_registry_credentials` | `NVCF_TOKEN` |
+| Self-hosted cluster register, list, rotate, delete | `NVCF_TOKEN` | `cluster-management` | `NVCF_TOKEN` |
+| `task create` | `NVCF_NVCT_API_KEY` | `launch_task` | `NVCF_NVCT_API_KEY` |
+| `task list` | `NVCF_NVCT_API_KEY` | `list_tasks` | `NVCF_NVCT_API_KEY` |
+| `task get` | `NVCF_NVCT_API_KEY` | `task_details` | `NVCF_NVCT_API_KEY` |
+| `task cancel` | `NVCF_NVCT_API_KEY` | `cancel_task` | `NVCF_NVCT_API_KEY` |
+| `task delete` | `NVCF_NVCT_API_KEY` | `delete_task` | `NVCF_NVCT_API_KEY` |
+| `task events` | `NVCF_NVCT_API_KEY` | `list_events` | `NVCF_NVCT_API_KEY` |
+| `task results` | `NVCF_NVCT_API_KEY` | `list_results` | `NVCF_NVCT_API_KEY` |
+| `task update-secrets` | `NVCF_NVCT_API_KEY` | `update_secrets` | `NVCF_NVCT_API_KEY` |
+
+For additional troubleshooting, see [self-hosted-troubleshooting](./troubleshooting.md).
diff --git a/docs/v0.6.1-rc/cluster-management/configuration.md b/docs/v0.6.1-rc/cluster-management/configuration.md
new file mode 100644
index 000000000..1a3c78214
--- /dev/null
+++ b/docs/v0.6.1-rc/cluster-management/configuration.md
@@ -0,0 +1,1073 @@
+# NVCA Configuration
+
+This page documents NVCA configuration options. For cluster registration and lifecycle
+operations, see [Self-Managed Clusters](./self-managed.md).
+
+## Advanced Settings
+
+
+Some of the options below are supported only on Cluster Agent Versions `2.50.0` or higher.
+
+
+
+See below for descriptions of all available configuration options.
+
+| Configuration | Description |
+| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Cluster Agent Version | Version of the cluster agent to be installed on the cluster. Defaults to the latest available. Recommended to use the latest version available at the time of registration unless there are business reasons to pick another version. |
+| Node Selector Key and Node Selector Value | This Key-Value pair is the label selector key to control the placement of the cluster agent and the cluster agent operator pods to specic nodes on the cluster. Not providing a value will allow these infrastructure components to be placed anywhere on the cluster. Ensure there are matching nodes in the cluster using `kubectl get node -l key=value` before registration as incorrect value will cause operational issues. For additional details: [Labels & Selectors](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors) |
+| Priority Class | Set appropriate kubernetes priority class name for cluster agent and the operator pod. Additional details: [Priority Class](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass) |
+| Model Cache Volume Mount Options | Configure the model cache volume mount options based on the CSI Driver capabilities on the cluster. Refer to the CSI Driver documentation. Defaults to `Enabled` and `ro,norecovery,nouuid` on an upgrade. Requires cluster reconfiguration after upgrade to prevent disruption.Additional details: [Mount options](https://man7.org/linux/man-pages/man8/mount.8.html) |
+| Network CIDR Range | Quoted & comma separated list of CIDR range for outbound network access for the infrastructure components & workloads on the cluster. |
+| Worker Degradation Period | Stabilization time (in minutes) before cluster agent fails to consider a worker as healthy and initiates a purge. |
+
+## Cluster Features
+
+Cluster Features allow enabling specific features on the cluster. Dynamic GPU Discovery is enabled by default.
+
+See below for descriptions of all cluster features.
+
+| Capability | Description |
+| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| Dynamic GPU Discovery | Enables automatic detection and management of allocatable GPU capacity within the cluster via the NVIDIA GPU Operator. This capability is **strongly recommended** and would only be disabled in cases where [Manual Instance Configuration](./configuration.md) is required. |
+| Caching Support | Enhances application performance by storing frequently accessed data (models, resources and containers) in a cache. See [cluster-caching](./configuration.md). |
+| Optimized AI Workload Scheduling | Enable support for optimized AI workload scheduling using [KAI Scheduler](https://github.com/kai-scheduler/KAI-Scheduler). Additional setup details: [KAI Scheduler](./kai-scheduler.md) |
+| Shared Cluster mode | Partitions the Kubernetes cluster's nodes into NVCF and non-NVCF pools. Set the label `nvca.nvcf.nvidia.io/schedule=true` on all nodes that can receive NVCF workload Pods. **Note**: this is an advanced use case and should not be used unless absolutely necessary |
+
+
+Removing the Dynamic GPU Discovery will require manual instance configuration. See [Manual Instance Configuration](./configuration.md).
+
+
+
+### Caching Support
+
+Enabling caching for models, resources and containers is recommended for optimal performance. You must create `StorageClass` configurations for caching within your cluster to fully enable "Caching Support" with the Cluster Agent. See examples below.
+
+
+Caching is not supported for Multi-Node Helm functions. If you attempt to deploy a Multi-Node Helm function with Caching Support enabled, the deployment will fail.
+
+Caching is also currently not supported for AWS EKS.
+
+
+
+**StorageClass Configurations in GCP**
+
+```yaml
+kind: StorageClass
+apiVersion: storage.k8s.io/v1
+metadata:
+ name: nvcf-sc
+provisioner: pd.csi.storage.gke.io
+allowVolumeExpansion: true
+volumeBindingMode: Immediate
+reclaimPolicy: Retain
+parameters:
+ type: pd-ssd
+ csi.storage.k8s.io/fstype: xfs
+```
+
+```yaml
+kind: StorageClass
+apiVersion: storage.k8s.io/v1
+metadata:
+ name: nvcf-cc-sc
+provisioner: pd.csi.storage.gke.io
+allowVolumeExpansion: true
+volumeBindingMode: Immediate
+reclaimPolicy: Retain
+parameters:
+ type: pd-ssd
+ csi.storage.k8s.io/fstype: xfs
+```
+
+
+GCP currently allows only [10 VM's](https://cloud.google.com/compute/docs/disks#:~:text=You%20can%20attach%20a%20balanced,VMs%20in%20read%2Donly%20mode) to mount a Persistent Volume in Read-Only mode.
+
+
+
+**StorageClass Configurations in Azure**
+
+```yaml
+kind: StorageClass
+apiVersion: storage.k8s.io/v1
+metadata:
+ name: nvcf-sc
+provisioner: file.csi.azure.com
+allowVolumeExpansion: true
+volumeBindingMode: Immediate
+reclaimPolicy: Retain
+parameters:
+ skuName: Standard_LRS
+ csi.storage.k8s.io/fstype: xfs
+```
+
+```yaml
+kind: StorageClass
+apiVersion: storage.k8s.io/v1
+metadata:
+ name: nvcf-cc-sc
+provisioner: file.csi.azure.com
+allowVolumeExpansion: true
+volumeBindingMode: Immediate
+reclaimPolicy: Retain
+parameters:
+ skuName: Standard_LRS
+ csi.storage.k8s.io/fstype: xfs
+```
+
+**StorageClass Configurations in Oracle Cloud**
+
+```yaml
+kind: StorageClass
+apiVersion: storage.k8s.io/v1
+metadata:
+ name: nvcf-sc
+provisioner: blockvolume.csi.oraclecloud.com
+allowVolumeExpansion: true
+volumeBindingMode: Immediate
+reclaimPolicy: Retain
+parameters:
+ csi.storage.k8s.io/fstype: xfs
+```
+
+```yaml
+kind: StorageClass
+apiVersion: storage.k8s.io/v1
+metadata:
+ name: nvcf-cc-sc
+provisioner: blockvolume.csi.oraclecloud.com
+allowVolumeExpansion: true
+volumeBindingMode: Immediate
+reclaimPolicy: Retain
+parameters:
+ csi.storage.k8s.io/fstype: xfs
+```
+
+**Apply the StorageClass Configurations**
+
+Save the StorageClass template to files `nvcf-sc.yaml` and `nvcf-cc-sc.yaml` and apply them as:
+
+```bash
+kubectl create -f nvcf-sc.yaml
+kubectl create -f nvcf-cc-sc.yaml
+```
+
+**Override the Default Mount Options for Cache Volumes**
+
+
+Supported in Cluster Agent Versions 2.45.21 or higher
+
+
+
+
+Please note this is a Post NVCA Install Operation and needs careful consideration to ensure there are no volume corruptions. Use with caution.
+
+
+
+Cluster Agent with caching support by default will enable linux mount-options with `ro,norecovery,nouuid`.
+
+If the CSI Driver in the cluster doesn't support mount options then you may apply the following command on the cluster to disable the mount options
+
+```bash
+nvcf_cluster_name="$(kubectl get nvcfbackends -n nvca-operator -o name | cut -d'/' -f2)"
+kubectl patch nvcfbackends.nvcf.nvidia.io -n nvca-operator "$nvcf_cluster_name" \
+ --type='merge' \
+ -p '{"spec":{"overrides":{"agentConfig":{"cacheMountOptionsEnabled":false}}}}'
+```
+
+If you want to update the mount-options to a different value for example: `ro,norecovery`. You may use the following command. Replace these options with desired value as dictated by CSI Driver Volume Mount Options.
+
+```bash
+nvcf_cluster_name="$(kubectl get nvcfbackends -n nvca-operator -o name | cut -d'/' -f2)"
+kubectl patch nvcfbackends.nvcf.nvidia.io -n nvca-operator "$nvcf_cluster_name" \
+ --type='merge' \
+ -p '{"spec":{"overrides":{"agentConfig":{"cacheMountOptionsEnabled":true,"cacheMountOptions":"ro,norecovery"}}}}'
+```
+
+### Cluster Maintenance Modes
+
+The Cluster Agent supports two maintenance modes that control how workloads are handled during cluster configuration changes. Configure maintenance mode via the feature flags `CordonMaintenance` or `CordonAndDrainMaintenance` respectively.
+
+**Cordon Maintenance**
+
+In this mode, existing workloads continue to run uninterrupted on the cluster. New workloads will not be scheduled until maintenance mode is cleared.
+
+**Cordon and Drain Maintenance**
+
+In this mode, all existing workloads in the cluster are terminated. No updates to the state of workloads will be effective while in this mode.
+
+
+Once maintenance mode is configured, it can take up to 10 minutes for the agent reconfiguration to take effect.
+
+
+
+### Account-Isolated Clusters
+
+
+Supported in Cluster Agent Versions 2.49.0 or higher
+
+
+
+Clusters with the `AccountIsolation` attribute have enhanced isolation between workloads, ensuring that function and task instances run on nodes isolated by NCAId. This is particularly important for customers with strict security requirements or those who want to ensure complete separation of workloads at the account level.
+
+
+In Account Isolated mode, the cluster might be inefficient in GPU utilization if workloads are not designed to utilize the full capacity of the isolated nodes.
+While toggling this attribute, the cluster workloads also have to be drained using [CordonAndDrainMaintenance](./configuration.md) mode to effectively re-balance the workloads as the attribute will not be applied retroactively.
+
+
+
+### NVLink-optimized Clusters
+
+Clusters with [MNNVL](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/dra-cds.html#dra-docs-compute-domains) GPUs like [GB200](https://www.nvidia.com/en-us/data-center/gb200-nvl72/) can run multi-node workloads that require inter-GPU data transfer with [large performance improvements](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/dra-cds.html#usage-example-a-multi-node-nvbandwidth-test) when properly configured.
+The Cluster Agent can be directed to configure multi-node workloads with their own [ComputeDomains](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/dra-cds.html#computedomains-multi-node-nvlink-simplified) automatically to optimize inter-GPU connections.
+
+Additional prerequisites:
+\- The [NVIDIA GPU DRA driver](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/dra-intro-install.html) must be installed.
+\- The `NVLinkOptimized` cluster attribute must be added during cluster registration.
+
+
+In NVLink-optimized mode, the NVIDIA GPU DRA driver currently limits one GPU-enabled Pod to a node. To optimally utilize these clusters, GPU-enabled Pods _should_ request a full node's worth of GPUs. For example, nodes in GB200 clusters have 4 GPUs each so all containers and all GPU-enabled Pods in a workload must request `nvidia.com/gpu` values that sum to 4.
+
+
+
+### Kata Container-Isolated Workloads
+
+Clusters that have this attribute run all function/task Pods in [Kata Containers](https://katacontainers.io/) without exception.
+
+Additional cluster restrictions to be aware of:
+
+- Pod containers _must_ at least have resource `limits` defined for `cpu` and `memory`. If unset, runtime behavior is undefined.
+
+- Object count limits are configured for resource fairness in these clusters:
+
+ - ConfigMaps: 20
+ - Secrets: 20
+ - Services: 20
+ - Pods: 100
+ - Jobs: 10
+ - CronJobs: 10
+ - Deployments: 10
+ - ReplicaSets: 10
+ - StatefulSets: 10
+
+## Network Configuration
+
+
+The network policies described in this section are only enforced if your cluster's Container Network Interface (CNI) supports Kubernetes Network Policies. Common CNIs that support network policies include:
+
+- Calico
+- Cilium
+- Weave Net
+- Antrea
+
+If your cluster uses a CNI that doesn't support network policies, the security controls described below will not be enforced, and pods will be able to communicate with each other without restrictions. This could lead to security vulnerabilities.
+
+
+
+The NVCA operator requires outbound network connectivity to pull images, charts, and report logs and metrics. During installation, the operator pre-configures the `nvca-namespace-networkpolicies` configmap with the following network policies:
+
+| Policy Name | Description |
+| ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| allow-egress-gxcache | Allows egress traffic to the GX Cache namespace for caching operations (only relevant for NVIDIA managed clusters) |
+| allow-egress-internet-no- internal-no-api | Allows egress traffic to the public internet (0.0.0.0/0) but blocks traffic to common private IP ranges. Also allows DNS resolution via kube-dns. |
+| allow-egress-intra-namespace | Controls pod-to-pod communication within the same namespace. This policy is only applied to function namespaces and not to shared pod instance namespaces. |
+| allow-egress-nvcf-cache | Allows egress traffic to NVCF cache services (only relevant for NVIDIA managed clusters) |
+| allow-egress-prometheus- nvcf-byoo | Allows egress traffic to Prometheus monitoring endpoints (only relevant for NVIDIA managed clusters) |
+| allow-ingress-monitoring | Allows ingress traffic for monitoring services |
+| allow-ingress-monitoring-dcgm | Allows ingress traffic for DCGM monitoring |
+| allow-ingress-monitoring- gxcache | Allows ingress traffic for GX Cache monitoring (only relevant for NVIDIA managed clusters) |
+
+## Key Network Requirements
+
+1. **Kubernetes API Access**
+
+ - NVCA requires access to the Kubernetes API
+ - Consult your cloud provider's documentation (e.g., Azure, AWS, GCP) for the Kubernetes API endpoint
+
+2. **Container Registry and NVCF Control Plane Access**
+
+ - NVCA requires access to your container registry to pull images and Helm charts.
+ - NVCA requires network access to NVCF control plane services (SIS, NATS, ESS) running in your cluster. The specific endpoints depend on your gateway configuration. See [gateway-routing](../gateway-routing.md) for details.
+
+3. **Monitoring and Logging**
+
+ - If your environment requires advanced monitoring or logging (e.g., sending logs to external endpoints), ensure your cluster's NetworkPolicy or firewall rules allow egress to the required monitoring/logging domains
+
+## Network Policy Customization via ConfigMap
+
+The NVCA operator pre-configures the `nvca-namespace-networkpolicies` configmap during installation. If you need to customize these policies for your cluster, you can use a configmap to override the default policies.
+
+To customize a network policy:
+
+1. Create a configmap with your custom network policy, for example:
+
+ ```yaml
+
+ apiVersion: v1
+ kind: ConfigMap
+ metadata:
+ name: demopatch-configmap
+ namespace: nvca-operator
+ labels:
+ nvca.nvcf.nvidia.io/operator-kustomization: enabled
+ data:
+ patches: |
+ - target:
+ group: ""
+ version: v1
+ kind: ConfigMap
+ name: nvca-namespace-networkpolicies
+ patch: |-
+ - op: replace
+ path: /data/allow-egress-internet-no-internal-no-api
+ value: |
+ apiVersion: networking.k8s.io/v1
+ kind: NetworkPolicy
+ metadata:
+ name: allow-egress-internet-no-internal-no-api
+ labels:
+ app.kubernetes.io/name: nvca
+ app.kubernetes.io/instance: nvca
+ app.kubernetes.io/version: "1.0"
+ app.kubernetes.io/managed-by: nvca-operator
+ spec:
+ podSelector: {}
+ policyTypes:
+ - Egress
+ egress:
+ - to:
+ - namespaceSelector: {}
+ podSelector:
+ matchLabels:
+ k8s-app: kube-dns
+ - to:
+ - namespaceSelector:
+ matchLabels:
+ kubernetes.io/metadata.name: gxcache
+ ports:
+ - port: 8888
+ protocol: TCP
+ - port: 8889
+ protocol: TCP
+```
+
+2. Apply the configmap:
+
+ ```bash
+
+ kubectl apply -f patchcm.yaml
+```
+
+3. Verify the changes:
+
+ ```bash
+
+ kubectl logs -n nvca-operator -l app.kubernetes.io/name=nvca-operator
+```
+
+ You should see a message indicating successful patching:
+ `configmap patched successfully`
+
+The changes will be applied to the `nvcf-backend` namespace and will be used for all new namespaces' network policies. The network policies will also be updated across all helm chart namespaces.
+
+## Network Policy Customization via clusterNetworkCIDRs Flag
+
+You can customize the `allow-egress-internet-no-internal-no-api` policy with helm, by adding on the `networkPolicy.clusterNetworkCIDRs` flag. For example:
+
+```bash
+helm upgrade nvca-operator -n nvca-operator --create-namespace -i --reuse-values --wait \
+ oci://${REGISTRY}/${REPOSITORY}/nvca-operator --version \
+ --set networkPolicy.clusterNetworkCIDRs="{10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/12}"
+```
+
+This command will override the default k8s networking CIDRs specified in the `allow-egress-internet-no-internal-no-api` with your input.
+
+## Advanced: Additional Configuration Options
+
+## CSI Volume Mount Options
+
+The NVIDIA Cluster Agent supports customizing CSI volume mount options for caching. This allows you to configure specific mount options for the CSI volumes used in your cluster.
+
+
+CSI volume mount options configuration is an **experimental feature** and may be subject to change in future releases.
+
+
+
+To configure CSI volume mount options:
+
+1. Get the NVCF cluster name:
+
+```bash
+nvcf_cluster_name="$(kubectl get nvcfbackends -n nvca-operator -o name | cut -d'/' -f2)"
+```
+
+2. View current mount options configuration:
+
+```bash
+kubectl get nvcfbackend -n nvca-operator "$nvcf_cluster_name" -o yaml | grep -A 5 "MountOptions"
+```
+
+1. Set mount options (example):
+
+```bash
+kubectl patch nvcfbackends.nvcf.nvidia.io -n nvca-operator "$nvcf_cluster_name" \
+ --type='merge' \
+ -p '{"spec":{"overrides":{"agentConfig":{"cacheMountOptionsEnabled":true,"cacheMountOptions":"ro,norecovery,nouuid"}}}}'
+```
+
+4. Verify the changes:
+
+```bash
+kubectl get nvcfbackend -n nvca-operator "$nvcf_cluster_name" -o yaml | grep -A 5 "MountOptions"
+```
+
+The default mount options are:
+\- `ro`: Read-only mount
+\- `norecovery`: Skip journal recovery
+\- `nouuid`: Ignore filesystem UUID
+
+You can modify these options based on your specific requirements. The configuration will be applied to all CSI volumes created by the NVIDIA Cluster Agent for caching purposes.
+
+## Node Selection for Cloud Functions
+
+By default, the cluster agent uses all nodes discovered with GPU resources to schedule Cloud Functions and there are no additional configuration required.
+
+In order to limit the nodes that can run Cloud Functions, you may use `nvca.nvcf.nvidia.io/schedule=true` label on the specific nodes.
+
+If there are no nodes in the cluster with the `nvca.nvcf.nvidia.io/schedule=true` label set, the cluster agent will switch to the default behavior of using all nodes with GPUs.
+
+For example, to mark specific nodes as schedulable in a cluster:
+
+```bash
+kubectl label node nvca.nvcf.nvidia.io/schedule=true
+```
+
+To mark a single node from the above set as unschedulable for nvcf workloads, you can unlabel using:
+
+```bash
+kubectl label node nvca.nvcf.nvidia.io/schedule-
+```
+
+## GPU Product Name Override
+
+The NVIDIA Cluster Agent supports GPU product name override via node label. This is useful for customers who want to use a custom product name or override the default GPU product name.
+
+For example, to set the GPU product name for a node, use the following command:
+
+```bash
+kubectl label node nvca.nvcf.nvidia.io/gpu.product=
+```
+
+
+The GPU Product Name Override via node labeling only takes effect when there are no pre-existing active instances in the cluster. If active instances already exist with the original GPU instance types, the override will not be applied.
+
+
+## Single MIG Mode Support
+
+The NVIDIA Cluster Agent supports GPUs configured in single
+[Multi-Instance GPU (MIG)](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/)
+mode. When the NVIDIA GPU Operator is configured with the `single` MIG strategy,
+each GPU is partitioned into multiple MIG instances of the same profile. NVCA
+automatically detects MIG profiles from the `nvidia.com/gpu.product` node label and
+generates a unique instance type for each profile.
+
+### How It Works
+
+In single MIG mode, the GPU Operator appends the MIG profile to the GPU product name
+in the node label. For example:
+
+```text
+nvidia.com/gpu.product=NVIDIA-H200-MIG-7g.141gb
+nvidia.com/gpu.product=NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition-MIG-2g.48gb
+nvidia.com/gpu.product=NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition-MIG-1g.24gb
+```
+
+NVCA parses the MIG suffix from the product name and produces a unique instance type
+for each profile. The MIG profile in the instance type name uses hyphens and lowercase
+(for example, `MIG-2g-48gb`).
+
+| GPU Product Label | Instance Type Name |
+| --- | --- |
+| `NVIDIA-H200-MIG-7g.141gb` | `NCP.GPU.H200-MIG-7g-141gb` |
+| `NVIDIA-RTX-PRO-6000-...-MIG-2g.48gb` | `ON-PREM.GPU.RTXPRO6000-MIG-2g-48gb` |
+| `NVIDIA-RTX-PRO-6000-...-MIG-1g.24gb` | `ON-PREM.GPU.RTXPRO6000-MIG-1g-24gb` |
+
+The cluster provider prefix, for example `NCP`, `ON-PREM`, or `AWS`, is determined
+by the cluster registration configuration.
+
+### Prerequisites
+
+- **GPU Operator** configured with MIG strategy `single`. See the
+ [GPU Operator MIG documentation](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-operator-mig.html)
+ for setup instructions.
+- **Dynamic GPU Discovery** enabled, the default for customer-managed clusters.
+- NVCA **v2.53.0** or later.
+
+
+Changing MIG mode or MIG profiles on a node is a disruptive operation. Nodes may
+reboot or terminate GPU clients while applying the new MIG geometry. Before
+reconfiguring MIG:
+
+1. **Cordon** the target nodes to prevent new workload scheduling.
+2. **Drain** all active GPU workloads from the nodes.
+3. Apply the MIG configuration change.
+4. Wait for the GPU Operator MIG Manager to finish, see verification below.
+5. **Uncordon** the nodes once verification passes.
+
+See the [GPU Operator MIG reconfiguration guide](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-operator-mig.html#configuring-mig-profiles)
+for details on the reconfiguration process.
+
+
+
+### Verification
+
+After configuring MIG on your cluster, first confirm the GPU Operator has finished
+applying the MIG profile before checking NVCA labels.
+
+**Step 1: Verify MIG Manager has converged**
+
+Check that MIG-configured nodes report `nvidia.com/mig.config.state=success`:
+
+```bash
+# Check MIG configuration state on MIG nodes
+kubectl get nodes -l nvidia.com/mig.strategy=single \
+ -o custom-columns='NODE:.metadata.name,MIG_CONFIG:.metadata.labels.nvidia\.com/mig\.config,STATE:.metadata.labels.nvidia\.com/mig\.config\.state'
+```
+
+All target nodes should show `success` in the STATE column. If any node shows
+`pending`, `rebooting`, or a failure state, wait for the GPU Operator to finish
+or investigate the MIG Manager logs before proceeding.
+
+**Step 2: Verify NVCA instance type labels**
+
+Once MIG Manager has converged, verify that NVCA has detected the MIG profiles and
+applied the correct instance type labels:
+
+```bash
+# Check the GPU product label on MIG nodes
+kubectl get nodes -l nvidia.com/mig.strategy=single \
+ -o custom-columns='NODE:.metadata.name,GPU_PRODUCT:.metadata.labels.nvidia\.com/gpu\.product'
+
+# Verify the instance type label assigned by NVCA
+kubectl get nodes -l nvidia.com/mig.strategy=single \
+ -o custom-columns='NODE:.metadata.name,INSTANCE_TYPE:.metadata.labels.nvca\.nvcf\.nvidia\.io/instance-type'
+```
+
+Each MIG profile should have a distinct instance type label. For example, a node with
+`MIG-2g.48gb` should show `ON-PREM.GPU.RTXPRO6000-MIG-2g-48gb` while a node with
+`MIG-1g.24gb` should show `ON-PREM.GPU.RTXPRO6000-MIG-1g-24gb`.
+
+
+Only the `single` MIG strategy is supported. In single mode, all GPUs on a node
+are partitioned with the same MIG profile. The `mixed` MIG strategy, with different
+profiles on the same node, is not currently supported by NVCA.
+
+
+
+## Managing Feature Flags
+
+The NVIDIA Cluster Agent supports various feature flags that can be enabled or disabled to customize its behavior. The following are some commonly used feature flags:
+
+| Feature Flag | Description |
+| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| DynamicGPUDiscovery | Dynamically discover GPUs and instance types on this cluster. This is enabled by default for customer-managed clusters. |
+| HelmSharedStorage | Configure Helm functions and tasks with shared read-only storage for ESS secrets. This is required for enabling Helm-based tasks in your cluster. Please note turning on this feature flag requires additional configuration, see [Helm Shared Storage section below](./configuration.md). |
+| LogPosting | Post instance logs to SIS directly. This is enabled by default for NVIDIA managed clusters. |
+| MultiNodeWorkloads | Instruct NVCA to report multi-node instance types to SIS during registration. |
+| SelfHosted | Enables local vault-based authentication for self-hosted deployments. Required when `ngcConfig.clusterSource` is `self-managed`. |
+| HelmAllowCPUNodes | Allow CPU-only pods (e.g. etcd, redis, envoy) from Helm-based functions to be scheduled on non-GPU nodes; GPU pods keep required instance-type affinity. Reduces cost and improves GPU utilization. Mutually exclusive with HelmResourceConstraints. See [helm-allow-cpu-nodes](./configuration.md). |
+
+### Setting Feature Flags at Install Time
+
+Feature flags can be set during the initial NVCA Operator installation through Helm values.
+
+**Standalone Helm**
+
+Set `selfManaged.featureGateValues` in your values file. The chart default is
+`["DynamicGPUDiscovery"]`.
+
+**In the values file:**
+
+```yaml
+selfManaged:
+ featureGateValues: ["DynamicGPUDiscovery", "SelfHosted", "LogPosting"]
+```
+
+**Or via** `--set` **during install:**
+
+```bash
+helm upgrade --install nvca-operator \
+ oci://${REGISTRY}/${REPOSITORY}/nvca-operator \
+ --version 1.2.7 \
+ --namespace nvca-operator --create-namespace \
+ -f nvca-operator-values.yaml \
+ --set 'selfManaged.featureGateValues={DynamicGPUDiscovery,SelfHosted,LogPosting}'
+```
+
+
+The `--set` flag **replaces** the entire list. You must include all desired flags,
+not just the new one.
+
+
+
+To update flags on an existing installation, run `helm upgrade` with the updated values
+file or `--set`:
+
+```bash
+helm upgrade nvca-operator \
+ oci://${REGISTRY}/${REPOSITORY}/nvca-operator \
+ --version 1.2.7 \
+ --namespace nvca-operator \
+ -f nvca-operator-values.yaml \
+ --set 'selfManaged.featureGateValues={DynamicGPUDiscovery,SelfHosted,LogPosting,MultiNodeWorkloads}'
+```
+
+**Helmfile**
+
+The Helmfile deployment uses the same `selfManaged.featureGateValues` chart value. By
+default, the helmfile does not set this field, so the chart default
+`["DynamicGPUDiscovery"]` applies.
+
+To override, add `featureGateValues` to the worker release values in
+`helmfile.d/03-worker.yaml.gotmpl`:
+
+```yaml
+- selfManaged:
+ featureGateValues: ["DynamicGPUDiscovery", "SelfHosted", "LogPosting"]
+ imageCredHelper:
+ imageRepository: {{ .Values.global.image.registry }}/{{ .Values.global.image.repository }}/nvcf-image-credential-helper
+```
+
+Alternatively, set it in an environment-specific values file (e.g.,
+`environments/.yaml`) under the same key path, which avoids editing the shared
+helmfile template.
+
+After changing, run `helmfile --selector release-group=workers sync` to apply.
+
+### Verifying Feature Flags
+
+After installing or upgrading, verify the active feature flags:
+
+```bash
+nvcf_cluster_name="$(kubectl get nvcfbackends -n nvca-operator -o name | cut -d'/' -f2)"
+kubectl get nvcfbackends -n nvca-operator "$nvcf_cluster_name" -o jsonpath='{.spec.featureGate.values}' && echo ""
+```
+
+The NVCA agent pod command-line args also reflect the active flags:
+
+```bash
+kubectl get pods -n nvca-system -o yaml | grep -i feature
+```
+
+### Modifying Feature Flags at Runtime
+
+Feature flags can also be modified at runtime by patching the NVCFBackend resource directly.
+This is useful for quick changes without running a `helm upgrade`.
+
+
+Prefer `helm upgrade` with updated values to change feature flags. Direct patches to the
+NVCFBackend will be overwritten on the next Helm upgrade.
+
+
+
+1. Get the NVCF cluster name:
+
+```bash
+nvcf_cluster_name="$(kubectl get nvcfbackends -n nvca-operator -o name | cut -d'/' -f2)"
+```
+
+2. View current feature flags:
+
+```bash
+kubectl get nvcfbackends -n nvca-operator -o yaml | grep -A 5 "featureGate:"
+```
+
+3. Patch the feature flags. Note that this will override all feature flags.
+
+
+When modifying feature flags, you must preserve any existing feature flags you want to keep. The patch command will override all feature flags, so you need to include all desired feature flags in the value array.
+
+
+
+```bash
+kubectl patch nvcfbackends.nvcf.nvidia.io -n nvca-operator "$nvcf_cluster_name" --type=merge -p '{"spec":{"overrides":{"featureGate":{"values":["LogPosting","CachingSupport"]}}}}'
+```
+
+As an alternative to the patch command, you can also modify the feature flags using the edit command:
+
+```bash
+kubectl edit nvcfbackend -n nvca-operator
+...
+spec:
+ featureGate:
+ values:
+ - LogPosting # Existing feature flag
+ overrides:
+ featureGate:
+ values:
+ - LogPosting # Existing feature flag copied over
+ - -CachingSupport # Caching support disabled
+ ...
+```
+
+4. Verify the changes:
+
+```bash
+kubectl get pods -n nvca-system -o yaml | grep -i feature
+```
+
+## CPU-only pod scheduling (HelmAllowCPUNodes)
+
+
+Supported in Cluster Agent 2.50.4 or higher.
+
+
+
+When `HelmAllowCPUNodes` is enabled, NVCA schedules CPU-only pods from Helm-based functions on
+non-GPU nodes while keeping GPU pods on GPU nodes with their required instance-type affinity.
+This reduces infrastructure cost and improves GPU utilization.
+
+**Scheduling behavior**
+
+| Pod type | Scheduling behavior |
+| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| GPU pods | Required instance-type node affinity (unchanged). |
+| CPU-only pods | Preferred anti-affinity (weight 100) for nodes with the instance-type label; schedules on CPU-only nodes when available, can fall back to GPU nodes if needed. |
+
+
+`HelmAllowCPUNodes` cannot be enabled when `HelmResourceConstraints` is enabled.
+`HelmResourceConstraints` is enabled by default. You must disable it first by adding
+`-HelmResourceConstraints` to the feature gate values, then add `HelmAllowCPUNodes`.
+
+
+
+**How to enable**
+
+1. Follow the steps in [Modifying Feature Flags at Runtime](./configuration.md).
+
+2. In `spec.overrides.featureGate.values`, include all existing flags you want to keep, add
+ `-HelmResourceConstraints`, then add `HelmAllowCPUNodes`.
+
+ Example (preserve your existing flags and add the two changes):
+
+ ```yaml
+ spec:
+ overrides:
+ featureGate:
+ values:
+ - # copy existing flags here
+ - -HelmResourceConstraints
+ - HelmAllowCPUNodes
+ ```
+
+**How to disable**
+
+Remove `HelmAllowCPUNodes` from the values list, or explicitly disable it:
+
+```yaml
+spec:
+ overrides:
+ featureGate:
+ values:
+ - -HelmAllowCPUNodes
+```
+
+## Enable Helm Shared Storage
+
+The NVIDIA Cluster Agent supports shared storage for Helm charts through the [SMB CSI driver](https://github.com/kubernetes-csi/csi-driver-smb/tree/master/charts#install-csi-driver-with-helm-3). This feature is required for enabling Helm-based tasks in your cluster.
+
+
+The Helm shared storage feature must be enabled before you can use Helm-based tasks in your cluster. This feature provides the necessary storage infrastructure for Helm chart operations.
+
+
+
+
+When enabling the Helm shared storage feature flag, you must preserve any existing feature flags. The patch command will override all feature flags, so you need to include all desired feature flags in the value array. If you already have other feature flags enabled, you should include them along with "HelmSharedStorage" in the value array.
+
+
+
+1. First, install the SMB CSI driver using Helm:
+
+```bash
+helm repo add csi-driver-smb https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts
+helm install csi-driver-smb csi-driver-smb/csi-driver-smb --namespace kube-system --version v1.16.0
+```
+
+2. Get the NVCF cluster name:
+
+```bash
+nvcf_cluster_name="$(kubectl get nvcfbackends -n nvca-operator -o name | cut -d'/' -f2)"
+```
+
+3. Enable the Helm shared storage feature flag:
+
+```bash
+kubectl patch nvcfbackends.nvcf.nvidia.io -n nvca-operator "$nvcf_cluster_name" --type=merge -p '{"spec":{"overrides":{"featureGate":{"values":["LogPosting","HelmSharedStorage", "CachingSupport"]}}}}'
+```
+
+4. Verify that the feature flag is enabled:
+
+```bash
+kubectl get pods -n nvca-system -o yaml | grep HelmSharedStorage
+```
+
+## Agent Config Merging
+
+The NVIDIA Cluster Agent supports merging custom configuration into the generated NVCA config
+via the `agentConfig.mergeConfig` Helm value. This allows you to override or extend NVCA
+runtime settings without modifying the operator's config generation logic.
+
+When `agentConfig.mergeConfig` is set, the Helm chart creates a ConfigMap called
+`agent-config-merge` containing the provided YAML. This ConfigMap is mounted into the NVCA
+pod and merged with the generated config at runtime.
+
+Example values:
+
+```yaml
+agentConfig:
+ mergeConfig: |
+ agent:
+ logLevel: debug
+```
+
+Local LLM worker transport example:
+
+```yaml
+agentConfig:
+ mergeConfig: |
+ workload:
+ stargateQUICInsecure: true
+```
+
+`workload.stargateQUICInsecure: true` makes generated LLM workers pass
+`--quic-insecure` to the `pylon` sidecar. Use it only for local or
+isolated test clusters that run the LLM request router tunnel without TLS. For
+the full LLM addon setup, see
+[LLM Function Enablement](../llm-function-enablement.md).
+
+Apply via Helm:
+
+```bash
+helm upgrade nvca-operator -n nvca-operator --create-namespace -i \
+ \
+ --set-file agentConfig.mergeConfig=my-nvca-config.yaml
+```
+
+Or include it in a values file passed to `helm upgrade -f values.yaml`.
+
+## Manual Instance Configuration
+
+
+It is **highly recommended** to rely on Dynamic GPU Discovery (and therefore the NVIDIA GPU Operator), as manual instance configuration is error-prone.
+
+This type of configuration is only necessary when the cluster cloud provider does not support the NVIDIA GPU Operator.
+
+
+
+Manual instance configuration allows you to disable Dynamic GPU Discovery and instead provide a static list of instance types that NVCA will register with the NVCF control plane. This is useful when:
+
+- You have a known, fixed set of GPU configurations
+- Dynamic GPU discovery isn't working correctly for your environment
+- You want to control exactly which instance types are available
+
+By default, NVCA uses Dynamic GPU Discovery to automatically detect GPUs on cluster nodes and register appropriate instance types. When this is disabled, NVCA instead reads a static GPU configuration from a ConfigMap.
+
+### Prerequisites
+
+- A working NVCF cluster with `nvca-operator` installed
+
+- Access to modify Helm values for `nvca-operator`
+
+- Since you are not using the GPU Operator, you must ensure each GPU node has the instance-type label that matches the "value" field in your manual configuration:
+
+ ```bash
+ kubectl label nodes nvca.nvcf.nvidia.io/instance-type=
+ ```
+
+ For example, if your configuration specifies `"value": "OCI.GPU.A10"`, you would label the node with:
+
+ ```bash
+ kubectl label nodes gpu-node-1 nvca.nvcf.nvidia.io/instance-type=OCI.GPU.A10
+ ```
+
+### Step 1: Create the GPU Configuration JSON
+
+Create a JSON file defining your GPU types and instance configurations. The configuration is an array of GPU types, each containing an array of instance types.
+
+**Example Configuration (gpu-config.json):**
+
+```json
+[
+ {
+ "name": "H100",
+ "capacity": 8,
+ "instanceTypes": [
+ {
+ "name": "ON-PREM.GPU.H100_1x",
+ "value": "ON-PREM.GPU.H100",
+ "description": "One Nvidia Hopper GPU",
+ "default": true,
+ "cpuCores": 16,
+ "systemMemory": "128G",
+ "gpuMemory": "80G",
+ "gpuCount": 1,
+ "os": "linux",
+ "driverVersion": "535.135.05",
+ "cpuArch": "amd64",
+ "storage": "1Ti"
+ },
+ {
+ "name": "ON-PREM.GPU.H100_8x",
+ "value": "ON-PREM.GPU.H100",
+ "description": "Eight Nvidia Hopper GPUs (Full Node)",
+ "default": false,
+ "cpuCores": 128,
+ "systemMemory": "1Ti",
+ "gpuMemory": "640G",
+ "gpuCount": 8,
+ "os": "linux",
+ "driverVersion": "535.135.05",
+ "cpuArch": "amd64",
+ "storage": "8Ti"
+ }
+ ]
+ }
+]
+```
+
+### Step 2: Base64 Encode the Configuration
+
+The GPU configuration must be Base64-encoded for the Helm values. Use the following command:
+
+```bash
+# Base64 encode the configuration (without line wrapping)
+GPU_CONFIG_B64=$(cat gpu-config.json | base64 -w 0)
+echo $GPU_CONFIG_B64
+```
+
+
+On macOS, use `base64` without the `-w 0` flag:
+
+```bash
+GPU_CONFIG_B64=$(cat gpu-config.json | base64)
+```
+
+
+
+### Step 3: Configure Helm Values
+
+Update your `nvca-operator` Helm values to disable Dynamic GPU Discovery and provide the manual configuration.
+
+**Example values.yaml:**
+
+```yaml
+selfManaged:
+ featureGateValues: []
+ gpuManualInstanceConfigB64: ""
+```
+
+
+By default `selfManaged.featureGateValues` is `["DynamicGPUDiscovery"]`. Set it to an empty list (`[]`) to disable dynamic discovery and use your manual configuration instead.
+
+
+
+### Step 4: Install or Upgrade the Operator
+
+Apply the configuration using Helm:
+
+```bash
+helm upgrade nvca-operator -n nvca-operator --create-namespace -i \
+ \
+ -f values.yaml
+```
+
+If you are using the NVCF self-hosted Helmfile, add your values file as an entry in the nvca-operator release `values:` list and run `helmfile sync` or `helmfile apply` instead.
+
+### Configuration Fields Reference
+
+**GPU Type Fields:**
+
+| Field | Required | Description |
+| --- | --- | --- |
+| `name` | Yes | GPU type name (e.g., "A100", "L40", "H100"). Must match the GPU product name reported by `nvidia-smi`. |
+| `capacity` | No | Total GPU capacity for this type. Used for resource accounting and quota management. |
+| `instanceTypes` | Yes | Array of instance type configurations for this GPU type. |
+
+**Instance Type Fields:**
+
+| Field | Required | Description |
+| --- | --- | --- |
+| `name` | Yes | Unique instance type identifier (e.g., "ON-PREM.GPU.H100_1x"). This is the name users select when deploying functions. |
+| `value` | Yes | Instance type value used for internal matching (e.g., "ON-PREM.GPU.H100"). Must match the `nvca.nvcf.nvidia.io/instance-type` node label. |
+| `description` | No | Human-readable description displayed in the UI. |
+| `default` | No | Whether this is the default instance type for this GPU. Only one instance type per GPU should be marked as default. |
+| `cpuCores` | Yes | Number of CPU cores allocated to workloads using this instance type. |
+| `systemMemory` | Yes | System RAM allocation (e.g., "28G", "128G", "1Ti"). Uses Kubernetes quantity format. |
+| `gpuMemory` | Yes | Total GPU memory for this instance type. For multi-GPU instances, this is the total across all GPUs. |
+| `gpuCount` | Yes | Number of GPUs in this instance type. |
+| `os` | No | Operating system (e.g., "linux"). |
+| `driverVersion` | No | NVIDIA driver version (e.g., "535.135.05"). |
+| `cpuArch` | No | CPU architecture (e.g., "amd64", "arm64"). |
+| `storage` | No | Storage allocation per instance (e.g., "512Gi", "1Ti"). Uses Kubernetes quantity format. |
+
+### Verification
+
+After applying the configuration, verify that NVCA is using the static configuration:
+
+1. **Check the NVCFBackend resource:**
+
+ ```bash
+ kubectl get nvcfbackend -n nvca-operator -o yaml
+ ```
+
+ Look for `-DynamicGPUDiscovery` in the feature gates and verify the GPU configuration is present.
+
+2. **Check the nvca-config ConfigMap:**
+
+ ```bash
+ kubectl get configmap nvca-config -n nvca-system -o yaml
+ ```
+
+ The `gpus` key should contain your JSON configuration.
+
+3. **Check NVCA logs for registration:**
+
+ ```bash
+ kubectl logs -n nvca-system -l app=nvca | grep -i "registration\|instance"
+ ```
+
+### Troubleshooting
+
+**Configuration Not Applied:**
+
+1. Verify Dynamic GPU Discovery is disabled in the feature gate values
+2. Ensure the Base64 encoding is correct and doesn't contain line breaks
+3. Check that the JSON is valid before encoding
+
+**Invalid JSON Format:**
+
+1. Validate your JSON using a JSON validator before encoding
+2. Ensure all required fields are present
+3. Check that numeric values (`cpuCores`, `gpuCount`) are not quoted as strings
+
+**Memory/Storage Format Errors:**
+
+Memory and storage values must use valid Kubernetes quantity format:
+
+- Valid: `"28G"`, `"128Gi"`, `"1Ti"`, `"512Mi"`
+- Invalid: `"28GB"`, `"128 Gi"`, `"1TB"`
+
+
+Use `G` or `Gi` for gigabytes, `T` or `Ti` for terabytes. The `i` suffix indicates binary units (1024-based).
+
+
+
+## Cloud Provider-Specific Notes
+
+## Oracle Cloud Infrastructure (OCI)
+
+When using Oracle Container Engine for Kubernetes (OKE), ensure that:
+
+- Your compute nodes and GPU nodes are in the same availability domain
+- This is required for proper network connectivity between the NVIDIA Cluster Agent and GPU nodes
+- Flannel CNI is the current recommended and validated CNI vs OCI native CNI for OKE cluster networking.
diff --git a/docs/v0.6.1-rc/cluster-management/container-cache.md b/docs/v0.6.1-rc/cluster-management/container-cache.md
new file mode 100644
index 000000000..d3543629d
--- /dev/null
+++ b/docs/v0.6.1-rc/cluster-management/container-cache.md
@@ -0,0 +1,473 @@
+# Container Cache
+
+This guide provides detailed information on the Container Cache component and its use in NVCF GPU clusters.
+
+Container Cache provides a container caching solution specifically optimized for NGC. It is designed to enhance the efficiency of Docker image pulls from NGC by caching the images locally, reducing network bandwidth and improving pull times for frequently accessed images.
+
+The Container Cache acts as a proxy between your Kubernetes cluster and the NGC registry, caching frequently accessed container images locally to reduce network bandwidth usage and improve deployment times.
+
+## Installation
+
+### Prerequisites
+
+- A running Kubernetes cluster with `kubectl` access
+- **Helm** >= 3.12
+- Credentials for the registry where your NVCF charts and images are stored
+
+### Step 1. Authenticate Helm to your chart registry
+
+Authenticate Helm to your OCI registry where the NVCF charts are stored:
+
+```bash
+echo "${REGISTRY_PASSWORD}" | helm registry login ${REGISTRY} \
+ --username '${REGISTRY_USERNAME}' --password-stdin
+```
+
+### Step 2. Create the namespace and image pull secret
+
+```bash
+kubectl create namespace container-caching
+```
+
+Create an image pull secret so that pods can pull container images from your registry.
+
+
+
+
+```bash
+kubectl create secret docker-registry nvcr-creds \
+ --docker-server=nvcr.io \
+ --docker-username='$oauthtoken' \
+ --docker-password="${NGC_API_KEY}" \
+ --namespace=container-caching
+```
+
+
+
+
+
+```bash
+kubectl create secret docker-registry nvcr-creds \
+ --docker-server=.dkr.ecr..amazonaws.com \
+ --docker-username=AWS \
+ --docker-password="$(aws ecr get-login-password --region )" \
+ --namespace=container-caching
+```
+
+
+
+
+
+```bash
+kubectl create secret docker-registry nvcr-creds \
+ --docker-server= \
+ --docker-username= \
+ --docker-password= \
+ --namespace=container-caching
+```
+
+
+
+
+
+
+The secret name `nvcr-creds` is referenced in the values file under `images.secrets`.
+If you use a different secret name, update the values file to match.
+
+
+
+### Step 3. Create a values file
+
+Create a `values.yaml` using the complete example in [Base Configuration] below.
+
+- **BYOC users** pulling directly from NGC can use the `nvcr.io/nvidia/nvcf-byoc/` image
+ paths without mirroring.
+- **Self-hosted users** should replace the `/` placeholders with
+ their mirrored registry path (see the Artifact Manifest in the self-hosted installation guide).
+
+Adjust `storageClassName` for your environment (e.g., `gp3` for AWS EKS).
+
+### Step 4. Install the chart
+
+
+
+
+```bash
+helm repo add nvcf https://helm.ngc.nvidia.com/nvidia/nvcf --force-update
+helm repo update
+
+helm upgrade --install container-cache \
+ nvcf/nvcf-container-cache \
+ --version 0.25.22 \
+ --namespace container-caching \
+ --values values.yaml
+```
+
+
+
+
+
+Replace `/` with your mirrored registry path.
+
+```bash
+helm upgrade --install container-cache \
+ oci:////nvcf-container-cache \
+ --version 0.25.22 \
+ --namespace container-caching \
+ --values values.yaml
+```
+
+
+
+
+
+### Step 5. Verify the installation
+
+Container Cache deploys two workloads:
+
+- A **StatefulSet** (`container-cache`) with the number of replicas set by `replicaCount`.
+ Each replica runs two containers (nginx proxy + prometheus exporter) and provisions two PVCs
+ (`cache` and `proxy-cache`).
+- A **DaemonSet** (`container-cache-cc`) that runs on every node and configures the container
+ runtime (containerd) to route image pulls through the cache.
+
+```bash
+# StatefulSet replicas and DaemonSet pods should all be Running
+kubectl get pods -n container-caching
+
+# Verify the StatefulSet is fully ready (READY should match replicaCount)
+kubectl get statefulset -n container-caching
+
+# Verify the DaemonSet is running on all nodes
+kubectl get daemonset -n container-caching
+
+# Check services are created
+kubectl get svc -n container-caching
+
+# Check persistent volume claims are Bound
+kubectl get pvc -n container-caching
+```
+
+## Base Configuration
+
+The following is a complete example `values.yaml` for deploying Container Cache.
+Copy this file and adjust values for your environment. Each section is explained in detail
+below.
+
+- **BYOC users** can use the `nvcr.io/nvidia/nvcf-byoc/` paths directly.
+- **Self-hosted users** should replace `/` with their mirrored
+ registry path (see the Artifact Manifest in the self-hosted installation guide for source image paths).
+
+```yaml
+replicaCount: 3
+
+targetHost: nvcr.io,docker.io
+
+images:
+ server: //nvcf-container-cache:v1.1.31
+ exporter: nginx/nginx-prometheus-exporter:1.0
+ certificates: //nvcf-proxy-tls-certs:1.2.0
+ secrets:
+ - nvcr-creds
+
+cache:
+ keyStorageSize: 50m
+ maxSize: 180g
+ inactive: 1d
+ valid: 1h
+
+persistentVolumeClaim:
+ sizeGB: 100
+ storageClassName: gp3 # Use gp3 for AWS EKS, adjust for other platforms
+ sizeProxyGB: 100
+
+service:
+ type: ClusterIP
+ port: 30345
+
+metrics:
+ cacheMetricsStorageSize: 300m
+ throughputHistogramBuckets: 25000000, 30000000, 35000000, 40000000, 50000000, 60000000, 80000000, 100000000
+
+resources:
+ requests:
+ memory: 2Gi
+ cpu: "1"
+ limits:
+ memory: 4Gi
+ cpu: "2"
+
+traces:
+ enabled: false
+
+nucleus:
+ enabled: false
+
+vault:
+ enabled: false
+
+monitoring:
+ enabled: false
+```
+
+## Configuration Sections
+
+### Replicas
+
+The number of Container Cache pods is controlled through the `replicaCount` value. Container Cache replicas operate independently and distribute requests from worker nodes.
+
+```yaml
+# values.yaml
+
+# Min Value: 1
+# Recommended Value: 3
+replicaCount: 3
+```
+
+
+Container Cache is designed to scale horizontally to handle increased load.
+
+
+
+### Node Selection
+
+Container Cache pods are scheduled on nodes with appropriate labels to ensure they run on compute nodes. Adjust the node selector based on your cluster's node labeling scheme.
+
+```yaml
+# values.yaml
+
+nodeSelector:
+ nvcf.nvidia.com/workload: gpu # Adjust based on your node labels, or remove if not using node labels
+```
+
+### Target Hosts
+
+The Container Cache can proxy requests to multiple container registries. The default configuration includes both NGC and Docker Hub.
+
+```yaml
+# values.yaml
+
+# Domain of target Host where the proxy passes the request to
+targetHost: nvcr.io,docker.io
+```
+
+### Image Configuration
+
+Container Cache uses specific images for the server, exporter, and certificates:
+
+```yaml
+# values.yaml
+
+images:
+ # Container Cache Nginx Proxy
+ server: //nvcf-container-cache:v1.1.31
+
+ # Nginx Prometheus Exporter (public image, no mirroring required)
+ exporter: nginx/nginx-prometheus-exporter:1.0
+
+ # TLS Certificates
+ certificates: //nvcf-proxy-tls-certs:1.2.0
+
+ # Image pull secret created in Step 2
+ secrets:
+ - nvcr-creds
+```
+
+Replace `/` with your registry path. BYOC users pulling
+directly from NGC can use `nvcr.io/nvidia/nvcf-byoc` as the registry/repo path.
+
+### Cache Configuration
+
+The cache behavior is controlled through several parameters:
+
+```yaml
+# values.yaml
+
+cache:
+ # Size for storing cache keys
+ keyStorageSize: 50m
+
+ # Maximum size of the cache
+ maxSize: 180g
+
+ # Period a resource can remain in cache without being accessed
+ inactive: 1d
+
+ # Period a cache is valid if resource doesn't become inactive first
+ valid: 1h
+```
+
+### Storage Configuration
+
+Container Cache requires persistent storage for caching container images:
+
+```yaml
+# values.yaml
+
+persistentVolumeClaim:
+ # Size of persistent volume
+ sizeGB: 100
+
+ # Storage class for persistent volume claim
+ storageClassName: gp3 # Use gp3 for Amazon EKS, adjust for other platforms
+
+ # Size of persistent volume for proxy cache
+ sizeProxyGB: 100
+```
+
+### Service Configuration
+
+The service type and port can be configured based on your access requirements:
+
+```yaml
+# values.yaml
+
+service:
+ # Service type: ClusterIP, NodePort, or LoadBalancer
+ type: ClusterIP
+
+ # Port for the Container Cache service
+ port: 30345
+```
+
+### Metrics Configuration
+
+Container Cache includes Prometheus metrics for monitoring cache performance:
+
+```yaml
+# values.yaml
+
+metrics:
+ # Size for storing cache metrics
+ cacheMetricsStorageSize: 300m
+
+ # Bucket configuration for throughput histogram
+ throughputHistogramBuckets: 25000000, 30000000, 35000000, 40000000, 50000000, 60000000, 80000000, 100000000
+```
+
+### Resource Requests and Limits
+
+Resource requests and limits for the Container Cache StatefulSet pods are **required**. The
+chart will fail to install without them.
+
+```yaml
+# values.yaml
+
+resources:
+ requests:
+ memory: 2Gi
+ cpu: "1"
+ limits:
+ memory: 4Gi
+ cpu: "2"
+```
+
+
+Adjust these values based on your cluster size and expected cache throughput. Larger
+deployments with high pull rates may need more memory and CPU.
+
+
+
+## Architecture
+
+### Container Cache Architecture
+
+Container Cache consists of several components:
+
+1. **Nginx Proxy Server**: Handles incoming requests and serves cached content
+2. **Prometheus Exporter**: Provides metrics for monitoring
+3. **Persistent Storage**: Stores cached container images
+4. **DaemonSet**: Configures containerd on worker nodes to use the cache
+
+### Data Flow
+
+1. **Initial Request**: Worker node requests a container image
+2. **Cache Check**: Container Cache checks if image is cached locally
+3. **Cache Hit**: If cached, serve image directly from local storage
+4. **Cache Miss**: If not cached, fetch from upstream registry and cache locally
+5. **Response**: Return image to requesting worker node
+
+## Performance Considerations
+
+### Cache Size
+
+The cache size should be sized based on your workload requirements:
+
+- **Small deployments**: 50-100GB
+- **Medium deployments**: 100-500GB
+- **Large deployments**: 500GB+
+
+### Storage Performance
+
+For optimal performance, use high-performance storage:
+
+- **AWS**: Use gp3 or io1/io2 EBS volumes
+- **Azure**: Use Premium SSD storage
+- **GCP**: Use SSD persistent disks
+
+### Network Configuration
+
+Container Cache should be deployed close to worker nodes to minimize network latency:
+
+- Deploy in the same availability zone as worker nodes
+- Use high-bandwidth network connections
+- Consider using dedicated network interfaces for cache traffic
+
+## Monitoring and Observability
+
+### Metrics
+
+Container Cache provides several key metrics:
+
+- **Cache hit ratio**: Percentage of requests served from cache
+- **Cache size**: Current size of cached data
+- **Request throughput**: Number of requests per second
+- **Response times**: Time to serve cached vs. uncached content
+
+### Logging
+
+Container Cache logs include:
+
+- Cache hit/miss events
+- Upstream registry communication
+- Error conditions and troubleshooting information
+- Performance metrics
+
+## Troubleshooting
+
+### Common Issues
+
+**Cache Not Working**
+: - Verify containerd configuration on worker nodes
+ - Check network connectivity to Container Cache service
+ - Ensure proper DNS resolution
+
+**Low Cache Hit Ratio**
+: - Review cache size configuration
+ - Check cache eviction policies
+ - Monitor storage performance
+
+**Storage Issues**
+: - Verify storage class availability
+ - Check persistent volume claims
+ - Monitor disk space usage
+
+## Best Practices
+
+### Deployment
+
+- Deploy Container Cache before deploying workloads
+- Use multiple replicas for high availability
+- Monitor cache performance and adjust configuration as needed
+
+### Configuration
+
+- Size cache appropriately for your workload
+- Use high-performance storage for better performance
+- Configure appropriate cache eviction policies
+
+### Maintenance
+
+- Monitor cache hit ratios and adjust cache size as needed
+- Regularly review and clean up unused cached images
+- Update Container Cache images regularly for security patches
diff --git a/docs/v0.6.1-rc/cluster-management/gxcache.md b/docs/v0.6.1-rc/cluster-management/gxcache.md
new file mode 100644
index 000000000..cca7ef80a
--- /dev/null
+++ b/docs/v0.6.1-rc/cluster-management/gxcache.md
@@ -0,0 +1,438 @@
+# Distributed Shader Cache (GXCache)
+
+This guide provides detailed information on the Distributed Shader Cache (GXCache) component and its use in NVCF GPU clusters.
+
+GXCache is a cloud-native shader cache for NVIDIA that provides distributed shader caching capabilities. It consists of the GXCache server and mutating webhook to optimize shader compilation and caching across your cluster.
+
+GXCache improves rendering performance by caching compiled shaders, reducing compilation time for frequently used shaders and enabling faster scene loading and rendering.
+
+## Installation
+
+### Prerequisites
+
+- A running Kubernetes cluster with `kubectl` access
+- **Helm** >= 3.12
+- Credentials for the registry where your NVCF charts and images are stored
+
+### Step 1. Authenticate Helm to your chart registry
+
+Authenticate Helm to your OCI registry where the NVCF charts are stored:
+
+```bash
+echo "${REGISTRY_PASSWORD}" | helm registry login ${REGISTRY} \
+ --username '${REGISTRY_USERNAME}' --password-stdin
+```
+
+### Step 2. Create the namespace and image pull secret
+
+```bash
+kubectl create namespace gxcache
+```
+
+Create an image pull secret so that pods can pull container images from your registry.
+
+
+
+
+```bash
+kubectl create secret docker-registry nvcr-creds \
+ --docker-server=nvcr.io \
+ --docker-username='$oauthtoken' \
+ --docker-password="${NGC_API_KEY}" \
+ --namespace=gxcache
+```
+
+
+
+
+
+```bash
+kubectl create secret docker-registry nvcr-creds \
+ --docker-server=.dkr.ecr..amazonaws.com \
+ --docker-username=AWS \
+ --docker-password="$(aws ecr get-login-password --region )" \
+ --namespace=gxcache
+```
+
+
+
+
+
+```bash
+kubectl create secret docker-registry nvcr-creds \
+ --docker-server= \
+ --docker-username= \
+ --docker-password= \
+ --namespace=gxcache
+```
+
+
+
+
+
+
+The secret name `nvcr-creds` is referenced in the values file under
+`webhook.deployment.secret` and `service.deployment.imageSecret`.
+If you use a different secret name, update the values file to match.
+
+
+
+### Step 3. Create a values file
+
+Create a `values.yaml` using the complete example in [Base Configuration] below.
+
+- **BYOC users** pulling directly from NGC can use the `nvcr.io/nvidia/nvcf-byoc/` image
+ paths without mirroring.
+- **Self-hosted users** should replace the `/` placeholders with
+ their mirrored registry path (see the Artifact Manifest in the self-hosted installation guide).
+
+At minimum, configure the storage class for your environment (e.g., `gp3` for AWS EKS)
+and the node selector for your GPU nodes.
+
+### Step 4. Install the chart
+
+
+
+
+```bash
+helm upgrade --install gxcache \
+ nvcf-byoc/gxcache \
+ --namespace gxcache \
+ --values values.yaml
+```
+
+
+
+
+
+Replace `/` with your mirrored registry path.
+
+```bash
+helm upgrade --install gxcache \
+ oci:////gxcache \
+ --version 0.8.2 \
+ --namespace gxcache \
+ --values values.yaml
+```
+
+
+
+
+
+### Step 5. Verify the installation
+
+```bash
+# All pods should reach Running status
+kubectl get pods -n gxcache
+
+# Check the service and webhook are available
+kubectl get svc -n gxcache
+
+# Check persistent volume claims are bound
+kubectl get pvc -n gxcache
+```
+
+## Base Configuration
+
+The following is a complete example `values.yaml` for deploying GXCache. Copy this file and
+adjust values for your environment. Each section is explained in detail below.
+
+- **BYOC users** can use the `nvcr.io/nvidia/nvcf-byoc/` paths directly.
+- **Self-hosted users** should replace `/` with their mirrored
+ registry path (see the Artifact Manifest in the self-hosted installation guide for source image paths).
+
+```yaml
+webhook:
+ nodeSelector:
+ node-type: compute # Adjust based on your node labels, or set to null
+ deployment:
+ image: //gxcache-webhook
+ version: 59bd8ec5
+ secret: nvcr-creds # Image pull secret created in Step 2
+ client:
+ version:
+ image: //gxcache-init
+ tag: 1e47f722
+ config:
+ tls:
+ enabled: false
+ metrics:
+ enabled: false # Requires monitoring.coreos.com/v1 CRD
+
+service:
+ nodeSelector:
+ node-type: compute # Adjust based on your node labels, or set to null
+ deployment:
+ image: //gxcache-service:b206ce39 # Tag is part of the image field
+ imageSecret: nvcr-creds # Image pull secret created in Step 2
+ vault:
+ enabled: false
+ kns:
+ enabled: true
+ keyset:
+ api: /.well-known/jwks.json
+ endpoint: http://notary.nvcf.svc.cluster.local:8080
+ persistence:
+ enabled: true
+ storageClass: gp3 # Use gp3 for AWS EKS, adjust for other platforms
+ accessMode: ReadWriteOnce
+ size: 20Gi
+ metrics:
+ enabled: false # Requires monitoring.coreos.com/v1 CRD
+```
+
+Replace `/` with your registry path. BYOC users pulling
+directly from NGC can use `nvcr.io/nvidia/nvcf-byoc` as the registry/repo path.
+
+
+If your cluster does not use node labels, set `nodeSelector` to `null`.
+
+
+
+## Configuration Sections
+
+### Webhook Configuration
+
+The GXCache webhook is responsible for intercepting and modifying pod specifications to enable shader caching:
+
+```yaml
+# values.yaml
+
+webhook:
+ nodeSelector:
+ node-type: compute # Adjust based on your node labels
+ deployment:
+ image: //gxcache-webhook
+ version: 59bd8ec5
+ secret: nvcr-creds # Registry credentials secret
+ client:
+ version:
+ image: //gxcache-init
+ tag: 1e47f722
+ config:
+ tls:
+ enabled: false
+ metrics:
+ enabled: false # Requires monitoring.coreos.com/v1 CRD
+```
+
+### Service Configuration
+
+The GXCache service provides the shader cache functionality:
+
+```yaml
+# values.yaml
+
+service:
+ nodeSelector:
+ node-type: compute # Adjust based on your node labels
+ deployment:
+ image: //gxcache-service:b206ce39
+ imageSecret: nvcr-creds # Registry credentials secret
+ vault:
+ enabled: false
+```
+
+### Node Selection
+
+GXCache components are scheduled on nodes with appropriate labels to ensure they run on GPU compute nodes. Adjust the node selector based on your cluster's node labeling scheme.
+
+```yaml
+# values.yaml
+
+webhook:
+ nodeSelector:
+ node-type: compute
+service:
+ nodeSelector:
+ node-type: compute
+```
+
+### Key Management
+
+GXCache uses NVIDIA's Key Notary Service (KNS) for secure key management.
+For self-hosted deployments use the in-cluster notary service:
+
+```yaml
+# values.yaml
+
+service:
+ kns:
+ enabled: true
+ keyset:
+ api: /.well-known/jwks.json
+ endpoint: http://notary.nvcf.svc.cluster.local:8080
+```
+
+### Storage Configuration
+
+GXCache requires persistent storage for caching shader data:
+
+```yaml
+# values.yaml
+
+service:
+ persistence:
+ enabled: true
+ storageClass: gp3 # Use gp3 for AWS EKS, adjust for other platforms
+ accessMode: ReadWriteOnce
+ size: 20Gi
+```
+
+### Metrics Configuration
+
+GXCache supports Prometheus metrics for monitoring:
+
+```yaml
+# values.yaml
+
+webhook:
+ metrics:
+ enabled: false # Requires monitoring.coreos.com/v1 CRD
+service:
+ metrics:
+ enabled: false # Requires monitoring.coreos.com/v1 CRD
+```
+
+## Architecture
+
+### GXCache Architecture
+
+GXCache consists of two main components:
+
+1. **GXCache Service**: Provides the shader cache storage and retrieval functionality
+2. **GXCache Webhook**: Intercepts pod creation to inject shader cache configuration
+
+### Data Flow
+
+1. **Pod Creation**: Application pod is created with GPU requirements
+2. **Webhook Interception**: GXCache webhook intercepts pod creation
+3. **Configuration Injection**: Webhook injects shader cache configuration
+4. **Shader Compilation**: Application compiles shaders during runtime
+5. **Cache Storage**: Compiled shaders are stored in GXCache
+6. **Cache Retrieval**: Subsequent requests retrieve cached shaders
+
+## Performance Considerations
+
+### Storage Performance
+
+For optimal shader cache performance:
+
+- **Storage Type**: Use high-performance SSD storage (gp3, io1, io2)
+- **Storage Size**: Size based on expected shader cache usage (20GB minimum)
+- **Access Mode**: ReadWriteOnce is sufficient for most deployments
+
+### Network Configuration
+
+GXCache should be deployed to minimize network latency:
+
+- Deploy in the same availability zone as GPU nodes
+- Use high-bandwidth network connections
+- Consider dedicated network interfaces for cache traffic
+
+### Cache Size Planning
+
+Plan cache size based on your workload:
+
+- **Small workloads**: 20-50GB
+- **Medium workloads**: 50-200GB
+- **Large workloads**: 200GB+
+
+## Monitoring and Observability
+
+### Metrics
+
+GXCache provides several key metrics:
+
+- **Cache hit ratio**: Percentage of shader requests served from cache
+- **Cache size**: Current size of cached shader data
+- **Request latency**: Time to serve cached shaders
+- **Storage utilization**: Disk space usage
+
+### Prometheus Integration
+
+To enable Prometheus metrics:
+
+1. **Install Prometheus Operator** (if not already installed):
+
+ ```bash
+ helm install --wait --timeout 15m \
+ --namespace monitoring --create-namespace \
+ --repo https://prometheus-community.github.io/helm-charts \
+ prometheus-agent kube-prometheus-stack
+ ```
+
+2. **Enable Metrics in GXCache**:
+
+ ```bash
+ helm upgrade gxcache -n gxcache . \
+ --values values.yaml \
+ --set webhook.metrics.enabled=true \
+ --set service.metrics.enabled=true
+ ```
+
+3. **Access Prometheus**:
+
+ ```bash
+ kubectl port-forward -n monitoring svc/prometheus-operated 9090:9090
+ ```
+
+### Logging
+
+GXCache logs include:
+
+- Cache hit/miss events
+- Shader compilation requests
+- Error conditions and troubleshooting information
+- Performance metrics
+
+## Troubleshooting
+
+### Common Issues
+
+**Webhook Not Working**
+- Verify webhook is running and healthy
+- Check webhook configuration and secrets
+- Ensure proper RBAC permissions
+
+**Cache Not Storing Shaders**
+- Verify GXCache service is running
+- Check storage configuration and persistent volumes
+- Review application logs for shader compilation errors
+
+**Low Cache Hit Ratio**
+- Review cache size configuration
+- Check cache eviction policies
+- Monitor storage performance
+
+**Storage Issues**
+- Verify storage class availability
+- Check persistent volume claims
+- Monitor disk space usage
+
+## Best Practices
+
+### Deployment
+
+- Deploy GXCache before deploying GPU workloads
+- Use high-performance storage for better cache performance
+- Monitor cache performance and adjust configuration as needed
+
+### Configuration
+
+- Size cache appropriately for your shader workload
+- Use high-performance storage for better performance
+- Enable metrics for monitoring and optimization
+
+### Security
+
+- Use proper RBAC permissions for webhook and service
+- Secure communication between components
+- Regularly update GXCache images for security patches
+
+### Maintenance
+
+- Monitor cache hit ratios and adjust cache size as needed
+- Regularly review and clean up unused cached shaders
+- Update GXCache images regularly for security patches
+- Monitor storage usage and plan for growth
diff --git a/docs/v0.6.1-rc/cluster-management/index.md b/docs/v0.6.1-rc/cluster-management/index.md
new file mode 100644
index 000000000..815c30da7
--- /dev/null
+++ b/docs/v0.6.1-rc/cluster-management/index.md
@@ -0,0 +1,45 @@
+# GPU Cluster Setup
+
+The NVIDIA Cluster Agent (NVCA) connects GPU clusters to the NVCF control plane, enabling them to act as deployment targets for Cloud Functions. NVCA is a function deployment orchestrator that registers a cluster's GPU resources, communicates with the control plane, and manages the lifecycle of function deployments on GPU nodes.
+
+For a fresh install, use the [Quickstart](../quickstart.md). The one-click CLI flow can register a GPU cluster as part of the install. Use this section for manual cluster registration, standalone NVCA installation, and day-two cluster configuration.
+
+
+If you pin NVCA separately from the recommended compute-plane stack, check the current NVCA version before upgrading to NVCA 3.x. Clusters running NVCA 2.51.0 or earlier have version-specific upgrade guidance. See the [0.6.0 upgrade notes](../release-notes/0.6.0.md#upgrade-notes).
+
+
+After installing NVCA on a cluster:
+
+- The registered cluster will show as a deployment option in the `GET /v2/nvcf/clusterGroups` API response.
+- Any functions under the cluster's authorized NCA IDs can now deploy on the cluster.
+
+## Authentication and Keys
+
+| Key Type | Description |
+| --- | --- |
+| NVCF API Key (NAK) | Used by NVCA to authenticate with the control plane. See [self-hosted-api](../api.md) for details on API key generation. |
+
+## Prerequisites
+
+- Access to a Kubernetes cluster including GPU-enabled nodes ("GPU cluster")
+
+ - The cluster must have a compatible version of [Kubernetes](https://kubernetes.io/releases/).
+
+ - The cluster must have the [NVIDIA GPU Operator](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/getting-started.html#operator-install-guide) installed.
+
+ - If your cloud provider does not support the NVIDIA GPU Operator, [Manual Instance Configuration](./configuration.md) is possible, but not recommended due to lack of maintainability.
+ - To get the most out of clusters with multi-node NVLink ([MNNVL](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/dra-cds.html#dra-docs-compute-domains)) GPUs like [GB200](https://www.nvidia.com/en-us/data-center/gb200-nvl72/), the [NVIDIA GPU DRA driver](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/dra-intro-install.html) must be installed. See the [nvlink-optimized-clusters](./configuration.md) for details.
+ - For development or testing environments without physical GPUs, install the [fake-gpu-operator](../fake-gpu-operator.md) instead.
+
+- Registering the cluster requires `kubectl` and `helm` installed.
+
+- The user registering the cluster must have the `cluster-admin` role privileges to install the NVIDIA Cluster Agent Operator (`nvca-operator`).
+
+### Supported Kubernetes Versions
+
+- Supported versions are the latest Kubernetes minor release and the two prior minor releases (N-2). See official Kubernetes docs for current supported [versions](https://kubernetes.io/releases/version-skew-policy/#supported-versions).
+
+### Considerations
+
+- The NVIDIA Cluster Agent currently only supports caching if the cluster is enabled with `StorageClass` configurations. If the "Caching Support" capability is enabled, the agent will make the best effort by attempting to detect storage during deployments and fall back on non-cached workflows.
+- Each function and task requires several infrastructure containers be deployed alongside workload containers. These infrastructure containers collectively need 6 CPU cores and 8 Gi of system memory to execute. Each GPU node must have at least this many resources, ideally significantly more for workload resource usage.
diff --git a/docs/v0.6.1-rc/cluster-management/kai-scheduler.md b/docs/v0.6.1-rc/cluster-management/kai-scheduler.md
new file mode 100644
index 000000000..809b8fd6b
--- /dev/null
+++ b/docs/v0.6.1-rc/cluster-management/kai-scheduler.md
@@ -0,0 +1,77 @@
+# KAI Scheduler Integration Guide
+
+[KAI Scheduler](https://github.com/kai-scheduler/KAI-Scheduler) is an open source Kubernetes Native scheduler for AI workloads at large scale.
+To use the KAI Scheduler for NVCF Workloads the following configuration should be applied post the installation of the KAI Scheduler in the cluster and the [Optimized AI Workload Scheduling](./configuration.md) enabled on the
+cluster. NVCF Workloads deployed will be automatically BinPacked upon this cluster configuration changes.
+
+**KAI Scheduler Installation**
+
+
+Upgrade to latest [KAI Scheduler release](https://github.com/kai-scheduler/KAI-Scheduler/releases) is recommended to get latest fixes and security patches
+
+
+
+NVCA's KAI scheduler integration expects default queues to exist with names `default-parent-queue` (parent) and `default-queue` (child);
+other queues may exist in the cluster.
+
+
+One caveat is that NVCA expects all queues used to create NVCF workloads to have unlimited (`-1`) quotas and limits
+to ensure full cluster capacity utilization and accurate usage tracking. If the cluster is partitioned to serve both NVCF and non-NVCF workloads
+and KAI scheduler queue quotas/limits are limited to reflect this, then [Shared Cluster mode](./configuration.md#cluster-features) must be enabled so non-NVCF workload nodes
+are accurately excluded from tracking and scheduling by NVCA.
+
+
+
+Create `values.yaml` with [default queue](https://raw.githubusercontent.com/NVIDIA/KAI-Scheduler/refs/heads/main/docs/quickstart/default-queues.yaml) attributes:
+
+
+```yaml title="kai-scheduler-queues.yaml"
+scheduler:
+ placementStrategy: binpack
+ plugins:
+ nodeplacement:
+ arguments:
+ gpu: binpack
+ cpu: spread
+ actions:
+ preempt:
+ enabled: false
+ consolidation:
+ enabled: false
+
+defaultQueue:
+ createDefaultQueue: true
+ parentName: default-parent-queue
+ childName: default-queue
+ parentResources:
+ cpu:
+ quota: -1
+ limit: -1
+ overQuotaWeight: 1
+ gpu:
+ quota: -1
+ limit: -1
+ overQuotaWeight: 1
+ memory:
+ quota: -1
+ limit: -1
+ overQuotaWeight: 1
+ childResources:
+ cpu:
+ quota: -1
+ limit: -1
+ overQuotaWeight: 1
+ gpu:
+ quota: -1
+ limit: -1
+ overQuotaWeight: 1
+ memory:
+ quota: -1
+ limit: -1
+ overQuotaWeight: 1
+```
+
+
+```bash
+helm install kai-scheduler oci://ghcr.io/kai-scheduler/kai-scheduler/kai-scheduler -f values.yaml -n kai-scheduler --create-namespace --version v0.14.0
+```
diff --git a/docs/v0.6.1-rc/cluster-management/monitoring.md b/docs/v0.6.1-rc/cluster-management/monitoring.md
new file mode 100644
index 000000000..c0fa6ae76
--- /dev/null
+++ b/docs/v0.6.1-rc/cluster-management/monitoring.md
@@ -0,0 +1,122 @@
+# Monitoring & Observability
+
+The NVIDIA Cluster Agent and Operator provide built-in monitoring through Prometheus metrics,
+structured logging, and OpenTelemetry tracing.
+
+## Monitoring Data
+
+### Metrics
+
+**Prerequisites**
+
+To use the PodMonitor and ServiceMonitor examples below, you must first install the Prometheus Operator. Follow the [Prometheus Operator installation guide](https://prometheus-operator.dev/docs/getting-started/installation/) to set this up in your cluster.
+
+The cluster agent and operator emit Prometheus-style metrics. The following metric labels are available by default. The full list of available metrics are updated regularly and therefore not listed.
+
+| Metric Label | Metric Label Description |
+| ------------------ | -------------------------------- |
+| nvca_event_name | The name of the event |
+| nvca_nca_id | The NCA ID of this NVCA instance |
+| nvca_cluster_name | The NVCA cluster name |
+| nvca_cluster_group | The NVCA cluster group |
+| nvca_version | The NVCA version |
+
+Cluster maintainers can scrape the available metrics. See a full example of how to do this with an OpenTelemetry Collector in the cluster [cluster monitoring example](https://github.com/NVIDIA/nvcf/tree/main/examples/cluster-monitoring-sample).
+
+Use the following examples of a PodMonitor for NVCA Operator and ServiceMonitor for NVCA for reference:
+
+**Sample NVCA Operator PodMonitor**
+
+```yaml
+apiVersion: monitoring.coreos.com/v1
+kind: PodMonitor
+metadata:
+ labels:
+ app.kubernetes.io/component: metrics
+ app.kubernetes.io/instance: prometheus-agent
+ app.kubernetes.io/name: metrics-nvca-operator
+ jobLabel: metrics-nvca-operator
+ release: prometheus-agent
+ prometheus.agent/podmonitor-discover: "true"
+ name: metrics-nvca-operator
+ namespace: monitoring
+spec:
+ podMetricsEndpoints:
+ - port: http
+ scheme: http
+ path: /metrics
+ jobLabel: jobLabel
+ selector:
+ matchLabels:
+ app.kubernetes.io/name: nvca-operator
+ namespaceSelector:
+ matchNames:
+ - nvca-operator
+```
+
+**Sample NVCA ServiceMonitor**
+
+```yaml
+apiVersion: monitoring.coreos.com/v1
+kind: ServiceMonitor
+metadata:
+ labels:
+ app.kubernetes.io/component: metrics
+ app.kubernetes.io/instance: prometheus-agent
+ app.kubernetes.io/name: metrics-nvca
+ jobLabel: metrics-nvca
+ release: prometheus-agent
+ prometheus.agent/servicemonitor-discover: "true"
+ name: prometheus-agent-nvca
+ namespace: monitoring
+spec:
+ endpoints:
+ - port: nvca
+ jobLabel: jobLabel
+ selector:
+ matchLabels:
+ app.kubernetes.io/name: nvca
+ namespaceSelector:
+ matchNames:
+ - nvca-system
+```
+
+### Logs
+
+Both the Cluster Agent and Cluster Agent Operator emit logs locally by default.
+
+Local logs for the NVIDIA Cluster Agent Operator can be obtained via `kubectl`:
+
+```bash
+kubectl logs -l app.kubernetes.io/instance=nvca-operator -n nvca-operator --tail 20
+```
+
+Similarly, NVIDIA Cluster Agent logs can be obtained with the following command via kubectl:
+
+```bash
+kubectl logs -l app.kubernetes.io/instance=nvca -n nvca-system --tail 20
+```
+
+
+Current function-level inference container logs are **not supported** for functions deployed on non-NVIDIA-managed clusters. Customers are encouraged to emit logs directly from their inference containers running on their own clusters to any third-party tool, there are no public egress limitations for containers.
+
+
+
+### Tracing
+
+The NVIDIA Cluster Agent provides OpenTelemetry integration for exporting traces and events to compatible collectors. As of agent version 2.0, the only supported collector receiver is Lightstep.
+
+**Enable Tracing with Lightstep**
+
+1. Get your [Lightstep access token](https://docs.lightstep.com/docs/create-and-manage-access-tokens) from the [Lightstep UI](https://app.lightstep.com) and set to `LS_ACCESS_TOKEN` environment variable.
+2. Get the NVCF cluster name:
+
+```bash
+nvcf_cluster_name="$(kubectl get nvcfbackends -n nvca-operator -o name | cut -d'/' -f2)"
+```
+
+3. Apply the tracing configuration:
+
+```bash
+kubectl patch nvcfbackends.nvcf.nvidia.io -n nvca-operator "$nvcf_cluster_name" --type=merge --patch="{\"spec\":{\"overrides\":{\"featureGate\":{\"otelConfig\":{\"exporter\":\"lightstep\",\"serviceName\":\"nvcf-nvca\",\"accessToken\":\"${LS_ACCESS_TOKEN}\"}}}}}"
+```
diff --git a/docs/v0.6.1-rc/cluster-management/reference.md b/docs/v0.6.1-rc/cluster-management/reference.md
new file mode 100644
index 000000000..d40bb2c54
--- /dev/null
+++ b/docs/v0.6.1-rc/cluster-management/reference.md
@@ -0,0 +1,248 @@
+# Helm Values Reference
+
+The `nvca-operator` Helm chart is configured through a standard Helm values file (`values.yaml`)
+passed to `helm upgrade -f values.yaml`. This page documents all available parameters.
+
+
+The parameters listed below are a snapshot and may not reflect the latest chart version.
+Always refer to the `values.yaml` and `values.schema.json` included in your chart
+version for the authoritative list of parameters and defaults:
+
+
+
+## How Values Are Structured
+
+The chart values are organized into two layers:
+
+1. **Shared parameters** (top-level) — These control the operator image, authentication,
+ node placement, network policies, observability, agent resources, and agent runtime config.
+ Examples: `image`, `ngcConfig`, `nodeSelector`, `networkPolicy`, `agent`,
+ `agentConfig`.
+2. **\`\`selfManaged.\*\`\`** — Used when `ngcConfig.clusterSource` is `"self-managed"`.
+ These define the backend configuration including NVCA version, feature gates, cluster
+ attributes, and manual GPU config.
+
+**The key field is \`\`ngcConfig.clusterSource\`\`:**
+
+- `"self-managed"` — The operator reads backend configuration from `selfManaged.*` values.
+
+```yaml
+# --- Shared parameters (all modes) ---
+image:
+ repository: "nvcr.io/nvidia/nvcf-byoc/nvca-operator"
+ngcConfig:
+ clusterSource: "helm-managed" # ← This determines which section below is used
+ serviceKey: ""
+nodeSelector:
+ key: "node.kubernetes.io/instance-type"
+ value: "m5.2xlarge"
+
+# --- Only read when clusterSource is "helm-managed" ---
+helmManaged:
+ cloudProvider: "aws"
+ clusterRegion: "us-west-2"
+ nvcaVersion: "2.97.0"
+ featureGateValues: ["DynamicGPUDiscovery"]
+
+# --- Only read when clusterSource is "self-managed" ---
+# selfManaged:
+# nvcaVersion: "2.97.0"
+# featureGateValues: ["DynamicGPUDiscovery"]
+```
+
+
+The `helmManaged` and `selfManaged` sections share many of the same fields
+(`nvcaVersion`, `featureGateValues`, `gpuManualInstanceConfigB64`,
+`clusterAttributes`). The difference is that `helmManaged` also requires cluster
+identity fields (`cloudProvider`, `clusterRegion`, `clusterGroupID`,
+`clusterGroupName`) that self-managed deployments get from their own control plane.
+
+
+
+## Shared Parameters
+
+These parameters apply to all deployment modes.
+
+### NVCA Operator
+
+```yaml
+## Container images
+image:
+ repository: "nvcr.io/nvidia/nvcf-byoc/nvca-operator" # Operator image path
+ tag: "" # Defaults to chart version
+ pullPolicy: IfNotPresent
+
+nvcaImage:
+ repositoryOverride: "" # Override NVCA agent image path (staging/testing only)
+ pullPolicy: IfNotPresent
+
+## Image pull secrets
+generateImagePullSecret: true # Auto-generate from ngcConfig.serviceKey
+imagePullSecretName: "nvca-operator-image-pull" # Name of the generated secret
+imagePullSecrets: [] # Additional pre-existing pull secrets
+
+## Service account
+serviceAccount:
+ create: true
+ annotations: {}
+ name: "" # Auto-generated if empty
+
+## Operator settings
+replicaCount: 1
+systemNamespace: nvca-operator
+logLevel: info # debug, info, warn, error
+priorityClassName: "" # K8s PriorityClassName for eviction preference
+k8sVersionOverride: "" # Override K8s version NVCA registers with
+enableGXCache: true # Enable GXCache support
+ddcsIPAllowList: "" # Comma-separated CIDRs for DDCS access control
+nvcaHelmRepositoryPrefix: "" # Restrict Helm repos to specific org/team
+```
+
+### NGC Authentication
+
+```yaml
+ngcConfig:
+ username: '$oauthtoken'
+ serviceKey: "" # NGC Cluster Key or NVCF API Key (NAK)
+ serviceKeySecretName: "ngc-service-key" # K8s Secret name if serviceKey not set inline
+ serviceKeySecretKeyName: "ngcServiceKey" # Key within the Secret
+ apiURL: https://api.ngc.nvidia.com # NGC API URL (override for self-hosted)
+ clusterSource: ngc-managed # "ngc-managed", "helm-managed", or "self-managed"
+```
+
+### Node Selector
+
+```yaml
+nodeSelector:
+ key: node.kubernetes.io/instance-type # Label key for operator pod placement
+ value: "" # Label value (empty = no constraint)
+```
+
+### Network Policies
+
+```yaml
+networkPolicy:
+ clusterNetworkCIDRs: # CIDRs that workload pods are NOT allowed to access
+ - "10.0.0.0/8"
+ - "172.16.0.0/12"
+ - "192.168.0.0/16"
+ - "100.64.0.0/12"
+ customPolicies: [] # Custom NetworkPolicy definitions for function namespaces
+```
+
+### OpenTelemetry
+
+```yaml
+otel:
+ enabled: false
+ lightstep:
+ serviceName: "" # Lightstep service name
+ accessToken: "" # Lightstep API token
+```
+
+### Agent Configuration
+
+```yaml
+agent:
+ cacheMountOptionsEnabled: true
+ cacheMountOptions: "ro,norecovery,nouuid"
+ workerDegradationPeriod: "" # e.g., "90m", "1h30m"
+ secretMirrorNamespace: nvca-operator # Namespace to mirror custom secrets from
+ secretMirrorLabelSelector: "" # Label selector for mirrored secrets
+ customAnnotations: {} # Extra annotations on the agent pod
+ functionEnvOverrides: {} # Override infra container images for functions
+ taskEnvOverrides: {} # Override infra container images for tasks
+ overrideEnvironmentVariables: {} # Override env vars on the NVCA agent container
+ resources:
+ limits:
+ cpu: 1000m
+ memory: 4Gi
+ requests:
+ cpu: 100m
+ memory: 200Mi
+
+## Merge custom YAML into the generated NVCA agent config at runtime
+agentConfig:
+ mergeConfig: ""
+ # Example:
+ # mergeConfig: |
+ # agent:
+ # logLevel: debug
+
+## OTel Collector sidecar (for K8s event collection)
+otelCollector:
+ enabled: false
+ imageRepository: "" # Auto-calculated if empty
+ imageTag: 0.143.2
+```
+
+## Helm-Managed Parameters
+
+Only used when `ngcConfig.clusterSource: "helm-managed"`.
+
+```yaml
+helmManaged:
+ ## Cluster identity (REQUIRED, immutable after initial registration)
+ cloudProvider: "" # e.g., "aws", "gcp", "azure", "ON-PREM", "NCP"
+ clusterRegion: "" # e.g., "us-west-2"
+ clusterGroupID: "" # Unique cluster group identifier
+ clusterGroupName: "" # Human-readable cluster group name
+
+ ## Backend configuration
+ nvcaVersion: "" # NVCA agent version (REQUIRED)
+ clusterDescription: "" # Defaults to cluster name if empty
+ featureGateValues: [] # e.g., ["DynamicGPUDiscovery", "CachingSupport"]
+ gpuManualInstanceConfigB64: "" # Base64-encoded GPU config (manual instance only)
+ clusterAttributes: [] # e.g., ["CacheOptimized", "NVLinkOptimized"]
+
+ ## Authentication (optional)
+ oAuthClientID: "" # OAuth2/OIDC client ID (for internal NVIDIA clusters)
+ oAuthClientSecretKey: "" # Secret key for OAuth client
+
+ ## Image overrides (advanced, usually auto-calculated)
+ imageCredHelper:
+ imageRepository: ""
+ imageTag: 0.10.2
+ otelCollector:
+ imageRepository: ""
+ imageTag: 0.143.2
+```
+
+## Self-Managed Parameters
+
+Only used when `ngcConfig.clusterSource: "self-managed"` (self-hosted NVCF).
+
+```yaml
+selfManaged:
+ nvcaVersion: "" # NVCA agent version (REQUIRED)
+ featureGateValues: ["DynamicGPUDiscovery"] # Default includes GPU discovery
+ gpuManualInstanceConfigB64: "" # Base64-encoded GPU config (manual instance only)
+ clusterAttributes: [] # e.g., ["CacheOptimized"]
+
+ ## Image overrides (advanced, usually auto-calculated)
+ imageCredHelper:
+ imageRepository: ""
+ imageTag: 0.10.2
+ otelCollector:
+ imageRepository: ""
+ imageTag: 0.143.2
+```
+
+
+The `selfManaged` and `helmManaged` sections share the same backend fields. The key
+differences are:
+
+- `selfManaged` does not have cluster identity fields (`cloudProvider`,
+ `clusterRegion`, `clusterGroupID`, `clusterGroupName`) — these come from the
+ self-hosted control plane.
+- `selfManaged.featureGateValues` defaults to `["DynamicGPUDiscovery"]`. To disable a
+ feature, remove it from the list (set to `[]`).
+- `helmManaged.featureGateValues` defaults to `[]`. To disable a feature, prefix it
+ with `-` (e.g., `["-DynamicGPUDiscovery"]`).
+
+
+
+## Related Documentation
+
+- [NVCA Configuration](./configuration.md) — how to use these values to configure specific features (caching, network policies, manual instance config, etc.)
+- [Agent config merging](./configuration.md) — using `agentConfig.mergeConfig` for runtime config overrides
diff --git a/docs/v0.6.1-rc/cluster-management/self-managed.md b/docs/v0.6.1-rc/cluster-management/self-managed.md
new file mode 100644
index 000000000..59586be2e
--- /dev/null
+++ b/docs/v0.6.1-rc/cluster-management/self-managed.md
@@ -0,0 +1,736 @@
+# Self-Managed Clusters
+
+GPU clusters are registered with the NVCF control plane and managed by the NVCA
+Operator. Use the compute-plane Makefile in
+`deploy/stacks/nvcf-compute-plane` for this workflow. It registers one GPU
+cluster at a time with `nvcf-cli`, then installs the operator with the cluster
+identity returned by registration. The operator reads that identity from a
+local ConfigMap and authenticates through the local OpenBao (Vault) instance.
+
+
+A running NVCF control plane (SIS, OpenBao, NATS, Cassandra, and all core
+services) is required. The [Quickstart](../quickstart.md) can install the
+control plane and register a GPU cluster in one flow. Use this page when you
+need to install or operate the NVCA Operator after using the Helmfile
+installation path.
+
+
+
+Clone the public repository and run the compute-plane commands from its root:
+
+```bash
+git clone https://github.com/nvidia/nvcf.git
+cd nvcf
+```
+
+## Prerequisites
+
+Before installing the NVCA Operator, ensure the following prerequisites are met:
+
+- The [control plane](../helmfile-installation.md) is installed and all core services are running.
+
+- The [NVIDIA GPU Operator](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/getting-started.html) is installed on the GPU cluster. The GPU Operator manages the NVIDIA drivers, device plugin, and GPU feature discovery required for workload scheduling. For development or testing environments without physical GPUs, see [fake-gpu-operator](../fake-gpu-operator.md).
+
+- (Optional) The [KAI Scheduler](./kai-scheduler.md) can be installed on the GPU cluster for optimized AI workload scheduling and bin-packing of GPU resources. It is required only when you enable the `KAIScheduler` feature flag. See [NVCA Configuration](./configuration.md).
+
+- `GPU Workload Components` must be available in a user-managed registry that your Kubernetes cluster can access. See `GPU Workload Components` under [self-hosted-artifact-manifest](../manifest.md) for necessary artifacts and [self-hosted-image-mirroring](../image-mirroring.md) for mirroring instructions.
+
+- `nvcf-cli` is available on the deployment machine. The compute-plane Makefile
+ calls it during cluster registration. See [self-hosted-cli](../cli.md) for
+ CLI installation and configuration.
+
+- The [SMB CSI driver](https://github.com/kubernetes-csi/csi-driver-smb) (`smb.csi.k8s.io`) must be installed on the GPU cluster. It is required for NVCA shared model cache storage (samba sidecar). Install it with:
+
+ ```bash
+ helm repo add csi-driver-smb \
+ https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts
+
+ helm install csi-driver-smb csi-driver-smb/csi-driver-smb \
+ -n kube-system --version v1.17.0
+ ```
+
+## How It Works
+
+In self-managed mode, the cluster identity comes from the CLI registration
+step, and the operator consumes it:
+
+1. You configure the compute-plane Helmfile environment with registry settings
+ and control-plane service URLs.
+2. From the repository root, you run
+ `make -C deploy/stacks/nvcf-compute-plane register-cluster` (see
+ [Register the cluster](#register-the-cluster)). The target runs
+ `nvcf-cli init`, then `nvcf-cli cluster register`. The CLI discovers the GPU
+ cluster's OIDC issuer and public JWKS, records them with the control plane
+ (ICMS), and writes the `clusterID`, `clusterGroupID`, identity source, and
+ service URLs to a registration values file.
+3. From the repository root, you run
+ `make -C deploy/stacks/nvcf-compute-plane install`. Helmfile installs
+ `helm-nvca-operator` with the compute-plane environment and registration
+ values for that cluster.
+4. The Helm chart renders a local ConfigMap (`nvcfbackend-self-managed`) holding the cluster
+ identity and the SIS, ReVal, and NATS endpoints (plus any host-header overrides). The
+ operator reads that ConfigMap and creates the NVCFBackend resource.
+5. The operator creates the NVCA agent pod. The agent authenticates to the control plane
+ with a projected service account token (PSAT), which the control plane validates against
+ the issuer and JWKS registered in step 1, and begins managing GPU workloads.
+6. Runtime secrets are injected by the OpenBao vault-agent sidecar, which
+ authenticates using Kubernetes service account JWT tokens against the local
+ OpenBao instance. Kubernetes image pull secrets are configured separately in
+ the compute-plane environment and namespaces.
+
+## Configure the compute plane
+
+Create the environment file from the repository root. It provides the chart
+registry, image registry, image pull secret name, and control-plane service URLs
+used by the NVCA operator.
+
+```bash
+cd path/to/nvcf
+touch deploy/stacks/nvcf-compute-plane/environments/.yaml
+```
+
+Use service URLs that are reachable from the GPU cluster. For a single
+load-balancer-fronted Gateway with hostname-based routing, the agent should dial
+the load balancer address and send route-matching host headers:
+
+```yaml title="deploy/stacks/nvcf-compute-plane/environments/.yaml"
+global:
+ helm:
+ sources:
+ registry:
+ repository:
+
+ image:
+ registry:
+ repository:
+
+ imagePullSecrets:
+ - name: nvcr-pull-secret
+
+ nvcaOperator:
+ selfManaged:
+ icmsServiceURL: "http://"
+ icmsServiceHostHeaderOverride: "sis."
+ revalServiceURL: "http://"
+ revalServiceHostHeaderOverride: "reval."
+ natsURL: "nats://:4222"
+ natsHostOverride: "nats."
+```
+
+If your GPU cluster can resolve per-service DNS names directly, set the service
+URLs to those names and omit the host-header override fields.
+
+Host-header overrides require `helm-nvca-operator` 1.12.0 or later. With proper
+per-service DNS and TLS (see [gateway-routing](../gateway-routing.md)), the
+service hostnames resolve directly and no host-header overrides are needed.
+
+## Register the cluster
+
+Register the GPU cluster with the control plane before installing the operator.
+The `nvcf-cli` discovers the cluster's OIDC issuer and JWKS and records them
+with the control plane, then returns the Helm values the operator needs. See
+[self-hosted-cli](../cli.md) for CLI installation and configuration, and the
+[Cluster Registration](../cli.md#cluster-registration) reference for full flag
+and output details.
+
+
+The control plane must expose its issuer for the CLI to discover. The Helmfile
+installation path enables this with
+`openbao.migrations.issuerDiscovery.enabled: true`.
+
+
+
+Create a CLI config that can reach the control-plane gateway. The
+`register-cluster` target runs `nvcf-cli init` before `cluster register`, so the
+config must include the API Keys endpoint and host-header overrides:
+
+```yaml title="nvcf-cli-gpu-register.yaml"
+base_http_url: "http://"
+invoke_url: "http://"
+api_keys_service_url: "http://"
+icms_url: "http://"
+
+api_keys_host: "api-keys."
+api_host: "api."
+icms_host: "sis."
+invoke_host: "invocation."
+
+api_keys_service_id: "nvidia-cloud-functions-ncp-service-id-aketm"
+api_keys_issuer_service: "nvcf-api"
+api_keys_owner_id: "svc@nvcf-api.local"
+client_id: ""
+```
+
+Run registration against the GPU cluster kubeconfig. This is required for
+multi-cluster installs because the CLI discovers the OIDC issuer and JWKS from
+the target Kubernetes cluster.
+
+```bash
+make -C deploy/stacks/nvcf-compute-plane register-cluster \
+ CLUSTER_NAME= \
+ NCA_ID= \
+ CLUSTER_REGION= \
+ ICMS_URL="http://" \
+ KUBECONFIG_FILE= \
+ NVCF_CLI= \
+ NVCF_CLI_CONFIG=
+```
+
+The target writes
+`registration/-register-values.yaml`. It carries the cluster
+identity and endpoints:
+
+```yaml
+clusterID:
+clusterGroupID:
+ncaID:
+region:
+selfManaged:
+ identitySource: psat
+ icmsServiceURL: "http://"
+ revalServiceURL: "http://"
+ natsURL: "nats://:4222"
+```
+
+The `template`, `install`, and `apply` targets copy this file into `out/` before
+running Helmfile.
+
+## Installing the NVCA Operator
+
+| Chart | `helm-nvca-operator` |
+| --- | --- |
+| Version | `1.12.7` |
+| Namespace | `nvca-operator` |
+| Depends on | All control-plane services and gateway must be running |
+
+The compute-plane Helmfile passes these values to the chart:
+
+| Value | Source |
+| --- | --- |
+| Cluster identity | `registration/-register-values.yaml` |
+| Operator, agent, image credential helper, and shared storage image repositories | `global.image.*` in `environments/.yaml` |
+| Control-plane service URLs and host-header overrides | `global.nvcaOperator.selfManaged.*` |
+| Image pull secrets | `global.imagePullSecrets` |
+
+Use `global.nvcaOperator.nodeSelector`, `global.nvcaOperator.tolerations`, and
+`global.nvcaOperator.agent.*` in the compute-plane environment when you need to
+place the operator, agent, or workloads on specific nodes.
+
+
+The `KAIScheduler` feature flag is optional. Enable it only if the
+[KAI Scheduler](./kai-scheduler.md) is installed on the GPU cluster. The flag has no effect,
+and GPU workload scheduling will not work as expected, if KAI Scheduler is not present. Omit
+`KAIScheduler` from `featureGateValues` when KAI Scheduler is not installed.
+
+
+
+
+For the full list of available feature flags and how to set or modify them, see
+[managing-feature-flags](./configuration.md).
+
+
+
+### Node inotify limits
+
+The NVCA operator and agent use file watchers for ConfigMap and Secret reconciliation.
+Some node images set `fs.inotify.max_user_instances` to `128`, which can be too low for
+nodes running the full NVCF stack. When the node exhausts inotify instances, NVCA logs
+errors such as `failed to create watcher` and `too many open files`. Function creation
+or deployment requests that wait for NVCA reconciliation can then return HTTP 500 or
+time out.
+
+Before installing the operator, set higher inotify limits on every node. The following
+DaemonSet sets the values on current nodes and on nodes added later:
+
+```yaml title="inotify-tuner.yaml"
+apiVersion: apps/v1
+kind: DaemonSet
+metadata:
+ name: node-inotify-tuner
+ namespace: kube-system
+spec:
+ selector:
+ matchLabels:
+ app.kubernetes.io/name: node-inotify-tuner
+ template:
+ metadata:
+ labels:
+ app.kubernetes.io/name: node-inotify-tuner
+ spec:
+ hostPID: true
+ tolerations:
+ - operator: Exists
+ priorityClassName: system-node-critical
+ terminationGracePeriodSeconds: 1
+ initContainers:
+ - name: set-sysctl
+ image: busybox:1.36
+ securityContext:
+ privileged: true
+ command:
+ - sh
+ - -c
+ - |
+ set -e
+ mkdir -p /host/etc/sysctl.d
+ {
+ echo 'fs.inotify.max_user_instances=8192'
+ echo 'fs.inotify.max_user_watches=524288'
+ } > /host/etc/sysctl.d/99-inotify.conf
+ nsenter -t 1 -m -u -i -n -p -- sysctl -w fs.inotify.max_user_instances=8192
+ nsenter -t 1 -m -u -i -n -p -- sysctl -w fs.inotify.max_user_watches=524288
+ volumeMounts:
+ - name: host
+ mountPath: /host
+ containers:
+ - name: pause
+ image: registry.k8s.io/pause:3.9
+ resources:
+ requests:
+ cpu: "1m"
+ memory: "1Mi"
+ limits:
+ cpu: "10m"
+ memory: "16Mi"
+ volumes:
+ - name: host
+ hostPath:
+ path: /
+```
+
+Apply the DaemonSet and wait for it to run on every node:
+
+```bash
+kubectl apply -f inotify-tuner.yaml
+kubectl -n kube-system rollout status ds/node-inotify-tuner --timeout=5m
+```
+
+If your cluster cannot pull public images, mirror `busybox:1.36` and
+`registry.k8s.io/pause:3.9` to your registry and update the image fields before applying
+the DaemonSet.
+
+### Image Pull Secrets
+
+The NVCA operator, NVCA agent, samba sidecar, and image-credential-helper all pull
+container images from the registry configured in your compute-plane environment
+file. If that registry is private, create the Kubernetes pull secret before
+installing the operator.
+
+
+Use a pre-existing pull secret through `global.imagePullSecrets`. Do not use
+`generateImagePullSecret`; it does not work in self-managed mode.
+
+
+
+Create the secret in the GPU cluster namespaces used by the operator and its
+managed resources:
+
+```bash
+for ns in nvca-operator nvca-system nvcf-backend; do
+ kubectl --kubeconfig \
+ create namespace "$ns" --dry-run=client -o yaml | kubectl --kubeconfig apply -f -
+ kubectl --kubeconfig \
+ create secret docker-registry nvcr-pull-secret \
+ --docker-server=${REGISTRY} \
+ --docker-username='$oauthtoken' \
+ --docker-password="$REGISTRY_PASSWORD" \
+ --namespace="$ns" \
+ --dry-run=client -o yaml | kubectl --kubeconfig apply -f -
+done
+```
+
+Replace `${REGISTRY}` with your container registry (e.g., `nvcr.io`). For non-NGC
+registries, replace `--docker-username` and `--docker-password` with your registry
+credentials. For NGC (`nvcr.io`), `$REGISTRY_PASSWORD` is your NGC Personal Key or
+API Key.
+
+Reference the secret in your compute-plane environment:
+
+```yaml
+global:
+ imagePullSecrets:
+ - name: nvcr-pull-secret
+```
+
+Helmfile passes this value to the operator chart. The operator adds the pull
+secret reference to the pods it manages. Pre-creating the secret in
+`nvca-system` and `nvcf-backend` prevents startup races for operator-managed
+resources that pull private images.
+
+### Install
+
+Install with the compute-plane Makefile. The command copies
+`registration/-register-values.yaml` into `out/`, then runs
+Helmfile against the GPU cluster kubeconfig:
+
+```bash
+make -C deploy/stacks/nvcf-compute-plane install \
+ CLUSTER_NAME= \
+ HELMFILE_ENV= \
+ NCA_ID= \
+ KUBECONFIG_FILE=
+```
+
+During installation, Helmfile will:
+
+1. Create the operator deployment with vault agent annotations for OpenBao auth.
+2. Render the `nvcfbackend-self-managed` ConfigMap from the register values (cluster
+ identity, SIS/ReVal/NATS endpoints, and any host-header overrides).
+3. Start the operator, which reads the ConfigMap and creates the NVCFBackend and NVCA agent
+ deployment.
+
+### Verify
+
+Check the operator pod is running. The pod runs the operator, the `nvca-mirror` sidecar, and
+the OpenBao vault-agent sidecar (and a `cluster-validator` init container when network
+validation is enabled):
+
+```bash
+kubectl get pods -n nvca-operator
+
+# Expected:
+# NAME READY STATUS RESTARTS AGE
+# nvca-operator-... Running 0 1m
+```
+
+Check the cluster identity ConfigMap was rendered from the register values:
+
+```bash
+kubectl get cm nvcfbackend-self-managed -n nvca-operator \
+ -o jsonpath='{.data.cluster-dto\.yaml}'
+
+# Expected: cluster-dto.yaml with non-empty clusterId and clusterGroupId
+```
+
+Check the NVCFBackend resource was created:
+
+```bash
+kubectl get nvcfbackends -n nvca-operator
+
+# Expected: one NVCFBackend resource with version and health status
+```
+
+Check the NVCA agent pod is running (the operator creates this automatically):
+
+```bash
+kubectl get pods -n nvca-system
+
+# Expected:
+# NAME READY STATUS RESTARTS AGE
+# nvca-... 3/3 Running 0 2m
+```
+
+
+The NVCA agent pod has 3 containers: the agent, the admission webhook, and the OpenBao vault agent sidecar.
+Both should show `Running`. If the vault agent sidecar is in `CrashLoopBackOff`, verify
+that OpenBao is healthy and the migration jobs completed successfully.
+
+
+
+Verify GPU discovery:
+
+```bash
+kubectl get nvcfbackends -n nvca-operator -o jsonpath='{.items[0].status}' | python3 -m json.tool
+
+# Look for GPU information in the status output
+```
+
+### Verify Workload Scheduling
+
+Use this check only when worker pods on the GPU cluster can reach the
+control-plane worker endpoints used by NVCF. For multi-cluster EKS installs,
+first validate the GPU cluster by checking `nvca-operator` rollout and
+`NVCFBackend` agent health. Function deployment and invocation are valid
+acceptance checks only after the worker endpoints are reachable from the GPU
+cluster.
+
+1. Set up environment variables:
+
+```bash
+# Get the Gateway address (from Step 1)
+export GATEWAY_ADDR=$(kubectl get gateway nvcf-gateway -n envoy-gateway -o jsonpath='{.status.addresses[0].value}')
+echo "Gateway Address: $GATEWAY_ADDR"
+```
+
+2. Generate an admin token:
+
+```bash
+# Generate an admin API token
+export NVCF_TOKEN=$(curl -s -X POST "http://${GATEWAY_ADDR}/v1/admin/keys" \
+ -H "Host: api-keys.${GATEWAY_ADDR}" \
+ | grep -o '"value":"[^"]*"' | cut -d'"' -f4)
+
+echo "Token generated: ${NVCF_TOKEN:0:20}..."
+```
+
+3. Create, deploy, and invoke a test function:
+
+```bash
+# Create a test function
+# Replace / with your container registry
+# This should match the registry you set in the secrets file
+curl -s -X POST "http://${GATEWAY_ADDR}/v2/nvcf/functions" \
+ -H "Host: api.${GATEWAY_ADDR}" \
+ -H "Content-Type: application/json" \
+ -H "Authorization: Bearer ${NVCF_TOKEN}" \
+ -d '{
+ "name": "my-echo-function",
+ "inferenceUrl": "/echo",
+ "healthUri": "/health",
+ "inferencePort": 8000,
+ "containerImage": "//load_tester_supreme:0.0.8"
+ }' | jq .
+
+# Extract function and version IDs from the response
+export FUNCTION_ID=
+export FUNCTION_VERSION_ID=
+
+# Deploy the function
+# Adjust instanceType and gpu based on your cluster configuration
+# Instance Type Examples: NCP.GPU.A10G_1x, NCP.GPU.H100_1x, NCP.GPU.L40S_1x, etc.
+# GPU Examples: A10G, H100, L40S, etc.
+curl -s -X POST "http://${GATEWAY_ADDR}/v2/nvcf/deployments/functions/${FUNCTION_ID}/versions/${FUNCTION_VERSION_ID}" \
+ -H "Host: api.${GATEWAY_ADDR}" \
+ -H "Content-Type: application/json" \
+ -H "Authorization: Bearer ${NVCF_TOKEN}" \
+ -d '{
+ "deploymentSpecifications": [
+ {
+ "instanceType": "NCP.GPU.A10G_1x",
+ "backend": "nvcf-default",
+ "gpu": "A10G",
+ "maxInstances": 1,
+ "minInstances": 1
+ }
+ ]
+ }' | jq .
+
+# Generate an API key for invocation. See the API scope reference for endpoint-specific scope requirements.
+# Set expiration to 1 day from now (required field)
+EXPIRES_AT=$(date -u -v+1d '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || date -u -d '+1 day' '+%Y-%m-%dT%H:%M:%SZ')
+SERVICE_ID="nvidia-cloud-functions-ncp-service-id-aketm"
+
+export API_KEY=$(curl -s -X POST "http://${GATEWAY_ADDR}/v1/keys" \
+ -H "Host: api-keys.${GATEWAY_ADDR}" \
+ -H "Content-Type: application/json" \
+ -H "Key-Issuer-Service: nvcf-api" \
+ -H "Key-Issuer-Id: ${SERVICE_ID}" \
+ -H "Key-Owner-Id: test@nvcf-api.local" \
+ -d '{
+ "description": "test invocation key",
+ "expires_at": "'"${EXPIRES_AT}"'",
+ "authorizations": {
+ "policies": [{
+ "aud": "'"${SERVICE_ID}"'",
+ "auds": ["'"${SERVICE_ID}"'"],
+ "product": "nv-cloud-functions",
+ "resources": [
+ {"id": "*", "type": "account-functions"},
+ {"id": "*", "type": "authorized-functions"}
+ ],
+ "scopes": ["invoke_function", "list_functions", "queue_details", "list_functions_details"]
+ }]
+ },
+ "audience_service_ids": ["'"${SERVICE_ID}"'"]
+ }' | jq -r '.value')
+
+echo "API Key: ${API_KEY:0:20}..."
+
+# Wait for deployment to be ready (list functions to see status), then invoke the function
+# Uses wildcard subdomain routing: .invocation.
+curl -s -X POST "http://${GATEWAY_ADDR}/echo" \
+ -H "Host: ${FUNCTION_ID}.invocation.${GATEWAY_ADDR}" \
+ -H "Content-Type: application/json" \
+ -H "Authorization: Bearer ${API_KEY}" \
+ -d '{"message": "hello world", "repeats": 1}'
+```
+
+
+The `backend` value should match the cluster group name registered by the NVCA operator.
+The `instanceType` and `gpu` values depend on the GPU types available in your cluster.
+
+For invocation, the Host header uses wildcard subdomain routing: `.invocation.`.
+The URL path should match the function's `inferenceUrl` (e.g., `/echo`).
+For full HTTP invocation behavior, streaming, and errors, see
+[Generic HTTP Function Invocation](../generic-http-function-invocation.md).
+
+
+
+You can also use the NVCF CLI for easier function management:
+
+- Create, deploy, and invoke functions with simple commands
+- Create or update registry credentials without manual API calls
+
+See [self-hosted-cli](../cli.md) for installation and usage instructions.
+
+## Re-registering a cluster
+
+Registration is performed by the CLI, not by the operator. To re-register (for
+example, after a failed install, or to refresh the recorded issuer and JWKS),
+re-run the compute-plane registration target and then re-apply the operator with
+the refreshed values:
+
+```bash
+make -C deploy/stacks/nvcf-compute-plane register-cluster \
+ CLUSTER_NAME= \
+ NCA_ID= \
+ CLUSTER_REGION= \
+ ICMS_URL="http://" \
+ KUBECONFIG_FILE= \
+ NVCF_CLI= \
+ NVCF_CLI_CONFIG=
+
+make -C deploy/stacks/nvcf-compute-plane install \
+ CLUSTER_NAME= \
+ HELMFILE_ENV= \
+ NCA_ID= \
+ KUBECONFIG_FILE=
+```
+
+The registration target rewrites
+`registration/-register-values.yaml`. The install target
+copies that file into `out/` before running Helmfile. The operator reads the
+cluster identity at startup, so re-running the install restarts it with the new
+values.
+
+## Uninstalling
+
+To fully remove the NVCA Operator and all associated resources:
+
+
+If functions are currently deployed on the cluster (pods in the `nvcf-backend` namespace),
+undeploy them through the NVCF API or CLI before uninstalling the operator. Attempting
+to delete NVCA while function pods are running can cause finalizers to block namespace
+deletion. If you encounter stuck resources, see [Handling Stuck Resources] below.
+
+
+
+1. Delete the NVCFBackend resource. This triggers operator-managed cleanup of
+ the agent deployment, NVCA system pods, and related resources:
+
+ ```bash
+ kubectl --kubeconfig \
+ delete nvcfbackends --all -n nvca-operator --timeout=60s
+ ```
+
+2. Verify the agent namespace is clean before proceeding:
+
+ ```bash
+ kubectl --kubeconfig get pods -n nvca-system
+
+ # Expected: "No resources found in nvca-system namespace."
+ ```
+
+3. Destroy the compute-plane Helmfile release:
+
+ ```bash
+ make destroy \
+ CLUSTER_NAME= \
+ HELMFILE_ENV= \
+ NCA_ID= \
+ KUBECONFIG_FILE=
+ ```
+
+
+ The cluster identity (`clusterID` and `clusterGroupID`) persists in the control plane
+ (ICMS) and in your registration values file. A reinstall reuses it by
+ re-applying that file. Pass `--ignore-existing` to `cluster register` if you re-register.
+
+
+
+4. Delete CRDs. This removes all NVCFBackend, MiniService, and StorageRequest
+ custom resources cluster-wide:
+
+ ```bash
+ kubectl --kubeconfig delete crd \
+ nvcfbackends.nvcf.nvidia.io \
+ miniservices.nvca.nvcf.nvidia.io \
+ storagerequests.nvca.nvcf.nvidia.io \
+ --ignore-not-found
+ ```
+
+5. Delete namespaces:
+
+ ```bash
+ kubectl --kubeconfig delete namespace \
+ nvca-operator nvca-system nvcf-backend nvca-modelcache-init \
+ --ignore-not-found
+ ```
+
+### Handling Stuck Resources
+
+If step 1 times out and namespaces remain stuck in `Terminating` state, or function pods in
+`nvcf-backend` prevent cleanup, use the [force-cleanup-script](../troubleshooting.md). This script removes
+finalizers on stuck NVCA resources, force-deletes function pods, and cleans up all NVCA
+namespaces.
+
+```bash
+# Preview what will be deleted
+./force-cleanup-nvcf.sh --dry-run
+
+# Execute the cleanup
+./force-cleanup-nvcf.sh
+```
+
+
+The force cleanup script bypasses normal cleanup procedures by removing finalizers. Always
+attempt the ordered uninstall steps above first.
+
+
+
+For the full script, download link, and detailed usage instructions, see the NVCA Force
+Cleanup Script appendix in the self-hosted troubleshooting guide.
+
+## Troubleshooting
+
+- Cluster IDs empty in the ConfigMap after install: The registration values were
+ not applied. Confirm
+ `registration/-register-values.yaml` exists and that its
+ `clusterID` and `clusterGroupID` are populated, then re-run `make install`:
+
+ ```bash
+ kubectl get cm nvcfbackend-self-managed -n nvca-operator \
+ -o jsonpath='{.data.cluster-dto\.yaml}'
+ ```
+
+- Operator pod not starting: Check the operator logs:
+
+ ```bash
+ kubectl logs -n nvca-operator -l app.kubernetes.io/name=nvca-operator -c nvca-operator --tail=100
+ ```
+
+- Operator or agent logs show `failed to create watcher` with `too many open files`:
+ Increase the node inotify limits with the `node-inotify-tuner` DaemonSet in
+ [Node inotify limits](#node-inotify-limits), then restart the affected pod.
+
+- NVCA agent pod not created: The operator creates the agent pod via the NVCFBackend
+ resource. Check the operator logs for reconciliation errors:
+
+ ```bash
+ kubectl describe nvcfbackends -n nvca-operator
+ ```
+
+- Agent fails to register with SIS (HTTP 401): The control plane could not
+ validate the agent's PSAT against the recorded issuer and JWKS. Re-run
+ `make register-cluster` for the cluster (see
+ [Re-registering a cluster](#re-registering-a-cluster)) so ICMS has the
+ current issuer and JWKS. Also verify the vault agent sidecar on the agent pod
+ is running and rendering the secrets file:
+
+ ```bash
+ kubectl logs -n nvca-system -l app.kubernetes.io/name=nvca -c vault-agent --tail=10
+ ```
+
+- Vault agent sidecar failing: The agent pod needs to authenticate with OpenBao. Verify
+ the vault system is healthy:
+
+ ```bash
+ kubectl exec -n vault-system openbao-server-0 -- bao status
+ ```
+
+- No GPUs discovered: Ensure the GPU Operator is installed and GPU nodes have the
+ `nvidia.com/gpu` resource advertised:
+
+ ```bash
+ kubectl get nodes -o custom-columns="NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu"
+ ```
diff --git a/docs/v0.6.1-rc/configure-autoscaling.md b/docs/v0.6.1-rc/configure-autoscaling.md
new file mode 100644
index 000000000..d7b2a720a
--- /dev/null
+++ b/docs/v0.6.1-rc/configure-autoscaling.md
@@ -0,0 +1,154 @@
+# Configure Function Autoscaling
+
+This page explains how to configure autoscaling on a deployed function using the NVCF API. For background on how the function autoscaler decides on instance counts, see the [Function Autoscaling Overview](./autoscaling/index.md). For the full schema of the request and response bodies referenced below, see the [NVCF OpenAPI specification](https://api.nvcf.nvidia.com/v3/openapi).
+
+Use these two endpoints:
+
+- `POST /v2/nvcf/deployments/functions/{functionId}/versions/{versionId}` sets the initial scaling bounds when you first deploy a function.
+- `PATCH /v2/nvcf/deployments/{deploymentId}/gpu-specifications/{gpuSpecificationId}` updates the bounds and the autoscaling policy on an existing deployment.
+
+All requests require an `NVCF_TOKEN` (JWT) with the `deploy_function` scope.
+
+## Scope of Autoscaling
+
+The function autoscaler operates per function version. It produces a single desired instance count for a given `(functionId, versionId)` pair; the GPU specification ID and deployment ID are not part of its decision. The `PATCH /v2/nvcf/deployments/.../gpu-specifications/{gpuSpecificationId}` endpoint is per-spec on the wire, but the resulting scaling decision applies at the function-version level. If a deployment has more than one GPU specification, set the autoscaling policy consistently across its specs.
+
+## Set Scaling Bounds at Deploy Time
+
+When creating a deployment, set `minInstances` and `maxInstances` on each entry in `deploymentSpecifications`. These are the only autoscaling fields accepted at create time.
+
+```bash
+curl -s -X POST "https://${GATEWAY_ADDR}/v2/nvcf/deployments/functions/${FUNCTION_ID}/versions/${FUNCTION_VERSION_ID}" \
+ -H "Authorization: Bearer $NVCF_TOKEN" \
+ -H "Content-Type: application/json" \
+ -d '{
+ "deploymentSpecifications": [{
+ "gpu": "H100",
+ "instanceType": "NCP.GPU.H100_1x",
+ "minInstances": 1,
+ "maxInstances": 4
+ }]
+ }'
+```
+
+Constraints:
+
+- `minInstances` must be `>= 0`. Setting `minInstances: 0` allows the function to scale to zero when idle. The scale-to-zero idle timeout is set on the platform and is not configurable per function.
+- `maxInstances` must be `> 0` and `>= minInstances`.
+
+With no other configuration, the deployment uses the platform's default scaling policy. The platform adds and removes instances within the `[minInstances, maxInstances]` bounds based on observed utilization.
+
+## Update Scaling Bounds on an Existing Deployment
+
+To change the bounds after deploy, look up the `deploymentId` and the `gpuSpecificationId` for the GPU spec you want to change, then `PATCH` that spec.
+
+Look up the IDs:
+
+```bash
+curl -s "https://${GATEWAY_ADDR}/v2/nvcf/deployments/functions/${FUNCTION_ID}/versions/${FUNCTION_VERSION_ID}" \
+ -H "Authorization: Bearer $NVCF_TOKEN"
+```
+
+The response includes `deployment.deploymentId` and a `gpuSpecificationId` on each entry in `deployment.deploymentSpecifications`.
+
+Update the bounds:
+
+```bash
+curl -s -X PATCH "https://${GATEWAY_ADDR}/v2/nvcf/deployments/${DEPLOYMENT_ID}/gpu-specifications/${GPU_SPEC_ID}" \
+ -H "Authorization: Bearer $NVCF_TOKEN" \
+ -H "Content-Type: application/json" \
+ -d '{
+ "minInstances": 0,
+ "maxInstances": 8
+ }'
+```
+
+The body must include at least one of `minInstances`, `maxInstances`, `autoscalingConfiguration`, or `autoscalingConfigurationPolicy`. You can't change `gpu`, `instanceType`, `backend`, `clusters`, `availabilityZones`, `preferredOrder`, or `maxRequestConcurrency` on an existing GPU spec.
+
+## Customize the Autoscaling Policy
+
+By default, every deployment uses the platform's autoscaling policy. To override it with per-function scale-up and scale-down behavior, send an `autoscalingConfiguration` block on the `PATCH` request.
+
+```bash
+curl -s -X PATCH "https://${GATEWAY_ADDR}/v2/nvcf/deployments/${DEPLOYMENT_ID}/gpu-specifications/${GPU_SPEC_ID}" \
+ -H "Authorization: Bearer $NVCF_TOKEN" \
+ -H "Content-Type: application/json" \
+ -d '{
+ "autoscalingConfigurationPolicy": "CUSTOM_CONFIGURATION",
+ "autoscalingConfiguration": {
+ "scaleUpDetails": {
+ "factor": 1.5,
+ "threshold": 75
+ },
+ "scaleDownDetails": {
+ "factor": 0.5,
+ "threshold": 25
+ }
+ }
+ }'
+```
+
+`autoscalingConfigurationPolicy` accepts two values:
+
+| Value | Effect |
+|-------|--------|
+| `CUSTOM_CONFIGURATION` | Apply the `autoscalingConfiguration` block on this request. |
+| `PLATFORM_CONFIGURATION` | Discard any custom configuration on this GPU spec and revert to the platform defaults. |
+
+`scaleUpDetails` and `scaleDownDetails` use the same shape:
+
+| Field | Type | Notes |
+|-------|------|-------|
+| `factor` | number | Multiplier applied to the current instance count. Must be `> 1.0` on `scaleUpDetails` and `< 1.0` on `scaleDownDetails`. |
+| `threshold` | integer | Utilization percent that triggers the scaling action. |
+| `stickiness` | object | Optional. See below. |
+
+## Stickiness
+
+Stickiness reduces churn by requiring utilization to stay past the threshold for a minimum window before a scaling action fires. Add a `stickiness` block to `scaleUpDetails` or `scaleDownDetails`:
+
+```json
+{
+ "autoscalingConfigurationPolicy": "CUSTOM_CONFIGURATION",
+ "autoscalingConfiguration": {
+ "scaleUpDetails": {
+ "factor": 1.5,
+ "threshold": 75,
+ "stickiness": {
+ "size": "PT10M",
+ "threshold": "PT3M"
+ }
+ }
+ }
+}
+```
+
+Both fields are ISO-8601 durations:
+
+- `size` is the length of the sliding window. Must be `<= PT1H`.
+- `threshold` is the amount of time within the window that utilization must stay past the scaling threshold. Must be `< size`.
+
+The threshold can be set to zero, but this would nullify the utilization metric. The scaling logic would effectively be collapsed into only scaling to/from zero/one instance(s).
+
+The example above scales up only if utilization exceeds 75 percent for at least 3 minutes in any 10-minute window.
+
+## Revert to Platform Defaults
+
+To discard a custom configuration without setting a new one:
+
+```bash
+curl -s -X PATCH "https://${GATEWAY_ADDR}/v2/nvcf/deployments/${DEPLOYMENT_ID}/gpu-specifications/${GPU_SPEC_ID}" \
+ -H "Authorization: Bearer $NVCF_TOKEN" \
+ -H "Content-Type: application/json" \
+ -d '{
+ "autoscalingConfigurationPolicy": "PLATFORM_CONFIGURATION"
+ }'
+```
+
+The GPU spec uses the platform's autoscaling policy on the next scaling cycle.
+
+## See Also
+
+- [Function Autoscaling Overview](./autoscaling/index.md) for what the function autoscaler does and what it depends on.
+- [CLI](./cli.md) for `nvcf-cli function deploy create` and `nvcf-cli function deploy update`, which wrap the same API surface.
+- [NVCF OpenAPI specification](https://api.nvcf.nvidia.com/v3/openapi) for the full request and response schema of the endpoints used on this page.
diff --git a/docs/v0.6.1-rc/container-functions.md b/docs/v0.6.1-rc/container-functions.md
new file mode 100644
index 000000000..7552a4dbd
--- /dev/null
+++ b/docs/v0.6.1-rc/container-functions.md
@@ -0,0 +1,325 @@
+# Container-Based Function Creation
+
+Container-based functions require building and pushing a Cloud Functions compatible [Docker](https://docker.com) container image to your container registry.
+
+## Resources
+
+- Example containers can be found [in the examples directory](https://github.com/NVIDIA/nvcf/tree/main/examples).
+
+- The repository also contains [helper functions](https://github.com/NVIDIA/nvcf/blob/main/src/libraries/python/nv-cloud-function-helpers/nv_cloud_function_helpers/nvcf_container/helpers.py) that are useful when authoring your container, including:
+
+ - Helpers that parse Cloud Functions-specific parameters on invocation
+ - Helpers that can be used to instrument your container with Cloud Functions compatible logs
+
+- It's always a **best practice to emit logs** from your inference container. Cloud Functions supports third-party logging and metrics emission from your container.
+
+
+Please note that container functions **should not run as root user**, running as root is not formally supported on any Cloud Functions backend.
+
+
+
+## Container Endpoints
+
+Any server can be implemented within the container, as long as it implements the following:
+
+- For HTTP-based functions, a health check endpoint that returns a 200 HTTP Status Code on success.
+- For gRPC-based functions, a standard gRPC health check. See [these docs for more info](https://github.com/grpc/grpc-proto/blob/master/grpc/health/v1/health.proto) also [gRPC Health Checking](https://grpc.io/docs/guides/health-checking/).
+- An inference endpoint (this endpoint will be called during function invocation)
+
+These endpoints are expected to be served on the same port, defined as the `inferencePort`.
+
+
+Cloud Functions reserves the following ports on your container for internal monitoring and metrics:
+
+- Port `8080`
+- Port `8010`
+
+Cloud Functions also expects the following directories in the container to remain read-only for caching purposes:
+
+- `/config/` directory
+- Nested directories created inside `/config/`
+
+
+
+## Composing a FastAPI Container
+
+It's possible to use any container with Cloud Functions as long as it implements a server with the above endpoints. The below is an example of a FastAPI-based container compatible with Cloud Functions. Clone the [FastAPI echo example](https://github.com/NVIDIA/nvcf/tree/main/examples/function-samples/fastapi-echo-sample).
+
+### Create the "requirements.txt" File
+
+```text
+fastapi==0.110.0
+uvicorn==0.29.0
+```
+
+### Implement the Server
+
+```python
+import os
+import time
+import uvicorn
+from pydantic import BaseModel
+from fastapi import FastAPI, status
+from fastapi.responses import StreamingResponse
+
+app = FastAPI()
+
+class HealthCheck(BaseModel):
+ status: str = "OK"
+
+# Implement the health check endpoint
+@app.get("/health", tags=["healthcheck"], summary="Perform a Health Check", response_description="Return HTTP Status Code 200 (OK)", status_code=status.HTTP_200_OK, response_model=HealthCheck)
+def get_health() -> HealthCheck:
+ return HealthCheck(status="OK")
+
+class Echo(BaseModel):
+ message: str
+ delay: float = 0.000001
+ repeats: int = 1
+ stream: bool = False
+
+# Implement the inference endpoint
+@app.post("/echo")
+async def echo(echo: Echo):
+ if echo.stream:
+ def stream_text():
+ for _ in range(echo.repeats):
+ time.sleep(echo.delay)
+ yield f"data: {echo.message}\n\n"
+ return StreamingResponse(stream_text(), media_type="text/event-stream")
+ else:
+ time.sleep(echo.delay)
+ return echo.message*echo.repeats
+
+# Serve the endpoints on a port
+if __name__ == "__main__":
+ uvicorn.run(app, host="0.0.0.0", port=8000, workers=int(os.getenv('WORKER_COUNT', 500)))
+```
+
+Note in the example above, the function's configuration during creation will be:
+
+- Inference Protocol: HTTP
+- Inference Endpoint: `/echo`
+- Health Endpoint: `/health`
+- Inference Port (also used for health check): `8000`
+
+### Create the Dockerfile
+
+```dockerfile
+FROM python:3.10.13-bookworm
+
+ENV WORKER_COUNT=10
+
+WORKDIR /app
+
+COPY requirements.txt ./
+
+RUN python -m pip install --no-cache-dir -U pip && \
+ python -m pip install --no-cache-dir -r requirements.txt
+
+COPY http_echo_server.py /app/
+
+CMD uvicorn http_echo_server:app --host=0.0.0.0 --workers=$WORKER_COUNT
+```
+
+### Build the Container & Create the Function
+
+See the [Create the Function] section below for the remaining steps.
+
+## Composing a PyTriton Container
+
+NVIDIA's [PyTriton](https://triton-inference-server.github.io/pytriton/) is a Python native solution of Triton inference server. A minimum version of 0.3.0 is required.
+
+### Create the "requirements.txt" File
+
+- This file should list the Python dependencies required for your model.
+- Add nvidia-pytriton to your `requirements.txt` file.
+
+Here is an example of a `requirements.txt` file:
+
+```text
+--extra-index-url https://pypi.ngc.nvidia.com
+opencv-python-headless
+pycocotools
+matplotlib
+torch==2.1.0
+nvidia-pytriton==0.3.0
+numpy
+```
+
+### Create the "run.py" File
+
+1. Your `run.py` file (or similar Python file) needs to define a PyTriton model.
+2. This involves importing your model dependencies, creating a PyTritonServer class with an `__init__` function, an `_infer_fn` function and a `run` function that serves the inference_function, defining the model name, the inputs and the outputs along with optional configuration.
+
+Here is an example of a `run.py` file:
+
+```python
+import numpy as np
+from pytriton.model_config import ModelConfig, Tensor
+from pytriton.triton import Triton, TritonConfig
+import time
+....
+class PyTritonServer:
+ """triton server for timed_sleeper"""
+
+ def __init__(self):
+ # basically need to accept image, mask(PIL Images), prompt, negative_prompt(str), seed(int)
+ self.model_name = "timed_sleeper"
+
+ def _infer_fn(self, requests):
+ responses = []
+ for req in requests:
+ req_data = req.data
+ sleep_duration = numpy_array_to_variable(req_data.get("sleep_duration"))
+ # deal with header dict keys being lowerscale
+ request_parameters_dict = uppercase_keys(req.parameters)
+ time.sleep(sleep_duration)
+ responses.append({"sleep_duration": np.array([sleep_duration])})
+
+ return responses
+
+ def run(self):
+ """run triton server"""
+ with Triton(
+ config=TritonConfig(
+ http_header_forward_pattern="NVCF-*", # this is required
+ http_port=8000,
+ grpc_port=8001,
+ metrics_port=8002,
+ )
+ ) as triton:
+ triton.bind(
+ model_name="timed_sleeper",
+ infer_func=self._infer_fn,
+ inputs=[
+ Tensor(name="sleep_duration", dtype=np.uint32, shape=(1,)),
+ ],
+ outputs=[Tensor(name="sleep_duration", dtype=np.uint32, shape=(1,))],
+ config=ModelConfig(batching=False),
+ )
+ triton.serve()
+if __name__ == "__main__":
+ server = PyTritonServer()
+ server.run()
+```
+
+### Create the "Dockerfile"
+
+1. Create a file named `Dockerfile` in your model directory.
+2. It's **strongly recommended to use NVIDIA-optimized containers like CUDA, Pytorch or TensorRT as your base container**. They can be downloaded from the [NGC Catalog](https://catalog.ngc.nvidia.com/).
+3. Make sure to install your Python requirements in your `Dockerfile`.
+4. Copy in your model source code, and model weights.
+
+Here is an example of a `Dockerfile`:
+
+```dockerfile
+FROM nvcr.io/nvidia/cuda:12.1.1-devel-ubuntu22.04
+RUN apt-get update && apt-get install -y \
+ git \
+ python3 \
+ python3-pip \
+ python-is-python3 \
+ libsm6 \
+ libxext6 \
+ libxrender-dev \
+ curl \
+ && rm -rf /var/lib/apt/lists/*
+WORKDIR /workspace/
+
+# Install requirements file
+COPY requirements.txt requirements.txt
+RUN pip install --no-cache-dir --upgrade pip
+RUN pip install --no-cache-dir -r requirements.txt
+ENV DEBIAN_FRONTEND=noninteractive
+
+# Copy model source code and weights
+COPY model_weights /models
+COPY model_source .
+COPY run.py .
+
+# Set run command to start PyTriton to serve the model
+CMD python3 run.py
+```
+
+### Build the Docker Image
+
+1. Open a terminal or command prompt.
+2. Navigate to the `my_model` directory.
+3. Run the following command to build the docker image:
+
+```bash
+docker build -t my_model_image .
+```
+
+Replace `my_model_image` with the desired name for your docker image.
+
+### Push the Docker Image
+
+Tag and push the docker image to your container registry.
+
+```bash
+docker tag my_model_image:latest ${REGISTRY}/${REPOSITORY}/my_model_image:latest
+docker push ${REGISTRY}/${REPOSITORY}/my_model_image:latest
+```
+
+### Create the Function
+
+Create the function via the NVCF API. In this example, we defined the inference port as `8000` and are using the default inference and health endpoint paths.
+
+```bash
+ curl -s -X POST "http://${GATEWAY_ADDR}/v2/nvcf/functions" \
+ -H "Host: api.${GATEWAY_ADDR}" \
+ -H 'Content-Type: application/json' \
+ -H "Authorization: Bearer $NVCF_TOKEN" \
+ -d '{
+ "name": "my-model-function",
+ "inferenceUrl": "/v2/models/my_model_image/infer",
+ "inferencePort": 8000,
+ "containerImage": "'${REGISTRY}'/'${REPOSITORY}'/my_model_image:latest",
+ "health": {
+ "protocol": "HTTP",
+ "uri": "/v2/health/ready",
+ "port": 8000,
+ "timeout": "PT10S",
+ "expectedStatusCode": 200
+ }
+ }'
+```
+
+### Additional Examples
+
+See more examples of containers that are Cloud Functions compatible [in the function samples directory](https://github.com/NVIDIA/nvcf/tree/main/examples/function-samples/).
+
+## Creating gRPC-based Functions
+
+Cloud Functions supports function invocation via gRPC. During function creation, specify that the function is a gRPC function by setting the `inferenceUrl` field to `/grpc`.
+
+### Prerequisites
+
+- The function container must implement a gRPC port, endpoint and health check. The health check is expected to be served by the gRPC inference port, there is no need to define a separate health endpoint path.
+
+ - See [gRPC health checking](https://grpc.io/docs/guides/health-checking/).
+ - See an [example container](https://github.com/NVIDIA/nvcf/blob/main/examples/function-samples/grpc-echo-sample/grpc_echo_server.py) with a gRPC server that is Cloud Functions compatible.
+
+### gRPC Function Creation via API
+
+When creating the gRPC function, set the `inferenceUrl` field to `/grpc`:
+
+```bash
+ curl -s -X POST "http://${GATEWAY_ADDR}/v2/nvcf/functions" \
+ -H "Host: api.${GATEWAY_ADDR}" \
+ -H 'Content-Type: application/json' \
+ -H "Authorization: Bearer $NVCF_TOKEN" \
+ -d '{
+ "name": "my-grpc-function",
+ "inferenceUrl": "/grpc",
+ "inferencePort": 8001,
+ "containerImage": "'${REGISTRY}'/'${REPOSITORY}'/grpc_echo_sample:latest"
+ }'
+```
+
+### gRPC Function Invocation
+
+For gRPC authentication metadata, gateway address configuration, and client
+examples, see [gRPC Function Invocation](./grpc-function-invocation.md).
diff --git a/docs/v0.6.1-rc/container-tasks.md b/docs/v0.6.1-rc/container-tasks.md
new file mode 100644
index 000000000..2f8c3f764
--- /dev/null
+++ b/docs/v0.6.1-rc/container-tasks.md
@@ -0,0 +1,135 @@
+# Container-Based Task Creation
+
+A container task runs a Docker image on a GPU instance until the process
+exits. Use this approach for training jobs, batch inference, data processing,
+or any workload that runs to completion.
+
+## Container requirements
+
+NVCT does not impose a server or health check requirement. The container only
+needs to:
+
+- Perform its workload.
+- Exit with code 0 on success, or a non-zero code on failure.
+
+GPU drivers and CUDA libraries are available on the host. Use an image based
+on an appropriate CUDA base image for your workload.
+
+## Creating a container task
+
+```bash
+# Minimal example using CLI flags
+nvcf-cli task create \
+ --name my-training-job \
+ --gpu H100 \
+ --instance-type GPU.H100_1x \
+ --image my-registry/training:latest
+
+# With arguments, environment variables, and secrets
+nvcf-cli task create \
+ --name my-training-job \
+ --gpu H100 \
+ --instance-type GPU.H100_1x \
+ --image my-registry/training:latest \
+ --container-args "--epochs 10 --batch-size 32" \
+ --container-env DATASET_PATH=/data/train \
+ --container-env LOG_LEVEL=info \
+ --secrets NGC_API_KEY=nvapi-... \
+ --max-runtime PT4H \
+ --result-strategy UPLOAD \
+ --results-location my-org/my-team/my-model
+
+# From a JSON file (recommended for repeatable configurations)
+nvcf-cli task create --input-file task.json
+```
+
+## Example JSON configuration
+
+```json
+{
+ "name": "my-training-job",
+ "gpuSpecification": {
+ "gpu": "H100",
+ "instanceType": "GPU.H100_1x",
+ "backend": "GFN"
+ },
+ "containerImage": "my-registry/training:latest",
+ "containerArgs": "--epochs 10 --batch-size 32",
+ "containerEnvironment": [
+ {"key": "DATASET_PATH", "value": "/data/train"},
+ {"key": "LOG_LEVEL", "value": "info"}
+ ],
+ "maxRuntimeDuration": "PT4H",
+ "maxQueuedDuration": "PT72H",
+ "resultHandlingStrategy": "UPLOAD",
+ "resultsLocation": "my-org/my-team/my-model",
+ "secrets": [
+ {"name": "NGC_API_KEY", "value": "nvapi-..."}
+ ]
+}
+```
+
+## GPU specification
+
+| Field | Description |
+| --- | --- |
+| `gpu` | GPU name, e.g. `H100`, `A100` |
+| `instanceType` | Instance type, e.g. `GPU.H100_1x`, `GPU.A100_8x` |
+| `backend` | Backend or CSP (optional) |
+| `clusters` | Specific cluster names to target (optional) |
+
+## Runtime limits
+
+| Field | Format | Default |
+| --- | --- | --- |
+| `maxRuntimeDuration` | ISO 8601 duration, e.g. `PT4H30M` | None |
+| `maxQueuedDuration` | ISO 8601 duration | `PT72H` |
+| `terminationGracePeriodDuration` | ISO 8601 duration | `PT1H` |
+
+A task that exceeds `maxRuntimeDuration` moves to
+`EXCEEDED_MAX_RUNTIME_DURATION` status. A task that is not scheduled within
+`maxQueuedDuration` moves to `EXCEEDED_MAX_QUEUED_DURATION` status.
+
+`maxRuntimeDuration` has no default or maximum: omit it and the task runs with
+no time limit.
+
+## Secrets
+
+Secrets are passed to the container as environment variables. Provide them via
+`--secrets NAME=value` on the CLI or as a `secrets` array in the JSON file.
+Secret values are stored encrypted and are not returned by default in task
+detail responses.
+
+To update secrets on a running task:
+
+```bash
+nvcf-cli task update-secrets --secrets NEW_KEY=new-value
+```
+
+## Model and resource artifacts
+
+Attach model or resource artifacts to a task using the `--models` and
+`--resources` flags (format: `name:version:uri`) or via the JSON `models` and
+`resources` arrays. These are made available to the container at runtime.
+
+## Result handling
+
+Note: result upload is not yet supported in this release.
+
+When `resultHandlingStrategy` is `UPLOAD`, the task uploads outputs to the
+registry location specified in `resultsLocation`. After the task completes,
+retrieve the results:
+
+```bash
+nvcf-cli task results
+```
+
+## Monitoring a task
+
+```bash
+# Check status
+nvcf-cli task get
+
+# Stream lifecycle events
+nvcf-cli task events
+```
diff --git a/docs/v0.6.1-rc/control-plane-installation.md b/docs/v0.6.1-rc/control-plane-installation.md
new file mode 100644
index 000000000..b1b3ecb7f
--- /dev/null
+++ b/docs/v0.6.1-rc/control-plane-installation.md
@@ -0,0 +1,18 @@
+---
+---
+
+# Control Plane Installation
+
+For a local k3d fresh install, start with the [Quickstart](./quickstart.md). The quickstart uses `nvcf-cli self-hosted up` to install the control plane, register the local k3d cluster, install NVCA, and run basic health checks.
+
+Use [Helmfile Installation](./helmfile-installation.md) when you need manual release control, partial recovery, or upgrade operations.
+
+## Installation guides
+
+- [Quickstart](./quickstart.md) - use `nvcf-cli self-hosted up` for local k3d one-click installation.
+- [Helmfile Installation](./helmfile-installation.md) - use `helmfile` for manual control-plane deployment.
+
+
+For the installation path overview, see [Deployment](./installation.md).
+
+
diff --git a/docs/v0.6.1-rc/control-plane-operations.md b/docs/v0.6.1-rc/control-plane-operations.md
new file mode 100644
index 000000000..a4e46ea95
--- /dev/null
+++ b/docs/v0.6.1-rc/control-plane-operations.md
@@ -0,0 +1,293 @@
+# Control Plane Operations
+
+This section provides runbooks for operating the self-hosted NVCF control plane, including encryption key rotation, service management, and upgrades.
+
+## Encryption Key Management
+
+Self-hosted NVCF uses a two-tier encryption hierarchy to protect secrets stored in the Encrypted Secret Store (ESS):
+
+| Key | Purpose |
+| --- | --- |
+| **MEK** (Master Encryption Key) | A single AES-256-GCM key stored in OpenBAO. The MEK encrypts (wraps) all NEKs. It is shared across NVCF services in the control plane. |
+| **NEK** (Namespace Encryption Key) | Per-namespace AES-256-GCM keys stored in Cassandra (encrypted by the MEK). NEKs directly encrypt user secrets such as API keys and registry credentials. |
+
+When a user stores a secret through the NVCF API, ESS encrypts it with the active NEK for that namespace. The NEK itself is stored in Cassandra, encrypted by the MEK. To decrypt a secret, ESS retrieves the NEK from Cassandra, decrypts it using the MEK from OpenBAO, then decrypts the secret.
+
+### Key Rotation Runbooks
+
+- [control-plane-runbook-mek-rotation](runbooks/control-plane-key-rotation-mek.md) — Rotate the master encryption key stored in OpenBAO.
+
+## Basic Operations
+
+### Service Reference
+
+The following table lists all NVCF control plane services with their namespace, resource name,
+and resource type. Use these values in the commands throughout this section.
+
+| Namespace | Service | Resource Name | Type |
+| --- | --- | --- | --- |
+| `nvcf` | NVCF API | `nvcf-api` | Deployment |
+| `nvcf` | Invocation Service | `invocation-service` | Deployment |
+| `nvcf` | gRPC Proxy | `grpc-proxy-deployment` | Deployment |
+| `nvcf` | Notary Service | `notary-service` | Deployment |
+| `sis` | Spot Instance Service | `spot-instance-service` | Deployment |
+| `api-keys` | API Keys Service | `api-keys` | Deployment |
+| `api-keys` | Admin Issuer Proxy | `admin-token-issuer-proxy` | Deployment |
+| `ess` | ESS API | `ess-api-helm-nvcf-ess-api-deployment` | Deployment |
+| `nats-system` | NATS | `nats` | StatefulSet |
+| `vault-system` | OpenBao | `openbao-server` | StatefulSet |
+| `cassandra-system` | Cassandra | `cassandra` | StatefulSet |
+| `nvca-operator` | NVCA Operator | `nvca-operator` | Deployment |
+| `envoy-gateway-system` | Envoy Gateway | `envoy-gateway` | Deployment |
+
+### Restarting a Service
+
+**Restarting a Deployment:**
+
+```bash
+kubectl rollout restart deployment/ -n
+
+# Example: restart the NVCF API
+kubectl rollout restart deployment/nvcf-api -n nvcf
+
+# Verify the rollout completes
+kubectl rollout status deployment/ -n --timeout=120s
+```
+
+**Restarting a StatefulSet:**
+
+StatefulSets perform a rolling restart, terminating and recreating one pod at a time in
+reverse ordinal order (highest first). For clustered services like NATS, OpenBao, and
+Cassandra, this preserves quorum as long as a majority of replicas remain available.
+
+```bash
+kubectl rollout restart statefulset/ -n
+
+# Example: restart Cassandra
+kubectl rollout restart statefulset/cassandra -n cassandra-system
+
+# Verify the rollout completes
+kubectl rollout status statefulset/ -n --timeout=300s
+```
+
+
+For OpenBao, verify the seal status after the rollout completes. Each pod must unseal
+before it can serve requests:
+
+```bash
+kubectl exec -n vault-system openbao-server-0 -- bao status
+```
+
+
+
+**Restarting all Deployments in a namespace:**
+
+```bash
+kubectl rollout restart deployment -n
+
+# Example: restart all services in the nvcf namespace
+kubectl rollout restart deployment -n nvcf
+```
+
+### Checking Service Health
+
+**List pods and their status:**
+
+```bash
+# All pods in a namespace
+kubectl get pods -n
+
+# All NVCF control plane pods at a glance
+for ns in nvcf sis api-keys ess nats-system vault-system cassandra-system; do
+ echo "=== $ns ==="
+ kubectl get pods -n $ns
+done
+```
+
+**Check logs for a service:**
+
+```bash
+# Recent logs
+kubectl logs -n -l app.kubernetes.io/name= --tail=50
+
+# Follow logs in real time
+kubectl logs -n -l app.kubernetes.io/name= -f
+```
+
+**Describe a pod for events and conditions:**
+
+```bash
+kubectl describe pod -n -l app.kubernetes.io/name=
+```
+
+### Scaling a Service
+
+To temporarily take a service offline (for example, during maintenance), scale it to zero,
+perform the work, then scale it back:
+
+```bash
+# Scale down
+kubectl scale deployment/ -n --replicas=0
+
+# ... perform maintenance ...
+
+# Scale back up
+kubectl scale deployment/ -n --replicas=1
+
+# Verify
+kubectl rollout status deployment/ -n
+```
+
+
+Scaling infrastructure StatefulSets (Cassandra, NATS, OpenBao) to zero will cause a full
+outage. Only do this if you understand the implications for data availability and quorum.
+
+
+
+## Upgrading Services
+
+
+**Upgrades are not officially supported during the Early Access period.** The self-hosted
+NVCF stack does not yet have a validated upgrade path. Even a full `helmfile sync` may
+introduce breaking changes between releases — there is no guarantee of backward
+compatibility for configuration, database schemas, or inter-service APIs at this stage.
+
+
+
+The guidance below is provided for **advanced users** who need to apply targeted fixes or
+hotfixes to individual services. It is not a substitute for a validated upgrade procedure.
+
+
+**Spot upgrades carry additional risk.** Beyond the general Early Access limitations above,
+spot-upgrading an individual Helm chart bypasses the Helmfile's version coordination and
+automatic database migrations. Proceed only when you understand the compatibility
+implications for the specific version you are upgrading to.
+
+
+
+### When to Spot Upgrade
+
+| Use a spot upgrade when | Use a full Helmfile upgrade when |
+| --- | --- |
+| Applying a patch release to a single service (e.g., `1.2.6` to `1.2.7`) | Upgrading the entire stack to a new minor or major version |
+| Applying a targeted hotfix provided by NVIDIA support | The new version includes Cassandra schema migrations |
+| You need to roll out a configuration change that requires a new chart version | Multiple services need to be upgraded together for compatibility |
+
+### Pre-Upgrade Checklist
+
+Before upgrading any chart:
+
+1. **Note the current chart version and app version:**
+
+ ```bash
+ helm list -n
+ ```
+
+2. **Back up the current Helm values:**
+
+ ```bash
+ helm get values -n -o yaml > -values-backup.yaml
+ ```
+
+3. **Review release notes** for the target version. Check for breaking changes, required
+ value changes, or new dependencies.
+
+4. **Verify the cluster is healthy** before starting — all pods running, no pending operations.
+
+### Spot Upgrading a Helm Chart
+
+The following commands work for any Deployment-based service. Replace the placeholders with
+values from the [Service Reference] table above.
+
+```bash
+# 1. Upgrade the chart
+helm upgrade \
+ oci://// \
+ --version \
+ --namespace \
+ --wait --timeout 5m \
+ -f -values.yaml
+
+# 2. Verify the rollout
+kubectl rollout status deployment/ -n --timeout=120s
+
+# 3. Confirm the new chart version
+helm list -n
+```
+
+**Example — upgrading the NVCF API chart:**
+
+```bash
+helm upgrade api \
+ oci://${REGISTRY}/${REPOSITORY}/helm-nvcf-api \
+ --version 2.1.0 \
+ --namespace nvcf \
+ --wait --timeout 5m \
+ -f nvcf-api-values.yaml
+
+kubectl rollout status deployment/api -n nvcf --timeout=120s
+```
+
+
+Always pass your values file (`-f values.yaml`) during upgrade. If you omit it, Helm
+resets all values to chart defaults, which can break your deployment. If you no longer
+have the original values file, back up the current values first with `helm get values`.
+
+
+
+### Upgrading StatefulSet-Based Services
+
+Cassandra, NATS, and OpenBao are deployed as StatefulSets. The `helm upgrade` command is the
+same, but the rollout behavior differs:
+
+- **Rolling update:** StatefulSets restart pods one at a time in reverse ordinal order,
+ waiting for each pod to become ready before proceeding to the next.
+- **Quorum preserved:** For 3-replica clusters, at most one pod is unavailable at a time,
+ maintaining quorum throughout the upgrade.
+
+```bash
+helm upgrade \
+ oci://// \
+ --version \
+ --namespace \
+ --wait --timeout 10m \
+ -f -values.yaml
+
+kubectl rollout status statefulset/ -n --timeout=300s
+```
+
+**Service-specific notes:**
+
+| **Cassandra** | After upgrading, check if the new version requires a schema migration. If a `cassandra-migrations` job exists in the chart, it will run automatically as a Helm post-upgrade hook. Verify it completes: `kubectl get jobs -n cassandra-system`. |
+| --- | --- |
+| **OpenBao** | After each pod restarts, verify it unseals successfully: `kubectl exec -n vault-system openbao-server-0 -- bao status`. If auto-unseal is configured, this happens automatically. Otherwise, you must unseal each pod manually. |
+| **NATS** | The NATS cluster maintains message availability during rolling updates as long as a majority of nodes remain online. Monitor the cluster state: `kubectl logs -n nats-system nats-0 --tail=10`. |
+
+### Rolling Back
+
+If an upgrade causes issues, roll back to the previous Helm revision:
+
+```bash
+# List revision history
+helm history -n
+
+# Roll back to the previous revision
+helm rollback -n
+
+# Verify
+kubectl rollout status deployment/ -n --timeout=120s
+
+# Or for StatefulSets
+kubectl rollout status statefulset/ -n --timeout=300s
+```
+
+
+`helm rollback` reverts both the chart version and the values. If you made intentional
+value changes alongside the version upgrade, you will need to re-apply them after the
+rollback.
+
+
+
+## Observability
+
+For observability configuration and reference architecture, see [self-hosted-observability](./observability.md).
diff --git a/docs/v0.6.1-rc/csp-end-to-end-example-installation.md b/docs/v0.6.1-rc/csp-end-to-end-example-installation.md
new file mode 100644
index 000000000..81a080aaf
--- /dev/null
+++ b/docs/v0.6.1-rc/csp-end-to-end-example-installation.md
@@ -0,0 +1,639 @@
+# CSP End-to-End Example Installation (Helmfile)
+
+This page provides a complete end-to-end example for installing NVCF on
+pre-provisioned managed Kubernetes clusters using the split Helmfile bundles.
+It covers both topologies:
+
+- Single-cluster: the control plane and the NVCA operator run on the same cluster.
+- Multi-cluster: the control plane runs on one cluster, and the NVCA operator is
+ registered and installed on a separate GPU (compute) cluster.
+
+The commands are written to work on any cloud provider (CSP). Amazon EKS is used
+as the worked example. The only provider-specific pieces are the load balancer
+annotations on the Gateway, the `storageClass` name, and the `kubectl` context
+names. Substitute the equivalents for GKE, AKS, or on-prem.
+
+For a deeper reference on each release and on values, see
+[Helmfile Installation](./helmfile-installation.md). For pulling and mirroring
+the bundles and images, see [Image Mirroring](./image-mirroring.md).
+
+
+This guide assumes you have already downloaded and extracted the control-plane
+Helmfile bundle and have a source checkout for the compute plane:
+
+- `nvcf-self-managed-stack` for the control plane.
+- `deploy/stacks/nvcf-compute-plane` in the source repository for the compute
+ plane (NVCA operator).
+
+Control-plane commands run from inside the `nvcf-self-managed-stack` directory.
+Compute-plane commands run from the repository root with `make -C`.
+
+```bash
+git clone https://github.com/nvidia/nvcf.git
+```
+
+
+## Installation order
+
+The order matters. Each step produces an input that the next step needs. The
+load balancer address, in particular, must exist before you configure the
+environment file, because it becomes `global.domain` and the NVCA Host headers.
+
+```
+1. Install the Gateway -> external load balancer address
+2. Configure the environment -> environments/.yaml + secrets/-secrets.yaml
+3. Install the control plane -> control-plane services + HTTPRoutes
+4. Author the nvcf-cli config -> points at the load balancer address
+5. Register the GPU cluster -> registration values file
+6. Install the NVCA operator -> agent connects back to the control plane
+7. Verify the agent is healthy
+```
+
+Single-cluster and multi-cluster share steps 1 through 4. Step 5 requires a
+compute-plane environment file for both topologies. Only the cluster target and
+control-plane endpoint values differ. See
+[Single-cluster vs multi-cluster](#single-cluster-vs-multi-cluster).
+
+## Prerequisites
+
+### Tools
+
+Install on the machine you run these commands from:
+
+- `kubectl`
+- `helm` (3.x)
+- `helmfile`
+- `nvcf-cli`
+
+### Clusters
+
+The clusters must be provisioned before you start. This guide does not create
+them. Each cluster needs:
+
+- A default-capable `StorageClass` with dynamic provisioning. On EKS this is
+ `gp3`, backed by the EBS CSI driver. Substitute your provider's class name.
+- The compute (GPU) cluster needs a GPU operator (real or the fake GPU operator
+ for non-GPU validation). See [Fake GPU Operator](./fake-gpu-operator.md).
+- The compute cluster needs the
+ [SMB CSI driver](https://github.com/kubernetes-csi/csi-driver-smb)
+ (`smb.csi.k8s.io`). NVCA uses it for shared model cache storage that function
+ worker pods mount. Install and verify the driver before registering the GPU
+ cluster. See the
+ [Self-Managed Clusters prerequisites](./cluster-management/self-managed.md#prerequisites)
+ for the installation command.
+
+Both clusters must be reachable through `kubectl` contexts:
+
+```bash
+kubectl --context "${CONTROL_PLANE_CONTEXT}" get nodes -o name
+# Multi-cluster only:
+kubectl --context "${COMPUTE_CONTEXT}" get nodes -o name
+```
+
+### Environment variables
+
+Set these once. In single-cluster, `COMPUTE_CONTEXT` equals
+`CONTROL_PLANE_CONTEXT`.
+
+```bash
+# Cluster targeting
+export CONTROL_PLANE_CONTEXT=""
+export COMPUTE_CONTEXT="