From 7ec0f4126deb8308209e10670ebaa256a3bc3909 Mon Sep 17 00:00:00 2001 From: Nick Larsen Date: Thu, 18 Dec 2025 12:06:25 +0100 Subject: [PATCH 01/10] Add guide for deploying operators and CSI drivers on separate nodes NOTE: This is work-in-progress --- modules/guides/nav.adoc | 1 + ...-operators-and-csi-drivers-separately.adoc | 353 ++++++++++++++++++ 2 files changed, 354 insertions(+) create mode 100644 modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc diff --git a/modules/guides/nav.adoc b/modules/guides/nav.adoc index bcf28f170..fe595dd2a 100644 --- a/modules/guides/nav.adoc +++ b/modules/guides/nav.adoc @@ -6,3 +6,4 @@ ** xref:viewing-and-verifying-sboms.adoc[] ** xref:enabling-verification-of-image-signatures.adoc[] ** xref:kubernetes-cluster-domain.adoc[] +** xref:deploy-operators-and-csi-drivers-separately.adoc[] diff --git a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc new file mode 100644 index 000000000..80a40c2b7 --- /dev/null +++ b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc @@ -0,0 +1,353 @@ += Deploying operators and CSI drivers on separate nodes +:related-issue: https://github.com/stackabletech/issues/issues/763 +:secret-operator-values: https://github.com/stackabletech/secret-operator/blob/main/deploy/helm/secret-operator/values.yaml +:listener-operator-values: https://github.com/stackabletech/listener-operator/blob/main/deploy/helm/listener-operator/values.yaml +:commons-operator-values: https://github.com/stackabletech/commons-operator/blob/main/deploy/helm/commons-operator/values.yaml +:nifi-operator-values: https://github.com/stackabletech/nifi-operator/blob/main/deploy/helm/nifi-operator/values.yaml +// TODO: Is there a way to make the links above go to the right place? +// Eg: we can search/replace "0.0.0-dev" with the release, but in the above case we would need to replace "main". + +Operators can be installed on nodes separate from where the workloads will run. +There is a caveat when it comes to two operators: Secret Operator and Listener Operator. +They make use of the Container Storage Interface (CSI) and have components that must run on nodes with workloads that mount CSI volumes. + +This guide will show how to schedule operators on one group of nodes (for example, a Karpenter NodePool), while scheduling applicable components where workload will run. + +== Setup + +You will need a Kubernetes cluster with multiple nodes split into two groups: + +* stackable-operators +* stackable-workloads + +TODO: tabs for kind or Karpenter NodePools + +[NOTE] +==== +This guide will use _KinD_ to demonstrate, but if you are using Karpenter (eg: AWK EKS), +you can adjust the labels to be based on the name of your NodePools. + +For example: + +* `karpenter.sh/nodepool: stackable-operators` +* `karpenter.sh/nodepool: stackable-workloads` + +==== + +Create a KinD config called `kind-config.yaml` containing: + +[source,yaml] +---- +kind: Cluster +apiVersion: kind.x-k8s.io/v1alpha4 +nodes: +- role: control-plane +- role: worker + labels: + nodepool: stackable-operators +- role: worker + labels: + nodepool: stackable-operators +- role: worker + labels: + nodepool: stackable-workloads +- role: worker + labels: + nodepool: stackable-workloads +---- + +Launch the cluster: + +[source,bash] +---- +kind create cluster --name stackable --config kind-config.yaml +---- + +You can see which nodes are in which _nodepool_ by using the following command: + +[source,bash] +---- +kubectl get nodes -o json | jq ' + .items[] + | .metadata.name as $name + | .metadata.labels["nodepool"] as $nodepool + | $nodepool//empty + | {"nodename": $name, "nodepool": $nodepool} +' +---- + +== Prepare Helm Values for the Stackable Operators + +[NOTE] +==== +Most Stackable operators use the same Helm Values structure, however Secret and +Listener operator differ slightly - which is what allows the components to be +configured independently of each other. +==== + +// TODO: Move these into files and include them (so we can run them easily) + +[tabs] +==== +Secret Operator:: ++ +-- +Store the values in a file called `stackable-secret-operator.yaml`. + +// TODO: Link to default values + +[source,yaml] +---- +controllerService: + nodeSelector: + nodepool: stackable-operators + +csiNodeDriver: + # Node Drivers need to run on the same nodes as the workloads using them + nodeSelector: + nodepool: stackable-workloads +---- + +-- + +Listener Operator:: ++ +-- +Store the values in a file called `stackable-listener-operator.yaml`. + +// TODO: Link to default values + +[source,yaml] +---- +csiProvisioner: + nodeSelector: + nodepool: stackable-operators + +csiNodeDriver: + # Node Drivers need to run on the same nodes as the workloads using them + nodeSelector: + nodepool: stackable-workloads +---- + +-- + +Remaining operators:: ++ +-- +Store the values in a file called `stackable-operators.yaml`. + +// TODO: Link to default values for remaining operators used in this guide + +[source,yaml] +---- +nodeSelector: + nodepool: stackable-operators +---- + +-- +==== + +NOTE: If you would like to run on nodes with taints, you can list `tolerations` next to the `nodeSelector`. + +== Install the Stackable Operators + +Now install the operators to the applicable node pools by using the Helm overrides + +[tabs] +==== +Secret Operator:: ++ +-- +NOTE: This operator uses a specific values file. + +[source,bash] +---- +helm install secret-operator \ + --version=0.0.0-dev \ + --values=stackable-secret-operator.yaml \ + oci://oci.stackable.tech/sdp-charts/secret-operator +---- + +-- + +Listener Operator:: ++ +-- +NOTE: This operator uses a specific values file. + +[source,bash] +---- +helm install listener-operator \ + --version=0.0.0-dev \ + --values=stackable-listener-operator.yaml \ + oci://oci.stackable.tech/sdp-charts/listener-operator +---- + +-- + +Remaining operators:: ++ +-- +NOTE: These operator use the same values file. + +[source,bash] +---- +helm install commons-operator \ + --version=0.0.0-dev \ + --values=stackable-operators.yaml \ + oci://oci.stackable.tech/sdp-charts/commons-operator + +helm install nifi-operator \ + --version=0.0.0-dev \ + --values=stackable-operators.yaml \ + oci://oci.stackable.tech/sdp-charts/nifi-operator +---- + +-- +==== + +You should now see that the operators are running on the `stackable-operator` nodes, while the CSI drivers are running on the `stackable-workload` nodes. + +Pods running on the `stackable-operators` node pool: + +[source,bash] +---- +OPERATORS_NODEPOOL=$(kubectl get nodes -l nodepool=stackable-operators -o jsonpath="{.items[*].metadata.name}" | tr ' ' ',') +echo "Nodes in operators pool: $OPERATORS_NODEPOOL\n" +kubectl get pods -o json | jq --raw-output --arg nodepool "$OPERATORS_NODEPOOL" '.items[] | .metadata.name as $podname | .spec.nodeName as $nodename | select($nodename | IN($nodepool | split(",")[])) | $podname' +---- + +You should see similar output showing the Stackable Operators are running only on nodes with the label `nodepool: stackable-operators`. + +[source] +--- +Nodes in operators pool: stackable-worker,stackable-worker2 + +commons-operator-deployment-674c469b47-nm5vb +listener-operator-csi-provisioner-85b686d48-hv5kf +nifi-operator-deployment-7c59778bb8-r26b8 +secret-operator-66b85c669d-7hsxs +--- + +Pods running on the `stackable-workloads` node pool: + +[source,bash] +---- +WORKLOADS_NODEPOOL=$(kubectl get nodes -l nodepool=stackable-workloads -o jsonpath="{.items[*].metadata.name}" | tr ' ' ',') +echo "Nodes in workloads pool: $WORKLOADS_NODEPOOL\n" +kubectl get pods -o json | jq --raw-output --arg nodepool "$WORKLOADS_NODEPOOL" '.items[] | .metadata.name as $podname | .spec.nodeName as $nodename | select($nodename | IN($nodepool | split(",")[])) | $podname' +---- + +You should see similar output showing the Stackable CSI Node Drivers are running only on nodes with the label `nodepool: stackable-workloads`. + +[source] +--- +Nodes in workloads pool: stackable-worker3,stackable-worker4 + +listener-operator-csi-node-driver-lv5r4 +listener-operator-csi-node-driver-vdzsq +secret-operator-csi-node-driver-d8sqw +secret-operator-csi-node-driver-zkrv6 +--- + +The CSI Node Drivers register as such. +This can be seen with the driver count being 2 (one for listener-operator volumes, and one for secret-operator volumes) for nodes in the workloads pool: + +[source,console] +---- +$ kubectl get csinodes +NAME DRIVERS AGE +stackable-control-plane 0 3h40m +stackable-worker 0 3h39m +stackable-worker2 0 3h39m +stackable-worker3 2 3h39m +stackable-worker4 2 3h39m +---- + +== Install a workload + +We'll install a NiFi cluster onto a `stackable-workload` node. Create a new file called `nifi.yaml` with the following contents: +// This is taken from the NiFi Getting Started, but with some modifications. +// TODO: Update nifi getting started to remove zookeeper. + +[source,yaml] +---- +--- +apiVersion: v1 +kind: Secret +metadata: + name: simple-admin-credentials +stringData: + admin: admin +--- +apiVersion: authentication.stackable.tech/v1alpha1 +kind: AuthenticationClass +metadata: + name: simple-nifi-users +spec: + provider: + static: + userCredentialsSecret: + name: simple-admin-credentials +--- +apiVersion: nifi.stackable.tech/v1alpha1 +kind: NifiCluster +metadata: + name: simple-nifi +spec: + image: + productVersion: 2.6.0 + clusterConfig: + authentication: + - authenticationClass: simple-nifi-users + sensitiveProperties: + keySecret: nifi-sensitive-property-key + autoGenerate: true + nodes: + roleGroups: + default: + replicas: 1 + config: + # Run NiFi nodes in the workloads pool + affinity: + nodeSelector: + nodepool: stackable-workloads +---- + +Apply it to Kubernetes: + +[source,console] +---- +$ kubectl apply -f nifi.yaml +---- + +Then take a look at the pods running on nodes with the label `nodepool: stackable-workloads`: + +[source,bash] +---- +WORKLOADS_NODEPOOL=$(kubectl get nodes -l nodepool=stackable-workloads -o jsonpath="{.items[*].metadata.name}" | tr ' ' ',') +echo "Nodes in workloads pool: $WORKLOADS_NODEPOOL\n" +kubectl get pods -o json | jq --raw-output --arg nodepool "$WORKLOADS_NODEPOOL" '.items[] | .metadata.name as $podname | .spec.nodeName as $nodename | select($nodename | IN($nodepool | split(",")[])) | $podname' +---- + +You should see similar output as last time, but now with the NiFi pod. + +[source] +--- +Nodes in workloads pool: stackable-worker3,stackable-worker4 + +listener-operator-csi-node-driver-lv5r4 +listener-operator-csi-node-driver-vdzsq +secret-operator-csi-node-driver-d8sqw +secret-operator-csi-node-driver-zkrv6 +simple-nifi-node-default-0 +--- + +== Cleanup + +Once done, you can delete the KinD cluster like so: + +[source,bash] +---- +kind delete cluster --name stackable +---- From 01da299e60f6245940f56a73fe971886c1d3dd46 Mon Sep 17 00:00:00 2001 From: Nick Larsen Date: Fri, 19 Dec 2025 16:47:25 +0100 Subject: [PATCH 02/10] wip --- .../pages/deploy-operators-and-csi-drivers-separately.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc index 80a40c2b7..a77e5575d 100644 --- a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc +++ b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc @@ -20,7 +20,7 @@ You will need a Kubernetes cluster with multiple nodes split into two groups: * stackable-operators * stackable-workloads -TODO: tabs for kind or Karpenter NodePools +// TODO: Add an image to illustrate what the guide aims to achieve [NOTE] ==== @@ -333,7 +333,7 @@ kubectl get pods -o json | jq --raw-output --arg nodepool "$WORKLOADS_NODEPOOL" You should see similar output as last time, but now with the NiFi pod. [source] ---- +---- Nodes in workloads pool: stackable-worker3,stackable-worker4 listener-operator-csi-node-driver-lv5r4 @@ -341,7 +341,7 @@ listener-operator-csi-node-driver-vdzsq secret-operator-csi-node-driver-d8sqw secret-operator-csi-node-driver-zkrv6 simple-nifi-node-default-0 ---- +---- == Cleanup From 5481429900a144c30c9a5aa850d553af68ddda12 Mon Sep 17 00:00:00 2001 From: Nick Larsen Date: Fri, 19 Dec 2025 16:48:21 +0100 Subject: [PATCH 03/10] wip --- .../deploy-operators-and-csi-drivers-separately.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc index a77e5575d..f17ec3375 100644 --- a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc +++ b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc @@ -220,14 +220,14 @@ kubectl get pods -o json | jq --raw-output --arg nodepool "$OPERATORS_NODEPOOL" You should see similar output showing the Stackable Operators are running only on nodes with the label `nodepool: stackable-operators`. [source] ---- +---- Nodes in operators pool: stackable-worker,stackable-worker2 commons-operator-deployment-674c469b47-nm5vb listener-operator-csi-provisioner-85b686d48-hv5kf nifi-operator-deployment-7c59778bb8-r26b8 secret-operator-66b85c669d-7hsxs ---- +---- Pods running on the `stackable-workloads` node pool: @@ -241,14 +241,14 @@ kubectl get pods -o json | jq --raw-output --arg nodepool "$WORKLOADS_NODEPOOL" You should see similar output showing the Stackable CSI Node Drivers are running only on nodes with the label `nodepool: stackable-workloads`. [source] ---- +---- Nodes in workloads pool: stackable-worker3,stackable-worker4 listener-operator-csi-node-driver-lv5r4 listener-operator-csi-node-driver-vdzsq secret-operator-csi-node-driver-d8sqw secret-operator-csi-node-driver-zkrv6 ---- +---- The CSI Node Drivers register as such. This can be seen with the driver count being 2 (one for listener-operator volumes, and one for secret-operator volumes) for nodes in the workloads pool: From 77c210f28e77f789b5c3fe8ef36656ef33d4c3d7 Mon Sep 17 00:00:00 2001 From: Nick <10092581+NickLarsenNZ@users.noreply.github.com> Date: Tue, 6 Jan 2026 09:24:05 +0100 Subject: [PATCH 04/10] Apply suggestions from code review Co-authored-by: Techassi --- ...-operators-and-csi-drivers-separately.adoc | 44 ++++++++++++++++--- 1 file changed, 37 insertions(+), 7 deletions(-) diff --git a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc index f17ec3375..99b8429e4 100644 --- a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc +++ b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc @@ -188,7 +188,7 @@ helm install listener-operator \ Remaining operators:: + -- -NOTE: These operator use the same values file. +NOTE: These operators use the same values file. [source,bash] ---- @@ -212,9 +212,19 @@ Pods running on the `stackable-operators` node pool: [source,bash] ---- -OPERATORS_NODEPOOL=$(kubectl get nodes -l nodepool=stackable-operators -o jsonpath="{.items[*].metadata.name}" | tr ' ' ',') +OPERATORS_NODEPOOL=$( + kubectl get nodes -l nodepool=stackable-operators -o jsonpath="{.items[*].metadata.name}" \ + | tr ' ' ',' +) + echo "Nodes in operators pool: $OPERATORS_NODEPOOL\n" -kubectl get pods -o json | jq --raw-output --arg nodepool "$OPERATORS_NODEPOOL" '.items[] | .metadata.name as $podname | .spec.nodeName as $nodename | select($nodename | IN($nodepool | split(",")[])) | $podname' +kubectl get pods -o json | jq --raw-output --arg nodepool "$OPERATORS_NODEPOOL" ' + .items[] + | .metadata.name as $podname + | .spec.nodeName as $nodename + | select($nodename | IN($nodepool | split(",")[])) + | $podname +' ---- You should see similar output showing the Stackable Operators are running only on nodes with the label `nodepool: stackable-operators`. @@ -233,9 +243,19 @@ Pods running on the `stackable-workloads` node pool: [source,bash] ---- -WORKLOADS_NODEPOOL=$(kubectl get nodes -l nodepool=stackable-workloads -o jsonpath="{.items[*].metadata.name}" | tr ' ' ',') +WORKLOADS_NODEPOOL=$( + kubectl get nodes -l nodepool=stackable-workloads -o jsonpath="{.items[*].metadata.name}" \ + | tr ' ' ',' +) + echo "Nodes in workloads pool: $WORKLOADS_NODEPOOL\n" -kubectl get pods -o json | jq --raw-output --arg nodepool "$WORKLOADS_NODEPOOL" '.items[] | .metadata.name as $podname | .spec.nodeName as $nodename | select($nodename | IN($nodepool | split(",")[])) | $podname' +kubectl get pods -o json | jq --raw-output --arg nodepool "$WORKLOADS_NODEPOOL" ' + .items[] + | .metadata.name as $podname + | .spec.nodeName as $nodename + | select($nodename | IN($nodepool | split(",")[])) + | $podname +' ---- You should see similar output showing the Stackable CSI Node Drivers are running only on nodes with the label `nodepool: stackable-workloads`. @@ -325,9 +345,19 @@ Then take a look at the pods running on nodes with the label `nodepool: stackabl [source,bash] ---- -WORKLOADS_NODEPOOL=$(kubectl get nodes -l nodepool=stackable-workloads -o jsonpath="{.items[*].metadata.name}" | tr ' ' ',') +WORKLOADS_NODEPOOL=$( + kubectl get nodes -l nodepool=stackable-workloads -o jsonpath="{.items[*].metadata.name}" + | tr ' ' ',' +) + echo "Nodes in workloads pool: $WORKLOADS_NODEPOOL\n" -kubectl get pods -o json | jq --raw-output --arg nodepool "$WORKLOADS_NODEPOOL" '.items[] | .metadata.name as $podname | .spec.nodeName as $nodename | select($nodename | IN($nodepool | split(",")[])) | $podname' +kubectl get pods -o json | jq --raw-output --arg nodepool "$WORKLOADS_NODEPOOL" ' + .items[] + | .metadata.name as $podname + | .spec.nodeName as $nodename + | select($nodename | IN($nodepool | split(",")[])) + | $podname +' ---- You should see similar output as last time, but now with the NiFi pod. From 758b3c9c3374bc46aa29bbb627f000e47ee5c708 Mon Sep 17 00:00:00 2001 From: Nick Larsen Date: Tue, 6 Jan 2026 12:32:06 +0100 Subject: [PATCH 05/10] Make an antora extension to derive the operator branches --- antora-playbook.yml | 1 + local-antora-playbook.yml | 1 + ...-operators-and-csi-drivers-separately.adoc | 14 ++++----- only-dev-antora-playbook.yml | 1 + .../lib/stackable-operator-branch.js | 29 +++++++++++++++++++ truly-local-playbook.yml | 1 + 6 files changed, 40 insertions(+), 7 deletions(-) create mode 100644 supplemental-ui/lib/stackable-operator-branch.js diff --git a/antora-playbook.yml b/antora-playbook.yml index bcf58d12a..30c06bc49 100644 --- a/antora-playbook.yml +++ b/antora-playbook.yml @@ -30,6 +30,7 @@ antora: - require: '@sntke/antora-mermaid-extension' mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs script_stem: mermaid-scripts + - ./supplemental-ui/lib/stackable-operator-branch.js content: sources: - url: . diff --git a/local-antora-playbook.yml b/local-antora-playbook.yml index 2f3285ae0..7190209fc 100644 --- a/local-antora-playbook.yml +++ b/local-antora-playbook.yml @@ -15,6 +15,7 @@ antora: - require: '@sntke/antora-mermaid-extension' mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs script_stem: mermaid-scripts + - ./supplemental-ui/lib/stackable-operator-branch.js content: sources: - url: ./ diff --git a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc index 99b8429e4..09c667505 100644 --- a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc +++ b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc @@ -1,9 +1,9 @@ = Deploying operators and CSI drivers on separate nodes :related-issue: https://github.com/stackabletech/issues/issues/763 -:secret-operator-values: https://github.com/stackabletech/secret-operator/blob/main/deploy/helm/secret-operator/values.yaml -:listener-operator-values: https://github.com/stackabletech/listener-operator/blob/main/deploy/helm/listener-operator/values.yaml -:commons-operator-values: https://github.com/stackabletech/commons-operator/blob/main/deploy/helm/commons-operator/values.yaml -:nifi-operator-values: https://github.com/stackabletech/nifi-operator/blob/main/deploy/helm/nifi-operator/values.yaml +:secret-operator-values: https://github.com/stackabletech/secret-operator/blob/{stackable-operator-branch}/deploy/helm/secret-operator/values.yaml +:listener-operator-values: https://github.com/stackabletech/listener-operator/blob/{stackable-operator-branch}/deploy/helm/listener-operator/values.yaml +:commons-operator-values: https://github.com/stackabletech/commons-operator/blob/{stackable-operator-branch}/deploy/helm/commons-operator/values.yaml +:nifi-operator-values: https://github.com/stackabletech/nifi-operator/blob/{stackable-operator-branch}/deploy/helm/nifi-operator/values.yaml // TODO: Is there a way to make the links above go to the right place? // Eg: we can search/replace "0.0.0-dev" with the release, but in the above case we would need to replace "main". @@ -94,7 +94,7 @@ Secret Operator:: -- Store the values in a file called `stackable-secret-operator.yaml`. -// TODO: Link to default values +See default Chart link:{secret-operator-values}[values]. [source,yaml] ---- @@ -115,7 +115,7 @@ Listener Operator:: -- Store the values in a file called `stackable-listener-operator.yaml`. -// TODO: Link to default values +See default Chart link:{listener-operator-values}[values]. [source,yaml] ---- @@ -136,7 +136,7 @@ Remaining operators:: -- Store the values in a file called `stackable-operators.yaml`. -// TODO: Link to default values for remaining operators used in this guide +See default Chart values for link:{commons-operator-values}[commons-operator] and link:{nifi-operator-values}[nifi-operator]. [source,yaml] ---- diff --git a/only-dev-antora-playbook.yml b/only-dev-antora-playbook.yml index 1171f8410..5951875db 100644 --- a/only-dev-antora-playbook.yml +++ b/only-dev-antora-playbook.yml @@ -15,6 +15,7 @@ antora: - require: '@sntke/antora-mermaid-extension' mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs script_stem: mermaid-scripts + - ./supplemental-ui/lib/stackable-operator-branch.js content: sources: - url: ./ diff --git a/supplemental-ui/lib/stackable-operator-branch.js b/supplemental-ui/lib/stackable-operator-branch.js new file mode 100644 index 000000000..e19366e27 --- /dev/null +++ b/supplemental-ui/lib/stackable-operator-branch.js @@ -0,0 +1,29 @@ +// This is so we can map `nightly` (from the release dropdown) to the `main` +// branch, and `YY.M` -> `release-YY.M`. +// +// In adocs, you can access it via {stackable-operator-branch}. +// +// Useful links: +// Extensions: https://docs.antora.org/antora/latest/extend/extensions/ +// Types of events: https://docs.antora.org/antora/latest/extend/generator-events-reference/ +module.exports.register = function () { + this.once('contentClassified', ({ playbook, contentCatalog }) => { + contentCatalog.getComponents().forEach((component) => { + component.versions.forEach((componentVersion) => { + const operatorBranch = componentVersion.version === 'nightly' + ? 'main' + : `release-${componentVersion.version}` + + // Not sure why we need a new object, but _they_ do it. + // See: https://github.com/couchbase/docs-site/blob/b7db9602fc035945ace72e3152e9fb83ef7cba51/lib/antora-component-version-rank.js + componentVersion.asciidoc = { + ...componentVersion.asciidoc, + attributes: { + ...componentVersion.asciidoc.attributes, + 'stackable-operator-branch': operatorBranch + } + } + }) + }) + }) +} diff --git a/truly-local-playbook.yml b/truly-local-playbook.yml index e9b1df55f..f9312d83e 100644 --- a/truly-local-playbook.yml +++ b/truly-local-playbook.yml @@ -15,6 +15,7 @@ antora: - require: '@sntke/antora-mermaid-extension' mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs script_stem: mermaid-scripts + - ./supplemental-ui/lib/stackable-operator-branch.js content: sources: - url: ./ From 5a9742a5d8cf67a6d3f4cfe751259c769aa8c5a7 Mon Sep 17 00:00:00 2001 From: Nick Larsen Date: Tue, 6 Jan 2026 12:52:38 +0100 Subject: [PATCH 06/10] Update stackable helper to set the operator version too --- antora-playbook.yml | 2 +- local-antora-playbook.yml | 2 +- ...loy-operators-and-csi-drivers-separately.adoc | 16 ++++++++-------- only-dev-antora-playbook.yml | 2 +- ...r-branch.js => stackable-operator-helpers.js} | 15 +++++++++------ truly-local-playbook.yml | 2 +- 6 files changed, 21 insertions(+), 18 deletions(-) rename supplemental-ui/lib/{stackable-operator-branch.js => stackable-operator-helpers.js} (56%) diff --git a/antora-playbook.yml b/antora-playbook.yml index 30c06bc49..457b55e00 100644 --- a/antora-playbook.yml +++ b/antora-playbook.yml @@ -30,7 +30,7 @@ antora: - require: '@sntke/antora-mermaid-extension' mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs script_stem: mermaid-scripts - - ./supplemental-ui/lib/stackable-operator-branch.js + - ./supplemental-ui/lib/stackable-operator-helpers.js content: sources: - url: . diff --git a/local-antora-playbook.yml b/local-antora-playbook.yml index 7190209fc..b0fefa006 100644 --- a/local-antora-playbook.yml +++ b/local-antora-playbook.yml @@ -15,7 +15,7 @@ antora: - require: '@sntke/antora-mermaid-extension' mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs script_stem: mermaid-scripts - - ./supplemental-ui/lib/stackable-operator-branch.js + - ./supplemental-ui/lib/stackable-operator-helpers.js content: sources: - url: ./ diff --git a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc index 09c667505..fb6195e3e 100644 --- a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc +++ b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc @@ -5,7 +5,7 @@ :commons-operator-values: https://github.com/stackabletech/commons-operator/blob/{stackable-operator-branch}/deploy/helm/commons-operator/values.yaml :nifi-operator-values: https://github.com/stackabletech/nifi-operator/blob/{stackable-operator-branch}/deploy/helm/nifi-operator/values.yaml // TODO: Is there a way to make the links above go to the right place? -// Eg: we can search/replace "0.0.0-dev" with the release, but in the above case we would need to replace "main". +// Eg: we can search/replace "{stackable-operator-version}" with the release, but in the above case we would need to replace "main". Operators can be installed on nodes separate from where the workloads will run. There is a caveat when it comes to two operators: Secret Operator and Listener Operator. @@ -160,10 +160,10 @@ Secret Operator:: -- NOTE: This operator uses a specific values file. -[source,bash] +[source,bash,subs="attributes"] ---- helm install secret-operator \ - --version=0.0.0-dev \ + --version={stackable-operator-version} \ --values=stackable-secret-operator.yaml \ oci://oci.stackable.tech/sdp-charts/secret-operator ---- @@ -175,10 +175,10 @@ Listener Operator:: -- NOTE: This operator uses a specific values file. -[source,bash] +[source,bash,subs="attributes"] ---- helm install listener-operator \ - --version=0.0.0-dev \ + --version={stackable-operator-version} \ --values=stackable-listener-operator.yaml \ oci://oci.stackable.tech/sdp-charts/listener-operator ---- @@ -190,15 +190,15 @@ Remaining operators:: -- NOTE: These operators use the same values file. -[source,bash] +[source,bash,subs="attributes"] ---- helm install commons-operator \ - --version=0.0.0-dev \ + --version={stackable-operator-version} \ --values=stackable-operators.yaml \ oci://oci.stackable.tech/sdp-charts/commons-operator helm install nifi-operator \ - --version=0.0.0-dev \ + --version={stackable-operator-version} \ --values=stackable-operators.yaml \ oci://oci.stackable.tech/sdp-charts/nifi-operator ---- diff --git a/only-dev-antora-playbook.yml b/only-dev-antora-playbook.yml index 5951875db..2248e9318 100644 --- a/only-dev-antora-playbook.yml +++ b/only-dev-antora-playbook.yml @@ -15,7 +15,7 @@ antora: - require: '@sntke/antora-mermaid-extension' mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs script_stem: mermaid-scripts - - ./supplemental-ui/lib/stackable-operator-branch.js + - ./supplemental-ui/lib/stackable-operator-helpers.js content: sources: - url: ./ diff --git a/supplemental-ui/lib/stackable-operator-branch.js b/supplemental-ui/lib/stackable-operator-helpers.js similarity index 56% rename from supplemental-ui/lib/stackable-operator-branch.js rename to supplemental-ui/lib/stackable-operator-helpers.js index e19366e27..ab8f7f5a7 100644 --- a/supplemental-ui/lib/stackable-operator-branch.js +++ b/supplemental-ui/lib/stackable-operator-helpers.js @@ -1,7 +1,8 @@ // This is so we can map `nightly` (from the release dropdown) to the `main` -// branch, and `YY.M` -> `release-YY.M`. +// branch and 0.0.0-dev version, and `YY.M` to the `release-YY.M` branch and +// YY.M.0 version. // -// In adocs, you can access it via {stackable-operator-branch}. +// In adocs, you can access it via {stackable-operator-branch} and {stackable-operator-version}. // // Useful links: // Extensions: https://docs.antora.org/antora/latest/extend/extensions/ @@ -10,9 +11,10 @@ module.exports.register = function () { this.once('contentClassified', ({ playbook, contentCatalog }) => { contentCatalog.getComponents().forEach((component) => { component.versions.forEach((componentVersion) => { - const operatorBranch = componentVersion.version === 'nightly' - ? 'main' - : `release-${componentVersion.version}` + const operatorInfo = componentVersion.version === 'nightly' + ? { branch: 'main', version: '0.0.0-dev' } + // TODO: Be clever about the patch level (eg: pull from the release repo: https://github.com/stackabletech/release/blob/main/releases.yaml) + : { branch: `release-${componentVersion.version}`, version: `${componentVersion.version}.0` } // Not sure why we need a new object, but _they_ do it. // See: https://github.com/couchbase/docs-site/blob/b7db9602fc035945ace72e3152e9fb83ef7cba51/lib/antora-component-version-rank.js @@ -20,7 +22,8 @@ module.exports.register = function () { ...componentVersion.asciidoc, attributes: { ...componentVersion.asciidoc.attributes, - 'stackable-operator-branch': operatorBranch + 'stackable-operator-branch': operatorInfo.branch, + 'stackable-operator-version': operatorInfo.version, } } }) diff --git a/truly-local-playbook.yml b/truly-local-playbook.yml index f9312d83e..a5ea96538 100644 --- a/truly-local-playbook.yml +++ b/truly-local-playbook.yml @@ -15,7 +15,7 @@ antora: - require: '@sntke/antora-mermaid-extension' mermaid_library_url: https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs script_stem: mermaid-scripts - - ./supplemental-ui/lib/stackable-operator-branch.js + - ./supplemental-ui/lib/stackable-operator-helpers.js content: sources: - url: ./ From fb9badc11f10e0291b9988b55b29d44ac32736c4 Mon Sep 17 00:00:00 2001 From: Nick <10092581+NickLarsenNZ@users.noreply.github.com> Date: Tue, 6 Jan 2026 12:55:30 +0100 Subject: [PATCH 07/10] Apply suggestions from code review --- .../pages/deploy-operators-and-csi-drivers-separately.adoc | 2 -- 1 file changed, 2 deletions(-) diff --git a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc index fb6195e3e..bfc480e9d 100644 --- a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc +++ b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc @@ -4,8 +4,6 @@ :listener-operator-values: https://github.com/stackabletech/listener-operator/blob/{stackable-operator-branch}/deploy/helm/listener-operator/values.yaml :commons-operator-values: https://github.com/stackabletech/commons-operator/blob/{stackable-operator-branch}/deploy/helm/commons-operator/values.yaml :nifi-operator-values: https://github.com/stackabletech/nifi-operator/blob/{stackable-operator-branch}/deploy/helm/nifi-operator/values.yaml -// TODO: Is there a way to make the links above go to the right place? -// Eg: we can search/replace "{stackable-operator-version}" with the release, but in the above case we would need to replace "main". Operators can be installed on nodes separate from where the workloads will run. There is a caveat when it comes to two operators: Secret Operator and Listener Operator. From ba16b34f1c227d545f0408eef5b1aebeac4f0c84 Mon Sep 17 00:00:00 2001 From: Nick Larsen Date: Tue, 6 Jan 2026 17:34:14 +0100 Subject: [PATCH 08/10] Add illustration --- .../topology.drawio.svg | 280 ++++++++++++++++++ ...-operators-and-csi-drivers-separately.adoc | 2 +- 2 files changed, 281 insertions(+), 1 deletion(-) create mode 100644 modules/guides/images/deploy-operators-and-csi-drivers-separately/topology.drawio.svg diff --git a/modules/guides/images/deploy-operators-and-csi-drivers-separately/topology.drawio.svg b/modules/guides/images/deploy-operators-and-csi-drivers-separately/topology.drawio.svg new file mode 100644 index 000000000..23b4fc563 --- /dev/null +++ b/modules/guides/images/deploy-operators-and-csi-drivers-separately/topology.drawio.svg @@ -0,0 +1,280 @@ + + + + + + + + + + +
+
+
+ nodepool: stackable-operators +
+
+
+ + + nodepool: stackable-operators + + + + + + + + + + + +
+
+
+ nodepool: stackable-workloads +
+
+
+ + + nodepool: stackable-workloads + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+ worker +
+
+
+ + + worker + + + + + + + + + + + + + +
+
+
+ worker2 +
+
+
+ + + worker2 + + + + + + + + + + + + + +
+
+
+ worker3 +
+
+
+ + + worker3 + + + + + + + + + + + + + +
+
+
+ worker4 +
+
+
+ + + worker4 + + + + + + + + + + + + + listener-operator-csi-provisioner-... + + + + + + + + + + + + listener-operator-csi-node-driver-... + + + + + + + + + + + + listener-operator-csi-node-driver-... + + + + + + + + + + + + secret-operator-... + + + + + + + + + + + + secret-operator-csi-node-driver-... + + + + + + + + + + + + secret-operator-csi-node-driver-... + + + + + + + + + + + + nifi-operator-... + + + + + + + + + + + + simple-nifi-node-default-0 + + + + + + + + + + + + +
+
+
+ + commons-operator-... + +
+
+
+ + + commons-... + + + + + + + + + + Text is not SVG - cannot display + + + + diff --git a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc index bfc480e9d..17b0415bb 100644 --- a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc +++ b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc @@ -18,7 +18,7 @@ You will need a Kubernetes cluster with multiple nodes split into two groups: * stackable-operators * stackable-workloads -// TODO: Add an image to illustrate what the guide aims to achieve +image::deploy-operators-and-csi-drivers-separately/topology.drawio.svg[] [NOTE] ==== From da803e3f754690210be8bb50f96e8b34bea87492 Mon Sep 17 00:00:00 2001 From: Nick Larsen Date: Tue, 6 Jan 2026 17:38:57 +0100 Subject: [PATCH 09/10] remove zookeeper znode for nifi --- modules/ROOT/pages/getting-started.adoc | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/modules/ROOT/pages/getting-started.adoc b/modules/ROOT/pages/getting-started.adoc index d14c340b9..e73c77cf1 100644 --- a/modules/ROOT/pages/getting-started.adoc +++ b/modules/ROOT/pages/getting-started.adoc @@ -162,14 +162,6 @@ We will next deploy an Apache NiFi server. ---- kubectl apply -f - < Date: Tue, 6 Jan 2026 17:41:17 +0100 Subject: [PATCH 10/10] Apply suggestions from code review --- .../pages/deploy-operators-and-csi-drivers-separately.adoc | 1 - 1 file changed, 1 deletion(-) diff --git a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc index 17b0415bb..9373aa2fc 100644 --- a/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc +++ b/modules/guides/pages/deploy-operators-and-csi-drivers-separately.adoc @@ -286,7 +286,6 @@ stackable-worker4 2 3h39m We'll install a NiFi cluster onto a `stackable-workload` node. Create a new file called `nifi.yaml` with the following contents: // This is taken from the NiFi Getting Started, but with some modifications. -// TODO: Update nifi getting started to remove zookeeper. [source,yaml] ----