Trident v25.10.0 (and also v25.06.3) fail to clone volumes.

[morete]

THIS ISSUE HAS A RELATED SUPPORT TICKET #2010595177

Describe the bug
Trident fails storage class name validation when trying to clone a volume.

Environment
Provide accurate information about the environment to help us reproduce the issue.

• Trident version: v25.10.0 or v25.06.3
• Trident installation flags used: Using helm chart with values tridentSilenceAutosupport: true, tridentDisableAuditLog: false
• Container runtime: containerd://1.7.27
• Kubernetes version: 1.32.8
• Kubernetes orchestrator: Vanilla Kubernetes
• Kubernetes enabled feature gates: No custom feature-gates
• OS: tried on 2 clusters, one on Ubuntu 22.04.5 and another on Ubuntu 24.04.3, in all cases with all the iscsi node preparations.
• NetApp backend types: ONTAP AFF-A90 with version 9.15.1P11
• Other:

To Reproduce
Steps to reproduce the behaviour:

• Create a Kubevirt VM.
• Snapshot the VM.
• Create a DataSource to use the created snapshot.
• Create a VM using the DataSource.

This will trigger Trident to clone a volume and fail the storage class name check.

Expected behaviour
Trident to successfully clone the volume and Kubevirt VM is successfully cloned as it works in Trident v24.10.1

Additional context
The target clone storage class is never specified so it is 100% on Trident to use the correct one or define one at all, could be that the storage class for the target volume is never specified and the storage class name validatino fails due to that...

Small log extract from Trident controller:

trident-controller-587894f986-5gdgh trident-main time="2026-01-15T16:22:32Z" level=debug msg="Found storage class for requested volume pvc-d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c." logLayer=csi_frontend name=san-xyz requestID=2c250f0c-951f-498e-bf35-ddad780fbafc requestSource=CSI workflow="volume=create"
trident-controller-587894f986-5gdgh trident-main time="2026-01-15T16:22:32Z" level=debug msg="<<<< CreateVolume" Method=CreateVolume Type=CSI_Controller logLayer=csi_frontend name=pvc-d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c requestID=2c250f0c-951f-498e-bf35-ddad780fbafc requestSource=CSI workflow="volume=create"
trident-controller-587894f986-5gdgh trident-main time="2026-01-15T16:22:32Z" level=error msg="GRPC error: rpc error: code = Unknown desc = clone volume pvc-d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c from source volume pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa with different storage classes is not allowed" logLayer=csi_frontend requestID=2c250f0c-951f-498e-bf35-ddad780fbafc requestSource=CSI
trident-controller-587894f986-5gdgh csi-provisioner I0115 16:22:32.574645       1 connection.go:270] "GRPC response" response="{}" err="rpc error: code = Unknown desc = clone volume pvc-d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c from source volume pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa with different storage classes is not allowed"
trident-controller-587894f986-5gdgh csi-provisioner I0115 16:22:32.574696       1 controller.go:846] CreateVolume failed, supports topology = true, node selected true => may reschedule = true => state = Finished: rpc error: code = Unknown desc = clone volume pvc-d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c from source volume pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa with different storage classes is not allowed
trident-controller-587894f986-5gdgh csi-provisioner I0115 16:22:32.574764       1 controller.go:1108] "Final error received, removing PVC from claims in progress" claimUID="d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c"
trident-controller-587894f986-5gdgh csi-provisioner I0115 16:22:32.574792       1 controller.go:965] "Retrying syncing claim" key="d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c" failures=2
trident-controller-587894f986-5gdgh csi-provisioner E0115 16:22:32.574836       1 controller.go:988] "Unhandled Error" err="error syncing claim \"d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c\": failed to provision volume with StorageClass \"san-xyz\": rpc error: code = Unknown desc = clone volume pvc-d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c from source volume pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa with different storage classes is not allowed" logger="UnhandledError"
trident-controller-587894f986-5gdgh csi-provisioner I0115 16:22:32.574896       1 event.go:389] "Event occurred" object="image-catalog/tmp-pvc-976b50b2-3ee1-47df-8f7c-465082d890d7" fieldPath="" kind="PersistentVolumeClaim" apiVersion="v1" type="Warning" reason="ProvisioningFailed" message="failed to provision volume with StorageClass \"san-xyz\": rpc error: code = Unknown desc = clone volume pvc-d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c from source volume pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa with different storage classes is not allowed"
trident-controller-587894f986-5gdgh csi-provisioner I0115 16:22:32.575278       1 event_expansion.go:99] "Request Body" body="{\"count\":3,\"lastTimestamp\":\"2026-01-15T16:22:32Z\",\"message\":\"failed to provision volume with StorageClass \\\"san-xyz\\\": rpc error: code = Unknown desc = clone volume pvc-d2a0c0a0-925b-44d6-ae7f-00ad14d2ee2c from source volume pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa with different storage classes is not allowed\"}"

Source PV exists, target gets never created:
pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa:

apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    pv.kubernetes.io/provisioned-by: csi.trident.netapp.io
    volume.kubernetes.io/provisioner-deletion-secret-name: ""
    volume.kubernetes.io/provisioner-deletion-secret-namespace: ""
  creationTimestamp: "2025-11-05T15:12:45Z"
  finalizers:
  - external-provisioner.volume.kubernetes.io/finalizer
  - kubernetes.io/pv-protection
  - external-attacher/csi-trident-netapp-io
  name: pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa
  resourceVersion: "4537665348"
  uid: e1d97ae2-f411-4334-8f2f-92c9dd79b0ad
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 500Gi
  claimRef:
    name: windows-golden-image
    namespace: image-catalog
    resourceVersion: "4537662238"
    uid: 20ca6d98-0ee5-48fd-af7e-2d3411eb1aec
  csi:
    driver: csi.trident.netapp.io
    volumeAttributes:
      backendUUID: 9b4b1f03-1094-4bf8-9fd0-c06b150542e8
      internalName: f457_b_d84d_pvc_5ddbe252_5f29_43e7_b3d6_c6e4e7e065fa
      name: pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa
      protocol: block
      storage.kubernetes.io/csiProvisionerIdentity: 1762353815195-1372-csi.trident.netapp.io
    volumeHandle: pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: topology.kubernetes.io/zone
          operator: In
          values:
          - b
        - key: topology.kubernetes.io/region
          operator: In
          values:
          - central
  persistentVolumeReclaimPolicy: Delete
  storageClassName: san-xyz
  volumeMode: Block

[morete]

I'm not an expert coder but by looking at the source, I think there might be potentially some issues at the comparisons or the data being compared on the functions cloneVolumeInitial and cloneVolumeRetry on https://github.com/NetApp/trident/blob/5e6ab2ae6b66e045d3fce943eee4502ff29632aa/core/orchestrator_core.go
and same logic repeats on https://github.com/NetApp/trident/blob/5e6ab2ae6b66e045d3fce943eee4502ff29632aa/core/concurrent_core.go

when I look at this condition:
if !isDockerPluginMode() && volumeConfig.StorageClass != sourceVolume.Config.StorageClass
I suspect the volumeConfig.StorageClass is empty... since the sourceVolume exists and is found and of course it must have a storage class, I think the volume that needs to be created is initiated without a storage class and then the comparison fails. #

[morete]

Here is a suggestion from github copilot on how to solve this updating the function cloneVolumeInitial to pick the source volume storage class before comparing IF the new volume comes without a storage class when cloning, probably the new check should also include the "if !isDockerPluginMode() && "

func (o *TridentOrchestrator) cloneVolumeInitial(
	ctx context.Context, volumeConfig *storage.VolumeConfig,
) (externalVol *storage.VolumeExternal, err error) {
	...
	// Get the source volume
	sourceVolume, found := o.volumes[volumeConfig.CloneSourceVolume]
	if !found {
		...
	}

	// If the incoming clone request doesn't specify a storage class, inherit
	// the source volume's storage class so later storage-class validation
	// compares equivalent values.
	if volumeConfig.StorageClass == "" {
		volumeConfig.StorageClass = sourceVolume.Config.StorageClass
	}

	// Check if the storage class of source and clone volume is different, ...
	if !isDockerPluginMode() && volumeConfig.StorageClass != sourceVolume.Config.StorageClass {
		return nil, errors.MismatchedStorageClassError("clone volume %s from source volume %s with"+
			" different storage classes is not allowed", volumeConfig.Name, volumeConfig.CloneSourceVolume)
	}
	...
}

[morete]

Interestingly the TridentVolume resource in namespace trident stated the volume pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa is in a different storageClass, the kubernetes PV reports the correct storageClass, I have manually modified the TridentVolume pvc-5ddbe252-5f29-43e7-b3d6-c6e4e7e065fa to have the correct storageClass and then had to restart all Trident pods to pick the change... since then, I can clone a VM again.

For some unknown reason Trident stored the wrong storage class for the volume and seems this storageClass validation was not performed in previous Trident versions...