I am not able to run examples/inference.py on release/2026-07-29-05a8aad6

[karshtharyani]

Hi,

I keep getting this error and I am not sure what's going on. I think my torch and CUDA versions are compatible.

RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasLtMatmulAlgoGetHeuristic( ltHandle, computeDesc.descriptor(), Adesc.descriptor(), Bdesc.descriptor(), Cdesc.descriptor(), Cdesc.descriptor(), preference.descriptor(), 1, &heuristicResult, &returnedResult)

[lfengad]

Hi, this could usually due to:

1. The memory is not enough, you may further check the GPU memory and its usage in your setting. (most likely)
2. The GPU card architecture you used is not supported with you torch version.
3. The GPU device visibility is not okay in your env setting, such as the driver not updated, /dev/shm too small, etc.

You might check the reason 1 first? Thx!

[karshtharyani]

Thu Jul 30 09:53:06 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.00                 Driver Version: 580.00         CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA Thor                    Off |   00000000:01:00.0 Off |                  N/A |
| N/A   37C  N/A               2W /  N/A  | Not Supported          |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA RTX PRO 6000 Blac...    Off |   00000004:09:00.0 Off |                  Off |
| 30%   30C    P8              8W /  300W |      15MiB /  97887MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A            4824      G   /usr/lib/xorg/Xorg                      154MiB |
|    0   N/A  N/A            5053      G   /usr/bin/gnome-shell                     51MiB |
|    1   N/A  N/A            4824      G   /usr/lib/xorg/Xorg                        4MiB |
+-----------------------------------------------------------------------------------------+

This is the NVIDIA thor. Is this not sufficient for Cosmos Nano?

[lfengad]

There are two GPUs in your system: Thor and RTX. If you use the GPU 0, then maybe you need to check whether it supports Thor (SM_110). Like to run python -c "import torch; print(torch.__version__, torch.version.cuda); print(torch.cuda.get_arch_list())" to check whether it includes sm_110. Thx!
Or, you could use GPU 1, which is RTX. You might also need to check your torch can support the SM version of it. Thx!