Scan vulkan devices in reverse order, to avoid main GPU

[xxxajk]

Summary

Chose a Vulkan device in reverse order, so that the main display GPU is not chosen by default in a multi-GPU setup.
Simple one-line fix.

Validation

• make -C c check
• CUDA changes were tested with make -C c cuda-test (if applicable)
• Performance claims include hardware, commands, and repeatable measurements

Compatibility

• The default CPU build remains dependency-free
• No model files, generated binaries, or benchmark artifacts are included

[JustVugg]

One line, and it changes device selection for everyone:

- for (uint32_t i = 0; i < count; i++)
+ for (int i = (int)count - 1; i >= 0; i--)

The problem is real — on a box with an iGPU and a discrete card, or two discrete cards, Vulkan enumerates in an order the driver chooses and colibri takes the first. If that is your display GPU, you get VRAM contention with the desktop. @Limalski hit an adjacent version of this in #848.

But reverse order is not a fix, it is a different arbitrary choice. It works on your machine because your enumeration happens to put the card you want last. On the next reporter's machine it will be first again, and the one-line change will need reverting for them.

What the engine should do is pick, not iterate:

• prefer VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU over INTEGRATED_GPU over the rest — the property Vulkan exposes precisely so callers do not have to guess from ordering
• break ties on device-local heap size, which is what actually matters for an expert tier
• and honour an explicit override, so someone with two identical cards can still say which

The last of those may already be most of the way there: please check whether COLI_VK_DEVICE or similar exists before writing anything, because selecting by index would let you solve your case today without changing anyone else's default.

If you want to take the typed selection, I would take that PR. If you would rather just report the environment, post vulkaninfo | grep -A2 deviceType and the enumeration order you see — with two of those from different vendors the heuristic writes itself, and right now nobody here has a multi-GPU Vulkan box to test on.

3 of 5 checklist items and CI has not reported yet; I have approved the workflow run so it will.

[xxxajk]

One line, and it changes device selection for everyone:

- for (uint32_t i = 0; i < count; i++)
+ for (int i = (int)count - 1; i >= 0; i--)

The problem is real — on a box with an iGPU and a discrete card, or two discrete cards, Vulkan enumerates in an order the driver chooses and colibri takes the first. If that is your display GPU, you get VRAM contention with the desktop. @Limalski hit an adjacent version of this in #848.

But reverse order is not a fix, it is a different arbitrary choice. It works on your machine because your enumeration happens to put the card you want last. On the next reporter's machine it will be first again, and the one-line change will need reverting for them.

What the engine should do is pick, not iterate:

* prefer `VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU` over `INTEGRATED_GPU` over the rest — the property Vulkan exposes precisely so callers do not have to guess from ordering

* break ties on device-local heap size, which is what actually matters for an expert tier

* and honour an explicit override, so someone with two identical cards can still say which

The last of those may already be most of the way there: please check whether COLI_VK_DEVICE or similar exists before writing anything, because selecting by index would let you solve your case today without changing anyone else's default.

If you want to take the typed selection, I would take that PR. If you would rather just report the environment, post vulkaninfo | grep -A2 deviceType and the enumeration order you see — with two of those from different vendors the heuristic writes itself, and right now nobody here has a multi-GPU Vulkan box to test on.

3 of 5 checklist items and CI has not reported yet; I have approved the workflow run so it will.

How about an option to choose which device then?

        deviceType        = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
        deviceName        = AMD Radeon RX 5700 XT (RADV NAVI10)
        pipelineCacheUUID = 81083e57-af5d-e002-1230-8be7d6da98cf
--
        deviceType        = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
        deviceName        = Tesla P40
        pipelineCacheUUID = 7cebaaca-c4fb-2b03-58ee-f08a3305aec7
--
        deviceType        = PHYSICAL_DEVICE_TYPE_CPU
        deviceName        = llvmpipe (LLVM 20.1.2, 256 bits)
        pipelineCacheUUID = 32352e32-2e38-2d30-7562-756e7475302e

[xxxajk]

I propose options can cure the issues, leaving defaults as is, and it's pretty simple.
A flag that specifies which GPU to use, if the GPU is not there, or does not have the resources, we fall back to scanning.

The scanning can rank each choice based on a few metrics, and choose the best one as an additive ranking.
During scan each ranking can simply add 1, and the highest overall value wins.
Each criteria could contain:
Is this CPU only? Skip it, that's the default.
Can the device do compute? Add 1
Dedicated, no video (like the P40)? Add 1
Minimum amount of free VRAM available that we can work with? add 1

When there is a tie amongst multiple devices, the device with the most free VRAM wins.
If there is STILL a tie, choose the first one.

So that means:
An if flag, we can default to something that's not going to be ever present, -1 for example would cause it to fall through to scanning.
A new int or byte array to hold the scanned ranking.
No extra loops, we only collect capabilities with a few if statements.
Finally three if statements for the scan validation, breaks any the ties and if nothing is good, we just use CPU.

That's how I think it could be solved, with minimal effort, and it would provide a few safety checks.

[JustVugg]

@xxxajk — eight days without an answer to a design proposal you wrote in good faith is on us, sorry. Here is where it stands, with the part you will want to know first: half of what you proposed is already on dev.

backend_vulkan.c now ranks devices instead of taking devs[0]:

int rank = p.deviceType == VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU   ? 4 :
           p.deviceType == VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU ? 3 :
           p.deviceType == VK_PHYSICAL_DEVICE_TYPE_VIRTUAL_GPU    ? 2 :
           p.deviceType == VK_PHYSICAL_DEVICE_TYPE_OTHER          ? 1 : 0; // CPU last

That is your "is this CPU only? skip it / can it do compute? +1 / dedicated? +1" additive ranking, in the shape it ended up taking — and it means the original symptom of this PR is likely fixed for the iGPU + discrete case, without either arbitrary enumeration order. Worth testing on your box before anything else: if dev already picks the right card for you, the remaining work is smaller than when you wrote the proposal.

What is still missing from your design, and is still wanted:

1. An explicit primary-device selector. There is COLI_VK_DEV2 for the second-device expert tier, but nothing to say "use device N as the primary" — so a user with two discrete cards (your P40 case, or two identical GPUs) still cannot choose, because the type ranking ties. This is the piece that actually needs your proposal.
2. Free VRAM as the tie-break. The VK_EXT_memory_budget extension is already detected at init for allocation pressure — the budget it exposes is exactly the number a tie-break would rank on, so the plumbing exists.
3. "Dedicated, no display output" as a distinct signal from DISCRETE_GPU. A P40 and a display-attached 4090 both report discrete; only the first is free of desktop contention. Vulkan does not expose "has a display" directly, so this one needs thought — that is a design question worth its own comment rather than an assumption.

So: this PR as written should close — the one-line reverse is superseded by the ranking, and we agreed it was a different arbitrary choice rather than a fix. Your proposal should become a new PR, scoped to (1) and (2): an explicit selector plus a free-VRAM tie-break on top of the existing ranking. That is small, testable, and does not need (3) to be useful.

Would you like to write it, or would you rather we do and have you test it on the multi-GPU box? Either is fine — you found the problem and designed the fix, so the choice is yours. Closing this one only when you have said which.

[xxxajk]

Cool, no rush, I'll give it a look asap, right now my GPUs are involved in a long horizon task.

[xxxajk]

BTW, finding out if the GPU is compute only, I did a search, and this is what I found:
To determine if a Vulkan physical device has video output (display presentation capability) versus being compute-only, inspect its queue families using vkGetPhysicalDeviceQueueFamilyProperties and check for presentation support via vkGetPhysicalDeviceSurfaceSupportKHR.

A compute-only device exposes queue families with compute/transfer flags but lacks support for presenting images to a window or screen surface.

HTH :-)

[ZacharyZcR]

The loop bound has an off-by-one that changes the semantics beyond the stated goal: for (uint32_t i = nd-1; i > 0; i--) never evaluates devs[0] at all — the condition excludes index 0, so device 0 is not merely deprioritized on ties, it is excluded from ranking entirely. Two concrete consequences:

• On a machine where devs[0] is the only discrete GPU (common: dGPU enumerates first, iGPU second), this selects the integrated GPU unconditionally — the rank comparison that is supposed to prefer discrete never sees the discrete device.
• nd == 1 works only by accident: i = 0, loop body never runs, bestrank stays -1, and the devs[0] fallback catches it.

If the intent is 'on equal rank, prefer the later-enumerated device' (avoid the display GPU), the full-coverage form does that with no exclusion:

for (uint32_t i = nd; i-- > 0;) {

— iterates nd-1 .. 0 inclusive, and with the existing strict > comparison the first-seen (highest-index) device wins ties, which is exactly the avoid-main-GPU behavior this PR wants, while a lone discrete GPU at index 0 still outranks everything. An explicit VK_DEVICE=<index> override might serve the underlying use case (compute on the non-display GPU) more predictably than enumeration-order heuristics, but that's the maintainer's call; the bound fix stands either way.

[xxxajk]

@ZacharyZcR nice catch.