Device selection by ID does not work

[henryruhs]

Describe the issue

Many users reach out that GPU.0, GPU.1 etc. is not working for them when creating a InferenceSession. According to the documentation this should work - but in fact only GPU does, which should just be an alias to GPU.0

...\rel-1.19_final\onnxruntime\onnxruntime\core\providers\openvino\openvino_execution_provider.cc:72 __cdecl onnxruntime::OpenVINOExecutionProvider::OpenVINOExecutionProvider(const struct onnxruntime::OpenVINOExecutionProviderInfo &) [ERROR] [OpenVINO] Specified device - GPU.0 is not available
 when using [('OpenVINOExecutionProvider', {'device_type': 'GPU.0', 'precision': 'FP32'})]
Falling back to ['CPUExecutionProvider'] and retrying.

A user with an Intel Arc 770 send me this, his device id is 0.

To reproduce

We found this script to debug his graphics card.

He then send me his output.

ZE_LOADER_DEBUG_TRACE:Using Loader Library Path: 
ZE_LOADER_DEBUG_TRACE:Tracing Layer Library Path: libze_tracing_layer.so.1
['CPU', 'GPU']

It seems this is an issue on OpenVINO's API itself or that different drivers return different information? Anyways, it could be resolved on the onnxruntime level bei hardcoding GPU.0 to GPU.

Urgency

High priority as this breaks apps.

Platform

Windows

OS Version

11

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.19.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

OpenVINO

Execution Provider Library Version

Used with OpenVino 2024.3.0

[n1harika]

Inferencing from OpenVINO depends on the available devices on your system- 'GPU.0' is equivalent to 'GPU' and 'GPU.1' can only be used if an additional dGPU driver is available on your system. Kindly use the latest release of OVEP and make sure your GPU drivers are updated and can be read by OpenVINO.

[henryruhs]

This does not explain this error:

[OpenVINO] Specified device - GPU.0 is not available

[hlevring]

Since its somewhat related I just wanted to note this PR
#614
passing LUID as device Id would be a nice and clean option (obviously still does not explain why GPU and GPU.0 is not the same in the context of the reported issue)

[henryruhs]

Intel seems busy to not support their devices.

[sfatimar]

Sorry for the late response. We are working on this feature and will get it upstreamed by WW16 at the latest.

[sfatimar]

PR to Microsoft has been raised for this issue: microsoft#24394

[henryruhs]

thanks for the effort