Add fp32 resnet50 model for vitisai ep demo#116
Open
Json288 wants to merge 2 commits intomicrosoft:mainfrom
Open
Add fp32 resnet50 model for vitisai ep demo#116Json288 wants to merge 2 commits intomicrosoft:mainfrom
Json288 wants to merge 2 commits intomicrosoft:mainfrom
Conversation
Signed-off-by: Song <jamesong@amd.com>
Signed-off-by: Song <jamesong@amd.com>
adrastogi
reviewed
Apr 7, 2026
| url: "webnn/efficientnet-lite4/resolve/main/onnx/model_fp16.onnx", | ||
| path: "./demos/image-classification/models/webnn/efficientnet-lite4/onnx", | ||
| }, | ||
| { |
Contributor
There was a problem hiding this comment.
I was testing out the changes in this branch, and it looks like I can't fetch the full suite of models (after the seventh one, the npm command exits and doesn't download the remaining ones). Are you seeing anything similar? Example output below.
PS C:\src\webnn-developer-preview-amd> npm run fetch-models
> webnn-developer-preview@1.0.0 fetch-models
> node fetch_models.js
[1/40] Downloading https://huggingface.co/xenova/resnet-50/resolve/main/onnx/model_fp16.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\xenova\resnet-50\onnx\model_fp16.onnx
-> downloading [========================================] 100% 0.0s
[1/40] Downloaded https://huggingface.co/xenova/resnet-50/resolve/main/onnx/model_fp16.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\xenova\resnet-50\onnx\model_fp16.onnx
[2/40] Downloading https://huggingface.co/webnn/mobilenet-v2/resolve/main/onnx/model_fp16.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\webnn\mobilenet-v2\onnx\model_fp16.onnx
-> downloading [========================================] 100% 0.0s
[2/40] Downloaded https://huggingface.co/webnn/mobilenet-v2/resolve/main/onnx/model_fp16.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\webnn\mobilenet-v2\onnx\model_fp16.onnx
[3/40] Downloading https://huggingface.co/webnn/efficientnet-lite4/resolve/main/onnx/model_fp16.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\webnn\efficientnet-lite4\onnx\model_fp16.onnx
-> downloading [========================================] 100% 0.0s
[3/40] Downloaded https://huggingface.co/webnn/efficientnet-lite4/resolve/main/onnx/model_fp16.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\webnn\efficientnet-lite4\onnx\model_fp16.onnx
[4/40] Downloading https://huggingface.co/amd/resnet50/resolve/main/webnn/onnx/model.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\amd\resnet50\onnx\model.onnx
-> downloading [========================================] 100% 0.0s
[4/40] Downloaded https://huggingface.co/amd/resnet50/resolve/main/webnn/onnx/model.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\amd\resnet50\onnx\model.onnx
[5/40] Downloading https://huggingface.co/amd/MobileNetV2/resolve/main/webnn/onnx/model.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\amd\MobileNetV2\onnx\model.onnx
-> downloading [========================================] 100% 0.0s
[5/40] Downloaded https://huggingface.co/amd/MobileNetV2/resolve/main/webnn/onnx/model.onnx to C:\src\webnn-developer-preview-amd\demos\image-classification\models\amd\MobileNetV2\onnx\model.onnx
[6/40] Downloading https://huggingface.co/microsoft/sd-turbo-webnn/resolve/main/text_encoder/model_layernorm.onnx to C:\src\webnn-developer-preview-amd\demos\sd-turbo\models\text_encoder\model_layernorm.onnx
-> downloading [========================================] 100% 0.0s
[6/40] Downloaded https://huggingface.co/microsoft/sd-turbo-webnn/resolve/main/text_encoder/model_layernorm.onnx to C:\src\webnn-developer-preview-amd\demos\sd-turbo\models\text_encoder\model_layernorm.onnx
[7/40] Downloading https://huggingface.co/microsoft/sd-turbo-webnn/resolve/main/unet/model_layernorm.onnx to C:\src\webnn-developer-preview-amd\demos\sd-turbo\models\unet\model_layernorm.onnx
PS C:\src\webnn-developer-preview-amd>
Author
There was a problem hiding this comment.
No, I did not see a similar issue with fetch_models.js. And based on the provided log, the amd models added have been downloaded without error. Could you check if amd's models exist in the corresponding folder? And could you also try to run the fetch again to see if the problem persist?
Contributor
There was a problem hiding this comment.
The new models exist, but I think the problem is that the rest of the models aren't getting downloaded for whatever reason. I'd expect to see ~40 models downloaded. So, I'm not sure if something regressed here, as I don't see this behavior on the main branch of the parent repo.
C:\src\webnn-developer-preview-amd>dir /s /b *.onnx
C:\src\webnn-developer-preview-amd\demos\image-classification\models\amd\MobileNetV2\onnx\model.onnx
C:\src\webnn-developer-preview-amd\demos\image-classification\models\amd\resnet50\onnx\model.onnx
C:\src\webnn-developer-preview-amd\demos\image-classification\models\webnn\efficientnet-lite4\onnx\model_fp16.onnx
C:\src\webnn-developer-preview-amd\demos\image-classification\models\webnn\mobilenet-v2\onnx\model_fp16.onnx
C:\src\webnn-developer-preview-amd\demos\image-classification\models\xenova\resnet-50\onnx\model_fp16.onnx
C:\src\webnn-developer-preview-amd\demos\sd-turbo\models\text_encoder\model_layernorm.onnx
C:\src\webnn-developer-preview-amd\demos\sd-turbo\models\unet\model_layernorm.onnx
Contributor
There was a problem hiding this comment.
Well, not sure what changed in the previous few hours (I restarted and changed networks in the interim, so maybe something there got things unstuck)- but at the moment, running the script seems to make it past the seventh model (it's downloading number 13 as I write this). We can consider this resolved.
Contributor
There was a problem hiding this comment.
I jinxed it. It bailed out after the 14th model finished downloading. Trying again. 😢
Contributor
There was a problem hiding this comment.
Hm, there's some degree of flakiness that I'm observing. At any rate, will treat this as a separate issue. Thanks for confirming behavior on your side.
[26/40] Downloading https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.encoder-fp16.onnx to C:\src\webnn-developer-preview-amd\demos\segment-anything\models\sam_vit_b_01ec64.encoder-fp16.onnx
Retrying download for https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.encoder-fp16.onnx (2 retries left)
Retrying download for https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.encoder-fp16.onnx (1 retries left)
Failed to download https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.encoder-fp16.onnx after multiple attempts Error: HTTP error! Status: 404 Not Found
at downloadFile (file:///C:/src/webnn-developer-preview-amd/fetch_models.js:206:19)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async file:///C:/src/webnn-developer-preview-amd/fetch_models.js:226:13
Failed to download https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.encoder-fp16.onnx Error: HTTP error! Status: 404 Not Found
at downloadFile (file:///C:/src/webnn-developer-preview-amd/fetch_models.js:206:19)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async file:///C:/src/webnn-developer-preview-amd/fetch_models.js:226:13
[27/40] Downloading https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.decoder-orig-img-size-dynamic-fp16.onnx to C:\src\webnn-developer-preview-amd\demos\segment-anything\models\sam_vit_b_01ec64.decoder-orig-img-size-dynamic-fp16.onnx
Retrying download for https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.decoder-orig-img-size-dynamic-fp16.onnx (2 retries left)
Retrying download for https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.decoder-orig-img-size-dynamic-fp16.onnx (1 retries left)
Failed to download https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.decoder-orig-img-size-dynamic-fp16.onnx after multiple attempts Error: HTTP error! Status: 404 Not Found
at downloadFile (file:///C:/src/webnn-developer-preview-amd/fetch_models.js:206:19)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async file:///C:/src/webnn-developer-preview-amd/fetch_models.js:226:13
Failed to download https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b_01ec64.decoder-orig-img-size-dynamic-fp16.onnx Error: HTTP error! Status: 404 Not Found
at downloadFile (file:///C:/src/webnn-developer-preview-amd/fetch_models.js:206:19)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async file:///C:/src/webnn-developer-preview-amd/fetch_models.js:226:13
[28/40] Downloading https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-encoder-int8.onnx to C:\src\webnn-developer-preview-amd\demos\segment-anything\models\sam_vit_b-encoder-int8.onnx
Retrying download for https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-encoder-int8.onnx (2 retries left)
Retrying download for https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-encoder-int8.onnx (1 retries left)
Failed to download https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-encoder-int8.onnx after multiple attempts Error: HTTP error! Status: 404 Not Found
at downloadFile (file:///C:/src/webnn-developer-preview-amd/fetch_models.js:206:19)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async file:///C:/src/webnn-developer-preview-amd/fetch_models.js:226:13
Failed to download https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-encoder-int8.onnx Error: HTTP error! Status: 404 Not Found
at downloadFile (file:///C:/src/webnn-developer-preview-amd/fetch_models.js:206:19)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async file:///C:/src/webnn-developer-preview-amd/fetch_models.js:226:13
[29/40] Downloading https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-decoder-orig-img-size-dynamic-int8.onnx to C:\src\webnn-developer-preview-amd\demos\segment-anything\models\sam_vit_b-decoder-orig-img-size-dynamic-int8.onnx
Retrying download for https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-decoder-orig-img-size-dynamic-int8.onnx (2 retries left)
Retrying download for https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-decoder-orig-img-size-dynamic-int8.onnx (1 retries left)
Failed to download https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-decoder-orig-img-size-dynamic-int8.onnx after multiple attempts Error: HTTP error! Status: 404 Not Found
at downloadFile (file:///C:/src/webnn-developer-preview-amd/fetch_models.js:206:19)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async file:///C:/src/webnn-developer-preview-amd/fetch_models.js:226:13
Failed to download https://huggingface.co/webnn/segment-anything-model-webnn/resolve/main/sam_vit_b-decoder-orig-img-size-dynamic-int8.onnx Error: HTTP error! Status: 404 Not Found
at downloadFile (file:///C:/src/webnn-developer-preview-amd/fetch_models.js:206:19)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async file:///C:/src/webnn-developer-preview-amd/fetch_models.js:226:13
[30/40] Downloading https://huggingface.co/webnn/whisper-base-webnn/resolve/main/whisper_base_decoder_static_kvcache_128_lm_fp16_layernorm_gelu_4dmask_iobinding.onnx to C:\src\webnn-developer-preview-amd\demos\whisper-base\models\whisper_base_decoder_static_kvcache_128_lm_fp16_layernorm_gelu_4dmask_iobinding.onnx
-> downloading [========================================] 100% 0.0s
[30/40] Downloaded https://huggingface.co/webnn/whisper-base-webnn/resolve/main/whisper_base_decoder_static_kvcache_128_lm_fp16_layernorm_gelu_4dmask_iobinding.onnx to C:\src\webnn-developer-preview-amd\demos\whisper-base\models\whisper_base_decoder_static_kvcache_128_lm_fp16_layernorm_gelu_4dmask_iobinding.onnx
[31/40] Downloading https://huggingface.co/webnn/whisper-base-webnn/resolve/main/whisper_base_decoder_static_non_kvcache_lm_fp16_layernorm_gelu_4dmask_iobinding.onnx to C:\src\webnn-developer-preview-amd\demos\whisper-base\models\whisper_base_decoder_static_non_kvcache_lm_fp16_layernorm_gelu_4dmask_iobinding.onnx
-> downloading [========================================] 100% 0.0s
[31/40] Downloaded https://huggingface.co/webnn/whisper-base-webnn/resolve/main/whisper_base_decoder_static_non_kvcache_lm_fp16_layernorm_gelu_4dmask_iobinding.onnx to C:\src\webnn-developer-preview-amd\demos\whisper-base\models\whisper_base_decoder_static_non_kvcache_lm_fp16_layernorm_gelu_4dmask_iobinding.onnx
[32/40] Downloading https://huggingface.co/webnn/whisper-base-webnn/resolve/main/whisper_base_encoder_lm_fp16_layernorm_gelu.onnx to C:\src\webnn-developer-preview-amd\demos\whisper-base\models\whisper_base_encoder_lm_fp16_layernorm_gelu.onnx
-> downloading [========================================] 100% 0.0s
[32/40] Downloaded https://huggingface.co/webnn/whisper-base-webnn/resolve/main/whisper_base_encoder_lm_fp16_layernorm_gelu.onnx to C:\src\webnn-developer-preview-amd\demos\whisper-base\models\whisper_base_encoder_lm_fp16_layernorm_gelu.onnx
adrastogi
reviewed
Apr 7, 2026
| @@ -0,0 +1,2026 @@ | |||
| { | |||
| "architectures": [ | |||
| "ResNetForImageClassification" | |||
Contributor
There was a problem hiding this comment.
I was trying the sample locally- I did get back an inference but did see some log output indicating attempts at local file system access... is this expected?
[60792:33108:0407/135409.409:ERROR:services\webnn\ort\environment.cc:89] : [ORT] [FATAL: onnxruntime, vaiml_mlopslib_partition_rule.cpp:1317 vaiml_mlopslib_partition_rule.cpp] Failed to open "C:\\temp\\adityar\\vaip\\.cache\\ef572deb32890331e9d986c1826163b6\\aie_unsupported_original_ops.json"
Contributor
There was a problem hiding this comment.
Including the logs from about://gpu here as well. It seems like that perhaps the computation in my local testing is happening on the CPU?
[53892:58412:0407/140321.915:ERROR:services\webnn\ort\environment.cc:89] : [ORT] [INFO: onnxruntime, stat.cpp:193 stat.cpp] [Vitis AI EP] No. of Operators :
[53892:58412:0407/140321.915:ERROR:services\webnn\ort\environment.cc:89] : [ORT] [INFO: onnxruntime, stat.cpp:204 stat.cpp] CPU 124
[53892:58412:0407/140321.915:ERROR:services\webnn\ort\environment.cc:89] : [ORT] [INFO: onnxruntime, stat.cpp:213 stat.cpp]
Author
There was a problem hiding this comment.
The failed file access caused the model to fall back to CPU. But that on-disk file access should not happen. Could you share the steps to reproduce this?
Contributor
There was a problem hiding this comment.
Sure- as prerequisites, you'll need to have the 1.8.59 AMD NPU EP installed, and then you'll need to have the 2.0 Preview2 Windows App SDK build installed (you can run the installer from here). I used Google Chrome Canary as my test browser.
I cloned your fork locally, pulled down the models locally, and ran the site via npm run dev.
Once I had the server launched, I launched Google Chrome Canary with the following incantation:
PS C:\Users\adityar\AppData\Local\Google\Chrome SxS\Application> .\chrome.exe --no-first-run --no-default-browser-check --enable-features="WebMachineLearningNeuralNetwork,WebNNOnnxRuntime" --webnn-ort-ignore-ep-blocklist --webnn-ort-logging-level=VERBOSE --webnn-ort-library-path-for-testing="C:\Program Files\WindowsApps\Microsoft.WindowsAppRuntime.2-preview2_2.0.0.2_x64__8wekyb3d8bbwe" --webnn-ort-ep-device="VitisAIExecutionProvider,0x1022,0x17F0" --allow-third-party-modules about:blank --webnn-ort-ep-library-path-for-testing=VitisAIExecutionProvider?"C:\Program Files\WindowsApps\MicrosoftCorporationII.WinML.AMD.NPU.EP.1.8_1.8.59.0_x64__8wekyb3d8bbwe\ExecutionProvider\onnxruntime_vitisai_ep.dll"
(More details on some of these arguments is available here and if you have any questions on any of them, let me know.)
I then navigated to the local web site (http://127.0.0.1:8080/), loaded the Image Classification sample, and tried running the FP32 model. I was inspecting the verbose ORT logs via the chrome://gpu page.
If I should be using a newer AMD NPU EP, please let me know (I know that there's been ongoing work to eliminate the file accesses during model compilation, maybe I am on a build that doesn't have the latest fixes).
|
@Json288 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why is this change being made?
To verify and demonstrate VitisAI EP 's support for WebNN with fp32 image_classification models: ResNet50 & MobileNet.
What changed?
Added
demos/image-classification/models/amd/resnet50/—config.json,preprocessor_config.json(local assets for AMD FP32 ResNet-50).demos/image-classification/models/amd/MobileNetV2/—config.json,preprocessor_config.json(local assets for AMD FP32 MobileNetV2).Modified
demos/image-classification/index.js— FP32 AMD paths (amd/resnet50,amd/MobileNetV2), WebNN Hub layout handling (webnn/JSON +webnn/onnxweights on remote vs flatonnx/locally),env.fetchrewrites for Hubconfig.json/preprocessor_config.json,options.subfolderfor remote/local, and AMDpixel_values→inputpatch for the classifier pipeline.demos/image-classification/index.html— UI wiring for FP32 / model selection as needed for the new paths.demos/image-classification/static/main.css— Styles for any new/updated controls.fetch_models.js— Extra download entries for AMD ResNet-50 / MobileNetV2 from the Hubwebnn/onnxpaths into the localonnx/mirror layout.How was the change tested?
./modelsflows; WebNN NPU + FP32 + MobileNet V2 / ResNet50.node fetch_modelsagainst Hub when validating AMDwebnn/asset URLs.