From d402ade1b9e45b1f516dd32205fd735cb60812f8 Mon Sep 17 00:00:00 2001 From: Lukasz Stolarczuk Date: Fri, 23 Oct 2026 18:13:57 +0200 Subject: [PATCH 1/3] chore(deps): add missing info about XPU Graph backend Signed-off-by: Lukasz Stolarczuk --- README.md | 3 ++- docs/guides/advanced/tuning_workflow.md | 4 +++- docs/guides/backends/xpu_graph_backend.md | 2 +- docs/guides/tune_strategies/tune_strategies.md | 7 ++++++- docs/learn/quick_start.md | 2 +- 5 files changed, 13 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 2cb5e5b..a3b4aaf 100644 --- a/README.md +++ b/README.md @@ -18,10 +18,11 @@ NOTE: This file has been modified by Intel Corporation. It provides model tuning capabilities through compilation and conversion paths that can significantly improve inference speed and efficiency across various AI workloads including Computer Vision, Natural Language Processing, Speech Recognition, and Generative AI. -The toolkit tunes PyTorch models and pipelines through one Python API. Four backends ship with the package: +The toolkit tunes PyTorch models and pipelines through one Python API. Five backends ship with the package: * **TorchAO** quantizes weights through PyTorch AO * **Torch Inductor** compiles with `torch.compile` +* **XPU Graph** records the module once on an Intel GPU during tuning and replays it for inference; input tensor shapes must remain static * **OpenVINO** converts a module to an OpenVINO model and runs it on an Intel CPU or GPU * **Torch Eager** runs the module with no compilation, as the baseline other backends are compared against diff --git a/docs/guides/advanced/tuning_workflow.md b/docs/guides/advanced/tuning_workflow.md index 4865810..48c9f1d 100644 --- a/docs/guides/advanced/tuning_workflow.md +++ b/docs/guides/advanced/tuning_workflow.md @@ -159,10 +159,11 @@ The tuning strategy determines which backend(s) to try and how to select the bes #### FirstWinsStrategy -Tries backends in priority order and returns the first one that builds and validates successfully. Not every backend can handle every model (e.g., OpenVINO may fail during model conversion, Torch Inductor may hit graph breaks), so this strategy provides automatic fallback instead of aborting. +Tries backends in priority order and returns the first one that builds and validates successfully. Not every backend can handle every model (e.g., OpenVINO may fail during model conversion, Torch Inductor may hit graph breaks, or XPU Graph will reject inputs with dynamic tensor shapes), so this strategy provides automatic fallback instead of aborting. ```python strategy = tt.FirstWinsStrategy([ + tt.backend.XPUGraphBackend(), tt.backend.TorchInductorBackend(), tt.backend.OpenVINOBackend(), tt.backend.TorchEagerBackend(), @@ -203,6 +204,7 @@ Before actual tuning, this strategy tries to estimate `max_batch_size`. It does strategy = tt.HighestThroughputStrategy([ tt.backend.TorchAOBackend(), tt.backend.TorchInductorBackend(), + tt.backend.XPUGraphBackend(), tt.backend.OpenVINOBackend(), tt.backend.TorchEagerBackend(), ]) diff --git a/docs/guides/backends/xpu_graph_backend.md b/docs/guides/backends/xpu_graph_backend.md index d3b40d0..4195619 100644 --- a/docs/guides/backends/xpu_graph_backend.md +++ b/docs/guides/backends/xpu_graph_backend.md @@ -28,7 +28,7 @@ strategy = HighestThroughputStrategy(backends=[XPUGraphBackend()]) The backend is first in the default `FirstWinsStrategy` list, before Torch Inductor. It is also in the default `HighestThroughputStrategy` list. -PyTorch records a native SYCL graph when it was built with a SYCL compiler of 2026.1 or newer, except on PVC. +PyTorch records a native SYCL graph when it was built with a SYCL compiler of 2026.1 or newer, except on Intel Data Center GPU Max Series. An older build records the legacy SYCL command graph. This backend uses the recording that build already performs. diff --git a/docs/guides/tune_strategies/tune_strategies.md b/docs/guides/tune_strategies/tune_strategies.md index 9215c13..2fe3757 100644 --- a/docs/guides/tune_strategies/tune_strategies.md +++ b/docs/guides/tune_strategies/tune_strategies.md @@ -24,6 +24,7 @@ Not every backend can successfully tune every model. Each backend relies on a di - **OpenVINO** requires converting the model to an OpenVINO representation. Models with unsupported operators, complex dynamic control flow, or symbolic shape constraints may fail during conversion or model compilation. Memory constraints can also prevent the model from being compiled. - **Torch Inductor** uses `torch.compile`, which may encounter *graph breaks* on unsupported Python constructs or operations, causing partial or failed compilation. +- **XPU Graph** records the module for replay, so it requires static tensor shapes. A failed recording or a mismatch against eager output rejects the backend for that module and input. - **TorchAO** applies quantization transformations that may not support all layer types or model architectures. - **Torch Eager** runs the module unchanged, so it always succeeds, which makes it a useful last-resort fallback. @@ -87,6 +88,7 @@ Tries backends in priority order and returns the first one that successfully bui ```python from torch_tweak.torch.backend import ( + XPUGraphBackend, TorchAOBackend, TorchAOBackendConfig, TorchInductorBackend, @@ -95,6 +97,7 @@ import torch_tweak.torch as tt # List backends in priority order (fastest → most compatible) backends = [ + XPUGraphBackend(), TorchAOBackend(config=TorchAOBackendConfig(quantization="fp8wo")), # Quantization: fast inference TorchInductorBackend(), # Good performance, broader compatibility ] @@ -147,12 +150,14 @@ from torch_tweak.torch.backend import ( TorchInductorBackend, TorchAOBackend, TorchAOBackendConfig, + XPUGraphBackend, ) import torch_tweak.torch as tt # List all candidate backends backends = [ TorchInductorBackend(config=TorchInductorBackendConfig(mode="default")), + XPUGraphBackend(), TorchAOBackend(config=TorchAOBackendConfig(quantization="fp8wo")), ] @@ -201,6 +206,6 @@ tt.tune(model, input_data) ## Next Steps -- Learn about specific backends: [TorchAO](../backends/torchao_backend.md), [Inductor](../backends/torch_inductor_backend.md) +- Learn about specific backends: [TorchAO](../backends/torchao_backend.md), [Inductor](../backends/torch_inductor_backend.md), [XPU Graph](../backends/xpu_graph_backend.md) - Explore [Deployment Guide](../deployment/deployment.md) - Review the [Tuning Guide](../tuning.md) diff --git a/docs/learn/quick_start.md b/docs/learn/quick_start.md index b485993..4760e88 100644 --- a/docs/learn/quick_start.md +++ b/docs/learn/quick_start.md @@ -113,7 +113,7 @@ Tuning gives you control over the process: * it detects the batch axis and dynamic axes (axes that change shape independently of batch size, e.g., sequence length in LLMs) * allows picking modules to tune * you can pick a tuning strategy (e.g., best throughput) for the whole process or per-module -* you can pick tuning backends (for example, TorchAO, Torch Inductor, OpenVINO, and eager) which will be used by the strategy +* you can pick tuning backends (for example, XPU graph, TorchAO, Torch Inductor, OpenVINO, and eager) which will be used by the strategy * you can mix different backends in the same model/pipeline * you can manually verify the tuning process (note: Torch Tweak performs basic checks for NaNs and errors) * you can save the resulting artifact and later read it from disk From ba2d3fb2bcc1530e0a75318aaf484419eb6fea1e Mon Sep 17 00:00:00 2001 From: Lukasz Stolarczuk Date: Mon, 26 Oct 2026 09:44:13 +0100 Subject: [PATCH 2/3] chore(docs): fix links in examples' README files Signed-off-by: Lukasz Stolarczuk --- examples/E5Large/README.md | 3 +-- examples/ParakeetCTC/README.md | 5 ++--- examples/ParakeetRNNT/README.md | 5 ++--- examples/ResNet/README.md | 5 ++--- 4 files changed, 7 insertions(+), 11 deletions(-) diff --git a/examples/E5Large/README.md b/examples/E5Large/README.md index 32783e8..72d64c4 100644 --- a/examples/E5Large/README.md +++ b/examples/E5Large/README.md @@ -69,5 +69,4 @@ uv run inference --prompt "query: What is the capital city of France?" ## Model Details -Can be found in following pages: -* https://huggingface.co/intfloat/e5-large-v2 +* [Hugging Face Model](https://huggingface.co/intfloat/e5-large-v2) diff --git a/examples/ParakeetCTC/README.md b/examples/ParakeetCTC/README.md index ae07ac4..13d3f77 100644 --- a/examples/ParakeetCTC/README.md +++ b/examples/ParakeetCTC/README.md @@ -72,6 +72,5 @@ uv run inference ## Model Details -Can be found in following pages: -* https://huggingface.co/nvidia/parakeet-ctc-0.6b -* https://docs.nvidia.com/nemo-framework/user-guide/24.09/nemotoolkit/asr/models.html +* [Hugging Face Model](https://huggingface.co/nvidia/parakeet-ctc-0.6b) +* [Nemo Documentation](https://docs.nvidia.com/nemo-framework/user-guide/24.09/nemotoolkit/asr/models.html) diff --git a/examples/ParakeetRNNT/README.md b/examples/ParakeetRNNT/README.md index b4685a2..e804063 100644 --- a/examples/ParakeetRNNT/README.md +++ b/examples/ParakeetRNNT/README.md @@ -88,6 +88,5 @@ The service uses dynamic batching - requests are grouped and processed together ## Model Details -Can be found in following pages: -* https://huggingface.co/nvidia/parakeet-rnnt-1.1b -* https://docs.nvidia.com/nemo-framework/user-guide/24.09/nemotoolkit/asr/models.html +* [Hugging Face Model](https://huggingface.co/nvidia/parakeet-rnnt-1.1b) +* [Nemo Documentation](https://docs.nvidia.com/nemo-framework/user-guide/24.09/nemotoolkit/asr/models.html) diff --git a/examples/ResNet/README.md b/examples/ResNet/README.md index 2fc1a57..0a4e046 100644 --- a/examples/ResNet/README.md +++ b/examples/ResNet/README.md @@ -60,6 +60,5 @@ uv run inference --model-name resnet50 --image-path your_image ## Model Details -Can be found in following pages: -* https://pytorch.org/vision/stable/models.html#classification -* https://huggingface.co/timm +* [Hugging Face Model](https://huggingface.co/timm) +* [Pytorch Documentation](https://pytorch.org/vision/stable/models.html#classification) From 6f498b7d81fb1264668c69f392fd2c1c687eb87a Mon Sep 17 00:00:00 2001 From: Lukasz Stolarczuk Date: Fri, 23 Oct 2026 18:14:47 +0200 Subject: [PATCH 3/3] chore(docs): add OpenSSF Scorecard badge to the README Signed-off-by: Lukasz Stolarczuk --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index a3b4aaf..a5fd264 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ NOTE: This file has been modified by Intel Corporation. [![PyTorch](https://img.shields.io/badge/PyTorch-2.13+-blue.svg)](https://pytorch.org/) [![Tests](https://github.com/intel/torch-tweak/actions/workflows/tests.yml/badge.svg)](https://github.com/intel/torch-tweak/actions/workflows/tests.yml) [![Nightly Tests](https://github.com/intel/torch-tweak/actions/workflows/nightly.yml/badge.svg?event=schedule)](https://github.com/intel/torch-tweak/actions/workflows/nightly.yml) +[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/intel/torch-tweak/badge)](https://securityscorecards.dev/viewer/?uri=github.com/intel/torch-tweak) **Torch Tweak** is an inference toolkit designed for tuning and deploying Deep Learning models with a focus on Intel XPU (GPU) accelerators. It provides model tuning capabilities through compilation and conversion paths that can significantly improve inference speed and efficiency