Skip to content

[CPU][ARM] Int8 conv swish fq fusion - #34931

Merged
alvoron merged 32 commits into
openvinotoolkit:masterfrom
Passavee-Losripat:int8-conv-swish-fq-fusion
Aug 3, 2026
Merged

[CPU][ARM] Int8 conv swish fq fusion#34931
alvoron merged 32 commits into
openvinotoolkit:masterfrom
Passavee-Losripat:int8-conv-swish-fq-fusion

Conversation

@Passavee-Losripat

@Passavee-Losripat Passavee-Losripat commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Details:

  • Re-enable ConvolutionTransformation on ARM by removing CPU_DISABLE_PASS_ARM in transformation_pipeline.cpp
  • Extend ConvMulAddFQBlock pattern matcher to optionally match an activation node between Add and FakeQuantize, enabling recognition of Conv -> Mul -> Add -> Activation -> FQ in addition to the existing Conv -> Mul -> Add -> FQ. Currently supports Swish and Relu via wrap_type<Swish, Relu>
    • designed to handle for other activation in addition to Swish support in the future without structural changes
  • Update ConvertConvolutionBias and FallbackUnsupportedLPConvToFP16 to retrieve and handle the optional activation anchor without breaking existing patterns
  • Improve ACLConvolutionExecutor to accept Activation + FakeQuantize as simultaneous post-ops, replacing the hard single-post-op limit with an iteration loop
  • Extend canFuse() in conv.cpp to allow FakeQuantize fusion after a single Eltwise activation is already fused, enabling the full post-op chain to reach the ACL executor
  • Introduce isI8DstF32 / isU8DstF32 flags and use_direct_i8_s8_f32 / use_direct_u8_u8_f32 paths in ACLConvolutionExecutor so Conv can output F32 directly, letting the activation run in floating point before FQ requantizes
  • Add subgraph tests covering static/dynamic shapes, per-channel and per-tensor weight scales, i8 and u8 quantized paths, with and without Swish.

Known Issues

  • Per-tensor quantization (i8 and u8) fails numerically and it's under investigation. - Per-channel quantization passes.

Tickets:

  • This PR is related to GSoC2026 Project 5 Optimize Quantized Model Inference Performance on ARM Devices with OpenVINO

AI Assistance:

  • AI assistance used: yes
  • Claude was used for writing external analyzing script and explaining codebase. All code in this PR was manually written and validated through YOLO26 detection accuracy on Apple M4 Max (ARM64). Analysis artifacts are not included in this PR.

@Passavee-Losripat
Passavee-Losripat force-pushed the int8-conv-swish-fq-fusion branch from 25f294b to c3d0c62 Compare March 25, 2026 16:54
@github-actions github-actions Bot added the category: CPU OpenVINO CPU plugin label Mar 28, 2026
@sys-openvino-ci sys-openvino-ci added the ExternalPR External contributor label Mar 28, 2026
@v-Golubev v-Golubev self-assigned this May 26, 2026
@Passavee-Losripat
Passavee-Losripat force-pushed the int8-conv-swish-fq-fusion branch from 3ea426d to d8eb502 Compare May 28, 2026 15:00
@github-actions github-actions Bot added the category: build OpenVINO cmake script / infra label Jun 11, 2026
@Passavee-Losripat
Passavee-Losripat force-pushed the int8-conv-swish-fq-fusion branch from 653e0c4 to fa7be2b Compare June 29, 2026 17:41
@Passavee-Losripat
Passavee-Losripat force-pushed the int8-conv-swish-fq-fusion branch from fa7be2b to d2b04d6 Compare June 29, 2026 18:04
@Passavee-Losripat
Passavee-Losripat marked this pull request as ready for review June 29, 2026 18:05
@Passavee-Losripat
Passavee-Losripat requested review from a team as code owners June 29, 2026 18:05
@alvoron
alvoron requested a review from Copilot June 29, 2026 18:10
Comment thread src/plugins/intel_cpu/tests/functional/custom/subgraph_tests/src/arm/conv_fq.cpp Outdated
Comment thread src/plugins/intel_cpu/src/transformations/utils.hpp Outdated
Comment thread src/plugins/intel_cpu/src/nodes/executors/acl/acl_conv.cpp Outdated

@alvoron alvoron left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Please fix CI pipelines and rollback ComputeLibrary submodule changes
Do we need to merge the PR as soon as we migrate to ACL with ARM-software/ComputeLibrary#1302 ?

@Passavee-Losripat

Passavee-Losripat commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Do we need to merge the PR as soon as we migrate to ACL with ARM-software/ComputeLibrary#1302 ?

Thank you for you review! No, the new ACL behavior only rejects the unsupported combination inside the new i8/i8/f32 path so it won't affect the existing code. Our OpenVINO code would also use fallback path until we later updates to an ACL version containing ACL#1302 as well so each PR could be updated before or after each other.

@v-Golubev

Copy link
Copy Markdown
Contributor

build_jenkins

@v-Golubev
v-Golubev enabled auto-merge July 31, 2026 10:27
@alvoron alvoron added the platform: arm OpenVINO on ARM / ARM64 label Jul 31, 2026
auto-merge was automatically disabled August 3, 2026 14:00

Head branch was pushed to by a user without write access

@alvoron
alvoron enabled auto-merge August 3, 2026 14:21
@alvoron
alvoron added this pull request to the merge queue Aug 3, 2026
Merged via the queue into openvinotoolkit:master with commit f0d5af7 Aug 3, 2026
209 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: build OpenVINO cmake script / infra category: CPU OpenVINO CPU plugin ExternalPR External contributor platform: arm OpenVINO on ARM / ARM64

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants