FP8 weight surpport for v79+ and dit optimizations

[happyyzy]

Overview

New operators

This change separates backend-neutral diffusion graph semantics from Hexagon-specific execution. New operations are defined in ggml/include/ggml.h and constructed in ggml/src/ggml.c;

• GGML_OP_MUL_MAT_SEGMENTED: Represents W * concat(x0, x1) without materializing the concatenated tensor; used by FLUX.2/Klein single-stream linear2 after attention and SwiGLU; Hexagon implementation: ggml/src/ggml-hexagon/htp/matmul-ops.c.

• GGML_OP_QKNORM_ROPE: Combines learned Q/K RMS normalization and RoPE before attention; Hexagon implementation: ggml/src/ggml-hexagon/htp/qknorm-rope-ops.c.

• GGML_OP_GROUP_NORM_AFFINE_SILU: Combines GroupNorm statistics, per-channel affine transformation, and SiLU, with optional in-place output; used by VAE residual blocks and the final decoder normalization; Hexagon implementation: ggml/src/ggml-hexagon/htp/groupnorm-ops.c.

• GGML_OP_CONV_2D_BIAS: Represents direct convolution with channel bias without a separate ADD tensor; Hexagon implementation: ggml/src/ggml-hexagon/htp/conv2d-ops.c.

• GGML_OP_CONV_2D_UPSCALE: Combines nearest-neighbor upscaling, direct convolution, and optional bias without materializing the enlarged activation; used by VAE upsampling blocks; Hexagon implementation: ggml/src/ggml-hexagon/htp/conv2d-ops.c.

The first two operations cover DiT attention and projection hotpaths. The remaining three cover VAE decoder hotpaths. Together, they represent common diffusion inference patterns with substantial memory io costs. Defining them as global GGML operations allows other backends(for example, fused QKNorm-RoPE on Adreno OpenCL) to implement the same optimized contracts, while unsupported backends retain the original graph path. This part are not tied to v79.

FP8 Data Path

The backend-neutral FP8 changes are intentionally minimal: GGML_TYPE_F8_E4M3, its type metadata, and a host conversion routine required for loading and fallback conversion.
The stable-diffusion.cpp safetensors loader maps standard F8_E4M3 tensors to GGML_TYPE_F8_E4M3. During Hexagon set_tensor, row-major FP8 data is repacked once into the HMX 32x32 tiled weight layout by repack_f8_e4m3_tiled() in ggml/src/ggml-hexagon/ggml-hexagon.cpp. Execution then routes directly to the v79+ W8A16 HMX GEMM in ggml/src/ggml-hexagon/htp/matmul-ops.c.

The kernel uses HMX activation.hf and weight.f8 instructions with FP32 accumulation. Model weight scales and graph scalar scales are folded into the HVX store epilogue. Because weights are already in the required HMX layout, inference does not pay runtime weight dequantization or HVX weight-shuffle costs. The e2e speed of this path is approximately one order of magnitude faster than the current standard Q4_0/Q8_0 Hexagon path, while the FP8 comparison images retain substantially better visual quality.

VAE Weight Path

Standard VAE checkpoints do not require a custom on-disk weight layout. stable-diffusion.cpp loads F32 or BF16 checkpoint values into F16 convolution tensors. For eligible 1x1 and 3x3 convolution weights, Hexagon set_tensor calls repack_conv2d_f16_hmx() in ggml/src/ggml-hexagon/ggml-hexagon.cpp, producing an HMX 32x32 tiled FP16 layout. The implicit-GEMM convolution in ggml/src/ggml-hexagon/htp/conv2d-ops.c consumes that layout without runtime weight rearrangement.

Related design discussion: #28904

Performance measurements and image-quality comparisons: leejet/stable-diffusion.cpp#1970

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes：codex helps with the optimization of servral kernels like implicit gemm/fused groupnorm-mul-add-silu.