[XPU] GGUF Q6_K dequantization segfault on Intel GPU during model loading

[tinafengfun]

[XPU] GGUF Q6_K dequantization segfault on Intel GPU during model loading

Summary

When loading a GGUF Q6_K quantized model via ComfyUI-GGUF and moving it to Intel XPU, the process segfaults. The crash occurs during model_management.load_model_gpu() when the on-the-fly weight dequantization (forward_ggml_cast_weights → dequantize_blocks_Q6_K) runs on the XPU device. The Q6_K dequantization uses bit-shift operations on uint8 tensors that segfault in the Intel XPU SYCL backend.

Environment

• ComfyUI version: v0.22.0-124-gc87384f2 (commit c87384f2b8f9e9a7f717e1a269b860698b6634bf, 2026-06-02)
• ComfyUI-GGUF: commit 6ea2651e7df66d7585f6ffee804b20e92fb38b8a (2026-01-12)
• PyTorch: 2.11.0+xpu
• Python: 3.13.3
• GPU: Intel(R) Graphics [0xe211] (PCI 8086:E211), 160 CUs, 24.4 GB VRAM
• Driver: xe kernel driver, Linux 6.14.0-1008-intel
• Model: qwen-image-2512-Q6_K.gguf (16.8 GB, 1933 state dict keys)
• GGUF quantization types in model: {Q6_K (14), BF16 (30), F32 (0)}

Bug Details

Code path

model_management.load_model_gpu(model)
→ ModelPatcher loads GGUF model weights to XPU
→ GGMLLayer.forward_ggml_cast_weights() accesses GGMLTensor weight
→ dequantize_tensor() → dequantize_blocks_Q6_K()    # dequant.py:141
→ bit-shift operations on uint8 tensors on XPU:
    ql = ql.reshape(...) >> torch.tensor([0, 4], ...)   # dequant.py:150
    qh = qh.reshape(...) >> torch.tensor([0, 2, 4, 6], ...) # dequant.py:152
→ SEGFAULT on Intel XPU

Root cause

The dequantize_blocks_Q6_K function (dequant.py:141-157) performs bit-shift operations on uint8 tensors:

ql = ql.reshape((n_blocks, -1, 1, 64)) >> torch.tensor([0, 4], device=d.device, dtype=torch.uint8).reshape((1, 1, 2, 1))
qh = qh.reshape((n_blocks, -1, 1, 32)) >> torch.tensor([0, 2, 4, 6], device=d.device, dtype=torch.uint8).reshape((1, 1, 4, 1))

These uint8 right-shift operations on the XPU device trigger a segfault in the Intel SYCL backend. The >> operator on uint8 tensors may lack a proper kernel implementation on XPU.

Existing Intel fix

ComfyUI-GGUF ops.py:78 has an "Intel Arc fix, ref#50" in the GGMLTensor.new_empty() method:

def new_empty(self, size, *args, **kwargs):
    # Intel Arc fix, ref#50
    new_tensor = super().new_empty(size, *args, **kwargs)
    return GGMLTensor(new_tensor, tensor_type=..., tensor_shape=size, patches=...)

This fix ensures new_empty returns a GGMLTensor wrapper on Intel GPUs, but it doesn't address the dequantization bit-shift crash.

Reproducer

import comfy.sd, comfy.model_management
from ComfyUI_GGUF.loader import gguf_sd_loader
from ComfyUI_GGUF.ops import GGMLOps

sd, extra = gguf_sd_loader("qwen-image-2512-Q6_K.gguf")
model = comfy.sd.load_diffusion_model_state_dict(
    sd, model_options={"custom_operations": GGMLOps()}
)
# SEGFAULT here — dequantization during GPU load
comfy.model_management.load_model_gpu(model)

Full reproducer: repro3_gguf_q6k_diffusion.py

Results

3/3 deterministic segfaults (exit code 139):

Stack trace from faulthandler:

Fatal Python error: Segmentation fault

Current thread (most recent call first):
  File "repro3_gguf_q6k_diffusion.py", line 123 in main
    comfy.model_management.load_model_gpu(model)

The segfault occurs inside native C code during the dequantization bit-shift operations on XPU, so the Python traceback stops at the load_model_gpu() call.

Impact

• All GGUF Q6_K (and likely Q5_K, Q4_K — same bit-shift pattern) models cannot run on Intel XPU
• This affects popular quantized models like Qwen-Image-2512-Q6_K, various LLMs in GGUF format
• Workaround: Use FP8 safetensors instead of GGUF Q6_K (fp8 e4m3fn works on XPU)

Suggested fix

Option A: Dequantize on CPU for XPU (recommended)

In dequantize_blocks_Q6_K and other K-quant dequant functions, move the uint8 bit-shift operations to CPU, then move the result back to XPU:

def dequantize_blocks_Q6_K(blocks, block_size, type_size, dtype=None):
    target_device = blocks.device
    # Perform bit-shift operations on CPU for XPU compatibility
    if target_device.type == 'xpu':
        blocks = blocks.to('cpu')
    # ... existing dequantization code ...
    return result.to(target_device)

Option B: Cast uint8 to int32 before shifts

Replace uint8 bit-shifts with int32 operations, which have proper XPU kernel support:

ql = ql.to(torch.int32).reshape(...) >> torch.tensor([0, 4], ...).to(torch.int32)

Option C: PyTorch XPU fix

File a bug against the PyTorch XPU backend for uint8 right-shift kernel support. The >> operator on uint8 tensors should be supported on all devices.

Related

• ComfyUI-GGUF issue tracker: https://github.com/city96/ComfyUI-GGUF
• ops.py:78 has a pre-existing "Intel Arc fix, ref#50" comment suggesting Intel GPU compatibility has been a concern
• The dequantize_blocks_Q5_K and dequantize_blocks_Q4_K functions use the same bit-shift pattern and likely have the same issue

[coderabbitai]

🔗 Related PRs

#13699 - fix: Hunyuan3D 2.1 batch size crashes in attention and forward pass [merged]
#14052 - fixup threaded loader with worksplit multi-gpu [merged]
#14116 - Threaded Loader performance fixes / improvements (+ Aimdo 0.4.6) [merged]
#14191 - fix: enable XPU MultiGPU CFG Split on Windows (CFG > 1) [merged]
#14359 - Revert "Add SeedVR2 support (CORE-6)" [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[tinafengfun]

Update: Resolved with ComfyUI-GGUF-XPU (Intel ESIMD kernels)

The Q6_K segfault is resolved by using ComfyUI-GGUF-XPU, an Intel XPU-optimized fork that uses ESIMD native kernels for dequantization instead of PyTorch uint8 bit-shifts.

Solution

# Install ComfyUI-GGUF-XPU
cd ComfyUI/custom_nodes
git clone https://github.com/analytics-zoo/ComfyUI-GGUF-XPU.git

# Install omni_xpu_kernel (ESIMD kernels)
git clone https://github.com/intel/llm-scaler.git
pip install llm-scaler/omni/omni_xpu_kernel/ --no-build-isolation

Verification

Same test (load_model_gpu with qwen-image-2512-Q6_K.gguf on Intel XPU):

	Original ComfyUI-GGUF	ComfyUI-GGUF-XPU
Q6_K dequantization	❌ Segfault (exit 139)	✅ PASSED (exit 0)
Backend	PyTorch uint8 bit-shifts	ESIMD native kernel

The ESIMD kernel uses omni_xpu_kernel.gguf.dequantize_q6_k() — a native SYCL implementation that avoids the problematic PyTorch uint8 right-shift operations on XPU entirely.

Remaining issue

The CLIP FP8 quantized tensor crash (comfy_kitchen clone() on XPU) is not affected by this fix — it's a separate comfy_kitchen issue (#14514).