Repository navigation

Make folding exact for GPTQ'd GGML weights, so the vLLM fake-quant export evaluates them - #2710

Merged

cjluo-nv merged 6 commits into

mainfrom

chenjiel/ggml-upcast-bf16

Oct 12, 2026

Merged

Make folding exact for GPTQ'd GGML weights, so the vLLM fake-quant export evaluates them#2710
cjluo-nv merged 6 commits into
mainfrom
chenjiel/ggml-upcast-bf16

Commits on Oct 12, 2026
- Add upcast_ggml="bf16" export for evaluating GGML checkpoints in vLLM
  
  cjluo-nv
  and
  claude
  committed
  97a2003
- Validate GGML layers before the BF16 upcast and reject ignored routes
  
  cjluo-nv
  and
  claude
  committed
  1a3840f
- Refuse grouped GGML layouts in the upcast and test it under FSDP2
  
  cjluo-nv
  and
  claude
  committed
  4827612
- Replace upcast_ggml with exact folding of GPTQ'd GGML weights
  
  cjluo-nv
  and
  claude
  committed
  61e93eb
- Bound the pinned-payload lookup's memory and test every GGML format
  
  cjluo-nv
  and
  claude
  committed
  7331bdf
- Hash pinned-payload rows a column chunk at a time
  
  cjluo-nv
  and
  claude
  committed
  72303f4