Skip to content

Latest commit

 

History

History
76 lines (61 loc) · 3.35 KB

File metadata and controls

76 lines (61 loc) · 3.35 KB

BigSmall vs DFloat11 — the benchmark page

DFloat11 is the best-known academic lossless bf16 weight format (entropy-coded exponent, raw sign + mantissa). This page compares it against BigSmall under the same ideal-coder accounting, on models measured on disk with research/compare_dfloat11.py (rerun for v3.15, 2026-06-11; CPU-feasible subset — the per-tensor entropy analysis is exact and dtype-grained).

Coding models compared (bits per element, bf16)

BigSmall :  H(sign, exp)  +  H(mantissa | exp)     joint-entropy floor
DFloat11 :  1  +  H(exp)  +  7                     sign and mantissa raw

Both assume an ideal arithmetic coder. BigSmall's measured implementation overhead over its bound is ~0.03 pp (Phi-3.5 measurement run) — effectively zero; DFloat11's huffman-coded exponent pays a small overhead over H(exp).

Results (lower = better; % of raw bf16 bytes)

model tensors raw BigSmall LB DFloat11 LB margin
Qwen2.5-0.5B 290 0.92 GiB 65.79% 66.35% +0.55 pp
Llama-3.2-1B 146 2.30 GiB 65.80% 66.25% +0.46 pp
Phi-3.5-mini-instruct 195 7.12 GiB 65.63% 66.11% +0.48 pp
Qwen3-8B 399 15.26 GiB 66.04% 66.49% +0.45 pp

Per-layer-type detail (v3.15 reruns): BigSmall wins every layer type of every model measured, with the margin concentrated where conditional mantissa coding pays —

layer type Qwen2.5-0.5B margin Llama-3.2-1B margin
mlp +0.55 pp +0.46 pp
embedding +0.56 pp +0.46 pp
attention +0.51 pp +0.41 pp
norm scales +12.5 pp +18.1 pp

The norm-scale gap is structural: DFloat11 spends 8 raw bits on sign + mantissa even where the value distribution makes them highly predictable; BigSmall's (mantissa | exponent) conditioning collapses them. Norm scales are a tiny byte fraction, so the model-level margin is ~0.5 pp — real, consistent, and measured at the entropy wall (see research.md: the floor guarantee).

On top of the format-level margin, the shipped BigSmall pipeline adds the column-major scan + 2D context + head-cluster coding (~66 → ~62% on typical checkpoints) and the per-tensor smallest-wins codec menu (A5 sparsity, FP2-residual, bf16-native-F32 splitting) — none of which the DFloat11 coding model can express.

Paper-claim cross-check

The DFloat11 paper's Table 1 reports Qwen2.5-14B-Instruct at 63.7% of raw. Our measured exponent entropy for the Qwen family (Qwen3-8B, 399 tensors) puts the DFloat11 coding model's own floor at 66.49% — the published number is ~2.8 pp below the floor of their stated coding model for this family. (Caveat: family proxy, not the same checkpoint — but H(exp) is stable across the family; see research/dfloat11_verify.json.) Either the accounting differs (e.g. excluding some tensors) or the described coding model is not what produced the number. BigSmall's numbers are reproducible from the shipped tooling: bigsmall compress + the research scripts measure the same bytes they ship.

Reproduce

python research/compare_dfloat11.py compare <model_dir> -o out.json
python research/compare_dfloat11.py verify

Raw rerun outputs: research/dfloat11_qwen05b.json, research/dfloat11_llama1b.json, research/dfloat11_verify.json (source distribution).