DFloat11 is the best-known academic
lossless bf16 weight format (entropy-coded exponent, raw sign + mantissa).
This page compares it against BigSmall under the same ideal-coder
accounting, on models measured on disk with
research/compare_dfloat11.py (rerun for v3.15, 2026-06-11; CPU-feasible
subset — the per-tensor entropy analysis is exact and dtype-grained).
BigSmall : H(sign, exp) + H(mantissa | exp) joint-entropy floor
DFloat11 : 1 + H(exp) + 7 sign and mantissa raw
Both assume an ideal arithmetic coder. BigSmall's measured implementation overhead over its bound is ~0.03 pp (Phi-3.5 measurement run) — effectively zero; DFloat11's huffman-coded exponent pays a small overhead over H(exp).
| model | tensors | raw | BigSmall LB | DFloat11 LB | margin |
|---|---|---|---|---|---|
| Qwen2.5-0.5B | 290 | 0.92 GiB | 65.79% | 66.35% | +0.55 pp |
| Llama-3.2-1B | 146 | 2.30 GiB | 65.80% | 66.25% | +0.46 pp |
| Phi-3.5-mini-instruct | 195 | 7.12 GiB | 65.63% | 66.11% | +0.48 pp |
| Qwen3-8B | 399 | 15.26 GiB | 66.04% | 66.49% | +0.45 pp |
Per-layer-type detail (v3.15 reruns): BigSmall wins every layer type of every model measured, with the margin concentrated where conditional mantissa coding pays —
| layer type | Qwen2.5-0.5B margin | Llama-3.2-1B margin |
|---|---|---|
| mlp | +0.55 pp | +0.46 pp |
| embedding | +0.56 pp | +0.46 pp |
| attention | +0.51 pp | +0.41 pp |
| norm scales | +12.5 pp | +18.1 pp |
The norm-scale gap is structural: DFloat11 spends 8 raw bits on sign + mantissa even where the value distribution makes them highly predictable; BigSmall's (mantissa | exponent) conditioning collapses them. Norm scales are a tiny byte fraction, so the model-level margin is ~0.5 pp — real, consistent, and measured at the entropy wall (see research.md: the floor guarantee).
On top of the format-level margin, the shipped BigSmall pipeline adds the column-major scan + 2D context + head-cluster coding (~66 → ~62% on typical checkpoints) and the per-tensor smallest-wins codec menu (A5 sparsity, FP2-residual, bf16-native-F32 splitting) — none of which the DFloat11 coding model can express.
The DFloat11 paper's Table 1 reports Qwen2.5-14B-Instruct at 63.7% of
raw. Our measured exponent entropy for the Qwen family (Qwen3-8B, 399
tensors) puts the DFloat11 coding model's own floor at 66.49% —
the published number is ~2.8 pp below the floor of their stated coding
model for this family. (Caveat: family proxy, not the same checkpoint —
but H(exp) is stable across the family; see research/dfloat11_verify.json.)
Either the accounting differs (e.g. excluding some tensors) or the
described coding model is not what produced the number. BigSmall's numbers
are reproducible from the shipped tooling: bigsmall compress + the
research scripts measure the same bytes they ship.
python research/compare_dfloat11.py compare <model_dir> -o out.json
python research/compare_dfloat11.py verifyRaw rerun outputs: research/dfloat11_qwen05b.json,
research/dfloat11_llama1b.json, research/dfloat11_verify.json (source
distribution).