Skip to content

encode: ordered dither in the NV12 conversion - #99

Closed
SirSicard wants to merge 1 commit into
shinyobjectz:mainfrom
SirSicard:nv12-dither
Closed

SirSicard wants to merge 1 commit into
shinyobjectz:mainfrom
SirSicard:nv12-dither

Conversation

@SirSicard

Copy link
Copy Markdown
Contributor

Requested by Mattias

Fixes #87.

nv12.k and nv12buf.k rounded RGB to BT.709 limited range with no dither, so slow dark gradients banded in the default 8-bit H.264. This adds a 4x4 ordered (Bayer) dither in place of the fixed +128 / +512 rounding. It's on by default, with no flag.

What changes

  • nv12.k, nv12buf.k and the CPU fallback nv12_cpu (core/k/src/native.rs) apply the same dither and stay bit-equal.
  • A test runs both kernels through the interpreter against nv12_cpu, and another checks that a slow dark ramp keeps its mean and loses its steps.

One correction to the issue's wording: the pattern is fixed per pixel position (x, y mod 4) and not seeded by frame. Play and render stay identical, and there's no temporal shimmer.

Measured on main 5b2c773

Dark page: the issue's 1920x1080, 60 fps, 4 s radial gradient.

main this branch
file size 641,121 bytes 4,013,807 bytes
raea check blotching error, 2 of 2 sampled frames, worst 3% error, 1 of 2 sampled frames, worst 1%

60 s 3D film (fog, bloom, dark background):

main this branch
file size 293,887,589 bytes 296,909,992 bytes
raea check blotching warn, 2 of 30 sampled frames, worst 4% warn, 3 of 30 sampled frames, worst 2%

The dark page grows because the encoder now keeps the gradient detail it was throwing away. The film already sits near the bitrate ceiling. It still gets a warning, and 10-bit HEVC remains the clean master. Constant quality is yours, as you said.

cargo test -p raea-core-k passes.

Also merged onto main 0ea4eab together with the other open PRs from this review: their crates' tests pass (118, 0 failed).


🤖 Generated with Claude Code

nv12.k and nv12buf.k rounded every sample with a fixed +128 (+512 for the
2x2 chroma sums), so slow dark gradients quantised into steps before the
encoder saw them. A 4x4 Bayer offset (mean unchanged, fixed per pixel so a
render stays a function of its frame) replaces it in both kernels and in
nv12_cpu, kept bit-equal: a new test runs both kernels through the
interpreter against nv12_cpu, and one checks a dark ramp keeps its mean.

Measured on a 1080p60 dark radial gradient (2D): raea check's blotching
goes from 3% of dark blocks on 2/2 sampled frames to 1% on 1/2 (still P1),
and the H.264 grows from 0.64 to 4.0 MB for 4 s, the encoder now keeping the
gradient instead of flattening it. 10-bit HEVC remains the full fix.
Fixes shinyobjectz#87

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shinyobjectz

Copy link
Copy Markdown
Owner

Thanks for this. We landed the same fix in 6177e27 while working through #87: both NV12 kernels and the CPU twin round through a 4x4 Bayer threshold, bit-equal, with black and white held at 16 and 235. Your report shaped it, and the commit says so. Closing this one as covered.

shinyobjectz added a commit that referenced this pull request Nov 18, 2026
…#103), after main's f37e5e3 (#99)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hshalab pushed a commit to hshalab/raea that referenced this pull request Nov 18, 2026
…jectz#171, shinyobjectz#172) on main for 0.1.8

raea export layers and layers.json, CapCut drafts written from layers.json with a captions track, raea export xml
(FCP7 XML for Premiere Pro and DaVinci Resolve), footage that moves, is cut or fades keyed in every export, and the
delivery and words guides teaching them (raea-skills shinyobjectz#99, merged onto raea-skills main).

Conflict: engine/cli/src/cli.rs's help, where main added raea wear and raea setup motion beside Zach's raea export
layers and raea export xml; both kept. The skills pin is raea-skills main with shinyobjectz#99 merged (f37e5e3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dark gradients band in 8-bit H.264: the RGB to NV12 step rounds without dither

2 participants