Skip to content

Share one subcode inflater across the CD codecs - #189

Merged
rtissera merged 1 commit into
masterfrom
share-subcode-inflater
Sep 10, 2026
Merged

Share one subcode inflater across the CD codecs#189
rtissera merged 1 commit into
masterfrom
share-subcode-inflater

Conversation

@rtissera

Copy link
Copy Markdown
Owner

cdzl, cdlz and cdfl each carry their own zlib_codec_data for the subcode stream, and each of those allocates an 8376-byte tinfl_decompressor. A CD image compressed with chdman's default trio therefore holds four inflaters — cdzl's sector-data one plus three identical subcode ones — even though a hunk is decoded by exactly one CD codec and only one subcode inflater is ever in flight.

The three now share a single instance owned by the chd_file. Nothing carries across, because zlib_codec_decompress() runs tinfl_init() on entry to every call. A borrowed flag keeps ownership explicit so only the chd_file frees it. cdzs is untouched — its subcode goes through zstd.

Seeding happens before each codec's own init, so nothing is allocated and then discarded, and it is insensitive to ordering — which matters because LOWRAM_TARGET readies codecs lazily, in whatever order the hunks ask for them. The system-zlib path is excluded: there zlib_codec_data holds a z_stream by value, with no pointer to share.

Effect

Peak heap under massif, same binary either side:

seed before after
cd_default (cdlz+cdzl+cdfl) 183532 166836 -16696, -9.1%
every single-codec seed +56

The +56 is the shared zlib_codec_data now living in chd_file even when one CD codec would have sufficed. Worth noting that the RAM-budget firmware uses one codec per file, so it cannot see this saving by construction — only the +32 it costs there.

On the CPU cost

Sharing base and subcode inside cdzl was tried before and reverted at 2.6% CPU — see the note in codec_cdzs.h. That does not transfer: those two streams alternate within a hunk, these three never do. Measured with cachegrind over a real disc: +1089 instructions out of 5.1 billion, and 7689 fewer D1 misses — three codecs touching one 8KB object have better locality than three separate ones. Wall-clock is unchanged within the ±4% noise floor of this machine.

Verification

Decoded output is byte-identical across the seed corpus, the CHDv3/v4 fixtures, real CHDv4 images, six real discs and an AVHuff image, in the default build, with CHDR_CD_SCRATCH_BUFFER=OFF, CHDR_LOWRAM_TARGET=ON and WITH_SYSTEM_ZLIB=ON — 144 comparisons, no difference, and no divergence in error codes.

Under ASan/UBSan over 820 corrupt and fuzzed images: no memory errors, and the leak count is identical either side (57 of 220, all pre-existing in chd_open's failure path — worth its own issue, not this one).

rv32imafc under qemu with LOWRAM_TARGET=ON: all nine codecs decode, every threshold held.

🤖 Generated with Claude Code

cdzl, cdlz and cdfl each carried their own zlib_codec_data for the subcode
stream, and each of those allocates an 8376-byte tinfl_decompressor. A CD image
compressed with chdman's default trio therefore held four inflaters - cdzl's
sector-data one plus three identical subcode ones - where a hunk is decoded by
exactly one CD codec and only one subcode inflater is ever in flight.

The three now share a single instance owned by the chd_file. Nothing carries
across, because zlib_codec_decompress() runs tinfl_init() on entry to every
call. The borrowed flag keeps the ownership explicit so only the chd_file frees
it. cdzs is untouched: its subcode goes through zstd.

Seeding happens before each codec's own init, so nothing is allocated and then
discarded, and it is insensitive to ordering - which matters because
LOWRAM_TARGET readies codecs lazily, in whatever order the hunks ask for them.
The system-zlib path is excluded: there zlib_codec_data holds a z_stream by
value, with no pointer to share.

Peak heap, massif, same binary either side:

  cd_default (cdlz+cdzl+cdfl)   183532 -> 166836   -16696, -9.1%
  every single-codec seed                          +56

The +56 is the shared zlib_codec_data now living in chd_file even when one CD
codec would have sufficed. Note the RAM-budget firmware uses one codec per
file, so it cannot see this saving by construction - only the +32 it costs
there.

Sharing base and subcode inside cdzl was tried before and reverted at 2.6% CPU
(see codec_cdzs.h). That does not transfer: those two alternate within a hunk,
these three never do. Cachegrind over a real disc, +1089 instructions out of
5.1 billion, and 7689 fewer D1 misses - the shared object has better locality
than three.

Decoded output is byte-identical across the seed corpus, the v3/v4 fixtures,
real CHDv4 images, six real discs and an AVHuff image, in the default build,
with CHDR_CD_SCRATCH_BUFFER=OFF, CHDR_LOWRAM_TARGET=ON and WITH_SYSTEM_ZLIB=ON
- 144 comparisons, no difference, and no divergence in error codes. Under
ASan/UBSan over 820 corrupt and fuzzed images the leak count is identical
either side (57 of 220, all pre-existing in chd_open's failure path).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01846EhHuAFk5qvxwEA5Gq6y
@rtissera
rtissera merged commit 2e5bf59 into master Sep 10, 2026
36 of 37 checks passed
@rtissera
rtissera deleted the share-subcode-inflater branch September 10, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant