Share one subcode inflater across the CD codecs - #189
Merged
Conversation
cdzl, cdlz and cdfl each carried their own zlib_codec_data for the subcode stream, and each of those allocates an 8376-byte tinfl_decompressor. A CD image compressed with chdman's default trio therefore held four inflaters - cdzl's sector-data one plus three identical subcode ones - where a hunk is decoded by exactly one CD codec and only one subcode inflater is ever in flight. The three now share a single instance owned by the chd_file. Nothing carries across, because zlib_codec_decompress() runs tinfl_init() on entry to every call. The borrowed flag keeps the ownership explicit so only the chd_file frees it. cdzs is untouched: its subcode goes through zstd. Seeding happens before each codec's own init, so nothing is allocated and then discarded, and it is insensitive to ordering - which matters because LOWRAM_TARGET readies codecs lazily, in whatever order the hunks ask for them. The system-zlib path is excluded: there zlib_codec_data holds a z_stream by value, with no pointer to share. Peak heap, massif, same binary either side: cd_default (cdlz+cdzl+cdfl) 183532 -> 166836 -16696, -9.1% every single-codec seed +56 The +56 is the shared zlib_codec_data now living in chd_file even when one CD codec would have sufficed. Note the RAM-budget firmware uses one codec per file, so it cannot see this saving by construction - only the +32 it costs there. Sharing base and subcode inside cdzl was tried before and reverted at 2.6% CPU (see codec_cdzs.h). That does not transfer: those two alternate within a hunk, these three never do. Cachegrind over a real disc, +1089 instructions out of 5.1 billion, and 7689 fewer D1 misses - the shared object has better locality than three. Decoded output is byte-identical across the seed corpus, the v3/v4 fixtures, real CHDv4 images, six real discs and an AVHuff image, in the default build, with CHDR_CD_SCRATCH_BUFFER=OFF, CHDR_LOWRAM_TARGET=ON and WITH_SYSTEM_ZLIB=ON - 144 comparisons, no difference, and no divergence in error codes. Under ASan/UBSan over 820 corrupt and fuzzed images the leak count is identical either side (57 of 220, all pre-existing in chd_open's failure path). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01846EhHuAFk5qvxwEA5Gq6y
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cdzl,cdlzandcdfleach carry their ownzlib_codec_datafor the subcode stream, and each of those allocates an 8376-bytetinfl_decompressor. A CD image compressed with chdman's default trio therefore holds four inflaters — cdzl's sector-data one plus three identical subcode ones — even though a hunk is decoded by exactly one CD codec and only one subcode inflater is ever in flight.The three now share a single instance owned by the
chd_file. Nothing carries across, becausezlib_codec_decompress()runstinfl_init()on entry to every call. Aborrowedflag keeps ownership explicit so only thechd_filefrees it.cdzsis untouched — its subcode goes through zstd.Seeding happens before each codec's own init, so nothing is allocated and then discarded, and it is insensitive to ordering — which matters because
LOWRAM_TARGETreadies codecs lazily, in whatever order the hunks ask for them. The system-zlib path is excluded: therezlib_codec_dataholds az_streamby value, with no pointer to share.Effect
Peak heap under massif, same binary either side:
cd_default(cdlz+cdzl+cdfl)The +56 is the shared
zlib_codec_datanow living inchd_fileeven when one CD codec would have sufficed. Worth noting that the RAM-budget firmware uses one codec per file, so it cannot see this saving by construction — only the +32 it costs there.On the CPU cost
Sharing base and subcode inside cdzl was tried before and reverted at 2.6% CPU — see the note in
codec_cdzs.h. That does not transfer: those two streams alternate within a hunk, these three never do. Measured with cachegrind over a real disc: +1089 instructions out of 5.1 billion, and 7689 fewer D1 misses — three codecs touching one 8KB object have better locality than three separate ones. Wall-clock is unchanged within the ±4% noise floor of this machine.Verification
Decoded output is byte-identical across the seed corpus, the CHDv3/v4 fixtures, real CHDv4 images, six real discs and an AVHuff image, in the default build, with
CHDR_CD_SCRATCH_BUFFER=OFF,CHDR_LOWRAM_TARGET=ONandWITH_SYSTEM_ZLIB=ON— 144 comparisons, no difference, and no divergence in error codes.Under ASan/UBSan over 820 corrupt and fuzzed images: no memory errors, and the leak count is identical either side (57 of 220, all pre-existing in
chd_open's failure path — worth its own issue, not this one).rv32imafc under qemu with
LOWRAM_TARGET=ON: all nine codecs decode, every threshold held.🤖 Generated with Claude Code