Add an optional micro-flac backend for the FLAC decoder - #185
Merged
Conversation
CHDR_FLAC_BACKEND=microflac swaps dr_flac for micro-flac behind the same flac_decoder interface, so exactly one backend is compiled and nothing else in the tree changes. dr_flac stays the default. micro-flac is Apache-2.0 and C++, so it is fetched at build time from a pinned commit rather than vendored: this tree stays BSD-3 for its other consumers, and a default build neither touches the network nor needs a C++ compiler - enable_language(CXX) is inside the backend's own branch, and the new workflow checks that by configuring a default build with a compiler path that does not exist. Point CHDR_MICROFLAC_SOURCE_DIR at a checkout to build offline. It fits one thing libchdr needs better than dr_flac does. dr_flac exposes no way to ask how many input bytes it consumed, so the dr_flac backend reconstructs that by reaching into the bit-reader's private cache state, and cdfl uses the answer to find the subcode that follows the audio in the hunk. micro-flac's streaming decode() returns bytes_consumed directly. It decodes into the caller's buffer, with no intermediate. micro-flac re-checks the output buffer against one whole block on every call, sized from the STREAMINFO block-size field - and the header libchdr synthesises carries block_size * num_channels where the format specifies inter-channel samples. Writing the specified value halves what micro-flac demands, which is what makes the caller's hunk large enough for every block of it. A buffer smaller than one block, or a final block that would overrun it, still falls back to a retained scratch; for CD geometry it is never allocated. That field cuts both ways and the comment says so: micro-flac also *rejects* any frame whose block size exceeds it, so the smaller value narrows what this backend accepts. It is safe against what chdman writes, because libchdr derives the field from the same geometry chdman encoded to, but it is not a margin. Measured on hardware, at the same point of the same image: 198 KB of heap in use against dr_flac's 210, and a largest free block of 52 KB against 39 - which is the figure that decides whether a later allocation of that size succeeds. On the desktop, peak heap on a cdfl image is 99488 bytes against dr_flac's 121640, excluding 73728 bytes of libstdc++ static initialisation the dynamic loader performs before main(); that is a hosted-link artifact and does not exist in a bare-metal link with the operator-new shim documented in contrib/tangcore-bl616/README.md. One incidental correctness gain. dr_flac writes int16_t straight into the caller's buffer, which is undefined on an odd pointer and is why chd_read documents a 2-byte alignment requirement. Under UBSan with a deliberately misaligned buffer, dr_flac aborts on all ten odd-aligned cases and micro-flac completes every one of them, with no sanitizer diagnostic at all. Three things this also fixes or closes: flac.h leaves the installed header set. It is included only from src/, and the backend makes struct _flac_decoder depend on a macro no consumer defines - so an installed copy would describe a different layout than the library was built with. flac_decoder_free() no longer destroys an object that was never constructed. chd_close() frees every codec slot, including ones whose init() never ran. generate.sh gains hd_flac.chd. The raw flac codec was reached only because it happens to be one of chdman's default hard-disk codecs, and relying on a default to cover a codec is how a codec stops being covered. Verified byte-identical against the dr_flac backend throughout: 18 seeds in three read orders, both settings of CHDR_CD_SCRATCH_BUFFER, and 4253 real images sampled 400 hunks each. 255 ASan/UBSan runs across seed x access pattern x alignment. AVHuff's audio streams also use this decoder and are not covered - that corpus is fetched rather than generated, so wiring it in is its own change.
rtissera
force-pushed
the
microflac-opt-in
branch
2 times, most recently
from
September 6, 2026 13:21
8616893 to
3079c6c
Compare
The operator-new shim is a GCC 10.2 artifact - -nostdlib++ replaces it in one flag and has existed since GCC 11 - and reads as chip-specific when it is not. Someone will copy it into a project on a modern compiler. And the reason the vendor toolchain cannot simply be swapped for upstream is not the T-Head extensions, which GCC has carried since 13. It is -mtune=e907, the unratified p extension, and zpsfoperand/xtheade, none of which upstream spells - still true in the GCC 15.2 and 16.1 manuals. Written down so the next person does not repeat the search.
rtissera
force-pushed
the
microflac-opt-in
branch
from
September 6, 2026 13:40
3079c6c to
5baeaa7
Compare
CMake's regex engine has no {n} repetition, so "^[0-9a-fA-F]{40}$" matches
nothing at all and the guard fired on the correct SHA the workflow passes.
Checked both ways this time: the pinned SHA configures, and "main" is still
refused. The first test only covered the branch name, which failed for the
wrong reason and looked like success.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CHDR_FLAC_BACKEND=microflacdecodes FLAC through micro-flac instead of dr_flac, behind the sameflac_decoderinterface. Exactly one backend is compiled. dr_flac stays the default and nothing changes for anyone who does not ask for this.micro-flac is Apache-2.0 and C++, so it is fetched at build time from a pinned commit rather than vendored: this tree stays BSD-3 for its other consumers, and a default build neither touches the network nor needs a C++ compiler.
CHDR_MICROFLAC_SOURCE_DIRpoints at a local checkout to build offline.Measured on an ESP32-S3, every hunk of five images from the card
15-18% where FLAC carries the image, neutral where it does not, and one file 1.2% slower. That one is not I/O — the card costs 0.875 ms/hunk on both sides, identically — it is 0.060 ms/hunk of decode CPU, on an image where cdfl is a minority codec. It is not explained, and it is stated rather than averaged away.
It uses less memory, which was not true of the first attempt
At the same point of the same image: 198 KB of heap in use against dr_flac's 210, and a largest free block of 52 KB against 39. Over the whole run the smallest largest-free-block is 36 KB against dr_flac's 31 — that figure, not the peak, is what decides whether a later allocation of that size succeeds.
That came from one field. micro-flac checks the caller's output buffer against a whole block on every call, sized from the STREAMINFO block-size field — and the header libchdr synthesises carries
block_size * num_channelswhere the format specifies inter-channel samples. It was therefore demanding 18816 bytes at a point where only 10176 remained, which forced every hunk through an intermediate buffer. Writing the specified value halves the demand to 9408, and the caller's hunk is then large enough for every block of it.Neither value changes the decoded bytes — each frame carries its own block size, and that is what both decoders read. A buffer smaller than one block, or a final block that would overrun it, still falls back to a retained scratch; for CD geometry it is never allocated.
Before this, the backend cost 218 KB and left 33 KB — worse than dr_flac. The three configurations are worth comparing directly, because they separate two effects that look like one: removing the copy is what fixed the memory and the small regression on non-FLAC images, while the 15-18% on cdfl images comes from the decoder itself and is unchanged by it.
One incidental correctness gain
dr_flac writes
int16_tstraight into the caller's buffer, which is undefined on an odd pointer and is whychd_readdocuments a 2-byte alignment requirement. Under UBSan with a deliberately misaligned buffer, dr_flac aborts on all ten odd-aligned cases and micro-flac completes every one, with no sanitizer diagnostic at all.Verification
Byte-identical to the dr_flac backend throughout:
CHDR_CD_SCRATCH_BUFFER— the branch this replaces predated the in-place CD spread and had never been run against itThe new workflow builds both backends in both scratch settings and diffs the seeds on every change — an opt-in C++ path fetched at a pinned SHA that nothing compiles would rot otherwise. It also configures a default build with a compiler path that does not exist, so
enable_language(CXX)cannot silently escape the backend's own branch.Not in this PR
The same header field costs dr_flac 18816 bytes of peak heap for the same reason, and correcting it there is byte-identical too. That is a separate change with its own blast radius — it affects every build, not an opt-in one — so it gets its own PR.
Re-measured after #188
The numbers above predate #188, which fixed the synthesised STREAMINFO block size and took 18816 bytes off dr_flac. That changed the comparison, so everything here was re-measured on the merged tree.
ESP32-S3, 11 real discs off a card, five codec profiles,
BENCH_HUNK_CAP=1500:cdlz cdzl cdflcdlz cdzl cdflcdlz cdzl cdflcdzs flaccdzs cdflcdlz cdzl cdfllzma zlib huff flaczstd flacThree independent GD-ROM images land within 0.4% of each other, so the CD gain is reproducible rather than one lucky file. The two
zstd flacimages are the only regression, and both spend 91-95% of their wall time in storage.On the flash-embedded corpus, where I/O is zero and the measurement is pure CPU:
hd_flac1.233,cd_cdfl1.198,cd_default1.179 — andcd_cdlz0.994,cd_cdzl0.991, so no regression on the codecs micro-flac does not touch. The slowdown on cdlz-heavy content reported earlier does not survive #188.RP2350 Cortex-M33, 7 real discs, two runs each with a spread under 0.17%: 1.032x overall, 1.12-1.14x on two images.
Memory. micro-flac's peak is 2-3 KB lower than dr_flac's on 8 of 11 discs, 1-2 KB higher on the rest. Worst-case smallest-largest-free-block - the metric that decided the S3's historical OOMs - is 47 KB for dr_flac against 51 KB for micro-flac.
What it costs
include/libchdr/flac.hleaves the installed public headers. The struct differs between backends, so installing it would be actively harmful, but it is a packaging change. No installed header includes it.src/libchdr_flac.candsrc/libchdr_flac_microflac.cpp. They already disagreed once - the.cppwas right before Declare the real FLAC block size in the synthesised STREAMINFO #188 fixed the.c.CHDR_MICROFLAC_SOURCE_DIRlimit the exposure, but it is a real dependency.Rebased on master (#186, #187, #188, #189); the conflict was the
hd_flaccorpus seed, which #188 added independently.