MCU targets: read-ahead budget, micro-flac by default, two dead ends removed - #192
Merged
Conversation
…ead ends The read-ahead budget was measured at 1.11x on an ESP32-S3 and 1.05-1.12x on an RP2350, both over SPI, and both contrib benchmarks have been setting it for months - but neither of the storage bridges a real integration copies did. chd_fatfs_open() and chd_esp_vfs_open() now call chd_set_cache_budget() with 32KB, the knee measured on the RP2350, adjustable through CHD_FATFS_CACHE_BUDGET and CHD_ESP_VFS_CACHE_BUDGET and off at 0. It is a ceiling, not an allocation request: an image whose hunks exceed it leaves caching off rather than over-allocating. micro-flac on BL616 was documented as three changes a glob cannot express, which left the reader to write them. It is now a real patch that applies on top of the integration one, and CI builds the firmware twice so the C++ backend is known to compile and link against the vendor GCC 10.2 and its bare-metal libc - the only thing that can be checked there without hardware. Both READMEs quoted 1.46x and 1.41x for micro-flac. Those predate the STREAMINFO block-size fix, which took an oversized decoded-sample buffer out of dr_flac and most of micro-flac's lead with it: the same measurement now reads 1.198x on an ESP32-S3 with I/O excluded, and 1.072x across eleven real discs. Replaced, with what has and has not actually been measured on each part said plainly - nothing at all on BL616, and nothing on the P4 since the fix. Two options removed rather than left as traps, both recorded with their numbers in docs/perf-esp32p4-findings.md: -Os is 1.12x slower than -O2 on an ESP32-S3. The decoders are loop-heavy and lose more to reduced unrolling than the smaller text wins back. Z7_LZMA_PROB32 costs 15,980 bytes per LZMA instance for a speedup the LZMA SDK only claims for "some CPUs", and was never measured on any target. The RP2350 option that exposed it is gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01846EhHuAFk5qvxwEA5Gq6y
libchdr's own default stays dr_flac, because a desktop consumer vendoring src/ must not have to fetch anything and must not be handed a C++ dependency. An MCU integrator is already cloning an SDK, a toolchain and a filesystem library, so one more pinned checkout costs them nothing - and on every board this was tried on, micro-flac is faster, uses less peak heap on most images, and leaves a larger worst-case free block. Measured, all post-#188. ESP32-S3 with I/O excluded: 1.198x on a CD-FLAC hunk, 1.233x on raw FLAC. Eleven real discs on the same board: 1.072x overall, 1.21x where the image is FLAC-heavy, 0.988x on one profile where FLAC barely appears. RP2350 Cortex-M33, seven real discs, two runs each within 0.17%: 1.032x overall, 1.12-1.14x on two of them, worst-case largest-free-block 51 KB against dr_flac's 47 KB. BL616 and ESP32-P4 have not been measured - said so in both READMEs. The three targets now fetch micro-flac at a pinned commit when no local checkout is given, the same shape libchdr's root CMakeLists already uses. CHDR_FLAC_BACKEND=drflac goes back, and CI builds both ways on all three so neither path rots. Two things this ran into: FetchContent's SOURCE_SUBDIR names a directory that does not exist on purpose. Without it MakeAvailable runs micro-flac's own CMakeLists, which configures its targets for the host and joins them to a cross build - visible as "microFLAC: Building for host platform" in the RP2350 configure. ESP-IDF runs every component CMakeLists in script mode first to collect REQUIRES, and FetchContent is not scriptable there: define_property fails. The backend does not affect requirements, so that pass takes the in-tree source and the fetch happens once, in the real configure. On BL616 the integration patch now carries micro-flac, and firmware-bl616-libchdr-drflac.patch reverts it for anyone Apache-2.0 does not suit. Both were generated against the pinned firmware-bl616 commit and checked to apply, rather than written by hand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01846EhHuAFk5qvxwEA5Gq6y
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three things, all on the MCU targets. Nothing in
src/changes.The read-ahead budget was measured and then not used
chd_set_cache_budget()is worth 1.11x on an ESP32-S3 and 1.05-1.12x on an RP2350, both reading over SPI, and both contrib benchmarks have been setting it for months. Neither of the storage bridges a real integration actually copies did.chd_fatfs_open()(BL616) andchd_esp_vfs_open()(ESP32) now set it to 32KB — the knee measured on the RP2350, where 64KB doubled the cost for under 0.7% more. Adjustable throughCHD_FATFS_CACHE_BUDGET/CHD_ESP_VFS_CACHE_BUDGET, off at 0. It is a ceiling rather than an allocation request, so an image whose hunks exceed it leaves caching off instead of over-allocating, and a failure to set it is not fatal to the open.This is the lever that attacks storage, which is where 49-95% of wall time goes on these parts.
micro-flac becomes the default on all four MCU targets
libchdr's own default stays dr_flac: a desktop consumer vendoring
src/must not have to fetch anything, and must not be handed a C++ dependency. An MCU integrator is already cloning an SDK, a toolchain and a filesystem library, so one more pinned checkout costs nothing — and on every board this was tried on, micro-flac is faster, uses less peak heap on most images, and leaves a larger worst-case free block.BL616, ESP32-P4/S3 and RP2350 now fetch micro-flac at a pinned commit when no local checkout is given, the same shape the root
CMakeLists.txtalready uses.CHDR_FLAC_BACKEND=drflacgoes back, and CI builds both ways on all three, so neither path rots. On BL616 that means the integration patch carries micro-flac andfirmware-bl616-libchdr-drflac.patchreverts it; both were generated against the pinned firmware-bl616 commit and checked to apply, not written by hand.Two things this ran into, both worth knowing:
SOURCE_SUBDIRnames a directory that does not exist on purpose. Without it,MakeAvailableruns micro-flac's own CMakeLists, which configures its targets for the host and joins them to a cross build — visible asmicroFLAC: Building for host platformin the RP2350 configure.REQUIRES, and FetchContent is not scriptable there (define_property command is not scriptable). The backend does not affect requirements, so that pass takes the in-tree source and the fetch happens once, in the real configure.Stale numbers replaced
Both READMEs quoted 1.46x and 1.41x. Those predate #188, which took an oversized decoded-sample buffer out of dr_flac and most of micro-flac's lead with it. The same measurement now reads:
Both files now say plainly what has and has not been measured on each part: nothing at all on BL616, and nothing on the ESP32-P4 since #188.
Two options removed
Kept in
docs/perf-esp32p4-findings.mdwith their numbers, under the existing "measured, do not re-try" section, so nobody repeats them:-Osis 1.12x slower than -O2 on an ESP32-S3. The decoders are loop-heavy and lose more to reduced unrolling than the smaller text wins back.Z7_LZMA_PROB32costs 15,980 bytes per LZMA instance for a speedup the LZMA SDK only claims for "some CPUs", and was never measured on any target. The RP2350 option exposing it is gone.Still to measure
🤖 Generated with Claude Code