Repository navigation
Read every image once for all channels in step 3 - #178
Merged
Merged
Conversation
Step 3 analysed the channels of an image one after another, and each channel opened and decoded the image again, so an RGB analysis decoded every RAW file three times. Nothing the decoding yields depends on the channel: the black level and the read noise are taken per Bayer color, and only the channel mask is built per channel. ImageChannels reads the file once and builds the array, mask and read noise of one channel at a time from it, so that besides the decoded image only the channel in work is held in memory. process_img_file reads the image before the channel loop and passes it to generate_analysis_data, which still reads the image itself when it gets none. read_channel_data_mask_and_noise_from_img uses ImageChannels as well, so the reads for one and for several channels cannot drift apart. An image that cannot be read is now skipped for all channels at once, before any result file is written. On a 24 MP CR2 of demo_2 reading the three channels takes 0.42 s instead of 1.22 s, with bit-identical arrays, masks and noise. An analysis of a single channel is unchanged. Unit tests pass, 378; the acceptance tests pass, 32. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Step 3 analysed the channels of an image one after another, and each channel opened and decoded the image again. An RGB analysis (
--rgb) therefore decoded every RAW file three times. Nothing the decoding yields depends on the channel: the black level and the read noise are taken per Bayer color, and only the channel mask is built per channel.ledsa/core/image_reading.py: the newImageChannelsreads the file once (decoding, black level and scaling, read noise, orientation).channel_data_mask_and_noise(channel)builds the array, mask and noise of one channel at a time from it, so that besides the decoded image only the channel in work is held in memory.read_channel_data_mask_and_noise_from_imguses it as well, so the reads for one and for several channels cannot drift apart.ledsa/data_extraction/DataExtractor.py:process_img_filereads the image before the channel loop and passes it to every channel.ledsa/data_extraction/step_3_functions.py:generate_analysis_datatakes the image as the optionalimg_channels; without it, it reads the image itself as before.ledsa/tests/UnitTests/test_image_channels.py: 10 tests. For RAW (upright and sideways) and PNG, the single read gives exactly what a read per channel gives. Step 3 opens the file once for three channels. A grayscale or missing image is still reported as a corrupted image. The step 3 test fails against the oldDataExtractor.Why
On a 24 MP CR2 of
demo_2, reading the three channels takes 0.42 s instead of 1.22 s per image, with bit-identical arrays, masks and noise for R, G and B. The share of this in the whole of step 3 was not measured: with-s3the LED fitting probably dominates, with-s3_fastthe read is a larger part.For the reviewer
--restartonly checks channel 0.postprocessing/simulation.pyare not touched.Tests
🤖 Generated with Claude Code