Skip to content

Read every image once for all channels in step 3 - #178

Merged
kriboe90 merged 1 commit into
masterfrom
claude/raw-images-single-opening-fff0f0
Oct 30, 2026
Merged

kriboe90 merged 1 commit into
masterfrom
claude/raw-images-single-opening-fff0f0

Conversation

@kriboe90

Copy link
Copy Markdown
Contributor

What changed

Step 3 analysed the channels of an image one after another, and each channel opened and decoded the image again. An RGB analysis (--rgb) therefore decoded every RAW file three times. Nothing the decoding yields depends on the channel: the black level and the read noise are taken per Bayer color, and only the channel mask is built per channel.

  • ledsa/core/image_reading.py: the new ImageChannels reads the file once (decoding, black level and scaling, read noise, orientation). channel_data_mask_and_noise(channel) builds the array, mask and noise of one channel at a time from it, so that besides the decoded image only the channel in work is held in memory. read_channel_data_mask_and_noise_from_img uses it as well, so the reads for one and for several channels cannot drift apart.
  • ledsa/data_extraction/DataExtractor.py: process_img_file reads the image before the channel loop and passes it to every channel.
  • ledsa/data_extraction/step_3_functions.py: generate_analysis_data takes the image as the optional img_channels; without it, it reads the image itself as before.
  • ledsa/tests/UnitTests/test_image_channels.py: 10 tests. For RAW (upright and sideways) and PNG, the single read gives exactly what a read per channel gives. Step 3 opens the file once for three channels. A grayscale or missing image is still reported as a corrupted image. The step 3 test fails against the old DataExtractor.

Why

On a 24 MP CR2 of demo_2, reading the three channels takes 0.42 s instead of 1.22 s per image, with bit-identical arrays, masks and noise for R, G and B. The share of this in the whole of step 3 was not measured: with -s3 the LED fitting probably dominates, with -s3_fast the read is a larger part.

For the reviewer

  • An analysis of a single channel (the default, red) is unchanged.
  • An image that cannot be read is now skipped for all channels at once, before any result file is written. Before, a failure in a later channel could leave the result files of the earlier ones, and --restart only checks channel 0.
  • Peak memory per worker stays as it was: the per-channel arrays are built one at a time, never all three together.
  • The single-channel reads in step 1 and in postprocessing/simulation.py are not touched.

Tests

  • Unit tests: 378 passed
  • Robot Framework acceptance tests: 32 passed

🤖 Generated with Claude Code

Step 3 analysed the channels of an image one after another, and each
channel opened and decoded the image again, so an RGB analysis decoded
every RAW file three times. Nothing the decoding yields depends on the
channel: the black level and the read noise are taken per Bayer color,
and only the channel mask is built per channel.

ImageChannels reads the file once and builds the array, mask and read
noise of one channel at a time from it, so that besides the decoded image
only the channel in work is held in memory. process_img_file reads the
image before the channel loop and passes it to generate_analysis_data,
which still reads the image itself when it gets none.
read_channel_data_mask_and_noise_from_img uses ImageChannels as well, so
the reads for one and for several channels cannot drift apart.

An image that cannot be read is now skipped for all channels at once,
before any result file is written.

On a 24 MP CR2 of demo_2 reading the three channels takes 0.42 s instead
of 1.22 s, with bit-identical arrays, masks and noise. An analysis of a
single channel is unchanged.

Unit tests pass, 378; the acceptance tests pass, 32.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kriboe90
kriboe90 merged commit f2104be into master Oct 30, 2026
5 checks passed
@kriboe90
kriboe90 deleted the claude/raw-images-single-opening-fff0f0 branch October 30, 2026 11:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant