Description of the issue
When reprocessing with a large date range, i.e. 20250924 to 20260510, processing can miss some days that should be processed based on available data.
Steps to reproduce the issue
When running the following command:
imap-data-access reprocess --instrument swapi --descriptor proton-sw --data-level l3a --start-date 20250924 --end-date 20260510
We encountered the following error:
IMAPDataAccessError('503 Service Unavailable: {"message":"Service Unavailable"}')
This command triggered reprocessing for 134/181 days but did not trigger reprocessing for the following 47/181 days:
20260127, 20260128, 20260214, 20260215, 20260216, 20260217, 20260218, 20260219, 20260220, 20260221, 20260222, 20260223, 20260224, 20260225, 20260228, 20260301, 20260302, 20260308, 20260313, 20260317, 20260318, 20260319, 20260320, 20260328, 20260329, 20260403, 20260404, 20260406, 20260412, 20260415, 20260416, 20260420, 20260425, 20260426, 20260427, 20260428, 20260429, 20260430, 20260502, 20260503, 20260504, 20260505, 20260506, 20260507, 20260508, 20260509, 20260510
We were able to successfully trigger reprocessing for some of these days individually (20260510 and 20260127) , so it does not seem to be a problem with missing dependencies.
Description of the issue
When reprocessing with a large date range, i.e. 20250924 to 20260510, processing can miss some days that should be processed based on available data.
Steps to reproduce the issue
When running the following command:
We encountered the following error:
This command triggered reprocessing for 134/181 days but did not trigger reprocessing for the following 47/181 days:
20260127, 20260128, 20260214, 20260215, 20260216, 20260217, 20260218, 20260219, 20260220, 20260221, 20260222, 20260223, 20260224, 20260225, 20260228, 20260301, 20260302, 20260308, 20260313, 20260317, 20260318, 20260319, 20260320, 20260328, 20260329, 20260403, 20260404, 20260406, 20260412, 20260415, 20260416, 20260420, 20260425, 20260426, 20260427, 20260428, 20260429, 20260430, 20260502, 20260503, 20260504, 20260505, 20260506, 20260507, 20260508, 20260509, 20260510
We were able to successfully trigger reprocessing for some of these days individually (20260510 and 20260127) , so it does not seem to be a problem with missing dependencies.