The projects:download scheduled command currently considers all tasks with BuildTracking enabled, regardless of when the task ended.
As a result, repositories belonging to tasks that ended several years ago can still be queued for download. This causes unnecessary requests to GitLab and has contributed to the GitLab instance being throttled due to excessive load.
The download process should be limited so that old, no-longer-relevant tasks and projects are not processed.
Expected behavior
Only projects belonging to tasks that are still relevant for downloading should be considered by projects:download.
Projects from tasks that ended several years ago should not result in new GitLab archive requests.
Current behavior
projects:download runs every 30 minutes and selects all tasks with BuildTracking enabled without restricting them based on ends_at. This can cause repositories from tasks more than four years old to be downloaded.
The
projects:downloadscheduled command currently considers all tasks withBuildTrackingenabled, regardless of when the task ended.As a result, repositories belonging to tasks that ended several years ago can still be queued for download. This causes unnecessary requests to GitLab and has contributed to the GitLab instance being throttled due to excessive load.
The download process should be limited so that old, no-longer-relevant tasks and projects are not processed.
Expected behavior
Only projects belonging to tasks that are still relevant for downloading should be considered by
projects:download.Projects from tasks that ended several years ago should not result in new GitLab archive requests.
Current behavior
projects:downloadruns every 30 minutes and selects all tasks withBuildTrackingenabled without restricting them based onends_at. This can cause repositories from tasks more than four years old to be downloaded.