Problem
sync_project_files and sync_catalog serialize concurrent callers on a mutex. When on-demand project sync becomes reachable from the serving path, every waiter behind that mutex will issue its own conditional request to the upstream, so a burst of requests for one cold project becomes a burst of upstream requests rather than one.
This is not reachable today. #1302 established that waited is never true in production: sync_project_files has exactly one production caller, the catalog job, which iterates distinct projects and cannot overlap for one repository because the scheduler admits on a (kind, scope) conflict key; and sync_catalog's second caller is reachable only through peryx prefetch, which opens the store writable and so cannot run beside a serving process.
#1302 fixed the correctness half by deleting a shortcut that reported NotModified to a waiter without observing any upstream response. It deliberately did not build coalescing, because building it for callers that do not exist is speculative. This issue records the requirement for when they do.
What this needs, when it is needed
A waiter must learn the producer's outcome, not merely that the producer finished. Three outcomes are distinct and the current ProjectSyncOutcome collapses at least two of them: the producer published new content, the producer received a genuine 304, the producer failed.
The counting collision is the trap, and it is why this is not a small change. catalog_job.rs does report.changed += 1 per outcome. If waiters receive Published, one publication counts as N changes. Fixing that means either a distinct "another flight published this" outcome or moving the counting decision out of the per-caller path. Decide which before writing the flight, not after — this is the naming-collision class that forced PR #2089 to rename expired to timed_out, where a field's condition became reachable by a second cause and silently started meaning two things.
Also settle what a waiter observes when the producer is cancelled, which is a fourth case and has no representation today.
Acceptance criteria
- Concurrent callers for one project produce one upstream request, and each caller learns the producer's actual outcome.
- A publication counts as one change in catalog reporting regardless of how many callers waited on it. Assert the count, not just the outcome.
- A producer failure is reported as a failure to every waiter, never as a reuse of an existing generation.
- A cancelled producer has a defined observable outcome, pinned by a test.
ProjectSyncOutcome's documented meanings stay true for every path that can return each variant.
Boundary
Coalescing and outcome propagation for the sync flights. Do not open this until on-demand project sync from the serving path is actually reachable — until then there is no herd to prevent, and the mutex is sufficient. Note in the pull request which caller made it reachable, so the justification is on the record.
Problem
sync_project_filesandsync_catalogserialize concurrent callers on a mutex. When on-demand project sync becomes reachable from the serving path, every waiter behind that mutex will issue its own conditional request to the upstream, so a burst of requests for one cold project becomes a burst of upstream requests rather than one.This is not reachable today. #1302 established that
waitedis never true in production:sync_project_fileshas exactly one production caller, the catalog job, which iterates distinct projects and cannot overlap for one repository because the scheduler admits on a(kind, scope)conflict key; andsync_catalog's second caller is reachable only throughperyx prefetch, which opens the store writable and so cannot run beside a serving process.#1302 fixed the correctness half by deleting a shortcut that reported
NotModifiedto a waiter without observing any upstream response. It deliberately did not build coalescing, because building it for callers that do not exist is speculative. This issue records the requirement for when they do.What this needs, when it is needed
A waiter must learn the producer's outcome, not merely that the producer finished. Three outcomes are distinct and the current
ProjectSyncOutcomecollapses at least two of them: the producer published new content, the producer received a genuine304, the producer failed.The counting collision is the trap, and it is why this is not a small change.
catalog_job.rsdoesreport.changed += 1per outcome. If waiters receivePublished, one publication counts as N changes. Fixing that means either a distinct "another flight published this" outcome or moving the counting decision out of the per-caller path. Decide which before writing the flight, not after — this is the naming-collision class that forced PR #2089 to renameexpiredtotimed_out, where a field's condition became reachable by a second cause and silently started meaning two things.Also settle what a waiter observes when the producer is cancelled, which is a fourth case and has no representation today.
Acceptance criteria
ProjectSyncOutcome's documented meanings stay true for every path that can return each variant.Boundary
Coalescing and outcome propagation for the sync flights. Do not open this until on-demand project sync from the serving path is actually reachable — until then there is no herd to prevent, and the mutex is sufficient. Note in the pull request which caller made it reachable, so the justification is on the record.