Skip to content

📝 docs(storage): define an absent artifact placement - #2161

Merged
gaborbernat merged 1 commit into
mainfrom
fix/placement-absence-meaning-2151
Sep 2, 2026
Merged

📝 docs(storage): define an absent artifact placement#2161
gaborbernat merged 1 commit into
mainfrom
fix/placement-absence-meaning-2151

Conversation

@gaborbernat

Copy link
Copy Markdown
Member

Seven consumers read the artifact placement projection, and nothing told any of them what a missing row means. Each decided on its own, so one refuses a publish on it, two read it as local bytes, two read it as none, one drops a transition, and one records a row of its own. A comment on the file view had already drifted into claiming the upload path records a row it never writes, while the comment on search says the opposite and is the correct one. Whoever read the first comment next would have derived an eighth answer from it. 🔍

One correction to the framing this issue opens with, since it set the priority: the readings are not five contradictory meanings of one thing. The two PyPI sites agree with each other, the two OCI sites agree with each other, and both pairs are right for their own write-side guarantee. OCI writes a manifest's row in the transaction that publishes it, so a missing row there does mean no local bytes. PyPI writes none for a hosted upload, so its views lean on the upload record and let the projection demote it. apply_placement_event has one caller, which handles the empty case itself, and the Proxy source the blob plane records is right for a replica whose bytes came from the primary. The defect is narrower. One rule sits behind all four, implemented by hand each time, stated nowhere, and carrying a false comment in one of them.

This change settles the meaning and leaves behaviour alone. A digest with no row is a digest the projection holds no opinion about. It is not evidence the bytes are gone, because paths commit bytes without recording, which #2141 enumerates. It is not evidence they are there either, because reclaiming an orphan deletes the row before the file on purpose, so an interruption understates what the node holds rather than promising bytes that are already gone. A caller therefore answers from the record that settles on its own whether this node wrote the bytes, and lets the projection demote that answer rather than establish it. A caller with no such record reports none. A caller that needs proof refuses.

That rule now sits on ArtifactPlacement, where every consumer already looks, with the read that returns the Option pointing at it, and in the artifact source and availability page for readers who never open the type. The file view and search had grown separate copies of the PyPI half and could drift apart on the question at issue, so both resolve through one function that carries the rule. It reproduces both copies case for case, including the one where a same-digest mirror's row is ignored because it describes the mirror rather than the upload.

finalize keeps refusing on a missing row, and the reason changes rather than the outcome. Its module doc said it rejects on "a placement the bytes never reached", which reads absence as proof. Absence proves nothing, so the refusal now rests on publication needing positive evidence that the bytes reached this home: it fails closed, and the retry succeeds once the blob plane records the arrival. Nothing an operator or a client sees moves, here or at the other six sites, which keep the answers they had and now carry a written reason.

Closes #2151

Seven consumers read the placement projection and each decided for itself what
a missing row meant. Nothing said, so one refuses a publish on it, two read it
as local bytes, two read it as none, one drops a transition, and one records a
row of its own. Most of those answers are right for the caller that makes them,
but each was reached privately, and a comment on the file view had already
drifted into claiming the upload path records a row it never writes. The next
consumer would have invented an eighth reading from that comment.

The projection holds no opinion about a digest it has no row for. It is not
evidence the bytes are gone, because paths commit bytes without recording, and
it is not evidence they are there, because reclaiming an orphan deletes the row
before the file so an interruption understates what the node holds. So a caller
answers from the record that independently establishes whether this node wrote
the bytes and lets the projection demote that, never establish it; a caller
with no such record reports none; a caller that needs proof refuses.

That rule now sits on ArtifactPlacement, where every consumer already looks,
with the read that returns the Option pointing at it. The file view and search
had grown separate copies of the PyPI half and could drift apart on exactly the
question at issue, so they resolve through one function that carries the rule.
Behaviour is unchanged everywhere: the shared function reproduces both copies
case for case, and the remaining consumers keep the answers they had, now with
the reason written down rather than inferred.

Finalize is the one worth naming. It still refuses on a missing row, but not
because absence proves the bytes never arrived. It refuses because publishing
needs positive evidence and a missing row is not evidence either way, so it
fails closed and the retry succeeds once the arrival is recorded.
@gaborbernat gaborbernat added bug Something isn't working documentation Improvements or additions to documentation labels Sep 2, 2026
@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 peryx | 🛠️ Build #34361367 | 📁 Comparing 3b06629 against latest (892226e)

  🔍 Preview build  

1 file changed
± core/repositories/artifact-source/index.html

@codspeed-hq

codspeed-hq Bot commented Sep 2, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 29 untouched benchmarks
⏩ 133 skipped benchmarks1


Comparing fix/placement-absence-meaning-2151 (3b06629) with main (892226e)

Open in CodSpeed

Footnotes

  1. 133 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@gaborbernat
gaborbernat merged commit 68b3d3c into main Sep 2, 2026
20 checks passed
@gaborbernat
gaborbernat deleted the fix/placement-absence-meaning-2151 branch September 2, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Define what an absent artifact placement means

1 participant