diff --git a/README.md b/README.md index a1342d7d..600a9c5d 100644 --- a/README.md +++ b/README.md @@ -102,6 +102,25 @@ The `huggingface` module takes over the uploads Microcosm made itself. Microcosm **Bundle uploads run on their own thread.** A bundle upload can take minutes, so the module uploads bundles on a worker thread of its own, while the host's worker thread commits run files and makes the commit at shutdown. A bundle upload to one repository therefore never delays run-file commits to another repository, the commit at shutdown, or a read-back. Once shutdown begins, the bundle worker starts no further upload, even to a repository left in its current pass: it checks for shutdown before each repository, again once it has leased the repository, and again once it has claimed a job, immediately before that upload begins, and it gives back a job claimed after shutdown began without attempting it. So during the commit at shutdown a repository can be held only by an upload that passed that last check before shutdown began, or, for the few database operations it takes to give it back, by a lease the worker took just before; the second commit at shutdown covers that repository if the worker stops within the shutdown period. Each delivery leases its repository first, so this machine never commits to one repository twice at once: run files for the repository a bundle is uploading to wait for that upload, and then for the 10-second spacing. +**Releases.** The release writer replaces the commit, tag and pointer-file writes of Microcosm's `release.py`. Nothing calls it yet; Microcosm switches to it in a later change. Each build is published side by side and never changed: + +``` +releases.json the only record of what is current +releases///release_manifest.json +releases///... the build's files, with their roles +``` + +- `publish_release` names the release, the build (Microcosm's attempt ID), whether to make the build current, the release's role, and each file with its role. `promote_build` changes a release's current build; `withdraw_build` marks a build withdrawn, and its folder stays. +- Every change is one commit: the writer reads `releases.json` and the file list at `main`'s head, applies the change, runs the shared checks in `microcosm_emitter_interface.releases`, and commits the new build folder and `releases.json` with `parent_commit` set to that head. If another commit lands first, it reads the new head and applies the change again, so concurrent publications keep each other's builds. +- The writer refuses a change unless: every listed build has its folder and a `release_manifest.json` naming the same release and build; `current` names a listed build that is not withdrawn; existing build entries change only `withdrawn`; and no file inside an existing build folder is added, changed or removed (so publishing refuses a build whose folder exists). It never writes a `latest*.json` file, and refuses to change `current` while any `latest*.json` file still exists, until the old pointer files are deleted. +- It creates no Hugging Face tags or branches, no `current/` folder and no mutable root file, and leaves existing tags and files untouched. It does not port `release.py`'s staging branch, release tag or pointer-file commit, which this layout replaces. +- Each request carries a `request_id`, a new random one unless the caller gives one, and the emitter applies each request once. Sending the same request again returns its receipt; a new request is applied even if it repeats an earlier change (promoting a build back after another was promoted, or retrying a withdrawal or a refused change once its cause is gone). +- A publication of a build already listed with a byte-identical `release_manifest.json` reports `published` with the commit that wrote it, as when a retried attempt finds that its earlier commit landed, even if another build has been made current since. It never makes that build current; `promote_build` does. Hub failures stay `pending` and are retried; broken rules are `refused` with the rule's code. +- Every release receipt has a `current` field. On a `published` receipt it is the release's current build in `releases.json` once the change was applied (or, when nothing needed writing, as the writer found it), and `null` when the release has none; on every other status it is `null`. So a publication with `make_current` made its build current exactly when its receipt is `published` and `current` equals its build. +- **A queued change never undoes a newer change to the same release, and never lands much later.** A change still `pending` an hour after its hand-over (a Hub outage, a missing credential, a token that cannot write) becomes `refused` with `REQUEST_EXPIRED`, so it cannot land days later, for example after `hf auth login`. Order is kept per release: changes to different releases of one repository never affect each other. Only changes that set `current` are ordered: a promotion is `refused` with `NEWER_CHANGE_LANDED` when a promotion or a publication that makes its build current, to the same release and requested after it, has already landed. A publication that makes its build current is still `published` in that case: it adds its build and leaves `current` as the newer change set it; the receipt's `current` names that build, and its `message` says why. So two publications that land at once both keep their builds, and the newer request's build is current. The writer knows this machine's own changes that landed, and every writer commit's title ends `[newest requests: =