Publish release builds side by side with releases.json as the only record of what is current

README.md

@@ -102,6 +102,25 @@ The `huggingface` module takes over the uploads Microcosm made itself. Microcosm
 
 **Bundle uploads run on their own thread.** A bundle upload can take minutes, so the module uploads bundles on a worker thread of its own, while the host's worker thread commits run files and makes the commit at shutdown. A bundle upload to one repository therefore never delays run-file commits to another repository, the commit at shutdown, or a read-back. Once shutdown begins, the bundle worker starts no further upload, even to a repository left in its current pass: it checks for shutdown before each repository, again once it has leased the repository, and again once it has claimed a job, immediately before that upload begins, and it gives back a job claimed after shutdown began without attempting it. So during the commit at shutdown a repository can be held only by an upload that passed that last check before shutdown began, or, for the few database operations it takes to give it back, by a lease the worker took just before; the second commit at shutdown covers that repository if the worker stops within the shutdown period. Each delivery leases its repository first, so this machine never commits to one repository twice at once: run files for the repository a bundle is uploading to wait for that upload, and then for the 10-second spacing.
 
+**Releases.** The release writer replaces the commit, tag and pointer-file writes of Microcosm's `release.py`. Nothing calls it yet; Microcosm switches to it in a later change. Each build is published side by side and never changed:
+
+```
+releases.json                                  the only record of what is current
+releases/<release>/<build>/release_manifest.json
+releases/<release>/<build>/...                 the build's files, with their roles
+```
+
+- `publish_release` names the release, the build (Microcosm's attempt ID), whether to make the build current, the release's role, and each file with its role. `promote_build` changes a release's current build; `withdraw_build` marks a build withdrawn, and its folder stays.
+- Every change is one commit: the writer reads `releases.json` and the file list at `main`'s head, applies the change, runs the shared checks in `microcosm_emitter_interface.releases`, and commits the new build folder and `releases.json` with `parent_commit` set to that head. If another commit lands first, it reads the new head and applies the change again, so concurrent publications keep each other's builds.
+- The writer refuses a change unless: every listed build has its folder and a `release_manifest.json` naming the same release and build; `current` names a listed build that is not withdrawn; existing build entries change only `withdrawn`; and no file inside an existing build folder is added, changed or removed (so publishing refuses a build whose folder exists). It never writes a `latest*.json` file, and refuses to change `current` while any `latest*.json` file still exists, until the old pointer files are deleted.
+- It creates no Hugging Face tags or branches, no `current/` folder and no mutable root file, and leaves existing tags and files untouched. It does not port `release.py`'s staging branch, release tag or pointer-file commit, which this layout replaces.
+- Each request carries a `request_id`, a new random one unless the caller gives one, and the emitter applies each request once. Sending the same request again returns its receipt; a new request is applied even if it repeats an earlier change (promoting a build back after another was promoted, or retrying a withdrawal or a refused change once its cause is gone).
+- A publication of a build already listed with a byte-identical `release_manifest.json` reports `published` with the commit that wrote it, as when a retried attempt finds that its earlier commit landed, even if another build has been made current since. It never makes that build current; `promote_build` does. Hub failures stay `pending` and are retried; broken rules are `refused` with the rule's code.
+- Every release receipt has a `current` field. On a `published` receipt it is the release's current build in `releases.json` once the change was applied (or, when nothing needed writing, as the writer found it), and `null` when the release has none; on every other status it is `null`. So a publication with `make_current` made its build current exactly when its receipt is `published` and `current` equals its build.
+- **A queued change never undoes a newer change to the same release, and never lands much later.** A change still `pending` an hour after its hand-over (a Hub outage, a missing credential, a token that cannot write) becomes `refused` with `REQUEST_EXPIRED`, so it cannot land days later, for example after `hf auth login`. Order is kept per release: changes to different releases of one repository never affect each other. Only changes that set `current` are ordered: a promotion is `refused` with `NEWER_CHANGE_LANDED` when a promotion or a publication that makes its build current, to the same release and requested after it, has already landed. A publication that makes its build current is still `published` in that case: it adds its build and leaves `current` as the newer change set it; the receipt's `current` names that build, and its `message` says why. So two publications that land at once both keep their builds, and the newer request's build is current. The writer knows this machine's own changes that landed, and every writer commit's title ends `[newest requests: <release>=<time>; ...]`, each release's newest request time among the changes on the record; a release's time leaves the titles once it is two hours older than the Hub's date for the last commit to `releases.json`, when every change requested before it has expired. A withdrawal never changes `current` (the current build cannot be withdrawn, and a withdrawn build cannot become current, whichever change lands first), so, like a publication that only adds a build, it is never refused for order and does not count as newer. One case remains for the expiry alone: another machine's newer change that left `releases.json` unchanged (promoting the build that is already current) makes no commit, so it records no time, and an older change to that release can still land within the hour. Request times come from each machine's clock, so when two machines' clocks differ, changes requested closer together than that difference can still land out of order. Every change is requested before its commit, so a time in a title more than five minutes after the Hub's date for that commit can only come from a clock that runs fast: it is ignored, and that machine's change is ordered only by its own queue and the expiry. One fast machine therefore neither blocks later changes to its release on other machines nor pushes other releases' times out of the titles.
+- A release change goes at the next cycle, even within 10 seconds of this machine's last commit to the repository; bundles and run files keep that spacing. Each cycle still makes at most one commit per repository.
+- `release_change_policy` is where a decision on who may promote or withdraw will plug in. It receives the validated change and its caller (`ReleaseCaller`: the user the emitter runs as and the build process that started it) when the change is handed over; a refusal is answered at once as `NOT_AUTHORIZED` and never queued. By default every change is allowed. `current` may stay `null`, for a release that is never promoted.
+
 ## Development
 
 Use Python 3.13 or 3.14 on Linux or macOS.

changelog.d/release-writer.added.md

@@ -0,0 +1 @@
+Add the release writer and the interface's `publish_release`, `promote_build`, `withdraw_build` and `release_receipt` calls. Each build is written to `releases/<release>/<build>/` with an updated `releases.json` in one commit pinned to `main`'s head, re-applied after a parent-commit conflict, and checked against the shared rules: no new tags, no `latest*.json` files, no change to an existing build folder, and no change of `current` while any `latest*.json` file exists. Each request carries a `request_id`, so a repeated change is applied again while a resent request is applied once. Nothing calls the writer yet. A change still pending an hour after its hand-over is refused (`REQUEST_EXPIRED`), and a promotion is refused (`NEWER_CHANGE_LANDED`) when a promotion, or a publication that makes its build current, to the same release and requested after it landed first, so a queued change never puts back an older current build. Withdrawals never change `current`, so they are never refused for order and never count as newer. Request times more than five minutes after the Hub's date for the commit that records them are ignored, so one machine whose clock runs fast cannot block or unprotect other machines' changes. A publication overtaken in that way still lists its build and leaves `current` as the newer change set it, so two publications that land at once both keep their builds. Every release receipt names the release's current build after the change (`current`), so a caller can tell whether a publication made its build current.

changelog.d/releases-record-checks.added.md

@@ -0,0 +1 @@
+Add `microcosm_emitter_interface.releases`: the shape of each dataset repository's `releases.json`, the changes the release writer makes (publish a build, make a build current, withdraw a build), and the checks every write must pass, so Microcosm and the emitter run the same code. A `releases.json` above 4 MiB, or nested too deeply to decode, is refused as invalid, and so is a manifest nested too deeply.

packages/microcosm-emitter-interface/README.md

@@ -32,8 +32,11 @@ No call raises into the build: not because the emitter is missing, slow or refus
 | Telemetry event | `client.telemetry.stage(...)`, `transition_stage`, `progress`, `calibration_progress`, `fail`, `refuse`, `complete`, `emit` (each one `emit_telemetry`) | `telemetry` |
 | Staging run files under `runs/<run_id>/` | `stage_run_files(StageRunFiles(...))`, `run_files_receipt`, `read_back_run_files` | `huggingface` |
 | Staged dataset bundle, each file with its role | `stage_dataset_bundle(StageDatasetBundle(...))`, `dataset_bundle_receipt`, `check_write_access` | `huggingface` |
+| Release publication: release, build, whether current, files with roles | `publish_release(PublishRelease(...))`, `promote_build`, `withdraw_build`, `release_receipt` | `huggingface` |
 | Orrery inputs | `orrery_inventory`, `hand_over_orrery_inputs`, `orrery_receipt`, `retry_orrery_inputs` | `orrery` |
 
+`microcosm_emitter_interface.releases` holds the shape of `releases.json` and the checks every release write must pass, so Microcosm, the emitter and a scheduled repository check run the same code.
+
 ## Compatibility
 
 Both packages are released together with one version number, and `microcosm-emitter==X` requires `microcosm-emitter-interface==X`. A build, though, may run with a different emitter from the interface it depends on: it keeps the emitter it started with, and the next build uses whichever emitter is installed then. So the socket protocol carries its own version:

packages/microcosm-emitter-interface/src/microcosm_emitter_interface/__init__.py

@@ -12,6 +12,12 @@
 )
 from microcosm_emitter_interface.orrery import OrrerySettings
 from microcosm_emitter_interface.protocol import PROTOCOL_VERSION
+from microcosm_emitter_interface.publication import (
+    PromoteBuild,
+    PublishRelease,
+    ReleaseFile,
+    WithdrawBuild,
+)
 from microcosm_emitter_interface.spool import (
     default_handover_database_path,
     default_spool_path,
@@ -33,11 +39,15 @@
     "HuggingFaceSettings",
     "OrrerySettings",
     "RequestFailure",
+    "PromoteBuild",
+    "PublishRelease",
+    "ReleaseFile",
     "RunFile",
     "StageDatasetBundle",
     "StageRunFiles",
     "TelemetryRun",
     "TelemetrySettings",
+    "WithdrawBuild",
     "default_handover_database_path",
     "default_spool_path",
 ]

packages/microcosm-emitter-interface/src/microcosm_emitter_interface/client.py

@@ -58,6 +58,14 @@
     STARTUP_TIMEOUT_SECONDS,
     TELEMETRY_MODULE,
 )
+from microcosm_emitter_interface.publication import (
+    PromoteBuild,
+    PublishRelease,
+    ReleaseChange,
+    ReleaseReceipt,
+    WithdrawBuild,
+    unavailable_release_receipt,
+)
 from microcosm_emitter_interface.spool import default_spool_path
 from microcosm_emitter_interface.staging import (
     HuggingFaceSettings,
@@ -486,6 +494,53 @@ def check_write_access(self, repository: str) -> WriteAccess:
             return cast(WriteAccess, value)
         return unavailable_write_access()
 
+    def _release_change(
+        self, request: ReleaseChange, wait_seconds: float
+    ) -> ReleaseReceipt:
+        response = self._query(
+            HUGGING_FACE_MODULE,
+            request.message,
+            kind="release",
+            timeout=FILE_REQUEST_TIMEOUT_SECONDS,
+        )
+        value = None if response is None else response.get("receipt")
+        if not isinstance(value, dict):
+            return unavailable_release_receipt(request)
+        return cast(ReleaseReceipt, self._wait_until_settled(value, wait_seconds))
+
+    def publish_release(
+        self, request: PublishRelease, *, wait_seconds: float = 300.0
+    ) -> ReleaseReceipt:
+        """Queue a build's publication and wait for it to land or be refused.
+
+        The emitter writes ``releases/<release>/<build>/`` and ``releases.json``
+        in one commit. After ``wait_seconds`` a ``pending`` receipt means the
+        emitter is still retrying; ask again with ``release_receipt``. A
+        change still pending an hour after it was handed over is ``refused``
+        with ``REQUEST_EXPIRED``.
+
+        ``published`` means the build is listed. Whether it is current is in
+        the receipt's ``current`` field, the release's current build after
+        this change: with ``make_current=True``, the build became current
+        exactly when ``receipt["current"] == request.build``. It does not when
+        a change to ``current`` requested after this one landed first, or when
+        the build was already published; ``message`` then says which.
+        """
+        return self._release_change(request, wait_seconds)
+
+    def promote_build(
+        self, request: PromoteBuild, *, wait_seconds: float = 60.0
+    ) -> ReleaseReceipt:
+        return self._release_change(request, wait_seconds)
+
+    def withdraw_build(
+        self, request: WithdrawBuild, *, wait_seconds: float = 60.0
+    ) -> ReleaseReceipt:
+        return self._release_change(request, wait_seconds)
+
+    def release_receipt(self, handover_id: str) -> ReleaseReceipt | None:
+        return cast(ReleaseReceipt | None, self._handover_receipt(handover_id))
+
     def close(self) -> None:
         """Ask the emitter to drain and stop; repeated calls do nothing."""
         if self._closed:

packages/microcosm-emitter-interface/src/microcosm_emitter_interface/fixtures/protocol-v1/huggingface-promote-build.json

@@ -0,0 +1,17 @@
+{
+  "protocol_version": 1,
+  "module": "huggingface",
+  "kind": "query",
+  "sequence": 9,
+  "body": {
+    "protocol_version": 1,
+    "command": "promote_build",
+    "request_id": "fixture-request",
+    "repository": "policyengine/fixture-private",
+    "release": "fixture-release",
+    "build": "fixture-build"
+  },
+  "response_keys": [
+    "receipt"
+  ]
+}

packages/microcosm-emitter-interface/src/microcosm_emitter_interface/fixtures/protocol-v1/huggingface-publish-release.json

@@ -0,0 +1,47 @@
+{
+  "protocol_version": 1,
+  "module": "huggingface",
+  "kind": "query",
+  "sequence": 8,
+  "files": {
+    "release_manifest.json": "{\"release\":\"fixture-release\",\"build\":\"fixture-build\"}",
+    "dataset.h5": "synthetic rows",
+    "calibration_diagnostics.json": "{\"schema_version\":8}"
+  },
+  "body": {
+    "protocol_version": 1,
+    "command": "publish_release",
+    "request_id": "fixture-request",
+    "repository": "policyengine/fixture-private",
+    "release": "fixture-release",
+    "build": "fixture-build",
+    "release_role": "national_default",
+    "make_current": true,
+    "files": [
+      {
+        "name": "release_manifest.json",
+        "role": "release_manifest",
+        "local_path": "{directory}/release_manifest.json",
+        "sha256": "9f82c735620892c3f85f288cb9f3e46f5d811c53d5da1745c80aade58b161c85",
+        "bytes": 53
+      },
+      {
+        "name": "dataset.h5",
+        "role": "dataset",
+        "local_path": "{directory}/dataset.h5",
+        "sha256": "5cc590aa15e97595e13963c72aea8d3fbe65fbf5dd5f8fc690ee479f73d45548",
+        "bytes": 14
+      },
+      {
+        "name": "calibration_diagnostics.json",
+        "role": "calibration_diagnostics",
+        "local_path": "{directory}/calibration_diagnostics.json",
+        "sha256": "2300b80de59210bc04c40b1aade68e7b5fc265901dbc9af4e9f506c0bcc36e7c",
+        "bytes": 20
+      }
+    ]
+  },
+  "response_keys": [
+    "receipt"
+  ]
+}

packages/microcosm-emitter-interface/src/microcosm_emitter_interface/fixtures/protocol-v1/huggingface-withdraw-build.json

@@ -0,0 +1,17 @@
+{
+  "protocol_version": 1,
+  "module": "huggingface",
+  "kind": "query",
+  "sequence": 10,
+  "body": {
+    "protocol_version": 1,
+    "command": "withdraw_build",
+    "request_id": "fixture-request",
+    "repository": "policyengine/fixture-private",
+    "release": "fixture-release",
+    "build": "fixture-build"
+  },
+  "response_keys": [
+    "receipt"
+  ]
+}