Repository navigation
Document the artifact submission contract - #6
Merged
Merged
Conversation
Builds will hand calibration diagnostics and Orrery inputs to the collector, which stores them privately and processes them in worker jobs. This records the contract both sides implement: the two kinds and their file roles, the inventory and provenance fields, identifiers derived by the collector, the statuses and receipt, and the limits. File bytes go to signed upload URLs because the API accepts at most 1 MiB per request body and runs with 512 MiB of memory and a 30 s timeout. For the same reason, completion checks only sizes and recorded digests, and the worker recomputes each file's SHA-256 from the stored bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Implementing the contract settled four details, recorded here: upload URLs also sign x-goog-if-generation-match: 0, so an object is created once and a repeated upload gets 412; a job waiting for another attempt shows as queued; the collector's rejection codes are content_mismatch, object_missing, worker_error and attempts_exhausted; and the default handler's result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the Orrery projection (#9) and the 0.3.0 release commit, so the branch is up to date with main as the ruleset requires. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The emitter builds submissions from the roles a staged bundle names. The calibration roles have the same names in both places; the Orrery inputs carry an orrery_ prefix in a bundle (except execution_overlay), which the emitter maps to schema, upstream and evidence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Map each bundle role emitter#27 sends to its contract role, and each of emitter#22's earlier Orrery publication roles to the contract role a re-submission would use. Give build_id, release, the Hugging Face repo and folder a character set and length, say that null provenance fields are hashed as JSON null, and that the commit is the one that added the folder. Rename inventory_sha256 to files_sha256, since the emitter's receipt field of that name is computed differently, and say that size is the emitter's bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… intake The token response now states its scope, an exchange or submission path on a collector with intake switched off answers 404 with the code intake_not_enabled, and a 404 means an unknown submission only with the code submission_not_found. A client keeps its files queued while intake is not available. Document the create and status bodies, each upload instruction with every signed header (Host included) and its expiry, 429, the maintenance-mode 503 and other 5xx answers, and the storage service's answers to an upload. A stored object with another size or SHA-256 now rejects the submission at completion, since it can never be replaced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…result The job ID now hashes a processing round, so an operator can process a final submission again: after K4 or O4 registers a handler, or when the projection or Orrery changes. Dispatches are numbered in each Cloud Task name. A recovery run every 10 minutes fails a job whose lease expired after its third attempt, dispatches again one whose attempt or execution ended early, and fails one that would need a fourth dispatch, so every job reaches a final status within its configured attempts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration data derived from licensed UK data may stay publicly readable (D10). Orrery outputs of licensed builds stay private until output clearance (D5), and native evidence is never published. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 20, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
pushed a commit
that referenced
this pull request
Oct 27, 2026
Brings in #6 (the submission contract, released as 0.4.0), #4, #11 and #23. No conflicts: README.md, pyproject.toml and uv.lock merged automatically, and the consumed fragment changelog.d/artifact-submission-contract.added.md is removed, leaving only this branch's artifact-intake.added.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5
Summary
Adds
docs/artifact-submissions.md, the contract for builds handing files to the collector. The collector's intake endpoints and worker framework (#12, stacked on this one) and the emitter's submission module (emitter#27) implement it.calibration-diagnostics(rolescalibration_diagnostics, plus optionalbuild_manifest,release_manifestanddemographics: the files the calibration-tree publisher reads for a build today) andorrery-inputs(schema,execution_overlay, any number ofupstream, and nativeevidencefiles that are never published).orrery_schema,execution_overlay,orrery_upstream,orrery_evidenceand the four calibration roles) to the contract role it puts infiles[].role. A second table maps emitter#22's earlier Orrery publication roles (PublicationFileRole:graph,schema,index,declaration,manifest,binding,summaries,upstream) for O7's re-submission of queued uploads: five of them becomeevidence, andgraphhas no contract role, because the collector projects its own Orrery artifact. That earlier path has no overlay file, which O7 has to obtain.nullfor a staged build), and the Hugging Face repo, commit and folder holding the same bytes. Each field has a character set and length (build_id,releaseand each folder segment use the emitter interface'ssafe_identifierrule: 128 characters, no colon). Null fields are hashed as JSONnulland are never left out. The commit is the one that added the folder.licensed_dataand the sorted inventory, so every request in the flow can be retried. The job ID also hashes a processing round, so a submission can be processed again. The digest of the files isfiles_sha256, which is a different value from the emitter receipt'sinventory_sha256.files[].sizeis the emitter'sbytes.Hostincluded. Completion checks each object's size and recorded digest from storage metadata; the worker job recomputes every SHA-256.artifacts:submitand every submission path answer 404 with the codeintake_not_enabled. A 404 means "unknown submission, create it again" only with the codesubmission_not_found. While intake is not available, the client keeps its files queued.codeand what the client does, 429, the maintenance-mode 503 the collector returns on every path, other 5xx answers, and the storage service's answers to an upload (an expired URL, a signature mismatch, 412).awaiting_upload,queued,processing, then one ofsucceeded,rejectedorfailed. A stored object with another size or SHA-256 rejects the submission at completion (stored_object_mismatch), because stored objects are never replaced.licensed_datais required with no default.artifacts:submittoken only when asked; existing clients still gettelemetry:writetokens, and neither scope works on the other's endpoints.Review fixes (2026-10-20)
Docs only, so there is no red/green step for any of these commits.
397d555Name the roles the emitter sends and the provenance syntaxinventory_sha256renamedfiles_sha256;sizedocumented as the emitter'sbytescc798faSpecify every answer, and how a client recognises a collector without intakescope,intake_not_enabledandsubmission_not_found, the client rule to keep files queued; response bodies, upload instruction fields and headers (withHost), 429, 5xx and the maintenance-mode 503, storage answers; size and SHA-256 mismatches made final rejections8e52d02Reprocess submissions and recover jobs whose attempt ended without a result673df3dKeep only licensed Orrery outputs private, per decision D10Testing
tools/bump-version.py --checkpasses: with this fragment, the next release is 0.4.0 (main is at 0.3.0). CI runs the full checks.Merging
Don't merge before the collector deployment switch-over is verified. A merge to
maincreates a release, and after the switch-over it also deploys.🤖 Generated with Claude Code