Skip to content

[Capability]: Make AutoGenesis 1.1.1 Skills Preview publication resumable #71

Closed

Description

@houlianpi

Problem

The existing main-only release workflow cannot yet satisfy #70's no-partial-success contract:

  1. public agent-contract-v2.0.0 reports immutable:false, while publish-skills directly creates a published release with assets and only afterward asserts .immutable == true;
  2. the release manifest records GITHUB_RUN_ATTEMPT, and consumer jobs compare it with their current attempt, so GitHub's rerun failed jobs increments the attempt and rejects the already reviewed attempt-1 candidate instead of safely resuming it; and
  3. MCP Registry publication runs Appium then Windows without exact same-version preflight, exact post-publication query, or an idempotent partial-rerun rule, so a first-package success followed by second-package failure may not be safely resumable.

PyPI already has exact-byte skip-existing behavior and its four trusted publishers were proven by the prior accepted release, but the complete cross-surface release still cannot be called atomic or safely recovered until the gaps above are closed.

User outcome

An authorized release operator can publish one previously reviewed candidate through the existing workflow, safely rerun only failed publication jobs without rebuilding or changing candidate identity, and prove that the immutable GitHub Skill release plus both exact MCP Registry versions match that candidate. A partial or mismatched surface fails closed and is never promoted as release success.

Scope

  • Keep .github/workflows/publish-mcp-registry.yml as the sole intended publication path.
  • Require repository immutable releases to be enabled before dispatch.
  • Change publish-skills to create a draft, upload and verify the complete asset set, publish the completed draft once, then assert the public release is immutable.
  • Make candidate provenance stable across workflow rerun attempts: later failed-job attempts must consume the exact originally reviewed build artifacts for the same workflow run and commit without rejecting them solely because GITHUB_RUN_ATTEMPT changed.
  • Continue to reject artifacts from a different workflow run, commit, version line, manifest, or digest.
  • Add exact MCP Registry preflight for each percent-encoded package-name/version endpoint:
    • absent → publish that exact candidate;
    • present and byte/field-equivalent to the candidate → skip safely;
    • present but mismatched → fail closed without overwrite.
  • After each Registry publication or exact skip, query the exact endpoint with bounded retry and verify package identity, version, transport/package coordinates, and candidate-bound metadata before proceeding/succeeding.
  • Make Appium-success/Windows-failure and other partial Registry states safely resumable under the same reviewed candidate.
  • Preserve PyPI's exact-byte preflight/skip-existing behavior and verify all four exact public distributions before release success.
  • Add focused workflow/verifier coverage and independently review correctness and security.

Non-goals

  • No package, GitHub release, or MCP Registry publication while implementing or accepting this capability.
  • No new release workflow or manual side channel.
  • No weakening of commit/run provenance, version, digest, OIDC, environment approval, or no-partial-success checks.
  • No reuse or mutation of a mismatched public version.
  • No root README, Skill behavior, Runtime behavior, or client compatibility changes.

Product version and artifact identity

The public product release is AutoGenesis 1.1.1 Skills Preview. This product version is intentionally decoupled from component/package versions: it names the coherent user-facing Preview, while each independently distributed technical artifact retains its frozen semantic identity for package resolution, protocol compatibility, and immutable verification. Do not rewrite package coordinates or the GitHub tag to 1.1.1.

Frozen interfaces

  • Existing workflow: .github/workflows/publish-mcp-registry.yml.
  • Intended release candidate: Agent Contract 2.1.0, Appium Runtime 0.5.0, Windows Runtime 0.5.0, CLI 0.2.0.
  • Intended Skill release: agent-contract-v2.1.0.
  • Publication success remains a single accepted unit across all four PyPI versions, the immutable GitHub Skill release, and both MCP Registry versions.
  • Rerun support applies only to the same reviewed candidate from the same workflow run/commit; it is not permission to rebuild or substitute bytes.

Acceptance scenarios

Scenario: Skill assets become public only when complete and immutable
  Given immutable releases are enabled for FSQ-lab/AutoGenesis
  And the workflow has one verified Skill candidate
  When publish-skills runs
  Then it creates a draft release
  And uploads and verifies every expected asset and digest while still draft
  And publishes the completed draft once
  And the public API reports immutable true
  And no mutable, orphaned, or asset-incomplete public release remains

Scenario: Failed-job rerun reuses the reviewed candidate
  Given attempt 1 built and verified a candidate for one workflow run and commit
  And a downstream publication job failed without invalidating that candidate
  When an authorized operator reruns failed jobs and GitHub increments GITHUB_RUN_ATTEMPT
  Then every rerun consumer accepts the exact attempt-1 candidate provenance and digests
  And no build artifact is regenerated or substituted
  And the workflow does not fail solely because the consumer attempt number changed

Scenario: Cross-run or changed candidate is rejected
  Given a publication consumer receives an artifact from a different workflow run, commit, manifest, version, or digest
  When it validates provenance
  Then it fails before publication
  And it does not treat rerun support as authorization for the changed artifact

Scenario: Exact Registry version is absent
  Given the percent-encoded exact package-name/version endpoint returns absent
  When the applicable Registry publication job runs
  Then it publishes the candidate once
  And queries the exact endpoint with bounded retry
  And succeeds only when the public record exactly matches the candidate identity and metadata

Scenario: Exact Registry version already matches after a partial run
  Given Appium or Windows was published by an earlier attempt of the same reviewed candidate
  And the exact endpoint returns an equivalent record
  When failed jobs are rerun
  Then the workflow skips that exact publication safely
  And re-verifies the public record
  And proceeds to the missing surface without changing candidate identity

Scenario: Exact Registry version conflicts
  Given the exact package-name/version endpoint exists with any mismatched coordinate, transport, metadata, or candidate identity
  When preflight runs
  Then the workflow fails closed before overwrite or later publication
  And #70 remains unaccepted pending rollback and a Human Product Owner version decision

Scenario: No partial state is called success
  Given any PyPI version, Skill release asset/immutability check, or Registry exact postcheck is missing or mismatched
  When the workflow completes
  Then the overall publication is failed
  And it emits the exact failed surface and recovery state
  And no onboarding or release issue may promote the candidate as available

Required evidence

  • Repository-setting evidence that immutable releases are enabled.
  • Focused review evidence for draft → upload all assets → verify → publish → assert immutable ordering.
  • A verifier scenario showing attempt-2 failed-job consumers accept exact attempt-1 reviewed artifacts from the same run/commit.
  • Negative provenance evidence for changed run, commit, version, manifest, and digest.
  • Registry verifier evidence for absent, exact-match, mismatch, delayed consistency/retry exhaustion, and Appium-first/Windows-failed partial recovery states using the exact percent-encoded name/version endpoint contract.
  • Evidence that PyPI exact-byte skip-existing behavior remains intact.
  • Independent correctness and security review. The implementer cannot approve this work.
  • Link all evidence to this issue; source inspection alone is not acceptance.

Security and rollback

The change must preserve GitHub OIDC/trusted publishing and environment approval boundaries and must not expose credentials in logs or artifacts. Exact-match recovery is allowed only for the same authenticated candidate; any uncertainty or mismatch fails closed.

Before any public write, rollback is cancellation. A failed Skill draft is deleted before publication. If a public partial state exists, #70's rollback applies: do not advertise it; yank rather than overwrite new PyPI versions, withdraw/deprecate Registry entries under Registry policy, mark immutable public releases failed/superseded without identity reuse, and require a new Human Product Owner version decision after any collision.

Dependencies and ownership

Definition of Ready

  • Release behavior and no-partial outcome are explicit.
  • Public interfaces and candidate identities are frozen.
  • Failure/recovery states are observable.
  • Security-sensitive provenance, OIDC, public immutability, and Registry overwrite surfaces are identified.
  • File ownership and serialization with [Capability]: Ship generated-code recording as a read-only safe Preview #69 are explicit.
  • No product decision remains for the implementer.

Activity

  1. added
    acceptance:pendingAcceptance evidence has not passed yet
    area:packagingPython packages, entrypoints, publishing, and registry metadata
    risk:highRequires explicit risk and rollback review
    security-reviewIndependent security review is required
    type:capabilityUser-visible capability with executable acceptance criteria
    priority:p0Must be addressed before the next Windows Runtime compatibility release
    on Nov 10, 2026
  2. houlianpi commented on Nov 10, 2026

    @houlianpi
    Author

    Refinement audit evidence (2026-11-10):

    • Appium exact target https://registry.modelcontextprotocol.io/v0.1/servers/io.github.FSQ-lab%2Fautogenesis-appium/versions/0.5.0: 404.
    • Windows exact target https://registry.modelcontextprotocol.io/v0.1/servers/io.github.FSQ-lab%2Fautogenesis-windows/versions/0.5.0: 404.
    • Both Registry latest endpoints currently resolve active 0.4.0 records with official isLatest=true.
    • All four intended PyPI version endpoints (/pypi/{name}/{version}/json) return 404.
    • GitHub agent-contract-v2.1.0 release API returns 404; existing agent-contract-v2.0.0 reports immutable:false.
    • Prior successful run https://github.com/FSQ-lab/AutoGenesis/actions/runs/40892135326 proves the four PyPI trusted-publisher mappings and MCP Registry GitHub-OIDC namespace.
    • Provenance defect is exact: an attempt-1 manifest is passed to _load_release_manifest(..., _github_provenance()), whose current GITHUB_RUN_ATTEMPT=2 comparison rejects a normal rerun-failed-jobs recovery.

    These observations freeze the pre-publication state; acceptance still requires behavioral verifier evidence and independent review, not source inspection alone.

  3. houlianpi commented on Nov 10, 2026

    @houlianpi
    Author

    Implementation is available for independent review in Draft PR #73: #73 at commit 76c7fe0.

    Observable result:

    • Same-run failed-job reruns accept the exact earlier reviewed attempt while repository/workflow/ref/commit/run ID remain exact; changed or future provenance fails before publication.
    • The complete Skill payload is recorded in a separate reviewed Actions artifact after the install prompt is generated. Publication verifies every filename, size, and SHA-256 before creating a draft.
    • Skill publication is draft -> exact upload/byte verification -> one-way publish -> same-commit/public immutable: true proof, with failed-draft cleanup and no direct mutable create path.
    • Registry preflight checks both exact percent-encoded endpoints before any write, returns only absent or exact, rejects collisions, skips exact records, publishes only absent records, postverifies each surface with bounded retry, and finishes with both exact active/latest records proven. Appium-success/Windows-failure is explicitly resumable and is not labeled atomic.
    • Registry/PyPI reads, timeouts, attempts, and delays are bounded; untrusted Registry response values are not used in commands or echoed.

    Non-goals remain unchanged: no publication/dispatch, Runtime/Contract/server.json/package-manifest/root-README change, side channel, version reuse, or provenance/OIDC/environment weakening.

    Main verification commands (not run by the implementer, per assignment):

    python -m unittest tools/test_verify_registry_metadata.py -v
    actionlint .github/workflows/publish-mcp-registry.yml

    Security/rollback:

    • Immutable releases must be enabled before dispatch. A failed draft is deleted; a just-published release reporting non-immutable is deleted and the job fails. Any uncertainty/collision/partial public surface remains failed and unadvertised under [Release]: Publish AutoGenesis 1.1.1 Skills Preview atomically #70 rollback, with no overwrite and a new Human Product Owner version decision where identity is burned.
    • OIDC and mcp-registry-publish approval remain intact; no secrets or raw headers are emitted.
    • The separate reviewed Skill manifest anchors the prompt/lock/release-manifest/Skill/SHA256SUMS payload before release, while GitHub immutable-release attestation remains the public external anchor. Root/onboarding digest guidance is outside [Capability]: Make AutoGenesis 1.1.1 Skills Preview publication resumable #71 ownership and remains an explicit coordination dependency.

    Required acceptance still pending: Main's focused commands, repository immutable-setting evidence, and independent correctness/security review. No release success is claimed.

  4. houlianpi commented on Nov 10, 2026

    @houlianpi
    Author

    Correctness/security REQUEST_CHANGES are remediated in Draft PR #73 commit a3f699b (still no dispatch/publication/validation).

    Review-driven observable changes, all within #71 release-integrity scope:

    • The workflow resolves the exact tag ref before any draft/write and peels bounded annotated-tag chains. A tag-only state is allowed only at the reviewed GITHUB_SHA; a wrong-commit tag fails before draft creation. An absent tag is created directly at the reviewed SHA and immediately reverified.
    • The exact tag and captured Release database ID/state/target/tag are reverified immediately before publication and again afterward.
    • Draft deletion is exclusively REST deletion by captured/revalidated Release database ID. Tag-only gh release delete --cleanup-tag is gone. Cleanup deletes a tag only if this run created the unchanged direct ref and exact draft deletion/absence was proven; ambiguity preserves evidence.
    • After the draft becomes public, cleanup is disabled. immutable:false preserves the burned tag/Release/evidence and reruns reject that public mutable identity; the workflow never erases it or permits same-version reuse.
    • Release asset API names/sizes are matched to the reviewed manifest before download, with count, individual size, aggregate size, file-output and exact-byte bounds. Publisher archive, Sigstore bundle, and extracted binary are separately byte-bounded before use.
    • Focused regressions now cover same/wrong commit tag-only states, absent/uncaptured draft cleanup, database-ID-bound cleanup, mutable burn/rerun behavior, immutable ordering, and Release/publisher download limits.

    The prior rollback note is superseded: a public mutable release is not deleted. It remains visible/burned and requires #70 rollback plus a new Human Product Owner version decision.

    Main still must run (implementer did not run validation, per assignment):

    python -m unittest tools/test_verify_registry_metadata.py -v
    actionlint .github/workflows/publish-mcp-registry.yml

    Fresh independent correctness/security review remains required.

  5. changed the title [-][Capability]: Make Skills Preview publication resumable and fail-closed[/-] [+][Capability]: Make AutoGenesis 1.1.1 Skills Preview publication resumable[/+] on Nov 10, 2026
  6. houlianpi commented on Nov 10, 2026

    @houlianpi
    Author

    Human Product Owner version decision applied in Draft PR #73 commit 74bd39a: the public GitHub Release title/notes now identify AutoGenesis 1.1.1 Skills Preview.

    Frozen technical identities are deliberately unchanged:

    • GitHub tag: agent-contract-v2.1.0
    • Agent Contract: 2.1.0
    • Appium Runtime: 0.5.0
    • Windows Runtime: 0.5.0
    • CLI: 0.2.0

    This is presentation/product identity only; it does not change provenance, assets, package coordinates, Registry records, rollback semantics, or any publication state. No validation or publication was run.

  7. houlianpi commented on Nov 10, 2026

    @houlianpi
    Author

    Product Owner version decision (2026-11-10): the public product release is AutoGenesis 1.1.1 Skills Preview. Product and component versions are deliberately decoupled. Technical artifact identities remain exactly Agent Contract 2.1.0, Appium Runtime 0.5.0, Windows Runtime 0.5.0, CLI 0.2.0, and GitHub tag agent-contract-v2.1.0; those coordinates must not be renamed to 1.1.1. The issue title/body and Project Target Release now reflect the public product name while preserving the frozen technical table and verification contract.

  8. houlianpi commented on Nov 10, 2026

    @houlianpi
    Author

    Independent pipeline validation — PASS

    Target

    • PR Harden resumable Skills Preview publication #73 exact head: 74bd39a0ec548e612cb520487b15011e23ddfcdd
    • gh pr view 73 --repo FSQ-lab/AutoGenesis --json headRefOid: exit 0; remote head matched exactly.
    • Clean detached worktree at that commit; workflow product title confirmed as AutoGenesis 1.1.1 Skills Preview.

    Focused verifier tests

    • Command: python -m unittest tools/test_verify_registry_metadata.py -v
    • Runtime: existing CPython 3.12.14 (temporary PATH alias outside the checkout; no installation)
    • Exit: 0
    • Result: 27 tests run in 0.302s; OK

    Workflow lint and parse

    • Fixed official actionlint release: v1.7.7, darwin/arm64, built with Go 1.23.4.
    • Downloaded the archive and official checksum file from the same official release tag into a task-only temporary directory.
    • Archive: actionlint_1.7.7_darwin_arm64.tar.gz
    • Official/observed SHA-256: 2693315b9093aeacb4ebd91a993fea54fc215057bf0da2659056b4bc033873db
    • Checksum command: sha256sum --check -; exit 0; archive OK before extraction/execution.
    • Version command: actionlint -version; exit 0; version 1.7.7.
    • Command: actionlint .github/workflows/publish-mcp-registry.yml
    • Exit: 0; no findings/output.
    • Read-only fallback syntax command: ruby -rpsych -e 'Psych.parse_file(ARGV.fetch(0)); puts "YAML parse OK"' .github/workflows/publish-mcp-registry.yml
    • Exit: 0; YAML parse OK.

    Cleanliness and boundaries

    • Final git rev-parse HEAD: exit 0; still 74bd39a0ec548e612cb520487b15011e23ddfcdd.
    • Final git status --porcelain=v1 --untracked-files=all: exit 0; empty.
    • Final working-tree and cached git diff --exit-code: exit 0 for both.
    • Generated Python bytecode and the temporary actionlint/tool directory were removed; temp-tool absence check exited 0.
    • Skipped requested checks: none. Workflow dispatch/publication was intentionally not performed because it is outside this read-only acceptance boundary.
    • No source edit, install, user configuration change, commit, push, merge, workflow dispatch, release/package/Registry publication, or GitHub setting change occurred.
  9. houlianpi commented on Nov 10, 2026

    @houlianpi
    Author

    Integration acceptance — PASS

    Accepted exact PR head: 74bd39a0ec548e612cb520487b15011e23ddfcdd.

    Evidence:

    • Independent correctness review: APPROVE.
    • Independent security review: SECURITY_APPROVE.
    • Focused release verifier: python -m unittest tools/test_verify_registry_metadata.py -v — 27/27 passed.
    • Pinned official actionlint v1.7.7, archive SHA-256 2693315b9093aeacb4ebd91a993fea54fc215057bf0da2659056b4bc033873db verified against the official checksum file — workflow lint clean.
    • Independent YAML parser — clean.
    • GitHub checks at the accepted head all passed: Python syntax, Documentation build, Build exact Candidate channel, Extension build, and Official schema/descriptors/wheels.
    • Product identity is AutoGenesis 1.1.1 Skills Preview while technical identities remain agent-contract-v2.1.0, Contract 2.1.0, Appium/Windows 0.5.0, and CLI 0.2.0.
    • No workflow dispatch, package/Registry/GitHub Release publication, or repository setting change occurred.

    This satisfies #71 implementation acceptance. Release dispatch remains blocked by #70's separate operational/admin/public-artifact gates, including enabling immutable releases.

  10. houlianpi commented on Nov 10, 2026

    @houlianpi
    Author

    Merged through repository convention in PR #73: #73

    • Accepted head: 74bd39a0ec548e612cb520487b15011e23ddfcdd
    • Main merge commit: 930d3fc2b06c9f34e3246172df8365863d147213
    • Final review: correctness APPROVE, security SECURITY_APPROVE
    • Verification: focused verifier 27/27, pinned verified actionlint v1.7.7 clean, YAML parse clean, required GitHub checks passed
    • Publication/settings: none; [Release]: Publish AutoGenesis 1.1.1 Skills Preview atomically #70 release/admin/public-artifact gates remain open

    #71 acceptance is Passed and implementation is Done.

  11. added
    acceptance:passedIndependent acceptance passed with linked evidence
    and removed
    acceptance:pendingAcceptance evidence has not passed yet
    on Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    acceptance:passedIndependent acceptance passed with linked evidencearea:packagingPython packages, entrypoints, publishing, and registry metadatapriority:p0Must be addressed before the next Windows Runtime compatibility releaserisk:highRequires explicit risk and rollback reviewsecurity-reviewIndependent security review is requiredtype:capabilityUser-visible capability with executable acceptance criteria

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions