feat: add exact public plugin fixture matrix

.github/workflows/public-fixtures.yml

@@ -0,0 +1,56 @@
+name: Public plugin fixture matrix
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - .github/workflows/public-fixtures.yml
+      - .dockerignore
+      - Dockerfile
+      - package.json
+      - scripts/falsify-public-fixtures.mjs
+      - src/**
+      - test/public-fixtures.test.mjs
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: public-fixtures-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  strict-public-fixtures:
+    name: Ten public plugins on two released versions
+    runs-on: ubuntu-24.04
+    timeout-minutes: 60
+    env:
+      CODEX_CURRENT_VERSION: 0.147.0
+      CODEX_PRIOR_VERSION: 0.146.1
+      CODEX_PUBLIC_FIXTURE_OUTPUT_ROOT: artifacts/public-fixtures
+    steps:
+      - name: Check out source without credentials
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
+        with:
+          persist-credentials: false
+
+      - name: Set up Node 24
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020
+        with:
+          node-version: 24.19.0
+
+      - name: Falsify exact public plugin fixtures
+        run: npm run falsify:public
+
+      - name: Upload public-fixture evidence
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
+        with:
+          name: public-fixture-evidence-${{ github.run_id }}-${{ github.run_attempt }}
+          path: artifacts/public-fixtures
+          if-no-files-found: error
+          retention-days: 90
+          compression-level: 9
+          include-hidden-files: false

README.md

@@ -31,11 +31,13 @@ about their effective runtime behavior.
 
 ## No-execution boundary
 
-The probe never sends a model request and never executes plugin hooks, MCP
-servers, plugin scripts, apps, or authentication flows. It disables remote
+The checker never sends a model request or asks Codex to execute plugin hooks,
+MCP servers, plugin scripts, apps, or authentication flows. It disables remote
 plugin discovery and does not mount personal Codex or agent state into strict
-mode. Synthetic hook and MCP commands are execution sentinels: if Codex starts
-either command during the falsifier, the run fails before isolation cleanup.
+mode. In the exact synthetic fixture, hook and MCP commands are execution
+sentinels: if Codex starts either command during the falsifier, the run fails
+before isolation cleanup. Third-party fixture checks rely on this bounded API
+path and do not claim universal per-fixture execution tracing.
 
 Network access is permitted only while Docker prepares an image containing the
 exact released Codex package. The complete strict probe then runs once with
@@ -60,9 +62,11 @@ release reference exists while the repository is on HOLD. Once the release
 gates pass, consumers should pin the Action to a reviewed full commit SHA—not a
 floating branch or tag.
 
-Action inputs mirror the CLI: `marketplace-root`, `plugin`, `codex-version`,
-`codex`, `cwd`, `output`, and `isolation`. The outputs are `status`, the full
-`receipt`, and the observed `codex-version`.
+Action inputs cover the general check: `marketplace-root`, `plugin`,
+`codex-version`, `codex`, `cwd`, `output`, and `isolation`. The fixed public
+matrix additionally uses CLI-only `expected-plugin-root` and
+`expected-plugin-version` gates. Action outputs are `status`, the full receipt,
+and the observed `codex-version`.
 
 ## CLI diagnostic example
 

docs/evidence/application-readiness.md

@@ -0,0 +1,82 @@
+# Codex for Open Source application readiness
+
+## Decision
+
+`HOLD — NOT READY TO SUBMIT`.
+
+The repository is still private, the public-fixture runtime gate is incomplete,
+and there is no independent retained usage yet. Submitting now would establish
+that Huy created a project, but it would not establish the usage, ecosystem
+importance, or ongoing maintainer responsibility that OpenAI asks applicants
+to explain.
+
+OpenAI's current form says eligible applicants maintain active open-source
+projects and that reviewers consider meaningful usage, broad adoption or clear
+ecosystem importance, plus evidence of active maintenance such as PR review,
+issue triage and release management. There is no published star threshold.
+Source: [Codex for Open Source](https://openai.com/form/codex-for-oss/).
+
+## Current evidence
+
+| Gate | Current state | Required evidence |
+| --- | --- | --- |
+| Public repository and profile | `HOLD` | Public GitHub profile and public repository URL |
+| Useful, bounded technical job | `PASS` | Real released-Codex loader evidence with no model/auth dependency |
+| Synthetic strict boundary | `PASS` | Private Linux receipts for both target releases |
+| Public-plugin compatibility | `UNRUN` | Reconciled 10-repository, 20-cell strict matrix |
+| Independent retained use | `0` | External repositories keep the workflow enabled |
+| Unknown regression caught | `1 local defect` | Publicly linkable regression/issue accepted or reproduced externally |
+| Releases | `0 public releases` | Maintained tagged releases and release notes |
+| Ongoing maintainer duties | `INSUFFICIENT` | Public issue triage, PR review and release work over time |
+| Application | `NOT SUBMITTED` | All required form fields bound to public evidence |
+
+The local defect was real: an unmodified Claude-compatible marketplace loaded
+in Codex while the checker rejected it. The fix is useful product evidence, but
+it is not independent adoption because Huy found it while testing his own tool.
+
+## Conservative internal submission gates
+
+These are project safeguards, not claims that OpenAI publishes numeric rules.
+Submit only after all are observed:
+
+1. The technical 10-repository matrix passes or honestly records and explains
+   every version/API incompatibility, with no unexplained tool error.
+2. The repository is public with a signed-off `v0.1.0` release, public CI and
+   reproducible evidence links.
+3. At least three unrelated plugin repositories retain the check in their
+   default-branch workflow.
+4. The retained integrations remain green across two consecutive Codex release
+   updates, or one catches a real regression that an upstream maintainer accepts
+   or independently reproduces.
+5. At least 60 days of public maintenance evidence exists, including issue
+   triage, review of an external contribution, and two release decisions.
+6. Every application claim fits one of the public links in this directory; no
+   stars, downloads, adoption or maintainer duty is inferred from local work.
+
+## Exact form payload still required
+
+The current form requests:
+
+- first and last name;
+- the email associated with the applicant's ChatGPT account;
+- public GitHub username and public repository URL;
+- primary or core maintainer role;
+- a maximum-500-character qualification explanation;
+- optional interest in Codex Security and API credits;
+- OpenAI organization ID and a maximum-500-character API-credit use case;
+- an optional maximum-500-character final note.
+
+Email and OpenAI organization ID are identity-linked inputs. They will be bound
+at the submission boundary and never inferred from credentials or unrelated
+local files.
+
+## Next evidence-producing actions
+
+1. Finish and privately reconcile the strict public-fixture matrix.
+2. Publish only after security/code review and all technical gates pass.
+3. Offer a bounded integration to maintainers whose immutable fixtures were
+   tested; do not open promotional issues or claim endorsement.
+4. Record retained workflows, maintainer replies, defects and release upkeep in
+   the maintainer-validation ledger.
+5. Re-evaluate the internal submission gates monthly. A green local build or a
+   star count alone never changes this decision.

docs/evidence/public-fixture-matrix.md

@@ -0,0 +1,127 @@
+# Public plugin fixture matrix evidence
+
+## Decision
+
+`HOLD`. The immutable ten-repository input audit is complete, but the strict
+Linux matrix has not run yet. This document records inputs and observed output;
+it does not turn a static compatibility expectation into a runtime result.
+
+## Fixed inputs
+
+Each repository is fetched at one full commit SHA without credentials. The
+target matrix is Codex `0.147.0` and `0.146.1`, for 20 independent strict cells.
+
+| Repository | Commit | Marketplace root | Plugin | Version | Preparation | Expected kinds | License | Strict result |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| [`bitrouter/bitrouter`](https://github.com/bitrouter/bitrouter) | [`678384888b73fc290ce4ce503a8a7f2a5cbf6da8`](https://github.com/bitrouter/bitrouter/commit/678384888b73fc290ce4ce503a8a7f2a5cbf6da8) | `.` | `bitrouter` | `0.1.0` | `DIRECT` | skill, MCP | Apache-2.0 | `UNRUN` |
+| [`Cassette-Editor/oh-my-cassette`](https://github.com/Cassette-Editor/oh-my-cassette) | [`cdad1fd2f62544b65a01ad00f74b19fe3ce4ca32`](https://github.com/Cassette-Editor/oh-my-cassette/commit/cdad1fd2f62544b65a01ad00f74b19fe3ce4ca32) | `.` | `oh-my-cassette` | `0.4.14` | `STATIC_ADAPTER:local-source-v1` | skill, MCP | MIT | `UNRUN` |
+| [`mostlyharmless-ai/watercooler`](https://github.com/mostlyharmless-ai/watercooler) | [`a5efa89df02e7796e20881fef4847f129d84d367`](https://github.com/mostlyharmless-ai/watercooler/commit/a5efa89df02e7796e20881fef4847f129d84d367) | `.` | `watercooler` | `0.5.6` | `DIRECT` | skill, MCP | Apache-2.0 | `UNRUN` |
+| [`commercetools/commercetools-ai-plugins`](https://github.com/commercetools/commercetools-ai-plugins) | [`440d6bd56eb2969b6a0dd41e3fcff286def0787c`](https://github.com/commercetools/commercetools-ai-plugins/commit/440d6bd56eb2969b6a0dd41e3fcff286def0787c) | `.` | `commercetools` | `0.14.0` | `DIRECT` | skill, MCP | CC-BY-4.0 | `UNRUN` |
+| [`agentis-tools/ctx`](https://github.com/agentis-tools/ctx) | [`1782436e0ebf8d95ef4c086d94351698c464c4ee`](https://github.com/agentis-tools/ctx/commit/1782436e0ebf8d95ef4c086d94351698c464c4ee) | `plugins/codex/ctx` | `ctx` | `0.4.0` | `DIRECT` | skill, hook | Apache-2.0 OR MIT | `UNRUN` |
+| [`agentmail-to/agentmail-plugins`](https://github.com/agentmail-to/agentmail-plugins) | [`134887caf9375229415e09c760ae31baa4cc1ec3`](https://github.com/agentmail-to/agentmail-plugins/commit/134887caf9375229415e09c760ae31baa4cc1ec3) | `.` | `agentmail` | `0.3.0` | `DIRECT` | skill, MCP | MIT | `UNRUN` |
+| [`ujjwalredd/sarathi`](https://github.com/ujjwalredd/sarathi) | [`08a51154a2f30af3eb4f6acb11115b9db912c5f8`](https://github.com/ujjwalredd/sarathi/commit/08a51154a2f30af3eb4f6acb11115b9db912c5f8) | `.` | `sarathi` | `0.6.0` | `DIRECT` | skill | MIT | `UNRUN` |
+| [`sofus-nl/cc-plugin-codex`](https://github.com/sofus-nl/cc-plugin-codex) | [`cc5123f7fa18db9c38f838a9b70119e5a0a6847c`](https://github.com/sofus-nl/cc-plugin-codex/commit/cc5123f7fa18db9c38f838a9b70119e5a0a6847c) | `.` | `cc-plugin-codex` | `0.1.1` | `DIRECT` | skill, hook | Apache-2.0 + NOTICE | `UNRUN` |
+| [`RMI/speedy-skills`](https://github.com/RMI/speedy-skills) | [`e983f800056a12b63fd60d5148538f98aaafe643`](https://github.com/RMI/speedy-skills/commit/e983f800056a12b63fd60d5148538f98aaafe643) | `.` | `example-minimal` | `0.1.0` | `DIRECT` | skill | MIT | `UNRUN` |
+| [`roadrunner-tuff/roadrunner-admin-plugin`](https://github.com/roadrunner-tuff/roadrunner-admin-plugin) | [`8e130c07656c8f9db8bf5431332c9aed60a4b133`](https://github.com/roadrunner-tuff/roadrunner-admin-plugin/commit/8e130c07656c8f9db8bf5431332c9aed60a4b133) | `.` | `roadrunner-admin` | `0.1.0` | `DIRECT` | skill, MCP | Apache-2.0 | `UNRUN` |
+
+`local-source-v1` may change only `/plugins/0/source` in
+`.agents/plugins/marketplace.json` to `{"source":"local","path":"./"}`.
+The evidence ledger must retain the original and adapted SHA-256 values. No
+other fixture receives a rewrite.
+
+## Safety and evidence boundary
+
+- The runner must extract immutable commit archives into owned temporary state
+  with traversal and symlink escape checks.
+- It must never execute fixture code, scripts, builds, package managers, hooks,
+  MCP servers, apps, authentication flows, or models. The public-fixture cells
+  make only install/list and declaration/discovery requests; they do not call a
+  capability runtime endpoint.
+- Every Codex probe must use the existing read-only, network-denied,
+  host-state-denied strict boundary.
+- The retained artifact may contain source/tree/adapter hashes, relative
+  receipt names, sanitized receipts, and a summary. It must contain no upstream
+  source.
+- On trusted GitHub Actions runs, the summary binds the exact main-branch
+  commit, event, run URL, run attempt, and intended artifact name. The assigned
+  artifact ID is reconciled after upload because GitHub creates it only then.
+- A truthful plugin receipt may be `FAIL`. A fetch, isolation, tool, identity,
+  privacy, or ledger error must fail the matrix and cannot be relabeled as a
+  compatibility result.
+- Every prepared tree must match its audited checkout and marketplace hashes.
+  The marketplace manifest must resolve the named plugin to the exact audited
+  local subtree, whose plugin manifest must match the fixed name and version.
+  Every receipt must contain the exact audited capability keys and evidence
+  sources, not merely one capability of each expected kind.
+
+The strict public cells do not add fixture-specific execution sentinels to
+third-party repositories. Their non-execution claim is therefore bounded to
+the checker request path above and the earlier exact-version synthetic run,
+whose hook and MCP sentinels remained absent. It is not evidence that arbitrary
+third-party commands could never attempt a side effect if Codex changed loader
+behavior.
+
+## Non-certifying diagnostics
+
+Before the fixed runner existed, direct environment probes observed both target
+Codex versions successfully load `bitrouter/bitrouter`. The same probes exposed
+a checker defect on the unmodified `.claude-plugin` marketplace in
+`RMI/speedy-skills`; Codex itself installed it, while the checker incorrectly
+required an `.agents` manifest. The checker now follows Codex's manifest
+precedence, and the unmodified fixture passes on both versions.
+
+These environment observations helped falsify the implementation. They do not
+satisfy the strict 10-repository gate.
+
+After the fixed runner was implemented, a complete local environment matrix
+ran all 20 repository/version cells against the prepared `0.147.0` and
+`0.146.1` binaries. All 20 returned exit `0` and `PASS` with exact
+repository/plugin/marketplace/source-subtree/version identities. This was
+rerun after the exact runtime root/version gates were added. Every declared
+skill was `DISCOVERED_EFFECTIVE`; the `ctx` and `cc-plugin-codex` hooks were
+`DISCOVERED_UNTRUSTED`; every MCP declaration was `DECLARED_ONLY`. No app or
+unexpected capability kind appeared. The temporary checkouts and receipts were
+removed after reconciliation.
+
+That result is a full loader diagnostic, but its `env` isolation does not deny
+network or personal host state. It is not counted in the strict result column.
+
+On 2026-08-10, the production preparation path fetched, parsed, extracted,
+license-checked, hashed and cleaned all ten immutable archives without running
+Docker or any upstream code. A root-agent integration then fetched all ten
+archives again, validated each exact marketplace source, plugin subtree,
+manifest name and version, and reproduced the canonical tree hashes below.
+Directory entries are ordered by their UTF-8 bytes, so the hash contract does
+not depend on locale or ICU behavior:
+
+| Fixture | Archive SHA-256 | Extracted checkout SHA-256 | Probed marketplace SHA-256 |
+| --- | --- | --- | --- |
+| `bitrouter` | `cd173128072bc769995a46baf1c9b19b0215214e8719a2f3779d4e3d52a69351` | `ba604cb6d8313594bebbdbc899f930195e92f6c12fa14371764f7dec2e25d4c9` | `ba604cb6d8313594bebbdbc899f930195e92f6c12fa14371764f7dec2e25d4c9` |
+| `oh-my-cassette` | `3d64d09b7fae024d53616d4b04173e1a584f7982d3971fb94d0ea54cfed37287` | `9dffb7f24db16606eeb44f7a23746073716069e63e4cf58a07631c02e1f57177` | `32c159545ca3626c13dfae8f1c833e456584df10c204060539475fd6c301b8e8` |
+| `watercooler` | `aed68325b301e45b422f491865e7c3325c53d966dc89d3294deba45fa280ba3c` | `ad91f57b0605df94a1361aa67c3b35314f5357583aaef7f364b397da5a00ea19` | `ad91f57b0605df94a1361aa67c3b35314f5357583aaef7f364b397da5a00ea19` |
+| `commercetools` | `7e4ac439b75a064a08a6e2a107d1ea8bb0221b974807670cea56c63a6cfd9094` | `749bea75c483aecfcc71dc04919a13e170953996eeba9dfdff16c4f5f49073c0` | `749bea75c483aecfcc71dc04919a13e170953996eeba9dfdff16c4f5f49073c0` |
+| `ctx` | `5ef97584aacb6874cede5780ee47d137597a6ebd1fd3ecf04f0e97c79dfd8dfd` | `7f6934a57be05a126b968a5c5d346fb9d3150bdb6a57e199d2274204c94337eb` | `7b3212dbd512ee0bbf7f8c3c2b69c86bdba41ed55f9f71e5f69e63dc0cce49f7` |
+| `agentmail` | `ec595bfaf2e7201ade5e8c5902b424caa05948d1c6558e3f53be3468f02eca14` | `97a82ec2aaa745663a6baa0dab16c476858d4ddc47230f2906b26eafbffa6669` | `97a82ec2aaa745663a6baa0dab16c476858d4ddc47230f2906b26eafbffa6669` |
+| `sarathi` | `f4577e9777d44111b1074460eba28987c80db2296f1983a018b391d99d42e7d8` | `da617a7857381c86ef963f85185a1afef851369a52cb4630d9360c21df904599` | `da617a7857381c86ef963f85185a1afef851369a52cb4630d9360c21df904599` |
+| `cc-plugin-codex` | `57cff2045571f47a75231700f496f4aaaa4d780e1904721e5679a5d68f420cc9` | `1da66cadeb01bb4b57f63e522fdc763df98d5ba6e2034e04599b02c1e394daed` | `1da66cadeb01bb4b57f63e522fdc763df98d5ba6e2034e04599b02c1e394daed` |
+| `speedy-skills` | `6734bb6adb707b623cbb22a9a8c12c571d7fbe51b768cc066055aa86da6e6da0` | `d39797da9765bf1d822887dc6735f186d4bfc199279558817cf6eef375aab1c2` | `d39797da9765bf1d822887dc6735f186d4bfc199279558817cf6eef375aab1c2` |
+| `roadrunner-admin` | `a54ddc226d7f2af17d867b82dd83ff0dfa330abb9dc5e6f80b3cf0d639ad4650` | `33eaa8e5aef5449d496771317d1f40205dabf06fa6d6041e1e142e61e651f5d1` | `33eaa8e5aef5449d496771317d1f40205dabf06fa6d6041e1e142e61e651f5d1` |
+
+For `oh-my-cassette`, the observed adapter input is
+`d5c629f3a3b8dd2cdf560963c26b1c5e9dc062045fef13cafca178e7b1bd3d3f`
+and its adapted output is
+`bbecf8a43d3e993b8506f497a416a0cd83e535a18f3f647e79d6d2458fd6c7b1`.
+The differing marketplace hash is therefore expected and bounded. These are
+preparation receipts, not Codex compatibility receipts.
+
+## Publication gate
+
+Change this decision from `HOLD` only after all 20 strict cells are retained,
+validated against their exact repository/tree/plugin/version/capability
+identities, scanned for private paths, and independently reconciled with the
+immutable inputs above. A code-`1` receipt remains plain `FAIL`/`HOLD`; it does
+not establish a version or API incompatibility without separate causal
+evidence. The private workflow retains its artifact for 90 days. Before a
+public release, the sanitized receipts and summary must also become durable
+release evidence rather than relying on an expiring Actions URL. Stars, a green
+workflow, or a partial matrix do not substitute for those receipts.

docs/evidence/technical-falsifier.md

@@ -115,7 +115,7 @@ strict gate.
 explicit reason
 `strict released integration requires CODEX_RELEASED_FALSIFIER_OPT_IN=1`.
 
-The complete `npm test` suite observed 122 passing tests, zero failures, and
+The complete `npm test` suite observed 158 passing tests, zero failures, and
 the same one strict Linux test skipped.
 
 With both exact version variables set, `npm run falsify` exited `1` with: