Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@ produces a deterministic JSON receipt from Codex-owned install, declaration,
skill-registry, and hook-registry responses.

This repository is in pre-release validation. The current project decision is
**HOLD**: the bounded synthetic falsifier has passed private strict Linux CI,
but the ten-public-fixture gate is **UNRUN (0/10)**. There is no adoption or
production-readiness claim.
**HOLD for adoption/application claims**: the bounded synthetic falsifier and
the ten-public-fixture strict matrix have passed private Linux CI (`10/10`
fixtures, `20/20` cells), but no public immutable release or independent use
exists yet. There is no adoption or production-readiness claim.

## What the receipt proves

Expand Down Expand Up @@ -145,10 +146,13 @@ unless a trusted operator explicitly sets
that expensive test; its single evidence-producing step is `npm run falsify`.

It runs positive and deliberately disabled-skill lanes for both versions and
writes an evidence ledger. Even after that bounded check passes, its summary
remains `HOLD` while the separate public-fixture matrix is `UNRUN (0/10)`.
writes an evidence ledger. That bounded artifact remains `HOLD` because it does
not ingest the separate public-fixture result. The current project ledger links
the independently reconciled public matrix `PASS` (`10/10`, `20/20`) while
retaining `HOLD` for publication history, adoption and application readiness.
See [`docs/evidence/technical-falsifier.md`](docs/evidence/technical-falsifier.md)
for the observed strict boundary and the remaining public-fixture hold.
and [`docs/evidence/public-fixture-matrix.md`](docs/evidence/public-fixture-matrix.md)
for the two evidence boundaries.

## Development

Expand Down
20 changes: 12 additions & 8 deletions docs/evidence/application-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@

`HOLD — NOT READY TO SUBMIT`.

The repository is still private, the public-fixture runtime gate is incomplete,
and there is no independent retained usage yet. Submitting now would establish
that Huy created a project, but it would not establish the usage, ecosystem
importance, or ongoing maintainer responsibility that OpenAI asks applicants
to explain.
The repository is still private and there is no independent retained usage or
public maintenance history yet. The technical strict-Linux and public-fixture
gates now pass, but submitting now would establish only that Huy created and
validated a project. It would not establish the usage, ecosystem importance,
or ongoing maintainer responsibility that OpenAI asks applicants to explain.

OpenAI's current form says eligible applicants maintain active open-source
projects and that reviewers consider meaningful usage, broad adoption or clear
Expand All @@ -23,13 +23,17 @@ Source: [Codex for Open Source](https://openai.com/form/codex-for-oss/).
| Public repository and profile | `HOLD` | Public GitHub profile and public repository URL |
| Useful, bounded technical job | `PASS` | Real released-Codex loader evidence with no model/auth dependency |
| Synthetic strict boundary | `PASS` | Private Linux receipts for both target releases |
| Public-plugin compatibility | `UNRUN` | Reconciled 10-repository, 20-cell strict matrix |
| Public-plugin compatibility | `PASS` | Reconciled 10-repository, 20-cell strict matrix |
| Independent retained use | `0` | External repositories keep the workflow enabled |
| Unknown regression caught | `1 local defect` | Publicly linkable regression/issue accepted or reproduced externally |
| Releases | `0 public releases` | Maintained tagged releases and release notes |
| Ongoing maintainer duties | `INSUFFICIENT` | Public issue triage, PR review and release work over time |
| Application | `NOT SUBMITTED` | All required form fields bound to public evidence |

The technical compatibility result is bound to the [reconciled private Linux
run and artifact ledger](public-fixture-matrix.md#strict-linux-observation).
It is publication evidence, not usage or maintainer-role evidence.

The local defect was real: an unmodified Claude-compatible marketplace loaded
in Codex while the checker rejected it. The fix is useful product evidence, but
it is not independent adoption because Huy found it while testing his own tool.
Expand Down Expand Up @@ -72,8 +76,8 @@ local files.

## Next evidence-producing actions

1. Finish and privately reconcile the strict public-fixture matrix.
2. Publish only after security/code review and all technical gates pass.
1. Publish the reviewed `v0.1.0` release with durable sanitized evidence.
2. Keep the application on hold while independent use and maintenance accrue.
3. Offer a bounded integration to maintainers whose immutable fixtures were
tested; do not open promotional issues or claim endorsement.
4. Record retained workflows, maintainer replies, defects and release upkeep in
Expand Down
75 changes: 53 additions & 22 deletions docs/evidence/public-fixture-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@

## Decision

`HOLD`. The immutable ten-repository input audit is complete, but the strict
Linux matrix has not run yet. This document records inputs and observed output;
it does not turn a static compatibility expectation into a runtime result.
`PASS — 10/10 fixtures, 20/20 strict cells`. The private main-branch Linux run
completed successfully and its retained artifact was independently reconciled
against every immutable input and production receipt validator. This clears the
technical public-fixture gate; it does not establish adoption, ecosystem
importance, or ongoing maintainer work.

## Fixed inputs

Expand All @@ -13,16 +15,16 @@ target matrix is Codex `0.147.0` and `0.146.1`, for 20 independent strict cells.

| Repository | Commit | Marketplace root | Plugin | Version | Preparation | Expected kinds | License | Strict result |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| [`bitrouter/bitrouter`](https://github.com/bitrouter/bitrouter) | [`678384888b73fc290ce4ce503a8a7f2a5cbf6da8`](https://github.com/bitrouter/bitrouter/commit/678384888b73fc290ce4ce503a8a7f2a5cbf6da8) | `.` | `bitrouter` | `0.1.0` | `DIRECT` | skill, MCP | Apache-2.0 | `UNRUN` |
| [`Cassette-Editor/oh-my-cassette`](https://github.com/Cassette-Editor/oh-my-cassette) | [`cdad1fd2f62544b65a01ad00f74b19fe3ce4ca32`](https://github.com/Cassette-Editor/oh-my-cassette/commit/cdad1fd2f62544b65a01ad00f74b19fe3ce4ca32) | `.` | `oh-my-cassette` | `0.4.14` | `STATIC_ADAPTER:local-source-v1` | skill, MCP | MIT | `UNRUN` |
| [`mostlyharmless-ai/watercooler`](https://github.com/mostlyharmless-ai/watercooler) | [`a5efa89df02e7796e20881fef4847f129d84d367`](https://github.com/mostlyharmless-ai/watercooler/commit/a5efa89df02e7796e20881fef4847f129d84d367) | `.` | `watercooler` | `0.5.6` | `DIRECT` | skill, MCP | Apache-2.0 | `UNRUN` |
| [`commercetools/commercetools-ai-plugins`](https://github.com/commercetools/commercetools-ai-plugins) | [`440d6bd56eb2969b6a0dd41e3fcff286def0787c`](https://github.com/commercetools/commercetools-ai-plugins/commit/440d6bd56eb2969b6a0dd41e3fcff286def0787c) | `.` | `commercetools` | `0.14.0` | `DIRECT` | skill, MCP | CC-BY-4.0 | `UNRUN` |
| [`agentis-tools/ctx`](https://github.com/agentis-tools/ctx) | [`1782436e0ebf8d95ef4c086d94351698c464c4ee`](https://github.com/agentis-tools/ctx/commit/1782436e0ebf8d95ef4c086d94351698c464c4ee) | `plugins/codex/ctx` | `ctx` | `0.4.0` | `DIRECT` | skill, hook | Apache-2.0 OR MIT | `UNRUN` |
| [`agentmail-to/agentmail-plugins`](https://github.com/agentmail-to/agentmail-plugins) | [`134887caf9375229415e09c760ae31baa4cc1ec3`](https://github.com/agentmail-to/agentmail-plugins/commit/134887caf9375229415e09c760ae31baa4cc1ec3) | `.` | `agentmail` | `0.3.0` | `DIRECT` | skill, MCP | MIT | `UNRUN` |
| [`ujjwalredd/sarathi`](https://github.com/ujjwalredd/sarathi) | [`08a51154a2f30af3eb4f6acb11115b9db912c5f8`](https://github.com/ujjwalredd/sarathi/commit/08a51154a2f30af3eb4f6acb11115b9db912c5f8) | `.` | `sarathi` | `0.6.0` | `DIRECT` | skill | MIT | `UNRUN` |
| [`sofus-nl/cc-plugin-codex`](https://github.com/sofus-nl/cc-plugin-codex) | [`cc5123f7fa18db9c38f838a9b70119e5a0a6847c`](https://github.com/sofus-nl/cc-plugin-codex/commit/cc5123f7fa18db9c38f838a9b70119e5a0a6847c) | `.` | `cc-plugin-codex` | `0.1.1` | `DIRECT` | skill, hook | Apache-2.0 + NOTICE | `UNRUN` |
| [`RMI/speedy-skills`](https://github.com/RMI/speedy-skills) | [`e983f800056a12b63fd60d5148538f98aaafe643`](https://github.com/RMI/speedy-skills/commit/e983f800056a12b63fd60d5148538f98aaafe643) | `.` | `example-minimal` | `0.1.0` | `DIRECT` | skill | MIT | `UNRUN` |
| [`roadrunner-tuff/roadrunner-admin-plugin`](https://github.com/roadrunner-tuff/roadrunner-admin-plugin) | [`8e130c07656c8f9db8bf5431332c9aed60a4b133`](https://github.com/roadrunner-tuff/roadrunner-admin-plugin/commit/8e130c07656c8f9db8bf5431332c9aed60a4b133) | `.` | `roadrunner-admin` | `0.1.0` | `DIRECT` | skill, MCP | Apache-2.0 | `UNRUN` |
| [`bitrouter/bitrouter`](https://github.com/bitrouter/bitrouter) | [`678384888b73fc290ce4ce503a8a7f2a5cbf6da8`](https://github.com/bitrouter/bitrouter/commit/678384888b73fc290ce4ce503a8a7f2a5cbf6da8) | `.` | `bitrouter` | `0.1.0` | `DIRECT` | skill, MCP | Apache-2.0 | `PASS` |
| [`Cassette-Editor/oh-my-cassette`](https://github.com/Cassette-Editor/oh-my-cassette) | [`cdad1fd2f62544b65a01ad00f74b19fe3ce4ca32`](https://github.com/Cassette-Editor/oh-my-cassette/commit/cdad1fd2f62544b65a01ad00f74b19fe3ce4ca32) | `.` | `oh-my-cassette` | `0.4.14` | `STATIC_ADAPTER:local-source-v1` | skill, MCP | MIT | `PASS` |
| [`mostlyharmless-ai/watercooler`](https://github.com/mostlyharmless-ai/watercooler) | [`a5efa89df02e7796e20881fef4847f129d84d367`](https://github.com/mostlyharmless-ai/watercooler/commit/a5efa89df02e7796e20881fef4847f129d84d367) | `.` | `watercooler` | `0.5.6` | `DIRECT` | skill, MCP | Apache-2.0 | `PASS` |
| [`commercetools/commercetools-ai-plugins`](https://github.com/commercetools/commercetools-ai-plugins) | [`440d6bd56eb2969b6a0dd41e3fcff286def0787c`](https://github.com/commercetools/commercetools-ai-plugins/commit/440d6bd56eb2969b6a0dd41e3fcff286def0787c) | `.` | `commercetools` | `0.14.0` | `DIRECT` | skill, MCP | CC-BY-4.0 | `PASS` |
| [`agentis-tools/ctx`](https://github.com/agentis-tools/ctx) | [`1782436e0ebf8d95ef4c086d94351698c464c4ee`](https://github.com/agentis-tools/ctx/commit/1782436e0ebf8d95ef4c086d94351698c464c4ee) | `plugins/codex/ctx` | `ctx` | `0.4.0` | `DIRECT` | skill, hook | Apache-2.0 OR MIT | `PASS` |
| [`agentmail-to/agentmail-plugins`](https://github.com/agentmail-to/agentmail-plugins) | [`134887caf9375229415e09c760ae31baa4cc1ec3`](https://github.com/agentmail-to/agentmail-plugins/commit/134887caf9375229415e09c760ae31baa4cc1ec3) | `.` | `agentmail` | `0.3.0` | `DIRECT` | skill, MCP | MIT | `PASS` |
| [`ujjwalredd/sarathi`](https://github.com/ujjwalredd/sarathi) | [`08a51154a2f30af3eb4f6acb11115b9db912c5f8`](https://github.com/ujjwalredd/sarathi/commit/08a51154a2f30af3eb4f6acb11115b9db912c5f8) | `.` | `sarathi` | `0.6.0` | `DIRECT` | skill | MIT | `PASS` |
| [`sofus-nl/cc-plugin-codex`](https://github.com/sofus-nl/cc-plugin-codex) | [`cc5123f7fa18db9c38f838a9b70119e5a0a6847c`](https://github.com/sofus-nl/cc-plugin-codex/commit/cc5123f7fa18db9c38f838a9b70119e5a0a6847c) | `.` | `cc-plugin-codex` | `0.1.1` | `DIRECT` | skill, hook | Apache-2.0 + NOTICE | `PASS` |
| [`RMI/speedy-skills`](https://github.com/RMI/speedy-skills) | [`e983f800056a12b63fd60d5148538f98aaafe643`](https://github.com/RMI/speedy-skills/commit/e983f800056a12b63fd60d5148538f98aaafe643) | `.` | `example-minimal` | `0.1.0` | `DIRECT` | skill | MIT | `PASS` |
| [`roadrunner-tuff/roadrunner-admin-plugin`](https://github.com/roadrunner-tuff/roadrunner-admin-plugin) | [`8e130c07656c8f9db8bf5431332c9aed60a4b133`](https://github.com/roadrunner-tuff/roadrunner-admin-plugin/commit/8e130c07656c8f9db8bf5431332c9aed60a4b133) | `.` | `roadrunner-admin` | `0.1.0` | `DIRECT` | skill, MCP | Apache-2.0 | `PASS` |

`local-source-v1` may change only `/plugins/0/source` in
`.agents/plugins/marketplace.json` to `{"source":"local","path":"./"}`.
Expand Down Expand Up @@ -114,14 +116,43 @@ and its adapted output is
The differing marketplace hash is therefore expected and bounded. These are
preparation receipts, not Codex compatibility receipts.

## Strict Linux observation

The [main-branch public fixture run 31387585244](https://github.com/builtbyhuy/codex-plugin-check/actions/runs/31387585244)
completed successfully on `ubuntu-24.04` and Node `24.19.0` at merge commit
`3285a65bab2ba805665c6be4e4349874fc7be417`. Its job ran from
`2026-08-10T12:20:54Z` to `2026-08-10T12:21:55Z`; the evidence-producing step
completed in 47 seconds. The same commit also passed [ordinary CI](https://github.com/builtbyhuy/codex-plugin-check/actions/runs/31387585232)
and the [released-Codex synthetic falsifier](https://github.com/builtbyhuy/codex-plugin-check/actions/runs/31387585316).

The strict run produced artifact `public-fixture-evidence-31387585244-1`
(artifact ID `9062332756`, GitHub digest
`sha256:f4a06bb022b4c916ff5b1db3b14c8514f69b327095d3f16a1303128952d948b9`,
expires `2026-11-08T12:20:52Z`). Its 21 files contain one summary and all 20
expected receipts. The summary SHA-256 is
`7db42975282375feb5294d5d2be0c3964bfda127b5de4a6c02745185449492cc`.

Independent reconciliation re-read every file through the production public
receipt validator and compared the summary to the fixed fixture registry. It
observed:

- `status: PASS`, `10/10` fixtures, and `20/20` passing strict cells;
- exact main-branch commit, event, run URL, run attempt, and artifact name;
- exact archive, checkout, adapted marketplace, plugin root and plugin version
identities for every fixture;
- exact capability counts, keys, evidence sources and allowed states for both
Codex versions;
- `DISCOVERED_EFFECTIVE` for every skill, `DISCOVERED_UNTRUSTED` for the five
declared hooks, and `DECLARED_ONLY` for every MCP declaration;
- no unexpected file, capability, app, personal path, temporary path, or
invented `VERSION_OR_API_INCOMPATIBLE` cause.

## Publication gate

Change this decision from `HOLD` only after all 20 strict cells are retained,
validated against their exact repository/tree/plugin/version/capability
identities, scanned for private paths, and independently reconciled with the
immutable inputs above. A code-`1` receipt remains plain `FAIL`/`HOLD`; it does
not establish a version or API incompatibility without separate causal
evidence. The private workflow retains its artifact for 90 days. Before a
public release, the sanitized receipts and summary must also become durable
release evidence rather than relying on an expiring Actions URL. Stars, a green
workflow, or a partial matrix do not substitute for those receipts.
The technical gate is `PASS`. Before the public release is final, attach the
sanitized receipts and summary as a durable release asset rather than relying
only on the 90-day Actions artifact. A code-`1` receipt must remain plain
`FAIL`/`HOLD`; it cannot establish a version or API incompatibility without
separate causal evidence. This result supports publication of the tool, but it
does not satisfy the independent-use or ongoing-maintainer gates for a Codex
for Open Source application.
25 changes: 14 additions & 11 deletions docs/evidence/technical-falsifier.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@

## Decision and gate ledger

`HOLD` for the project/publication decision. The bounded synthetic falsifier's
orchestration, environment lanes, and private strict Linux run are verified,
but the required ten-public-fixture specification matrix is still `UNRUN`
(`0/10`). A successful bounded command must therefore report
`status: "HOLD"` with `boundedFalsifier: "PASS"`; it must not report the project
as `PASS`.
`PASS` for this bounded synthetic falsifier and `HOLD` for adoption/application
claims. Its orchestration, environment lanes, and private strict Linux run are
verified. The bounded artifact intentionally records the separate public
fixture matrix as `UNRUN (0/10)` because it does not ingest that external
ledger; the separately retained matrix now passes `10/10` fixtures and `20/20`
strict cells. The bounded command must still report `status: "HOLD"` with
`boundedFalsifier: "PASS"` rather than claiming whole-project readiness.

| Gate | Observed status | Evidence boundary |
| --- | --- | --- |
Expand All @@ -16,8 +17,8 @@ as `PASS`.
| Synthetic orchestration and failure gates | `PASS` | Unit/integration tests, without claiming Docker observation |
| Strict Linux boundary | `PASS` | Private GitHub Actions run on `ubuntu-24.04` |
| Strict positive and negative lanes | `PASS` | Four exact receipts across both released versions |
| Public fixture specification matrix | `UNRUN` | `0/10`; outside this bounded falsifier |
| Publication | `HOLD` | Repository remains private until the public-fixture gate passes |
| Public fixture specification matrix | `PASS` | Separate reconciled ledger: `10/10`, `20/20` |
| Publication | `HOLD` | Repository remains private until the reviewed release evidence is durable |

## Version and variant contract

Expand Down Expand Up @@ -172,6 +173,8 @@ Artifact SHA-256 values:
| `codex-0.146.1-negative.json` | `9ac7c8e980bc7157d3125f64204e97e836b1bad316f9c61a18cb32b36c2ba5ea` |
| `falsifier-summary.json` | `dacbcfa08a7888f7fbad27f9ba1855fe759aca87d8c1bebd64f150c947ad3e8d` |

This closes the bounded synthetic strict-Linux uncertainty. It does not close
the product/publication decision: the project remains `HOLD` until the ten
public fixtures are run and honestly reconciled.
This closes the bounded synthetic strict-Linux uncertainty. The later
[public fixture matrix](public-fixture-matrix.md) also closes the technical
`10/10` compatibility gate. The project remains `HOLD` for adoption and Codex
for Open Source application claims until public release and maintenance
evidence exist.
4 changes: 2 additions & 2 deletions docs/superpowers/plans/2026-08-10-codex-plugin-check.md
Original file line number Diff line number Diff line change
Expand Up @@ -522,14 +522,14 @@ Use the prepared released `0.147.0` and `0.146.1` binaries to reconcile each
observed capability set against the immutable source before strict CI. Record
these only as diagnostics; they cannot satisfy the strict gate.

- [ ] **Step 5: Run the strict matrix in the private remote**
- [x] **Step 5: Run the strict matrix in the private remote**

The workflow is main-push/manual only, least privilege, Node 24, pinned Actions,
and always uploads a relative-path evidence ledger. Observe all 20 cells inside
the existing network/host-state-denied boundary. Retain source/tree/adapter
hashes, sanitized receipts, run SHA/URL, and artifact identity.

- [ ] **Step 6: Reconcile and review before publication**
- [x] **Step 6: Reconcile and review before publication**

Independently compare the artifact to immutable source expectations. Mark the
technical public-fixture gate `PASS` only when all ten fixtures have complete
Expand Down