Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b736d20
docs: specify GSAE-E0 Stage-A conformance harness
ndrorchestration Sep 16, 2026
c26dad9
docs: plan GSAE-E0 Stage-A conformance harness
ndrorchestration Sep 16, 2026
52ea1aa
test: include GSAE-E0 experiment tests
ndrorchestration Sep 16, 2026
0dd2b1c
test: define GSAE-E0 fixture schema expectations
ndrorchestration Sep 16, 2026
c472b5c
feat: initialize GSAE-E0 research package
ndrorchestration Sep 16, 2026
a23d1e8
feat: add GSAE-E0 fixture schema
ndrorchestration Sep 16, 2026
d1afe2d
test: define frozen GSAE-E0 fixture identity
ndrorchestration Sep 16, 2026
e4f8f5f
feat: add deterministic GSAE-E0 fixture loader
ndrorchestration Sep 16, 2026
02ac674
feat: freeze GSAE-E0 Stage-A fixture candidate
ndrorchestration Sep 16, 2026
d49fde4
test: define GSAE-E0 semantic classification rules
ndrorchestration Sep 16, 2026
4b7191b
feat: add GSAE-E0 semantic classification
ndrorchestration Sep 16, 2026
2fa683b
test: define GSAE-E0 native observation and source-binding rules
ndrorchestration Sep 16, 2026
59133dd
feat: add GSAE-E0 native conformance observations
ndrorchestration Sep 16, 2026
81f480d
ci: fetch history for GSAE-E0 source binding
ndrorchestration Sep 16, 2026
2d5dc2b
test: define deterministic GSAE-E0 evidence bundle rules
ndrorchestration Sep 16, 2026
40fd990
feat: add deterministic GSAE-E0 evidence bundles
ndrorchestration Sep 16, 2026
e0749fa
test: assert blank result identity at schema boundary
ndrorchestration Sep 16, 2026
903ec63
test: define guarded GSAE-E0 Stage-A runner
ndrorchestration Sep 16, 2026
269b766
feat: add guarded GSAE-E0 Stage-A runner
ndrorchestration Sep 16, 2026
74a10ea
test: preserve overridden synthetic runner manifests
ndrorchestration Sep 16, 2026
2548ec7
docs: bound GSAE-E0 apparatus evidence ceiling
ndrorchestration Sep 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ jobs:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,273 @@
# GSAE-E0 Stage-A Conformance Harness — Design

Date: 2026-09-16
Status: APPROVED DESIGN / SPEC REVIEW PENDING
Repository: `ndrorchestration/agent-control-plane`
Source-under-test lineage: PR #5 exact head `07a09698ca66e8837d04e6ec05b4de3448eced04`
Candidate schema: `agent-control-plane.execution.v1`

## 1. Purpose

Build a research-scoped, deterministic conformance harness for `GSAE-E0` that tests whether the current ACP execution contract can represent frozen ordinary and consequential governance states without semantic overload, runtime-specific reinterpretation, or a core-schema fork.

This apparatus is evidence-producing research infrastructure only. It must not change the ACP contract under test, authorize GSAE-E0 execution, establish portability, establish DGAF/GSAE efficacy, or imply production/security readiness.

## 2. Design principle

The harness measures the contract before attempting to improve it. Stage A imports the exact bound contract implementation and treats missing or ambiguous semantics as results. It must not encode missing governance meanings into free-form `detail`, opaque artifact references, or runtime-specific strings and then count those encodings as structured coverage.

## 3. Scope

In scope:

- deterministic fixture definitions;
- exact source/schema/fixture identity binding;
- native ACP conformance fixtures;
- governance-authority semantic probes `AUTH-01` through `AUTH-10`;
- negative controls for malformed or mismatched inputs;
- explicit exception classification;
- deterministic result-matrix generation;
- machine-readable evidence manifests;
- fail-closed fixture/result validation;
- tests for determinism, identity binding, classification, and negative controls.

Out of scope:

- changing `agent-control-plane.execution.v1` during Stage A;
- adding a second runtime or claiming portability;
- provider/network integrations;
- DGAF/GSAE authorization or efficacy conclusions;
- production/security certification;
- durable or tamper-evident provenance claims;
- autonomous/self-improving runtime behavior;
- schema redesign before the Stage-A evidence package is frozen.

## 4. Repository layout

Use a research-isolated package:

```text
experiments/gsae_e0/
__init__.py
README.md
schema.py
fixtures.py
classify.py
runner.py
manifest.py
fixtures/
stage_a_v1.json
tests/
test_fixture_schema.py
test_classification.py
test_runner.py
test_manifest.py
```

The package may import public ACP contract types from `agent_control_plane.contract`. ACP production modules must not import from `experiments.gsae_e0`.

## 5. Fixed source binding

Every Stage-A evidence bundle must bind repository identity, exact source-under-test SHA, schema version, fixture-set version, fixture-manifest SHA-256, harness commit SHA, test command, and run/result identity.

A source-under-test SHA change invalidates the previous bundle for current-state claims and requires explicit rebind/re-run. Historical results remain provenance.

Initial source under test:

`07a09698ca66e8837d04e6ec05b4de3448eced04`

Its open PR status is metadata only and is not evidence that the implementation is accepted on `main`.

## 6. Fixture model

Each fixture is immutable by identifier within a fixture-set version and contains at minimum:

- `fixture_id`;
- `family`;
- `title`;
- `purpose`;
- `criticality` (`critical` or `noncritical`);
- `required_semantics`;
- `input_spec`;
- `expected_classification_domain`;
- optional expected exception class for negative controls.

Malformed fixture definitions fail closed before execution and produce no conformance result.

### Native conformance fixtures

1. allowed ordinary completion;
2. policy denial;
3. unknown-capability rejection;
4. handler/runtime failure;
5. cancellation;
6. cooperative budget exhaustion;
7. input/output artifact linkage;
8. trace parent/child linkage;
9. policy-decision reference binding;
10. legacy provenance to execution-contract mapping;
11. provenance/contract run-ID mismatch rejection;
12. malformed schema/identity/timestamp/hash/nested-contract rejection.

### Governance-authority probes

- `AUTH-01`: principal / acting identity;
- `AUTH-02`: requested capability;
- `AUTH-03`: resource / target scope;
- `AUTH-04`: operation / action semantics;
- `AUTH-05`: policy identity plus version/hash binding;
- `AUTH-06`: decision identity and explicit allow/deny/conditional semantics;
- `AUTH-07`: lease / authority expiry;
- `AUTH-08`: delegation chain / delegator scope;
- `AUTH-09`: permitted/forbidden actions or authority conditions;
- `AUTH-10`: reason codes plus decision-to-execution provenance linkage.

## 7. Structured-coverage rule

A semantic requirement counts as `STRUCTURED_COVERAGE` only when it has a stable, machine-addressable field or typed composition in the contract under test with a meaning that does not depend on free-form prose or runtime-specific reinterpretation.

The following do not count as structured coverage by themselves:

- `detail` strings;
- arbitrary artifact payload contents;
- overloaded `status` values;
- runtime-specific conventions embedded in identifiers;
- external documentation not represented in the contract instance.

This prevents the harness from manufacturing coverage through generic escape hatches.

## 8. Exception taxonomy

Every non-covered or failed fixture receives exactly one primary class:

- `MISSING_CORE_SEMANTIC`;
- `AMBIGUOUS_SEMANTIC`;
- `RUNTIME_SPECIFIC`;
- `ADAPTER_COMPLEXITY`;
- `NONCRITICAL_EXTENSION`;
- `MALFORMED_INPUT`;
- `IMPLEMENTATION_DEFECT`;
- `PROVENANCE_GAP`.

Explanatory notes may accompany the result, but the primary class remains machine-readable.

## 9. Result model

Each fixture result records result identity, fixture identity, source/schema/fixture binding, disposition, primary exception class when applicable, structured fields used, deterministic evidence summary, error information for rejected/malformed cases, and immutable retry linkage.

A retry receives a new result identity and must never overwrite prior evidence. Timestamps may be retained as provenance metadata but are not semantic ordering authority.

## 10. Aggregate disposition

Stage A may produce only:

- `FEASIBLE_FOR_FROZEN_SCOPE`;
- `CONDITIONALLY_FEASIBLE_NARROW`;
- `NOT_FEASIBLE_FOR_FROZEN_SCOPE`;
- `NOT_ESTABLISHED`.

Rules:

- missing required evidence yields `NOT_ESTABLISHED`;
- a critical `MISSING_CORE_SEMANTIC` or `AMBIGUOUS_SEMANTIC` cannot be overridden by a high coverage percentage;
- `FEASIBLE_FOR_FROZEN_SCOPE` requires structured coverage for every critical frozen semantic plus passing required negative controls;
- `CONDITIONALLY_FEASIBLE_NARROW` requires uncovered items to be noncritical to a clearly stated narrower scope;
- `NOT_FEASIBLE_FOR_FROZEN_SCOPE` applies when one or more critical frozen semantics cannot be represented without violating the structured-coverage rule.

The aggregate is a contract-feasibility result only, never an authorization, safety, efficacy, portability, or production-readiness conclusion.

## 11. Runner behavior

The runner must:

1. validate the fixture manifest;
2. verify exact source/schema binding;
3. compute the fixture-manifest hash;
4. execute native conformance fixtures deterministically;
5. evaluate authority-semantic probes against the typed contract surface;
6. classify each fixture without mutating the contract implementation;
7. emit a stable machine-readable result matrix;
8. derive the bounded aggregate disposition;
9. emit an evidence manifest linking component identities and hashes.

No network access or provider dependency is permitted in Stage A.

## 12. Evidence bundle

The canonical Stage-A evidence bundle consists of:

- frozen fixture manifest;
- fixture-manifest SHA-256;
- source-under-test identity;
- harness identity;
- result matrix;
- aggregate disposition;
- test-output summary;
- evidence manifest linking component identities/hashes.

The bundle format must be deterministic apart from explicitly excluded runtime metadata. Timestamps, if present, must not affect semantic equality of result content.

## 13. Fail-closed rules

- malformed fixture manifest: reject run as `NOT_ESTABLISHED`;
- source/schema mismatch: reject run as `NOT_ESTABLISHED`;
- unexpected fixture execution exception: `IMPLEMENTATION_DEFECT` unless evidence establishes another class;
- missing required result field: reject evidence bundle;
- invalid result classification: reject evidence bundle;
- unknown fixture ID in output: reject evidence bundle;
- duplicate result identity: reject evidence bundle;
- missing critical fixture: aggregate `NOT_ESTABLISHED`;
- attempted free-form escape-hatch substitution: classify the semantic `AMBIGUOUS_SEMANTIC` or `MISSING_CORE_SEMANTIC`, never PASS.

## 14. Testing strategy

Tests establish apparatus behavior only, not GSAE-E0 feasibility.

Required tests:

- valid frozen fixture schema accepted;
- malformed/duplicate fixture definitions rejected where required;
- manifest hash deterministic;
- source-binding mismatch fails closed;
- every taxonomy class reachable through deterministic synthetic cases;
- structured-coverage rule rejects generic `detail`/opaque-artifact substitution;
- native fixtures reproduce already-established ACP contract behavior without changing ACP code;
- result ordering deterministic;
- aggregate disposition obeys criticality rules;
- retries create distinct identities and preserve prior evidence;
- evidence manifest rejects missing/mismatched identities.

Implementation must use TDD: failing apparatus tests precede implementation for each behavior slice.

## 15. Relationship to PR #5

PR #5 remains the exact source-under-test candidate. Stage A does not modify that PR's contract implementation.

The harness branch is based on PR #5 exact head so it can import the candidate contract directly. Harness findings must be reviewed separately from the contract candidate. If Stage A later motivates schema changes, those changes belong in a subsequent explicit contract revision after the Stage-A evidence bundle is frozen.

## 16. Governance relationship

Notion remains the interpreted GSAE-E0 research/governance SSoT. GitHub remains authoritative for exact code, commit, issue, PR, CI, and evidence-artifact identities.

Apparatus completion may advance only to `APPARATUS_READY`. It must not mark:

- `PROTOCOL FROZEN` unless the owning GSAE record explicitly records the freeze;
- `EXECUTION AUTHORIZED` without a separate authorization event;
- `RUN` or scientific `N>0` before an actually authorized experiment;
- portability, efficacy, safety, or production-readiness claims.

## 17. Completion criteria

The subsystem is implementation-complete when:

1. all native fixture families exist in the frozen manifest;
2. `AUTH-01` through `AUTH-10` are represented;
3. deterministic tests pass on the exact apparatus head;
4. source/schema/fixture binding is enforced fail-closed;
5. the evidence manifest is deterministic and validated;
6. negative controls demonstrate malformed evidence rejection;
7. no ACP core/schema file is modified by the harness implementation;
8. repository documentation states the evidence ceiling;
9. GitHub issue #6 records exact apparatus head, test command/result, fixture hash, and `APPARATUS_READY` or narrower status.

These criteria establish measurement-apparatus readiness only.
114 changes: 114 additions & 0 deletions experiments/gsae_e0/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# GSAE-E0 Stage-A Conformance Harness

**Status:** `RESEARCH APPARATUS / EXECUTION NOT AUTHORIZED / NOT RUN / N=0`

**Source under test:** `07a09698ca66e8837d04e6ec05b4de3448eced04`
**Schema:** `agent-control-plane.execution.v1`
**Frozen fixture candidate:** `fixtures/stage_a_v1.json`

This package measures a bounded contract-feasibility question. Passing apparatus tests establish only that the measurement apparatus behaves as tested. They do **not** establish GSAE-E0 feasibility, cross-runtime portability, governance efficacy, safety, security, production readiness, or execution authorization.

## Purpose

Stage A asks whether the bound ACP execution contract can represent the frozen ordinary and consequential governance states with stable structured semantics, without semantic overload, runtime-specific reinterpretation, or a core-schema fork.

The harness deliberately measures the existing contract before any schema repair. Missing or ambiguous governance meanings remain evidence. Values hidden only in free-form `detail`, opaque artifact payloads, overloaded status strings, or runtime-specific identifier conventions do not count as structured authority coverage.

## Apparatus verification

Install the repository and test dependency:

```bash
python -m pip install -e . pytest
```

Verify the research apparatus:

```bash
python -m pytest experiments/gsae_e0/tests -v
```

Verify the full repository:

```bash
python -m pytest
```

The repository CI runs the full suite on Python 3.10, 3.11, 3.12, 3.13, and 3.14.

These commands test the apparatus and already-established ACP behavior. They do not execute the frozen canonical Stage-A experiment.

## Source binding

Before an evidence-producing Stage-A run can proceed, the runner requires the exact source-under-test commit to be an ancestor of the current apparatus head and requires zero changes under `src/agent_control_plane/` relative to that source commit.

The CI workflow fetches full repository history so this binding can be proven rather than assumed.

Changing the source-under-test SHA, schema version, or ACP source package invalidates the previous source binding for current-state claims and requires an explicit rebind/retest. Historical evidence remains provenance.

## Fixture identity

The frozen candidate manifest contains:

- 10 native ACP conformance fixtures (`NATIVE-01` through `NATIVE-10`);
- 8 negative controls (`NEG-01` through `NEG-08`);
- 10 governance-authority semantic probes (`AUTH-01` through `AUTH-10`).

Its canonical SHA-256 is computed from validated, canonical JSON. The content identity is an apparatus/protocol input identity, not an experiment result.

## Execution authorization boundary

`run_stage_a(...)` requires explicit protocol, authorization-record, run, harness-commit, source, schema, and fixture identities. It fails closed when required identities or bindings are absent.

The function records an authorization-record identifier; it does **not** decide whether that external governance record is valid. The owning GSAE control record remains the authority for whether execution is authorized.

Canonical Stage-A execution must not occur until:

1. the owning GSAE record accepts the protocol freeze;
2. a separate execution-authorization record is issued;
3. the exact apparatus/source/fixture identities are bound to that authorization.

No canonical execution command is documented here while the study remains unauthorized.

## Result dispositions

Stage A can produce only one aggregate contract-feasibility disposition:

- `FEASIBLE_FOR_FROZEN_SCOPE`;
- `CONDITIONALLY_FEASIBLE_NARROW`;
- `NOT_FEASIBLE_FOR_FROZEN_SCOPE`;
- `NOT_ESTABLISHED`.

A high numerical coverage rate cannot override a critical missing or ambiguous semantic. Missing required evidence produces `NOT_ESTABLISHED`; it is never imputed.

These dispositions are not authorization, safety, efficacy, portability, or production-readiness conclusions.

## Evidence bundle

The in-memory evidence bundle binds, at minimum:

- experiment and protocol identities;
- explicit authorization-record identity;
- source-under-test and harness commit identities;
- schema and fixture-set identities;
- fixture-manifest SHA-256;
- deterministic per-fixture result identities and retry lineage;
- aggregate disposition;
- test command and test summary;
- explicit evidence ceiling.

The semantic bundle hash excludes wall-clock timestamps. Retries receive distinct deterministic result identities and must preserve the immediately prior result identity.

The runner returns the bundle in memory; the apparatus does not silently write or publish canonical result artifacts.

## Known boundary of the current contract candidate

The Stage-A fixture set intentionally probes governance meanings that may be narrower than the current framework-neutral execution contract, including principal identity, resource scope, distinct operation semantics, policy identity/version, explicit decision outcome/reason codes, authority expiry, delegation, and authority conditions.

Their absence from the current typed surface is not pre-declared as an experiment result. The canonical disposition remains unestablished until a separately authorized run evaluates the frozen fixture set.

If an authorized Stage-A run later finds a missing or ambiguous critical semantic, freeze and retain that evidence before proposing a schema extension. Do not repair the contract first and erase the failure that motivated the change.

## Stage B

Cross-runtime portability is a separate Stage-B question. It remains `NOT ESTABLISHED` until a materially different runtime/adapter is selected, bound, and tested under a separate admitted protocol slice. A Stage-A result alone cannot establish portability.
Loading
Loading