feat(pi-admission): add minimal harness-level admission spike

[johnnygreco]

Summary

Adds projects/research/pi-admission/: a research spike demonstrating why network-only content policy is too late to protect an agent's session history.

The example uses unmodified Pi and OpenShell libraries/releases. A small TypeScript harness controls when candidate messages enter Pi's native history and JSONL. One standalone Rust service makes admission decisions and verifies signed receipts at OpenShell's pre-credentials egress boundary. It has no Egress Gate dependency.

The deliberately synthetic regex policy replaces addresses ending in @example.com with [EMAIL] and denies SSN-shaped values. This is a teaching policy, not production DLP.

How the pieces fit

flowchart LR
    D[Private message or completed tool result] --> A[Admission HTTPS]
    A -->|deny| X[Discard candidate]
    A -->|allow or redact| H[Pi history and JSONL]
    H --> C[Approve outgoing user and tool context]
    C --> R[Signed receipt]
    R --> O[OpenShell pre-credentials middleware]
    O -->|valid receipt and clean request| P[Attach provider credential and send]
    O -->|missing or invalid receipt| B[Block request]

Loading

Piece	Responsibility
Pi	Native TUI, tools, model serialization, reasoning controls, session persistence, and compaction machinery
pi-harness	Hold candidate content privately; admit complete messages and tool-result batches before publication; obtain a receipt before each model call
Rust service	Apply the fixed regex policy, authenticate callers, sign Ed25519 receipts, and check actual provider requests against those receipts
OpenShell	Sandbox isolation, authenticated middleware calls, provider interception and credential delivery, native network-decision logs

Network denial alone would happen after the local transcript had already been contaminated. The harness boundary prevents that earlier write; the network boundary prevents a request without valid admission evidence from reaching the provider.

Deliberately bounded behavior

• Fresh native Pi session per launch; no imports or unchecked history restoration.
• Native read, bash, edit, and write execution, sequentially. No custom tool factory.
• Complete assistant responses are buffered, including reasoning. An assistant tool-call message and its results are published only after the entire batch is admitted.
• Pi's native argument preparation runs privately; changed executable arguments are admitted again.
• Edit rendering uses admitted text, not a file-derived preview outside admission.
• Manual /compact uses the same receipt-backed model path and admits the completed summary before persistence. Automatic compaction and retry are disabled.
• Busy input is rejected without queuing or crashing the TUI. Unsupported session changes warn and cancel.
• Project instructions, skills, extensions, images, model switching, and direct ! commands are intentionally outside this spike.
• Receipts bind ordered user/tool text, destination, sandbox, middleware, policy, and expiry—not the full transcript. Assistant/reasoning text is admitted locally and scanned at egress, but is not receipt-bound.
• Tool side effects cannot be rolled back when result admission fails. This is a controlled-launcher demonstration, not protection against compromised same-authority code.

Independent hands-on QA

Merge conflicts are resolved against main 16f1ec5; the project and CI paths now use projects/research/. QA ran in isolated checkouts, with a separate gateway and sandbox rather than altering the shared gateway.

Session	What actually ran
History and edge cases	Ten real-HTTPS/native-Pi scenarios: real coding tools, prepared edits, redaction/denial, reasoning, pending visibility, cancellation, busy input, multi-tool batch publication, and manual compaction. Controlled provider responses make the edge cases reproducible.
Native TUI	Real PTY keyboard interactions, native read/edit/bash, live and saved history, busy input, Escape, unsupported /new, disabled resource loading, and normal exit. Controlled provider/admission fixtures; not a claim of live gateway coverage.
Live deployment	Existing OpenShell 0.0.116, production Rust process, Docker sandbox, native TUI, real model provider, and native egress logs. Final outcome recorded below.

QA found and fixed actual startup/usage defects: ambiguous TLS backend, missing JWT backend, supervisor manifest discovery, missing sandbox networking tools, busy-input and /new crashes, unchecked edit previews, and unintended project-resource loading. These are bounded corrections, not new product features.

All current CI checks pass. Local validation includes Rust formatting/Clippy/four focused tests, TypeScript typecheck/build/integration tests, Ruff, shell syntax, and every side-effect-free demo.sh --print workflow. A signed-token regression covers the newly exercised authentication path.

The unchanged ./demo.sh verify completed successfully against the real model endpoint: receipt-less egress denied, denied input absent from history, redacted content in live history and JSONL, successful native write and continuation, and real manual compaction. Native OpenShell logs show middleware_denied:pi_admission:receipt_missing alongside allowed requests. The real TUI read and edited both sample files; native bash ran the updated test successfully.

The selected provider was NVIDIA's OpenAI-compatible endpoint serving aws/anthropic/bedrock-claude-opus-5. Its first run returned an upstream HTTP 408 Bedrock connection timeout; one unchanged retry passed. No retry logic or provider-specific workaround was added. OpenRouter remains the documented example configuration, not a provider claimed as live-tested here. Live verification lowers the compaction retention threshold only in its test session to exercise compaction on a short conversation; the launcher does not change normal manual-compaction retention.

The live TUI displayed reasoning controls, but this provider configuration did not emit reasoning; reasoning-content admission was exercised with controlled responses instead. Cleanup passed: QA sandbox/providers were removed and its isolated gateway/service stopped, with shared resources untouched.

Reproduce the reviewed diff with git diff 16f1ec5242ca7210f0799a5a47c23dea189f17dd bc1d26ca3d12e8e2edaf109d393a91002f00c563 -- projects/research/pi-admission .github/workflows/pi-admission.yml projects/README.md. TUI/history reruns used c502dd1; subsequent fixes affect only Rust authentication, container dependencies, and documentation. Live deployment and final CI used bc1d26c.

Size

Physical lines in the added project, with generated material separated:

Category	Lines
TypeScript harness and live verifier	1,113
Rust service, including inline tests	1,296
Setup helpers and Dockerfile	521
TypeScript integration tests	210
Coding sample	9
Documentation	169
Configuration and metadata	246
Protocol and dependency lockfiles	4,928
License	203
Total project files	8,695

The existing OMM-managed protocol is unchanged. Dependency lock changes are generated by the package managers; the JWT backend is required for actual gateway authentication.

[github-actions]

Documentation preview

The preview has been removed.

[github-actions]

New project review

Revision: cf5083cf9b9b8e2ca5253d5645c2126e6a6eda0c · Workflow and result artifacts

Review findings are advisory. Required checks remain separate merge gates.

Execution: success

Reviewer and guidelines revision: 16f1ec5242ca7210f0799a5a47c23dea189f17dd

Reviews

Project	Verdict	Guidelines	Findings
projects/research/pi-admission	✅ Pass	✅ Pass	3

projects/research/pi-admission

`projects/research/pi-admission` is a research spike arguing that content policy must be enforced at the agent harness, not only at network egress: a thin TypeScript launcher around unmodified Pi admits each complete user/assistant/tool-result candidate (and a compaction summary) to a standalone Rust admission service before it enters Pi's live history and JSONL, while the same service signs Ed25519 receipts over the ordered user/tool projection and re-verifies them at OpenShell's pre-credentials middleware boundary so a provider request without valid admission evidence never receives credentials. The README is unusually complete for a spike (fixed synthetic email/SSN policy, prerequisites, gateway registration caveats, a five-step first run, a side-effect-free `--print` mode, an explicit ordinary-Pi + network-regex baseline comparison with its pattern-mismatch confounder disclosed, cleanup, and a Limits section on receipt coverage, non-transactional tool effects and same-authority trust). Manifests (pyproject + uv.lock, Cargo.toml + Cargo.lock, package.json + package-lock.json), Apache-2.0 license, `.env.example` with no secrets, and a bounded verification suite (four Rust unit tests, two harness end-to-end tests over Pi's public stream seam, and a live `./demo.sh verify` asserting deny/redact/write/compaction/receipt-less-403) are present and consistent with the documented claims. Main residual gap: the project documentation states expected outcomes but records no observed results from the live run, so the spike's evidence trail lives only in the PR narrative.

Project guidelines: ✅ Pass

Location matches kind (`projects/research/`). Purpose/scope, OpenShell usage, and non-goals are explicit in README ("## Scope", "## Limits"), with working behavior separated from excluded features. Layout is self-contained (harness, middleware, setup helpers, sandbox image, sample workspace) with only two READMEs and no empty docs scaffolding. Reproducible first run: prerequisites (OpenShell 0.0.116, uv 0.11+, Rust 1.90+, Docker, Python 3.11+), the smallest run commands, expected observable outputs, and disclosed provider costs/credentials. Evidence: runnable checks (`cargo test --locked`, `npm test`, `./demo.sh verify`) appropriate to the claims, which the guidelines accept in place of recorded artifacts. Dependencies/configuration: Python uses uv + pyproject + committed uv.lock; Cargo.lock and package-lock.json committed; `.env`/`models.json`/`.key`/`.pem` gitignored and documented in `.env.example`; Apache-2.0 LICENSE with SPDX headers. Engineering is proportionate: no plugin framework or speculative abstraction, unsupported native entry points fail closed rather than being reimplemented. Research-kind expectations (clear question/method, repeatable execution detail, honest uncertainty) are met; the only softness is the absence of a recorded results summary, reported as a low finding rather than a violation. Not verified in this bounded review: the added CI workflow (`.github/workflows/pi-admission.yml`) and `projects/README.md` edit are outside the supplied snapshot.

Findings

• low: No recorded results for the spike's central claim in project documentation — projects/research/pi-admission/README.md:215
  • Evidence: The comparison section states expected outcomes only — "| Ordinary Pi + network-only regex | Fake token redacted | Original fake token remains |" and "The email should appear as `[EMAIL]` in the admitted conversation" — and the document contains no results/findings subsection, logged middleware decisions, or session excerpt from an executed run.
  • Recommendation: Add a short Results section to the README recording one executed run: OpenShell/provider versions, the observed `middleware_denied:pi_admission:receipt_missing` log line, a redacted history/JSONL excerpt, and which behaviors were exercised with controlled fixtures rather than live.
  • Impact: A reader must stand up a gateway, Docker sandbox, and a paid provider before seeing any outcome, and cannot tell from the project which assertions were observed live versus only encoded in tests, weakening the spike's evidence trail once the PR narrative is out of view.
• low: Live verifier discards the underlying failure cause — projects/research/pi-admission/pi-harness/src/verify.ts:91
  • Evidence: `verify().catch(() => { console.error("FAIL end-to-end verification. Check service availability, credentials, model compatibility, and the last PASS line."); process.exitCode = 1; });`
  • Recommendation: Log the caught error (message or `AdmissionError.kind`) alongside the FAIL line before setting the exit code.
  • Impact: For the paid, gateway-dependent check that produces the spike's primary evidence, an assertion diff, HTTP status, or AdmissionError kind is dropped, so a failed reproduction attempt gives the researcher only a generic message.
• low: README does not say where to inspect the admitted session JSONL — projects/research/pi-admission/README.md:205
  • Evidence: The baseline section instructs "search its saved session JSONL for the fake token", but the admission-demo steps never name the in-sandbox session location (`/sandbox/sessions`, set in pi-harness/src/cli.ts) or how to read it; only `verify` prints "Saved evidence: ...".
  • Recommendation: Mention the sandbox session directory and a concrete inspection command (for example an `openshell sandbox exec` read of `/sandbox/sessions`) in step 4.
  • Impact: A reader following the interactive `launch` path cannot easily confirm the key claim — that denied content never reaches saved history — without reading the source.

Strengths

• Two-boundary method is coherent and fail-closed: publication-time admission plus receipt verification re-derived from the actual outbound body (middleware/src/lib.rs, policy.rs `inspect_provider_request`), with send-only replacement explicitly refused so saved history cannot diverge from what was sent.
• Honest, specific limitation disclosure: receipts bind only ordered user/tool text, destination, sandbox, middleware, policy and expiry; assistant/reasoning text is not receipt-bound; tool side effects are non-transactional; the guarantee excludes compromised same-authority code (README "## Limits").
• The baseline comparison openly names its own confounder — the network-regex demo and the admission demo use different fixed patterns, and layering them would invalidate receipts — instead of implying an identical-input result.
• First-run documentation anticipates real operational hazards: append-only gateway registration warnings, remote/containerized gateway handling, public-CA-only copying, restart-before-stop cleanup ordering, and a credential-free `./demo.sh --print` mode.
• Verification is small but targeted at experimental validity: batch-publication ordering, denial absence from live and saved history, receipt tamper/content/context mismatch, gateway token binding, escaped-entity detection at egress, and a receipt-less 403 in the live check.

Limitations

• No commands were executed: no `cargo test`, `npm test`, `./demo.sh verify`, Docker build, or gateway interaction, so all documented outcomes are assessed from source and documentation only.
• Implementation sampling was representative: pi-harness/src/{admission,session,cli,model,network,verify}.ts, middleware/src/{lib,policy,receipt,auth,admission,main}.rs partially, prepare.py, demo.sh, Dockerfile and both test files were read; agent.ts batching internals, the generated protobuf, and all lockfile contents were not audited.
• The added CI workflow `.github/workflows/pi-admission.yml` and the `projects/README.md` index edit listed in the change summary are outside the supplied project snapshot and were not reviewed.
• Omitted symlinks (`CLAUDE.md -> AGENTS.md` and one under `projects/tools/openshell-agent-runner/`) are outside this project; no repository-root AGENTS.md conventions were available to cross-check, and this project adds no nested AGENTS.md.
• External claims could not be verified: OpenShell 0.0.116 middleware contract semantics, the pinned proto sha256, Pi 0.85.1 SDK behavior, and provider endpoint behavior are taken as stated.