Skip to content

Complete evidence-gated service validation journey (#285) - #301

Draft
sodejm wants to merge 13 commits into
mainfrom
codex/issue-285-service-validation
Draft

sodejm wants to merge 13 commits into
mainfrom
codex/issue-285-service-validation

Conversation

@sodejm

@sodejm sodejm commented Oct 14, 2026 •

Copy link
Copy Markdown
Owner

Problem and resulting behavior

A discovered service could previously stop at an evidence-bound specialist handoff. This change adds the public, bounded validation journey required by #285: discovery → selection/routing → authorization → protocol probe → independent verification → report → cleanup.

The disposable loopback lab registers seven explicitly lab-only adapters on nonstandard ports:

  • HTTP: GET /health
  • SSH: identification exchange
  • Redis: PING; a separate authorization gates ECHO authorized-use
  • SMTP: greeting and EHLO
  • Docker Engine API: GET /version
  • DNS: length-prefixed DNS-over-TCP query
  • SOCKS: SOCKS5 method negotiation

Provider and verifier open separate connections and bind protocol, effect, and cleanup evidence to the exact endpoint, version, plan, attempt, timestamp, and evidence digest. Candidate, verified finding, and successful authorized use remain separate states. The #143 reproduction is loaded as trusted immutable data and its digest is bound to the exact Redis version, endpoint, controlled effect, and cleanup result. Authorization failure before dispatch releases the reserved attempt; replayed, stale, fabricated, mismatched, missing, or invalid evidence fails closed and remains insufficient_evidence.

Operator path and outcomes

Run:

python3.11 scripts/service_validation_loopback_example.py

The JSON report covers all seven routed families across eight loopback endpoints:

  • benign SSH, SMTP, Docker API, DNS, and SOCKS results;
  • a versioned Redis [Proposal]: [E14.01] Build versioned vulnerability reproduction workflows #143 candidate followed by separately authorized controlled proof;
  • a remediated Redis endpoint;
  • an HTTP endpoint that intentionally remains insufficient_evidence;
  • bounded attempts and approvals, cleanup receipts, and confirmation that every fixture stopped.

These adapters are registered only inside the disposable lab. They do not provide or claim production external-target support. Provider and verifier use separate connections, but share the protocol parsing implementation; the journey therefore does not claim an independently implemented parser.

Validation

Reviewed and pushed revision: 999f25065059f3ae836f4abf3caa221a2ef63199 (signed).

  • Focused service-validation suite: python3.11 -m pytest -q tests/test_service_entry_validation.py tests/test_service_validation_execution.py tests/test_service_validation_lab.py tests/step_defs/test_service_entry_validation_steps.py — 52 passed.
  • Public subprocess journey: python3.11 scripts/service_validation_loopback_example.py — all seven families and all expected positive, remediated, and insufficient paths completed; every fixture cleaned up.
  • Normal pre-push gate — passed in 233.1s:
    • make check passed;
    • issue coverage passed for 22 changed files, including 7 documentation files and 6 test/spec files;
    • Ruff and changed-file formatting passed;
    • 1,908 tests passed, 20 skipped, and 286 subtests passed;
    • 79% aggregate branch coverage;
    • dependency audit found no known vulnerabilities.

Hosted checks are evaluated separately.

Draft integration state

This PR remains draft until #299 merges. After that dependency lands, latest main will be merged without rewriting signed history, the overlapping discovery contract will be reconciled, and the focused suite, public subprocess journey, make check, and normal push gate will be rerun at the final exact head.

Refs #285

Require a unique assessment for each canonical completed probe key before preparing specialist handoffs. Add a fail-closed regression and document the checkpoint rule. Forty-six focused tests and Ruff checks passed. Revert this commit to restore the prior handoff behavior; no external execution occurs.
Signed-off-by: Justin Soderberg <723679+sodejm@users.noreply.github.com>
@sodejm sodejm changed the title Add evidence-gated service validation handoffs (#285) Complete evidence-gated service validation journey (#285) Oct 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant