Skip to content

[FLO-13.3] Prove compatibility, artifact, and provider-failure scenarios #30

Closed

Description

@szmyty

Parent: #13
Depends on: #28, #29, #25
Coordinates with: #3

Outcome

Use the versioned scenario contract and hermetic provider kit to prove Flow fails closed across compatibility, contract, artifact, and provider-evidence boundaries.

This checkpoint covers acceptance and refusal around one execution boundary. Lifecycle recovery and resume are owned by the next checkpoint.

Required scenario families

Compatibility and contract

  • compatible, unavailable, missing, stale, future, and mismatched provider/capability versions;
  • valid and invalid manifests, provider locks, execution reports, provenance, output declarations, and schema versions;
  • configuration, publisher, executable, capability, authorization, and integrity disagreement;
  • deterministic conflict and fallback behavior, with fallback forbidden after invocation begins.

Artifact acceptance

  • complete, partial, missing, extra, stale, corrupt, and contradictory artifact sets;
  • absolute paths, parent traversal, path escape, symlinks, file/directory mismatch, duplicate identity, digest conflict, and undeclared output;
  • provider success or exit zero followed by failed Flow-owned validation.

Diagnostics and privacy

  • typed, actionable explanations naming the failing boundary;
  • bounded and redacted operational evidence;
  • canaries proving argv, environment, secrets, and private content do not leak into portable results.

Acceptance criteria

Non-goals

  • Durable interruption, retry, checkpoint, or resume.
  • Real provider algorithms or broad raw-format coverage.
  • Publication, upload, signing, or destructive mutation.

Verified closeout — 2026-09-25

PR #61 merged as dfb16b347975e3292dc2928c8c48463f51dcf6d1. Default-branch CI passed on that exact commit. The 81-case acceptance suite, two fresh-workspace repetitions, Rust 1.85/stable checks, and repository validators are complete. #49 now supplies durable state before #31 extends lifecycle coverage. #62 owns the later documentation showcase.

Activity

  1. self-assigned this
    on Sep 20, 2026
  2. szmyty commented on Sep 24, 2026

    @szmyty
    ContributorAuthor

    The #29 → #30 handoff is prepared in draft PR #60, but #30 is not yet unblocked: the gate is PR #60 merged plus green default-branch CI.

    After that gate, #30 inherits the stable package source/finalization procedure, four synthetic capability IDs, closed behavior modes, typed expected outcomes, and the public resolve → observe subjects → authorize → run → observe artifacts → accept harness. #30 still owns executable machine-readable coverage for compatibility/schema disagreements, conflict/fallback, the broader path/symlink/kind/duplicate/digest/undeclared-output families, diagnostic/privacy canaries, and explicit PR-tier budgets.

  3. szmyty commented on Sep 26, 2026

    @szmyty
    ContributorAuthor

    Implementation is ready for review in PR #61, head 20dcfaea55c8fd9f82f299b8ce4c79942b369f85.

    The executable catalog has 81 stable scenarios (19 resolution, 22 contract, 22 artifact, 14 provider, 4 privacy). Each runs twice in fresh roots with exact typed-outcome checks, normalized receipts, actual package/executable identities, and explicit budgets/gaps. A real child probe covers literal argv, authorized environment/secret values, ambient HOME exclusion, and private stderr. Five Python regressions reject incomplete, duplicate, unknown, false-acceptance, and oversized report evidence.

    Local Rust 1.85/stable suites, strict lint, formatting, docs/examples, package verification, and repository validators passed. PR CI checks the pushed head and retains portable JSON reports. The remotely published Git tree ff36ecae83076a2e7f68a53fbd3e3dd2fb6a0af9 exactly matches the locally tested tree.

    No production API change or durable-state claim. Raw provider diagnostics and authority profiles remain local inputs/evidence; the normalized report is the portable surface, not a general redactor. Keep this issue open until merge. After merge plus green default-branch CI, #49 is the next central checkpoint; #31 follows it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions