Skip to content

Measure the podman side of rule two, the way conformance measures the docker side #109

Description

@lesnik512

tests/conformance/ measures the docker side of the parity rule continuously: every key is probed
against a real docker compose config, so a claim about what Docker accepts cannot go stale
unnoticed. The podman side has no equivalent. Rule two ("a document Docker accepts, compose2pod
accepts whenever podman can express it") rests on statements no test ever runs, and real podman is
exercised only by tests/integration/, which covers happy paths.

That gap produced a wrong fix this week. Issue 86 asserted "Docker accepts it, podman can express
it, so compose2pod should too" for volumes: ['C:\data:/var']. Nobody had run podman. #104 acted
on it and shipped an acceptance whose generated script dies at podman run with exit 125; #107
reversed it once the measurement existed, and #108 widened the refusal to the rest of the family.
Two of those three PRs are the cost of the missing oracle.

Proposed shape. A table of the documented rule-two refusals, one row each, plus a gate that
every such refusal in the source has a row.

The awkward part, and the reason this is not just another conformance suite: compose2pod refuses
these documents, so there is no generated script to run. Each row has to carry a counterfactual --
the podman invocation that would express Docker's meaning -- and that argv is authored by hand, not
derived. Keeping the authored surface small and self-checking is the design problem.

Refusal(
    id="volume-single-letter-source",
    compose={"services": {"app": {"image": "x", "volumes": ["v:/data"]}}},
    refusal_match="Windows drive path",           # compose2pod must refuse, with this reason
    podman_argv=["-v", "v:/data"],                # expressing Docker's meaning needs this
    control_argv=["-v", "/tmp/probe-src:/data"],  # same harness, must succeed
)

Three assertions per row: compose2pod refuses the document with that message; podman run <podman_argv> alpine true exits nonzero; podman run <control_argv> alpine true exits zero. The
control is what stops a green for the wrong reason -- a missing image or a broken harness fails the
control too, so "podman rejected the spec" cannot be confused with "nothing ran".

Assert the exit code, print the message. Asserting podman's error text pins the suite to a
podman version and will churn on a rewording. Collect the messages and print them in a
pytest_terminal_summary section instead, the way tests/conformance/conftest.py already prints
its over-rejection list via the _OVER_REJECTIONS stash -- drift stays visible without CI going red
on cosmetics.

Placement. The probes belong under tests/integration/, whose conftest.py auto-marks by
location and skips the whole directory when podman is absent, so they need no new marker and run in
the CI job that already has podman. The completeness gate is a plain unit test (no podman), modelled
on tests/test_adr_citations.py.

Phases.

  1. Harness plus the volume family -- the rows measured in issue 105, so the first commit encodes
    facts already established rather than new claims.
  2. Backfill the rest: create_host_path and nocopy (parsing.py), a relative --mount target,
    image subpath, the cluster/npipe long-form types, deploy.resources.reservations.*
    (resources.py), and network_mode, which ADR-0006 names but which is refused through the
    generic unsupported-key path rather than a site of its own. Expect one or two to come back
    expressible; each of those is an issue filed the way issue 86 should have been -- with the
    measurement attached.
  3. The completeness gate, once every site has a row.

The gate needs one convention first. Refusals citing podman are scattered msg = ... strings,
not a registry: (podman cannot express it) appears verbatim at parsing.py:278 and
parsing.py:296, while resources.py:64 says (no podman equivalent) and the volume-source
refusal spells its reason inline. Normalising on one suffix is what gives the gate something stable
to scan for, and is the smallest piece of this that has to land before phase 3.

Version. Verdicts are per podman version -- CI runs 4.9.3 on ubuntu-latest today. The summary
section should print podman --version, and ADR-0006 should record which version its rulings were
measured against, the way it already does for docker compose config v5.1.2.

Out of scope. This does not prove the converse -- that everything compose2pod accepts, podman can
express. That is tests/integration/'s job, and only for the paths it covers.

Definition of done for phase 1: a Refusal table with the volume-family rows, the three
assertions, the summary print, green in the integration CI job, and the 100% line-coverage gate
still met.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions