cli-contract drives built Node command-line tools and their MCP stdio servers from the outside.
Every WireCat CLI keeps its plans and reviewed snapshots in contracts/ in its own repository: MAX, Telegram,
WhatsApp, Zoom, Drive and Memo, and runs them in full validation. A CLI on cli-base node-ci.yml with cli: true
runs them as the named steps contracts:check, contracts:upgrade and security:no-leak. The shared runner is also exported as runContracts from @wirecat/cli-testing.
Repositories pin @wirecat/cli-testing from npm as a development dependency, with
its SHA512 integrity recorded in each CLI lockfile. Trusted publishing was verified by
release run 38092344081,
which published the exact validated artifact and created tag v0.2.0. Repository installs
and contract checks do not require a local npm login.
The initial rollout used the 0.2.0 toolkit preview while publisher configuration was pending. That preview remains available for provenance; its package integrity matches the npm release.
- Help output matches a reviewed snapshot.
- Discovery is exactly one JSON object or array on stdout, with empty stderr.
- Invalid options and command paths return the documented exit code and one JSON error on stderr, with empty stdout and the expected error code.
- MCP initialization and
tools/listwork through the SDK. Tool names, the structure of each input schema, and annotations match a reviewed snapshot. Descriptions (of tools and inside schemas),$schema, output schemas and tool ordering are not frozen: they change with each cli-core release and only made every CLI adopt it. A snapshot written with output schemas still passes;--updaterewrites it in the shorter form.
The first plans cover shared metadata commands and errors. They do not claim coverage of every command, store migration, messenger protocol, HTTP MCP transport, or live-account behavior. Those remain separate testing tasks.
Each run starts with a fresh temporary home, config/state/cache directories, and synthetic store path. Tokens, application credentials, proxy configuration, and the owner's keyring environment are not inherited. A Node preload guard rejects common network, keyring, and child-process entry points, and native addons are disabled. An attempted external operation fails the run even when the command catches its exception. The runner accepts only the current Node executable.
This is a regression-test guard for trusted CLI builds, not a sandbox for arbitrary untrusted code.
Plans must use offline metadata operations. The MCP client initializes and lists tools; it calls a
tool only in the deny check, for a command that profile denies. Temporary directories and server processes are closed afterward.
Cases default to a five-second process limit; MCP uses one five-second request budget across
initialization and pagination. Process output has an eight-megabyte limit.
From a CLI repository with its build present:
pnpm contracts:check
pnpm exec cli-contract contracts/plan.json --update
git diff -- contracts/Ordinary checks never update snapshots. A missing or changed snapshot fails validation.
Use --update only when deliberately reviewing a contract change. Output normalization is limited
to line endings and JSON object-key ordering; renamed tools and changed parameter schemas remain
visible. Help snapshots retain command and option descriptions.
Contract checks run during full release, monthly, and manual validation. Local commit/push hooks and ordinary PR CI keep their existing budgets. The runner returns exit 0 for a passing report, 1 for contract failures, or 2 for setup errors. Reports contain case identifiers and failure reasons; they do not copy CLI stdout/stderr into public logs.
{
"command": "node",
"args": ["../dist/bin/example.js"],
"cases": [
{
"id": "discovery",
"args": ["commands", "--json"],
"exit": 0,
"stream": "stdout",
"kind": "json"
}
],
"mcp": {
"args": ["mcp"],
"snapshot": "mcp-tools.json"
}
}Relative executable arguments and snapshot paths resolve from the plan directory.
Add contains for a required output fragment, errorCode for a structured error, and snapshot
to freeze text such as help. MCP is optional for command-line tools without a server.
"deny": { "profile": "personal" } checks that a deny in the profile stops each command that reaches the
service, on the command line and over MCP (WC-PERM-23).
It reads commands --json and takes each leaf whose effects.remote is read or write, except a persistent
one such as mcp serve. Its arguments are the words after its path in its first example that has no --dry-run,
or x for each required argument and option. <profile> <path> --dry-run --json names the command's key;
housekeeping such as session and doctor names none and is left out. <profile> config show --json names the
config file, inside the fresh root.
deny-keysfails when no read names a key, or no write does in a CLI that has writes: the program then dropped its permission table, as one CLI's did.deny:<path>writesdenyfor that key alone and runs the command: exit 5,permission_erroron stderr, nothing on stdout, and no network, keyring or child-process access.deny-mcpdenies every key and starts<profile> mcp serve. Each command not markedmcp: falseorinteractiveis called through the server's*_reador*_writetool and must come backpermission_error. Avalidation_errorthat carries the arguments' schema is retried once with the required arguments filled; a secondvalidation_errorfails, since it hides whether the deny holds.
fixtures seeds {path, text} files inside the fresh profile before cases. json checks object
subsets with exact array lengths; capture maps names to JSON pointers such as /items/0/id.
Use {{name}} in later arguments and expectations. Whole-value JSON placeholders preserve the
captured string/number type; {{root}} identifies the fresh root. Missing/repeated names fail.
Per-case files check reviewed text, JSON subsets, original fixture bytes (unchanged: true) or
SHA256. Paths cannot escape the root or pass through symlinks.
previous: {} tests the upgrade from the newest published release at or below the version in the
package.json above the plan: cli-contract asks npm for the versions, installs that one with pnpm
into node_modules/.cache/wirecat-contract-previous/ (once per version, without install scripts),
and reports it as previous in its result. A release candidate whose package.json already holds the
next version tests from the last release; a pull request tests from the version on npm. Name the
executable with previous: {"bin": "<name>"} when the package has more than one target. This step
needs the registry; the cases themselves stay offline. previous: {entry, version} still pins an
installed Node CLI entry, for a plan whose seed script imports a release's own modules. A case with
previous: true uses that binary. entry can name a reviewed local synthetic fixture generator. Entry paths resolve
relative to the plan, while fixture and artifact paths stay relative to the temporary root.
diagnostic permits one exact reviewed stderr string on a stdout case; leak checks still apply.
diagnosticPattern does the same with a regular expression, for a message that carries a number or a version:
it is matched against the whole normalized stderr (anchored, no flags), and a mismatch names the pattern and the
stderr received. It replaces diagnostic, which it cannot be combined with, and needs stream: "stdout". An
invalid expression or both fields fail the whole plan before any case runs. In JSON the backslash is doubled:
{
"id": "store-newer-than-cli",
"args": ["upgrade", "--check", "--json"],
"exit": 0,
"stream": "stdout",
"kind": "json",
"diagnosticPattern": "the store moved to schema \\d+; every other WireCat CLI that uses it needs a version that speaks it — run `<cli> upgrade` for each\n"
}canaries names synthetic sensitive values and artifacts selects generated logs to scan.
allowSensitiveOutput: true permits only canaries in requested stdout data; credential and phone
patterns are still checked. See upgrade checks for the concrete CLI coverage.