diff --git a/README.md b/README.md index 410e5f3..84e3eab 100644 --- a/README.md +++ b/README.md @@ -168,7 +168,7 @@ SwiftUI and React Native. | `expectation_service` | [crates/service](crates/service) | The external things a run needs (a server, a database, a container): started once, stopped after, and "not supported yet" with how to get them when they can't be had | | `expectation_harness` | [crates/harness](crates/harness) | The `Driver` trait, closure `Bindings`, the runner, and the `Acceptance` pipeline | | `expectation_remote` | [crates/remote](crates/remote) | Driving an app through a driver embedded in it (Swift, JavaScript and Rust clients), and `expectation-run` | -| `expectation_cli` | [crates/cli](crates/cli) | The `expectation` command: `check`, `graph`, `plan`, `docs` | +| `expectation_cli` | [crates/cli](crates/cli) | The `expectation` command: `check`, `graph`, `states`, `plan`, `docs`, `screenshots` | ## The language @@ -263,8 +263,8 @@ outcome, so "Then the kettle is dry" reads "The display says DRY". phrase of `Given`s is a setup ("the kettle is plugged in and full"), usable in a Background, a Given or a scenario's start. A phrase of `When`s is a sequence of actions ("the user draws two shapes", one `When` per action); an expectation can use one that's a single action, and -a story step can use any. A phrase's `Say:` is how the docs put it. Phrases can use the -phrases declared before them. +a story step can use any. A phrase's `Say:` is how the docs put it. A phrase can use any +other phrase, whichever is written first; two that use each other are an error. **Steps.** A step is either an event or a fact ` is [not] `. The predicate is one or more values (`off or heating`) or, for numbers, a comparison: `80 %`, @@ -300,6 +300,12 @@ phrase of several actions becomes several test steps but stays one numbered step Each top-level section is a system of its own: a story only goes through its own section's expectations. +**More in the guide.** [The language](https://expectation.sim001.net/guide/language.html) is +the full reference. It also covers `Approval: approved | proposed` (whether the specification +is agreed, and approved-only manuals), `Launch:` (how to start the program under test), +`Follows:` and `Hands over to:` (how one section's setup is reached from another), and +`Scopes: sections` (each top-level section with its own vocabulary). + ## Platforms One suite can describe a system that exists on several platforms (a desktop app and a phone @@ -518,6 +524,9 @@ cargo run -p expectation_cli -- plan [--coverage] # the acceptance test cargo run -p expectation_cli -- docs # docs from the results saved in /results ``` +[The command's chapter](https://expectation.sim001.net/guide/cli.html) has every option, and +the `screenshots` commands. + Or install it: `cargo install --git https://github.com/ptescher/expectation expectation_cli`. ## License diff --git a/book/src/cli.md b/book/src/cli.md index bed4ae2..175350d 100644 --- a/book/src/cli.md +++ b/book/src/cli.md @@ -87,6 +87,9 @@ error. - `--approved-only` writes just what the suite marks `Approval: approved` (see [Approved-only manuals](manual.md#approved-only-manuals)); `--results ` reads the saved results from there instead of `/results`. +- `--reader` writes the edition for people who read the manual, not test it: no "How this is + tested" blocks and no screenshot placeholders (see + [The reader edition](manual.md#the-reader-edition)). - `--stylesheet ` restyles the HTML site with your CSS file (see [Your own theme](manual.md#your-own-theme)). A missing, empty or malformed file stops the command before anything is written. diff --git a/book/src/diagnostics.md b/book/src/diagnostics.md index 7ae48ad..c957b6d 100644 --- a/book/src/diagnostics.md +++ b/book/src/diagnostics.md @@ -39,6 +39,23 @@ error[bad-value]: `boiling` isn't a value of the kettle | `maybe-step` | warning | Description text that looks like a misspelled step ("Gvien the kettle is off"). | | `says-nothing` | warning | A step that's true of every value ("off or heating" of a state with only those). | | `shows-what` | warning | A `Shows:` line that doesn't say which value it's for. | +| `reference` | error | A term's `Reference:` that isn't a label and an `http` or `https` URL, or a second one in the same term. | +| `launch-in-step` | warning | A `Say:` that holds a shell command in backticks. Starting the program is not a step: put it in `Launch:` under the `Section:`. | + +## The size of the pages + +These warn when the manual's pages would be badly sized. The limits are constants in +[`sizing.rs`](https://github.com/ptescher/expectation/blob/main/crates/language/src/sizing.rs). + +| Code | | Means | +| --- | --- | --- | +| `many-scenarios` | warning | A story has more than one scenario. Each scenario is a tutorial page of its own. | +| `short-story` | warning | A scenario has fewer than 3 numbered steps (each `When` starts one). | +| `long-story` | warning | A scenario has more than 8 numbered steps. | +| `big-feature` | warning | A feature has more than 8 expectations on its topic page (`Docs: hidden` ones don't count). | +| `small-feature` | warning | A feature has 1 expectation on its page, and its section has other features. | +| `crowded-section` | warning | A section (not counting its subsections) has more than 8 tutorials or more than 8 topics. | +| `no-tutorial` | warning | A section has 2 or more features and no story. | ## Platforms diff --git a/book/src/drivers.md b/book/src/drivers.md index a70f693..7d08640 100644 --- a/book/src/drivers.md +++ b/book/src/drivers.md @@ -93,7 +93,9 @@ Say which vocabulary event using a control makes happen with `Control::mapped_to presses lock")`, and the report joins the control to the behaviour specified for that event and the stories about it. Each control then shows its gaps, with the reason: *no event mapping*, *event not in the vocabulary*, *no expectation*, *only pending expectations*, *no authored -story*, *never used by a test*, *no passing execution*. A story that performs the event in a +story*, *never used by a test*, *no passing execution* (in `interactions.json`: +`no-event-mapping`, `unknown-event`, `no-expectation`, `only-pending`, `no-authored-story`, +`never-used`, `no-passing-execution`). A story that performs the event in a section named like the control's view documents the control even if it couldn't run; it still isn't verification. The same event on a control in another view is a different row, and what covers one doesn't cover the other. An integration that knows controls no run will reach can diff --git a/book/src/getting-started.md b/book/src/getting-started.md index 7bcc9c3..b047579 100644 --- a/book/src/getting-started.md +++ b/book/src/getting-started.md @@ -86,13 +86,20 @@ warning[untested]: no story exercises «Pushing a locked turnstile does nothing | Expectation: Pushing a locked turnstile does nothing = help: a story that would: When a visitor pushes the arm -0 errors, 1 warning +warning[short-story]: scenario «A day at the zoo» has 2 steps + --> turnstile.expect:30 + | Story: A day at the zoo + = help: a tutorial needs 3 to 8 steps; with fewer it is a topic, so let the expectations say it, or add the steps a reader takes before and after + +0 errors, 2 warnings Turnstile: 3 expectations in 1 feature, 1 story (1 test); 0 exceptions; flow DAG 3 edges, 2 levels deep (search truncated) ``` Nothing ran, but the suite was checked: every step uses the vocabulary, no two expectations -disagree, and the story only relies on behaviour the expectations specify. The warning says -one expectation has no story, and how a story could reach it. +disagree, and the story only relies on behaviour the expectations specify. The first warning +says one expectation has no story, and how a story could reach it. The second is about the +manual: the story becomes a tutorial, and two steps make a thin one. A suite this small can +live with both. "Search truncated" is normal here: every coin makes a new state, so the search for what leads to what stops at its depth limit instead of counting to 1000. It has already found diff --git a/book/src/remote.md b/book/src/remote.md index 6dd4869..e9b2031 100644 --- a/book/src/remote.md +++ b/book/src/remote.md @@ -121,6 +121,10 @@ Bind only what the UI really shows and lets a person do. Anything else stays unb reported as "not supported yet", which is honest: a binding that reaches round the UI into the app's internals is an embedded driver in disguise, and should be reported as one. +## In the runner + +Wherever the client lives, the runner's side is the same: a `RemoteDriver` that listens for +it and runs the suite through it. ```rust use expectation_remote::RemoteDriver;