Add pipeline recipes: other programs' output, relix into relix, loops, CI

docs/guide/command-line/README.md

@@ -35,7 +35,7 @@ What a directory gives the commands run in it.
 
 | Page | What it covers |
 |---|---|
-| [Pipelines](pipelines.md) | With `jq`, `awk`, `sort` and `head`; with `find` and `xargs`; scripts in here-documents; a data check in CI; optimising a script and reviewing the change |
+| [Pipelines](pipelines.md) | With `jq`, `awk`, `sort` and `head`; another program's output as a table; one `relix` into another; `tee`; with `find`, `xargs` and shell loops; scripts in here-documents; data checks in CI, comparing snapshots, and a GitHub Actions workflow; optimising a script and reviewing the change |
 
 ## Command reference
 

docs/guide/command-line/index.json

@@ -6,7 +6,7 @@
     {"path": "output.md", "chapter": "The command line", "title": "Output formats", "summary": "`table`, `tsv`, `csv`, `ndjson`, `json` and `markdown`; NULLs and headers; scripts with several queries; `--fail-empty` and `--fail-rows`; messages; closed pipes and interrupts"},
     {"path": "catalog.md", "chapter": "Projects on the command line", "title": "The catalog", "summary": "`.relix/` directories and how they are found; trusting a project; a nearer declaration wins; inspecting the catalog; `--catalog`; `relixrc`"},
     {"path": "secrets.md", "chapter": "Projects on the command line", "title": "Secrets and profiles", "summary": "`${NAME}` placeholders; `profiles.json` and `-P`; the environment and `-D`; private profile files; what is never printed"},
-    {"path": "pipelines.md", "chapter": "Command-line recipes", "title": "Pipelines", "summary": "With `jq`, `awk`, `sort` and `head`; with `find` and `xargs`; scripts in here-documents; a data check in CI; optimising a script and reviewing the change"},
+    {"path": "pipelines.md", "chapter": "Command-line recipes", "title": "Pipelines", "summary": "With `jq`, `awk`, `sort` and `head`; another program's output as a table; one `relix` into another; `tee`; with `find`, `xargs` and shell loops; scripts in here-documents; data checks in CI, comparing snapshots, and a GitHub Actions workflow; optimising a script and reviewing the change"},
     {"path": "commands/relix.md", "chapter": "Command reference", "title": "relix", "summary": "The global options every command takes; the exit statuses; the environment variables and files", "command": "relix"},
     {"path": "commands/run.md", "chapter": "Command reference", "title": "relix run", "summary": "Runs scripts and writes their queries' rows", "command": "relix run"},
     {"path": "commands/check.md", "chapter": "Command reference", "title": "relix check", "summary": "Analyses scripts without running them, and reports their problems", "command": "relix check"},

docs/guide/command-line/pipelines.md

@@ -1,8 +1,9 @@
 # Pipelines
 
 `relix` reads text and writes text, so it works with the programs a shell already has.
-This page is a set of recipes: `relix` with `jq` for JSON, with `find` and `xargs` for
-many files, with `awk`, `sort` and `head` for its rows, and as a check in CI.
+This page is a set of recipes: `relix` with `jq` for JSON, with `awk`, `sort` and `head`
+for its rows and for other programs' output, with another `relix`, with `find`, `xargs`
+and shell loops for many files, and as a check in CI.
 
 The rule that makes all of them work: **standard output carries rows and nothing else.**
 Messages go to standard error, the format in a pipe is tab-separated values unless `-o`
@@ -50,6 +51,68 @@ completed 400
 pending 50
 ```
 
+### Another program's output, as a table
+
+Any program that prints columns can feed a query: turn its output into tab-separated
+values with a header line, and bind standard input as `tsv`. Here `awk` splits a web
+server's log into its method and path, and `relix` counts the requests to each path:
+
+```shell
+printf 'GET /orders\nGET /customers\nGET /orders\nPOST /orders\n' > access.log
+awk 'BEGIN { print "method\tpath" } { print $1 "\t" $2 }' access.log \
+  | relix -i Hits=tsv:- -e 'SORT n DESC, path (GROUP path, COUNT(*) -> n (Hits))'
+```
+```
+── <expression 1> ──────────────────────────────────────────────────────────────
+ path        n
+ ──────────  ─
+ /orders     3
+ /customers  1
+(2 rows)
+```
+
+The last stage writes to the terminal, so it prints a table. The column types are inferred
+from the values, so `n` is a number to sort on rather than text. `--schema` gives them instead, as [Binding inputs](inputs.md#the-heading-of-an-input)
+shows.
+
+## One relix into another
+
+A stage's rows are the next stage's input, so two runs of `relix` compose like any other
+two programs. Each run reads the catalog of the directory it runs in, which makes a pipe
+the way to join tables two projects declare. The `reports` project declares its own
+`Orders`, a fixed sample; the shop's run joins it to the shop's `Customers`:
+
+```shell
+(cd reports && relix -e 'Orders') | relix -i January=tsv:- -e 'PROJECT name, order_id, amount (Customers JOIN January)' -o csv
+```
+```
+name,order_id,amount
+Alice,101,500
+Dan,102,250
+```
+
+The first run writes tab-separated values because its output is a pipe, and the second
+reads them because `-i` says `tsv`. Within one project a view is the better tool: it is
+one query, so the optimiser sees all of it.
+
+### Keep the rows, print a summary
+
+`tee` writes a copy of what passes through it to a file. Here the detail is kept in
+`completed.csv`, and a second `relix` summarises it on the way past:
+
+```shell
+relix -e 'Completed' -o csv | tee completed.csv | relix -i C=csv:- -e 'GROUP COUNT(*) -> orders, SUM(amount) -> total (C)' -o tsv
+cat completed.csv
+```
+```
+orders	total
+3	400
+order_id,customer_id,amount,status,placed
+1,1,120,completed,2026-09-02
+2,1,80,completed,2026-09-14
+4,3,200,completed,2026-10-01
+```
+
 ## Stopping early
 
 When the reader of a pipe stops, `relix` stops too: it cancels the query rather than
@@ -98,6 +161,32 @@ data/orders.csv	4
 reports/sample-orders.csv	2
 ```
 
+### One output file per value
+
+A shell loop runs a script once for each value another query lists. The first run prints
+each city, one per line with no header; the loop runs the
+[parameterised](inputs.md#a-name-value-query-parameters) `by-city.relix` for each one and
+writes its customers to a file of its own:
+
+```shell
+relix -e 'PROJECT city (Customers)' --no-header | while read -r city; do
+  relix -a city="$city" scripts/by-city.relix -o csv > "customers-$city.csv"
+done
+ls customers-*.csv
+cat customers-London.csv
+```
+```
+customers-Leeds.csv
+customers-London.csv
+customers-York.csv
+name
+Alice
+Carol
+```
+
+The city goes in as a parameter, so a name with a quote or a space in it is a value to
+compare with, never part of the script.
+
 ### Check every script's formatting
 
 `relix fmt --check` prints the name of each file that is not formatted canonically, and
@@ -168,6 +257,72 @@ A CI job runs in a fresh container where the project's `.relix/` cannot have bee
 trusted by hand; `RELIX_TRUST_ALL=1` trusts it, and a profile or the job's secret
 environment variables supply the [placeholders](secrets.md).
 
+### Has anything changed since yesterday?
+
+`EXCEPT` keeps the rows of one input that are not in the other, so over two snapshots of
+the same export it lists what is new. With `--fail-rows` the exit status says whether
+there was anything:
+
+```shell
+cp data/orders.csv yesterday.csv
+{ cat data/orders.csv; echo '5,4,90,pending,2026-10-12'; } > today.csv
+relix -i Before=yesterday.csv -i After=today.csv --fail-rows -e 'After EXCEPT Before' -o tsv; echo "exit $?"
+```
+```
+order_id	customer_id	amount	status	placed
+5	4	90	pending	2026-10-12
+exit 1
+```
+
+Swap the two inputs to list the rows that went away.
+
+### In a workflow
+
+The checks above are each one line of a job, which is all a CI system asks of them. A
+GitHub Actions workflow, for instance, installs the Linux archive of the latest release
+with `gh`, which its runners carry, then runs the checks, and adds a table to the run's
+summary page, a file that takes Markdown:
+
+```yaml
+name: data checks
+on: [push, pull_request]
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    env:
+      RELIX_TRUST_ALL: "1"
+      GH_TOKEN: ${{ github.token }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install relix
+        run: |
+          gh release download -R DarkCollective/relix-cli -p 'relix-*-linux-x86_64.tar.gz'
+          tar xzf relix-*-linux-x86_64.tar.gz
+          echo "$PWD"/relix-*/bin >> "$GITHUB_PATH"
+      - name: Every script analyses
+        run: find scripts -name '*.relix' -print0 | xargs -0 relix check
+      - name: Every order has a positive amount
+        run: relix --fail-rows -e 'SELECT amount <= 0 (Orders)' -o tsv
+      - name: Big spenders
+        run: relix -e 'BigSpenders' -o markdown >> "$GITHUB_STEP_SUMMARY"
+```
+
+To pin a release instead of taking the latest, name its tag after `download`, as in
+`gh release download v0.1.1 …`. The Markdown that last step appends is this:
+
+```shell
+relix -e 'BigSpenders' -o markdown
+```
+```
+### <expression 1>
+
+| name | total |
+| --- | --- |
+| Alice | 200 |
+| Carol | 200 |
+```
+
 ## Optimise a script, and review the change
 
 `relix optimize` prints a script with each query rewritten by the optimiser, so it works