From 0f08e804823f749ba56c52255a29662a9fd88bfd Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 11:24:53 +0200 Subject: [PATCH 01/10] docs: add design spec for file-level @var directive (#60) Co-Authored-By: Claude Sonnet 5 --- ...6-09-17-file-level-var-directive-design.md | 114 ++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-17-file-level-var-directive-design.md diff --git a/docs/superpowers/specs/2026-09-17-file-level-var-directive-design.md b/docs/superpowers/specs/2026-09-17-file-level-var-directive-design.md new file mode 100644 index 0000000..9029eef --- /dev/null +++ b/docs/superpowers/specs/2026-09-17-file-level-var-directive-design.md @@ -0,0 +1,114 @@ +# File-level `@var` directive — design + +Issue: https://github.com/sleipi/cli-t/issues/60 + +## Problem + +`--var NAME=VALUE` feeds `{{NAME}}` but is global to the run. Sibling +`.clitest` files that differ only in one value (a directory name, a +tool version, …) end up repeating that value on every line that needs +it. Renaming the shared value means editing every occurrence, in every +file, and it is easy to miss one silently. + +## Goal + +A frontmatter directive, `@var KEY=VALUE`, that sets a value scoped to +the file it's declared in, feeding the existing `{{...}}` template +system. Sibling files then differ only in their frontmatter line. + +## Non-goals + +- **Entry-level `@var`.** Would require moving `{{...}}` substitution + from a single whole-file text pass (before parsing) to a per-entry + pass (after parsing), since two entries in one file could need the + same placeholder resolved differently — a plain string replace over + the raw file text can't do that. No use case for this is described + in the issue; out of scope here, open a follow-up issue if needed. +- **Rejecting `@var` placed after the frontmatter.** Unknown + directives in entry position are already silently ignored today + (`interpretEntryDirectives` has no `default` error case) — this is + existing, consistent behavior for every directive, not something + this feature special-cases. +- **`@env`/`$VAR` integration.** `@env` feeds child-process environment + variables (`$VAR` in commands), a separate mechanism from `{{...}}`. + `@var` does not touch it. + +## Resolution order + +``` +--var (CLI) > [captures] > @var (file) +``` + +`--var` is resolved today via a single `vars.Substitute(content, cliVars)` +pass over the raw file text, before parsing — unchanged by this +feature. Whatever `{{...}}` placeholders that pass didn't resolve stay +literal in the parsed `entry.Command` string. + +`[captures]` are resolved at runtime, per entry, via +`vars.SubstituteCaptures(entry.Command, captures)` — the only place +`{{...}}` is substituted after parsing today. This already only +touches `entry.Command`, not asserts/body — an existing limitation, +unchanged here. + +`@var` slots in at that same runtime step: the map passed to +`SubstituteCaptures` becomes `merge(fileVars, captures)`, with +`captures` overwriting `fileVars` on key collision. Because `--var` +already consumed matching placeholders at load time (before this step +ever runs), and `captures` overwrite `@var` in the merge, the net +effect is exactly `--var > captures > @var` with no change to the +load-time substitution pass and no per-entry re-architecture. + +## Changes + +**`internal/types/types.go`** +- `FileDirectives.Var map[string]string` (mirrors `.Env`). No + `EntryDirectives.Var`. + +**`internal/parser/directives.go`** +- `interpretFileDirectives`: new `case "var"`. Same malformed-value + check as `@env` (`KEY=VALUE`, non-empty key, `DirectiveError` if no + `=`). Additionally: a second `@var` for the same key within one + frontmatter block is a hard `DirectiveError` (not last-write-wins, + unlike `@env`, since a duplicate file-level default is almost + certainly a copy-paste mistake worth surfacing). +- `interpretEntryDirectives`: no change. + +**`internal/vars/vars.go`** +- `MergeCaptures(fileVars, captures map[string]string) map[string]string` + — returns a new map, `captures` entries win over `fileVars` on + collision. Used at all 5 existing `SubstituteCaptures` call sites + (`cmd/clitest/run.go` ×2, `internal/executor/executor.go` ×2, + `internal/executor/background.go` ×1) in place of the bare + `captures` map. + +**`SPEC.md`** +- Variables → priority list: insert `@var` between `[captures]` and + environment-variable expansion. +- Directives → Available Directives table: add `@var` row, scope + `file` only. +- New `#### @var` subsection (pattern matches `#### @env`), with a + worked example mirroring the sibling-file case from the issue. + +## Testing + +- **Unit** — `internal/parser/directives_test.go`: parses `@var`, + rejects malformed (no `=`), rejects duplicate key. + `internal/vars/vars_test.go`: `MergeCaptures` precedence. +- **e2e** — `test/e2e/var/` (pattern: `test/e2e/env/`): + - `it_sets_file_level_var.clitest` + - `it_var_available_to_all_entries.clitest` + - `it_captures_override_var.clitest` + - `it_cli_var_overrides_var.clitest` + - `it_rejects_duplicate_var.clitest` + - `it_rejects_malformed_var.clitest` +- **examples** — `examples/09_var.clitest`: the sibling-file scenario + from the issue (one shared value driving several paths/commands via + one frontmatter line). + +## Error handling + +Both new failure modes (malformed value, duplicate key) surface as +`*parser.DirectiveError` — already handled end-to-end in +`cmd/clitest/run.go` (formatted with file:line prefix, same path as +existing `@env`/`@timeout` validation errors). No new error-handling +code needed there. From b16062ff42f33f752a8bc1121baecbd69dff28f4 Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 11:28:39 +0200 Subject: [PATCH 02/10] docs: add implementation plan for file-level @var directive (#60) Co-Authored-By: Claude Sonnet 5 --- .../2026-09-17-file-level-var-directive.md | 704 ++++++++++++++++++ 1 file changed, 704 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-17-file-level-var-directive.md diff --git a/docs/superpowers/plans/2026-09-17-file-level-var-directive.md b/docs/superpowers/plans/2026-09-17-file-level-var-directive.md new file mode 100644 index 0000000..3a2eafc --- /dev/null +++ b/docs/superpowers/plans/2026-09-17-file-level-var-directive.md @@ -0,0 +1,704 @@ +# File-level `@var` directive Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a frontmatter directive `@var KEY=VALUE` that sets a file-scoped default for the `{{...}}` template system, so sibling `.clitest` files that differ by one value need only change one frontmatter line instead of repeating the value on every line that uses it. + +**Architecture:** `@var` is parsed exactly like the existing `@env` directive (file-level only, into a new `FileDirectives.Var map[string]string`), with a duplicate-key check `@env` doesn't have. Resolution needs no change to the existing load-time `{{...}}` substitution pass (`--var` + `$ENV`, unchanged). Instead, the single `captures` map created once per file run and threaded by reference through every executor call is seeded with the file's `@var` values at creation. Real captures then overwrite seeded values on the same map via ordinary map-write semantics — giving `--var > [captures] > @var` for free, with zero changes inside the executor package. + +**Tech Stack:** Go (stdlib only — `strings`, `strconv`), existing `.clitest`-file-based test suite (unit tests via `go test`, e2e via the `clitest` binary running `.clitest` fixture files against itself). + +**Spec:** `docs/superpowers/specs/2026-09-17-file-level-var-directive-design.md` + +## Global Constraints + +- File-level only — no `EntryDirectives.Var`. An `@var` line placed after the frontmatter is silently ignored, same as any other unknown directive (`SPEC.md:636`, unchanged). +- Duplicate `@var` key within one frontmatter block is a hard parse error (`*parser.DirectiveError`), not last-write-wins. +- Malformed `@var` (no `=`) is a hard parse error, same message shape as `@env`'s. +- `@var` never touches `@env`/`$VAR` — those remain a fully separate mechanism. + +--- + +### Task 1: Parse `@var` directive (file-level, with duplicate detection) + +**Files:** +- Modify: `internal/types/types.go` (`FileDirectives` struct, ~line 18-25) +- Modify: `internal/parser/directives.go` (`interpretFileDirectives`, ~line 93-126) +- Test: `internal/parser/parser_test.go` + +**Interfaces:** +- Produces: `types.FileDirectives.Var map[string]string` — nil if no `@var` in the file, otherwise `KEY -> VALUE`. Consumed by Task 3 (`cmd/clitest/root.go`, reading `parsed.Directives.Var`). + +- [ ] **Step 1: Add the `Var` field to `FileDirectives`** + +In `internal/types/types.go`, add to the `FileDirectives` struct (next to the existing `Env` field): + +```go +// FileDirectives holds interpreted directives for a file. +type FileDirectives struct { + Groups []string + Skip bool + SkipReason string + Timeout *int // @timeout in ms (nil = not set) + Workdir string // @workdir path (empty = not set) + Env map[string]string // @env KEY=VALUE (nil = not set) + Var map[string]string // @var KEY=VALUE (nil = not set, file-level only) +} +``` + +- [ ] **Step 2: Write the failing tests** + +Add to `internal/parser/parser_test.go` (near the existing `TestParseEnvDirective_*` tests, ~line 850): + +```go +func TestParseVarDirective_FileLevel(t *testing.T) { + input := `--- +@var SCENARIO=checkout/case-a +@var PHP=php81 +--- + +echo test +EXIT 0 +` + f, errs := ParseFile(input) + if len(errs) > 0 { + t.Fatalf("unexpected error: %v", errs) + } + v := f.Directives.Var + if len(v) != 2 { + t.Fatalf("expected 2 file-level vars, got %d", len(v)) + } + assertEqual(t, v["SCENARIO"], "checkout/case-a") + assertEqual(t, v["PHP"], "php81") +} + +func TestParseVarDirective_ValueContainsEquals(t *testing.T) { + input := `--- +@var CONN=host=localhost;port=5432 +--- + +echo test +EXIT 0 +` + f, errs := ParseFile(input) + if len(errs) > 0 { + t.Fatalf("unexpected error: %v", errs) + } + assertEqual(t, f.Directives.Var["CONN"], "host=localhost;port=5432") +} + +func TestParseVarDirective_EmptyValue(t *testing.T) { + input := `--- +@var KEY= +--- + +echo test +EXIT 0 +` + f, errs := ParseFile(input) + if len(errs) > 0 { + t.Fatalf("unexpected error: %v", errs) + } + val, exists := f.Directives.Var["KEY"] + if !exists { + t.Fatal("expected KEY to exist in var") + } + if val != "" { + t.Errorf("expected empty value, got %q", val) + } +} + +func TestParseVarDirective_NoEquals_Error(t *testing.T) { + input := `--- +@var NOVALUE +--- + +echo test +EXIT 0 +` + _, errs := ParseFile(input) + if len(errs) == 0 { + t.Fatal("expected error for @var without =, got none") + } + if !strings.Contains(errs[0].Error(), "@var") { + t.Fatalf("expected @var error, got: %v", errs[0]) + } +} + +func TestParseVarDirective_DuplicateKey_Error(t *testing.T) { + input := `--- +@var X=first +@var X=second +--- + +echo test +EXIT 0 +` + _, errs := ParseFile(input) + if len(errs) == 0 { + t.Fatal("expected error for duplicate @var key, got none") + } + if !strings.Contains(errs[0].Error(), "@var") || !strings.Contains(errs[0].Error(), "X") { + t.Fatalf("expected @var duplicate-key error mentioning X, got: %v", errs[0]) + } +} + +func TestParseVarDirective_EntryLevelIgnored(t *testing.T) { + // @var after the frontmatter is an unknown entry directive — silently + // ignored, same as any other unrecognized @directive (SPEC.md). + input := `@var NAME=foo +echo {{NAME}} +EXIT 0 +` + f, errs := ParseFile(input) + if len(errs) > 0 { + t.Fatalf("unexpected error: %v", errs) + } + if f.Entries[0].Command != "echo {{NAME}}" { + t.Errorf("expected placeholder left literal, got %q", f.Entries[0].Command) + } +} +``` + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `go test ./internal/parser/... -run TestParseVarDirective -v` +Expected: FAIL — `f.Directives.Var` doesn't compile yet (undefined field). + +- [ ] **Step 4: Implement the `var` case in `interpretFileDirectives`** + +In `internal/parser/directives.go`, add a `case "var":` inside the `switch d.Name` in `interpretFileDirectives` (after the existing `case "env":` block, ~line 122). Match the existing `if`/`else` style used by the `env`/`timeout` cases right above it (reading `f.Directives.Var[parts[0]]` on a possibly-nil map is safe in Go — it returns `"", false`): + +```go + case "var": + parts := strings.SplitN(d.Value, "=", 2) + if len(parts) < 2 || parts[0] == "" { + errs = append(errs, &DirectiveError{Line: d.Line, Directive: d.Name, Message: fmt.Sprintf("value must be KEY=VALUE with non-empty key, got %q", d.Value)}) + } else if _, exists := f.Directives.Var[parts[0]]; exists { + errs = append(errs, &DirectiveError{Line: d.Line, Directive: d.Name, Message: fmt.Sprintf("duplicate key %q (already defined earlier in this file)", parts[0])}) + } else { + if f.Directives.Var == nil { + f.Directives.Var = make(map[string]string) + } + f.Directives.Var[parts[0]] = parts[1] + } +``` + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `go test ./internal/parser/... -run TestParseVarDirective -v` +Expected: PASS (all 6 subtests) + +- [ ] **Step 6: Run the full parser package test suite to check for regressions** + +Run: `go test ./internal/parser/...` +Expected: PASS + +- [ ] **Step 7: Commit** + +```bash +git add internal/types/types.go internal/parser/directives.go internal/parser/parser_test.go +git commit -m "feat: parse file-level @var directive" +``` + +--- + +### Task 2: `vars.SeedCaptures` helper + +**Files:** +- Modify: `internal/vars/vars.go` +- Test: `internal/vars/vars_test.go` + +**Interfaces:** +- Consumes: nothing new. +- Produces: `vars.SeedCaptures(fileVars map[string]string) map[string]string` — returns a new map containing a copy of `fileVars` (empty, non-nil map if `fileVars` is nil or empty). Consumed by Task 3. + +- [ ] **Step 1: Write the failing tests** + +Add to `internal/vars/vars_test.go`: + +```go +func TestSeedCaptures_CopiesValues(t *testing.T) { + fileVars := map[string]string{"NAME": "world"} + got := SeedCaptures(fileVars) + if got["NAME"] != "world" { + t.Errorf("expected 'world', got %q", got["NAME"]) + } +} + +func TestSeedCaptures_DoesNotAliasInput(t *testing.T) { + fileVars := map[string]string{"NAME": "world"} + got := SeedCaptures(fileVars) + got["NAME"] = "mutated" + if fileVars["NAME"] != "world" { + t.Errorf("expected original map untouched, got %q", fileVars["NAME"]) + } +} + +func TestSeedCaptures_NilInput(t *testing.T) { + got := SeedCaptures(nil) + if got == nil { + t.Fatal("expected non-nil map for nil input") + } + if len(got) != 0 { + t.Errorf("expected empty map, got %d entries", len(got)) + } +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `go test ./internal/vars/... -run TestSeedCaptures -v` +Expected: FAIL — `SeedCaptures` undefined. + +- [ ] **Step 3: Implement `SeedCaptures`** + +Add to `internal/vars/vars.go` (after `SubstituteCaptures`): + +```go +// SeedCaptures returns a new map pre-populated with fileVars, for use as the +// starting point of an entry's captures map. The returned map never aliases +// fileVars, so later writes (real captures) never mutate the caller's map. +func SeedCaptures(fileVars map[string]string) map[string]string { + seeded := make(map[string]string, len(fileVars)) + for k, v := range fileVars { + seeded[k] = v + } + return seeded +} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `go test ./internal/vars/... -run TestSeedCaptures -v` +Expected: PASS (all 3 subtests) + +- [ ] **Step 5: Run the full vars package test suite to check for regressions** + +Run: `go test ./internal/vars/...` +Expected: PASS + +- [ ] **Step 6: Commit** + +```bash +git add internal/vars/vars.go internal/vars/vars_test.go +git commit -m "feat: add vars.SeedCaptures helper" +``` + +--- + +### Task 3: Wire `@var` into the run — seed the captures map from `parsed.Directives.Var` + +**Files:** +- Modify: `cmd/clitest/run.go` (`runEntries`, `runEntriesVerbose`, `runEntriesCompact`) +- Modify: `cmd/clitest/root.go` (`processFile`, both call sites) +- Modify: `cmd/clitest/run_test.go` (existing `runEntries` call sites) +- Test: `cmd/clitest/run_test.go` (new end-to-end test) + +**Interfaces:** +- Consumes: `types.FileDirectives.Var` (Task 1), `vars.SeedCaptures` (Task 2). +- Produces: `runEntries(cfg *runConfig, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry, fileVars map[string]string, onResult func(entryOutcome)) (pass, fail, skip int, details []display.CompactFailure)` — new `fileVars` parameter inserted before `onResult`. Same signature-shape addition on `runEntriesVerbose` and `runEntriesCompact` (see steps below for exact placement). + +- [ ] **Step 1: Write the failing end-to-end test** + +Add to `cmd/clitest/run_test.go`: + +```go +func TestRunEntries_FileVarResolvesInCommand(t *testing.T) { + entries := []types.Entry{ + {Command: "echo {{NAME}}", Asserts: []types.Assert{{Query: "stdout", Predicate: "contains", Value: "world"}}}, + } + fileVars := map[string]string{"NAME": "world"} + + cancelled := &atomic.Bool{} + cfg := &runConfig{FailFast: false, Cancelled: cancelled} + pd := newTestProgressDisplay() + + pass, fail, _, _ := runEntries(cfg, pd, 0, entries, fileVars, nil) + + if fail != 0 { + t.Errorf("expected 0 failures, got %d", fail) + } + if pass != 1 { + t.Errorf("expected 1 pass, got %d", pass) + } +} + +func TestRunEntries_CaptureOverridesFileVar(t *testing.T) { + entries := []types.Entry{ + { + Command: "echo captured_val", + Captures: []types.Capture{{Name: "NAME", Source: types.CaptureStdout}}, + Asserts: []types.Assert{{Query: "exit", Predicate: "eq", Value: "0"}}, + }, + { + Command: "echo {{NAME}}", + Asserts: []types.Assert{{Query: "stdout", Predicate: "contains", Value: "captured_val"}}, + }, + } + fileVars := map[string]string{"NAME": "default_val"} + + cancelled := &atomic.Bool{} + cfg := &runConfig{FailFast: false, Cancelled: cancelled} + pd := newTestProgressDisplay() + + pass, fail, _, _ := runEntries(cfg, pd, 0, entries, fileVars, nil) + + if fail != 0 { + t.Errorf("expected 0 failures, got %d", fail) + } + if pass != 2 { + t.Errorf("expected 2 pass, got %d", pass) + } +} +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `go test ./cmd/clitest/... -run TestRunEntries_FileVarResolvesInCommand -v` +Expected: FAIL — compile error, `runEntries` called with wrong number of arguments. + +- [ ] **Step 3: Update `runEntries` signature and seed `captures`** + +In `cmd/clitest/run.go`, change the `runEntries` signature (~line 44) and the `captures` initialization (~line 47): + +```go +func runEntries(cfg *runConfig, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry, fileVars map[string]string, onResult func(entryOutcome)) (pass, fail, skip int, details []display.CompactFailure) { + regular, defers := executor.SplitDeferEntries(entries) + pd.UpdateProgress(fileIdx, 0, len(regular)) + captures := vars.SeedCaptures(fileVars) + var backgrounds []*executor.BackgroundResult +``` + +- [ ] **Step 4: Thread `fileVars` through `runEntriesVerbose` and `runEntriesCompact`** + +In `cmd/clitest/run.go`, update `runEntriesVerbose` (~line 149) — add the parameter and pass it through to `runEntries` (~line 168): + +```go +func runEntriesVerbose(cfg *runConfig, vd *display.VerboseDisplay, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry, fileVars map[string]string) (pass, fail, skip int) { +``` + +```go + pass, fail, skip, _ = runEntries(cfg, pd, fileIdx, entries, fileVars, onResult) + return +``` + +And `runEntriesCompact` (~line 173): + +```go +func runEntriesCompact(cfg *runConfig, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry, fileVars map[string]string) (pass, fail, skip int, details []display.CompactFailure) { + return runEntries(cfg, pd, fileIdx, entries, fileVars, nil) +} +``` + +- [ ] **Step 5: Update the two call sites in `cmd/clitest/root.go`** + +In `processFile` (~line 234 and ~line 240): + +```go + pass, fail, skip := runEntriesVerbose(cfg, vd, pd, idx, entries, parsed.Directives.Var) +``` + +```go + pass, fail, skip, details := runEntriesCompact(cfg, pd, idx, entries, parsed.Directives.Var) +``` + +- [ ] **Step 6: Update the existing `runEntries` call sites in `cmd/clitest/run_test.go`** + +`TestRunEntries_FailFastCancels` (~line 70) and `TestRunEntries_PreCancelledSkipsAll` (~line 97) currently call `runEntries(cfg, pd, 0, entries, nil)` — insert `nil` for `fileVars` before the trailing `onResult` argument: + +```go + pass, fail, skip, _ := runEntries(cfg, pd, 0, entries, nil, nil) +``` + +(apply to both test functions) + +- [ ] **Step 7: Run the new tests to verify they pass** + +Run: `go test ./cmd/clitest/... -run TestRunEntries -v` +Expected: PASS (including the two pre-existing `TestRunEntries_*` tests and the two new ones) + +- [ ] **Step 8: Run the full repo test suite to check for regressions** + +Run: `go test ./...` +Expected: PASS + +- [ ] **Step 9: Commit** + +```bash +git add cmd/clitest/run.go cmd/clitest/root.go cmd/clitest/run_test.go +git commit -m "feat: seed captures map from file-level @var, wire through run path" +``` + +--- + +### Task 4: Document `@var` in `SPEC.md` and add `examples/09_var.clitest` + +**Files:** +- Modify: `SPEC.md` +- Create: `examples/09_var.clitest` + +**Interfaces:** +- Consumes: the working `@var` feature from Tasks 1-3 (this task's example file is executed by `make examples`, which is a form of test). +- Produces: nothing consumed by later tasks. + +- [ ] **Step 1: Update the Variables priority list** + +In `SPEC.md`, replace the priority list at line 432-436: + +```markdown +Variables are resolved from (in priority order): +1. `--var NAME=VALUE` CLI flags +2. `[captures]` from previous entries +3. `@var NAME=VALUE` file-level frontmatter directive +4. Environment variables (via `$VAR` or `${VAR}` syntax) +``` + +- [ ] **Step 2: Add `@var` to the Available Directives table** + +In `SPEC.md`, add a row after the `@env` row (line 499): + +```markdown +| `@var` | file | `@var KEY=VALUE` | Set a default for `{{KEY}}` template substitution, scoped to this file | +``` + +- [ ] **Step 3: Add the `#### @var` subsection** + +In `SPEC.md`, insert after the `#### @env` subsection (after line 622, before `#### Directive Validation`): + +```markdown +#### `@var` + +Sets a file-scoped default value for `{{KEY}}` template substitution. File-level only (frontmatter) — placing `@var` after the frontmatter has no effect (unknown entry directives are silently ignored). One `KEY=VALUE` per directive. A duplicate key within the same file's frontmatter is a parse error. + +**Precedence:** `--var` (CLI) > `[captures]` > `@var` (file) — see [Variables](#variables). + +**Values:** The value is everything after the first `=`. Values may contain `=`, spaces, and special characters. Empty values (`@var KEY=`) are valid. Malformed directives without `=` are rejected at parse time, as is a repeated key. + +**Use case:** sibling `.clitest` files that differ by exactly one value (a scenario directory, a tool version) can express that value once, instead of repeating it on every line that needs it: + +``` +--- +@var SCENARIO=checkout/case-a +--- + +hurl --test tests/{{SCENARIO}}/stubs/_register.hurl +EXIT 0 + +php bin/console app:seed tests/{{SCENARIO}}/seller.json +EXIT 0 +``` + +A sibling file for `case-b` differs from this one only in the frontmatter line. +``` + +- [ ] **Step 4: Add `@var` to the Directive Validation table** + +In `SPEC.md`, add a row to the validation table (after line 634): + +```markdown +| `@var` | `KEY=VALUE` with non-empty key | Empty value after `=` is valid. Missing `=` is rejected. Duplicate key within one file is rejected. | +``` + +- [ ] **Step 5: Write `examples/09_var.clitest`** + +``` +--- +@var SCENARIO=checkout/case-a +--- + +# All entries in this file see {{SCENARIO}} +echo "running {{SCENARIO}}" +EXIT 0 +running checkout/case-a + +# A sibling file for case-b would differ only in the frontmatter line above +echo "path: tests/{{SCENARIO}}/seller.json" +EXIT 0 +path: tests/checkout/case-a/seller.json +``` + +- [ ] **Step 6: Run the examples suite to verify it passes** + +Run: `make build examples` +Expected: all examples pass, including `09_var.clitest` + +- [ ] **Step 7: Commit** + +```bash +git add SPEC.md examples/09_var.clitest +git commit -m "docs: document @var directive, add example" +``` + +--- + +### Task 5: e2e positive-behavior tests for `@var` + +**Files:** +- Create: `test/e2e/var/it_sets_file_level_var.clitest` +- Create: `test/e2e/var/it_var_available_to_all_entries.clitest` +- Create: `test/e2e/var/it_captures_override_var.clitest` +- Create: `test/e2e/var/it_cli_var_overrides_var.clitest` + +**Interfaces:** +- Consumes: the working `@var` feature from Tasks 1-3. +- Produces: nothing consumed by later tasks. + +- [ ] **Step 1: `it_sets_file_level_var.clitest`** + +``` +--- +@var NAME=world +--- + +# File-level var resolves in the command +echo "hello {{NAME}}" +hello world +``` + +- [ ] **Step 2: `it_var_available_to_all_entries.clitest`** + +``` +--- +@var SCENARIO=case-a +--- + +# First entry sees the file-level var +echo "{{SCENARIO}}" +case-a + +# Second entry also sees it +echo "path/{{SCENARIO}}/file" +path/case-a/file +``` + +- [ ] **Step 3: `it_captures_override_var.clitest`** + +``` +--- +@var NAME=default_val +--- + +# Capture a value under the same key as the file-level var +echo "captured_val" +EXIT 0 +[captures] +NAME stdout + +# Subsequent entries see the captured value, not the file-level default +echo "{{NAME}}" +captured_val +``` + +- [ ] **Step 4: `it_cli_var_overrides_var.clitest`** + +``` +--- +@var NAME=file_value +--- + +# --var (passed on the command line invoking this fixture) beats @var +echo "{{NAME}}" +cli_value +``` + +This fixture is invoked with `--var NAME=cli_value` by a wrapper entry in `test/e2e/syntax/` — **do not** add it directly to `test/e2e/var/` for standalone execution, since `make e2e` runs `./clitest --silent test/e2e/` with no `--var` flag and this fixture would then see `file_value`, failing the assertion. Instead: + +- [ ] **Step 4a: Move `it_cli_var_overrides_var.clitest` under `test/_fixtures/syntax/`** + +Move the file to `test/_fixtures/syntax/var_cli_override.clitest` (same content as Step 4). + +- [ ] **Step 4b: Add the wrapper entry** + +Append to `test/e2e/syntax/it_validates_directives.clitest`: + +``` + +# Validates that --var overrides file-level @var +./clitest --var NAME=cli_value test/_fixtures/syntax/var_cli_override.clitest +EXIT 0 +``` + +- [ ] **Step 5: Run the e2e suite to verify all new fixtures pass** + +Run: `make build e2e` +Expected: all e2e tests pass, including the 3 files under `test/e2e/var/` and the new wrapper entry in `test/e2e/syntax/it_validates_directives.clitest` + +- [ ] **Step 6: Commit** + +```bash +git add test/e2e/var/ test/_fixtures/syntax/var_cli_override.clitest test/e2e/syntax/it_validates_directives.clitest +git commit -m "test: add e2e coverage for @var resolution and precedence" +``` + +--- + +### Task 6: e2e negative-case tests — malformed and duplicate `@var` + +**Files:** +- Create: `test/_fixtures/syntax/bad_var.clitest` +- Create: `test/_fixtures/syntax/duplicate_var.clitest` +- Modify: `test/e2e/syntax/it_validates_directives.clitest` + +**Interfaces:** +- Consumes: the parse-time validation from Task 1. +- Produces: nothing consumed by later tasks. + +- [ ] **Step 1: `test/_fixtures/syntax/bad_var.clitest`** + +``` +--- +@var NOEQUALS +--- + +echo hello +EXIT 0 +``` + +- [ ] **Step 2: `test/_fixtures/syntax/duplicate_var.clitest`** + +``` +--- +@var X=first +@var X=second +--- + +echo hello +EXIT 0 +``` + +- [ ] **Step 3: Append validation entries to `test/e2e/syntax/it_validates_directives.clitest`** + +``` + +# Validates that malformed @var is rejected at parse time +./clitest test/_fixtures/syntax/bad_var.clitest +EXIT 1 +[asserts] +stdout contains "@var" +stdout contains "KEY=VALUE" + +# Validates that duplicate @var key is rejected at parse time +./clitest test/_fixtures/syntax/duplicate_var.clitest +EXIT 1 +[asserts] +stdout contains "@var" +stdout contains "duplicate" +``` + +- [ ] **Step 4: Run the e2e suite to verify the new negative cases pass** + +Run: `make build e2e` +Expected: all e2e tests pass, including the 2 new validation entries + +- [ ] **Step 5: Run the full repo test suite one final time** + +Run: `make test e2e examples` +Expected: everything passes — unit tests, e2e tests, examples + +- [ ] **Step 6: Commit** + +```bash +git add test/_fixtures/syntax/bad_var.clitest test/_fixtures/syntax/duplicate_var.clitest test/e2e/syntax/it_validates_directives.clitest +git commit -m "test: add e2e coverage for malformed and duplicate @var" +``` From 4437f803d6bd1bc830d91910dd3c7a032e3fd092 Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 11:40:28 +0200 Subject: [PATCH 03/10] feat: parse file-level @var directive Co-Authored-By: Claude Haiku 4.5 --- internal/parser/directives.go | 12 ++++ internal/parser/parser_test.go | 108 +++++++++++++++++++++++++++++++++ internal/types/types.go | 1 + 3 files changed, 121 insertions(+) diff --git a/internal/parser/directives.go b/internal/parser/directives.go index 171d27e..0de6bb7 100644 --- a/internal/parser/directives.go +++ b/internal/parser/directives.go @@ -120,6 +120,18 @@ func interpretFileDirectives(f *types.File, directives []directive) []error { } f.Directives.Env[parts[0]] = parts[1] } + case "var": + parts := strings.SplitN(d.Value, "=", 2) + if len(parts) < 2 || parts[0] == "" { + errs = append(errs, &DirectiveError{Line: d.Line, Directive: d.Name, Message: fmt.Sprintf("value must be KEY=VALUE with non-empty key, got %q", d.Value)}) + } else if _, exists := f.Directives.Var[parts[0]]; exists { + errs = append(errs, &DirectiveError{Line: d.Line, Directive: d.Name, Message: fmt.Sprintf("duplicate key %q (already defined earlier in this file)", parts[0])}) + } else { + if f.Directives.Var == nil { + f.Directives.Var = make(map[string]string) + } + f.Directives.Var[parts[0]] = parts[1] + } } } return errs diff --git a/internal/parser/parser_test.go b/internal/parser/parser_test.go index 69437a5..6381d8c 100644 --- a/internal/parser/parser_test.go +++ b/internal/parser/parser_test.go @@ -881,6 +881,114 @@ EXIT 0 assertEqual(t, f.Entries[0].Directives.Env["X"], "second") } +func TestParseVarDirective_FileLevel(t *testing.T) { + input := `--- +@var SCENARIO=checkout/case-a +@var PHP=php81 +--- + +echo test +EXIT 0 +` + f, errs := ParseFile(input) + if len(errs) > 0 { + t.Fatalf("unexpected error: %v", errs) + } + v := f.Directives.Var + if len(v) != 2 { + t.Fatalf("expected 2 file-level vars, got %d", len(v)) + } + assertEqual(t, v["SCENARIO"], "checkout/case-a") + assertEqual(t, v["PHP"], "php81") +} + +func TestParseVarDirective_ValueContainsEquals(t *testing.T) { + input := `--- +@var CONN=host=localhost;port=5432 +--- + +echo test +EXIT 0 +` + f, errs := ParseFile(input) + if len(errs) > 0 { + t.Fatalf("unexpected error: %v", errs) + } + assertEqual(t, f.Directives.Var["CONN"], "host=localhost;port=5432") +} + +func TestParseVarDirective_EmptyValue(t *testing.T) { + input := `--- +@var KEY= +--- + +echo test +EXIT 0 +` + f, errs := ParseFile(input) + if len(errs) > 0 { + t.Fatalf("unexpected error: %v", errs) + } + val, exists := f.Directives.Var["KEY"] + if !exists { + t.Fatal("expected KEY to exist in var") + } + if val != "" { + t.Errorf("expected empty value, got %q", val) + } +} + +func TestParseVarDirective_NoEquals_Error(t *testing.T) { + input := `--- +@var NOVALUE +--- + +echo test +EXIT 0 +` + _, errs := ParseFile(input) + if len(errs) == 0 { + t.Fatal("expected error for @var without =, got none") + } + if !strings.Contains(errs[0].Error(), "@var") { + t.Fatalf("expected @var error, got: %v", errs[0]) + } +} + +func TestParseVarDirective_DuplicateKey_Error(t *testing.T) { + input := `--- +@var X=first +@var X=second +--- + +echo test +EXIT 0 +` + _, errs := ParseFile(input) + if len(errs) == 0 { + t.Fatal("expected error for duplicate @var key, got none") + } + if !strings.Contains(errs[0].Error(), "@var") || !strings.Contains(errs[0].Error(), "X") { + t.Fatalf("expected @var duplicate-key error mentioning X, got: %v", errs[0]) + } +} + +func TestParseVarDirective_EntryLevelIgnored(t *testing.T) { + // @var after the frontmatter is an unknown entry directive — silently + // ignored, same as any other unrecognized @directive (SPEC.md). + input := `@var NAME=foo +echo {{NAME}} +EXIT 0 +` + f, errs := ParseFile(input) + if len(errs) > 0 { + t.Fatalf("unexpected error: %v", errs) + } + if f.Entries[0].Command != "echo {{NAME}}" { + t.Errorf("expected placeholder left literal, got %q", f.Entries[0].Command) + } +} + // --- Multi-line assert (triple-quote) tests --- func TestParseMultilineAssert_Basic(t *testing.T) { diff --git a/internal/types/types.go b/internal/types/types.go index 079a24a..070cc02 100644 --- a/internal/types/types.go +++ b/internal/types/types.go @@ -22,6 +22,7 @@ type FileDirectives struct { Timeout *int // @timeout in ms (nil = not set, 0 = no timeout) Workdir string // @workdir path (empty = not set) Env map[string]string // @env KEY=VALUE (nil = not set) + Var map[string]string // @var KEY=VALUE (nil = not set, file-level only) } // Finally represents a [Finally] section for EXIT NEVER entries. From cbae1172d77c51103a58adcef6a26ff130c684d0 Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 11:43:04 +0200 Subject: [PATCH 04/10] feat: add vars.SeedCaptures helper Adds SeedCaptures function that returns a copy of fileVars for seeding per-run captures map. Ensures returned map never aliases input map. Co-Authored-By: Claude Haiku 4.5 --- internal/vars/vars.go | 11 +++++++++++ internal/vars/vars_test.go | 27 +++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/internal/vars/vars.go b/internal/vars/vars.go index 6030d4d..a218501 100644 --- a/internal/vars/vars.go +++ b/internal/vars/vars.go @@ -30,6 +30,17 @@ func SubstituteCaptures(input string, captures map[string]string) string { return result } +// SeedCaptures returns a new map pre-populated with fileVars, for use as the +// starting point of an entry's captures map. The returned map never aliases +// fileVars, so later writes (real captures) never mutate the caller's map. +func SeedCaptures(fileVars map[string]string) map[string]string { + seeded := make(map[string]string, len(fileVars)) + for k, v := range fileVars { + seeded[k] = v + } + return seeded +} + // ResolveCapture extracts a value from a runner.Result based on the capture definition. func ResolveCapture(c types.Capture, r runner.Result) string { switch c.Source { diff --git a/internal/vars/vars_test.go b/internal/vars/vars_test.go index 3d89a53..5b9dc0d 100644 --- a/internal/vars/vars_test.go +++ b/internal/vars/vars_test.go @@ -162,3 +162,30 @@ func TestResolveCapture_StderrRegex(t *testing.T) { t.Errorf("expected '42', got %q", got) } } + +func TestSeedCaptures_CopiesValues(t *testing.T) { + fileVars := map[string]string{"NAME": "world"} + got := SeedCaptures(fileVars) + if got["NAME"] != "world" { + t.Errorf("expected 'world', got %q", got["NAME"]) + } +} + +func TestSeedCaptures_DoesNotAliasInput(t *testing.T) { + fileVars := map[string]string{"NAME": "world"} + got := SeedCaptures(fileVars) + got["NAME"] = "mutated" + if fileVars["NAME"] != "world" { + t.Errorf("expected original map untouched, got %q", fileVars["NAME"]) + } +} + +func TestSeedCaptures_NilInput(t *testing.T) { + got := SeedCaptures(nil) + if got == nil { + t.Fatal("expected non-nil map for nil input") + } + if len(got) != 0 { + t.Errorf("expected empty map, got %d entries", len(got)) + } +} From 77840cc73fd8013a926d2fb6637ff3cb318d50ce Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 11:46:33 +0200 Subject: [PATCH 05/10] feat: seed captures map from file-level @var, wire through run path Co-Authored-By: Claude Sonnet 5 --- cmd/clitest/root.go | 4 ++-- cmd/clitest/run.go | 12 +++++----- cmd/clitest/run_test.go | 52 +++++++++++++++++++++++++++++++++++++++-- 3 files changed, 58 insertions(+), 10 deletions(-) diff --git a/cmd/clitest/root.go b/cmd/clitest/root.go index abc3cf2..aec67a0 100644 --- a/cmd/clitest/root.go +++ b/cmd/clitest/root.go @@ -236,13 +236,13 @@ func processFile(cfg *runConfig, f resolve.ScanFile, idx int, pd *display.Progre var buf bytes.Buffer vd := display.NewVerboseDisplay(&buf, true) vd.BeginFile(f.Display) - pass, fail, skip := runEntriesVerbose(cfg, vd, pd, idx, entries) + pass, fail, skip := runEntriesVerbose(cfg, vd, pd, idx, entries, parsed.Directives.Var) vd.EndFile() pd.FinishFile(idx, fail == 0, buf.String()) return fileResult{pass: pass, fail: fail, skip: skip, display: f.Display} } - pass, fail, skip, details := runEntriesCompact(cfg, pd, idx, entries) + pass, fail, skip, details := runEntriesCompact(cfg, pd, idx, entries, parsed.Directives.Var) pd.FinishFile(idx, fail == 0, "") return fileResult{pass: pass, fail: fail, skip: skip, display: f.Display, failures: details} } diff --git a/cmd/clitest/run.go b/cmd/clitest/run.go index 3c3665a..46a25b6 100644 --- a/cmd/clitest/run.go +++ b/cmd/clitest/run.go @@ -41,10 +41,10 @@ type entryOutcome struct { // runEntries executes entries, reports progress, and returns counters + failure details. // The onResult callback is invoked for each entry outcome (including skipped entries). // In verbose mode, pass a callback that writes to VerboseDisplay; in compact mode, pass nil. -func runEntries(cfg *runConfig, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry, onResult func(entryOutcome)) (pass, fail, skip int, details []display.CompactFailure) { +func runEntries(cfg *runConfig, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry, fileVars map[string]string, onResult func(entryOutcome)) (pass, fail, skip int, details []display.CompactFailure) { regular, defers := executor.SplitDeferEntries(entries) pd.UpdateProgress(fileIdx, 0, len(regular)) - captures := map[string]string{} + captures := vars.SeedCaptures(fileVars) var backgrounds []*executor.BackgroundResult for i, entry := range regular { @@ -146,7 +146,7 @@ func runEntries(cfg *runConfig, pd *display.ProgressDisplay, fileIdx int, entrie } // runEntriesVerbose executes entries and reports to VerboseDisplay. -func runEntriesVerbose(cfg *runConfig, vd *display.VerboseDisplay, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry) (pass, fail, skip int) { +func runEntriesVerbose(cfg *runConfig, vd *display.VerboseDisplay, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry, fileVars map[string]string) (pass, fail, skip int) { onResult := func(o entryOutcome) { if o.SkipReason == "defer" { vd.DeferResult(o.Command) @@ -165,13 +165,13 @@ func runEntriesVerbose(cfg *runConfig, vd *display.VerboseDisplay, pd *display.P Stderr: o.Stderr, }) } - pass, fail, skip, _ = runEntries(cfg, pd, fileIdx, entries, onResult) + pass, fail, skip, _ = runEntries(cfg, pd, fileIdx, entries, fileVars, onResult) return } // runEntriesCompact executes entries and reports progress to ProgressDisplay. -func runEntriesCompact(cfg *runConfig, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry) (pass, fail, skip int, details []display.CompactFailure) { - return runEntries(cfg, pd, fileIdx, entries, nil) +func runEntriesCompact(cfg *runConfig, pd *display.ProgressDisplay, fileIdx int, entries []types.Entry, fileVars map[string]string) (pass, fail, skip int, details []display.CompactFailure) { + return runEntries(cfg, pd, fileIdx, entries, fileVars, nil) } // processBackgrounds evaluates later asserts and finally sections for all kept-alive background processes. diff --git a/cmd/clitest/run_test.go b/cmd/clitest/run_test.go index 20df04a..9d3d817 100644 --- a/cmd/clitest/run_test.go +++ b/cmd/clitest/run_test.go @@ -67,7 +67,7 @@ func TestRunEntries_FailFastCancels(t *testing.T) { cfg := &runConfig{FailFast: true, Cancelled: cancelled} pd := newTestProgressDisplay() - pass, fail, skip, _ := runEntries(cfg, pd, 0, entries, nil) + pass, fail, skip, _ := runEntries(cfg, pd, 0, entries, nil, nil) if fail != 1 { t.Errorf("expected 1 failure, got %d", fail) @@ -94,7 +94,7 @@ func TestRunEntries_PreCancelledSkipsAll(t *testing.T) { cfg := &runConfig{FailFast: false, Cancelled: cancelled} pd := newTestProgressDisplay() - pass, fail, skip, _ := runEntries(cfg, pd, 0, entries, nil) + pass, fail, skip, _ := runEntries(cfg, pd, 0, entries, nil, nil) if skip != 2 { t.Errorf("expected 2 skipped, got %d", skip) @@ -107,6 +107,54 @@ func TestRunEntries_PreCancelledSkipsAll(t *testing.T) { } } +func TestRunEntries_FileVarResolvesInCommand(t *testing.T) { + entries := []types.Entry{ + {Command: "echo {{NAME}}", Asserts: []types.Assert{{Query: "stdout", Predicate: "contains", Value: "world"}}}, + } + fileVars := map[string]string{"NAME": "world"} + + cancelled := &atomic.Bool{} + cfg := &runConfig{FailFast: false, Cancelled: cancelled} + pd := newTestProgressDisplay() + + pass, fail, _, _ := runEntries(cfg, pd, 0, entries, fileVars, nil) + + if fail != 0 { + t.Errorf("expected 0 failures, got %d", fail) + } + if pass != 1 { + t.Errorf("expected 1 pass, got %d", pass) + } +} + +func TestRunEntries_CaptureOverridesFileVar(t *testing.T) { + entries := []types.Entry{ + { + Command: "echo captured_val", + Captures: []types.Capture{{Name: "NAME", Source: types.CaptureStdout}}, + Asserts: []types.Assert{{Query: "exit", Predicate: "==", Value: "0"}}, + }, + { + Command: "echo {{NAME}}", + Asserts: []types.Assert{{Query: "stdout", Predicate: "contains", Value: "captured_val"}}, + }, + } + fileVars := map[string]string{"NAME": "default_val"} + + cancelled := &atomic.Bool{} + cfg := &runConfig{FailFast: false, Cancelled: cancelled} + pd := newTestProgressDisplay() + + pass, fail, _, _ := runEntries(cfg, pd, 0, entries, fileVars, nil) + + if fail != 0 { + t.Errorf("expected 0 failures, got %d", fail) + } + if pass != 2 { + t.Errorf("expected 2 pass, got %d", pass) + } +} + func TestProcessBackgrounds_FailFastOnLaterFailure(t *testing.T) { // We cannot easily construct a real BackgroundResult with a running process, // so we test with an empty slice to verify no panic, and rely on E2E tests From a49d5fbf839b258c94ce07bcd48736980fa4f723 Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 11:49:18 +0200 Subject: [PATCH 06/10] docs: document @var directive, add example Co-Authored-By: Claude Haiku 4.5 --- SPEC.md | 29 ++++++++++++++++++++++++++++- examples/09_var.clitest | 13 +++++++++++++ 2 files changed, 41 insertions(+), 1 deletion(-) create mode 100644 examples/09_var.clitest diff --git a/SPEC.md b/SPEC.md index 62ab3a1..f394542 100644 --- a/SPEC.md +++ b/SPEC.md @@ -432,7 +432,8 @@ EXIT 0 Variables are resolved from (in priority order): 1. `--var NAME=VALUE` CLI flags 2. `[captures]` from previous entries -3. Environment variables (via `$VAR` or `${VAR}` syntax) +3. `@var NAME=VALUE` file-level frontmatter directive +4. Environment variables (via `$VAR` or `${VAR}` syntax) ### Environment Variable Expansion @@ -497,6 +498,7 @@ EXIT 0 | `@defer` | entry | `@defer` | Marks entry as cleanup — runs at file end (LIFO), always, not a test | | `@workdir` | file, entry | `@workdir ` | Run command in specified directory | | `@env` | file, entry | `@env KEY=VALUE` | Set environment variable for child process | +| `@var` | file | `@var KEY=VALUE` | Set a default for `{{KEY}}` template substitution, scoped to this file | #### `@group` @@ -621,6 +623,30 @@ EXIT 0 stdout == "false" ``` +#### `@var` + +Sets a file-scoped default value for `{{KEY}}` template substitution. File-level only (frontmatter) — placing `@var` after the frontmatter has no effect (unknown entry directives are silently ignored). One `KEY=VALUE` per directive. A duplicate key within the same file's frontmatter is a parse error. + +**Precedence:** `--var` (CLI) > `[captures]` > `@var` (file) — see [Variables](#variables). + +**Values:** The value is everything after the first `=`. Values may contain `=`, spaces, and special characters. Empty values (`@var KEY=`) are valid. Malformed directives without `=` are rejected at parse time, as is a repeated key. + +**Use case:** sibling `.clitest` files that differ by exactly one value (a scenario directory, a tool version) can express that value once, instead of repeating it on every line that needs it: + +``` +--- +@var SCENARIO=checkout/case-a +--- + +hurl --test tests/{{SCENARIO}}/stubs/_register.hurl +EXIT 0 + +php bin/console app:seed tests/{{SCENARIO}}/seller.json +EXIT 0 +``` + +A sibling file for `case-b` differs from this one only in the frontmatter line. + #### Directive Validation All directives are validated at parse time. If any directive has an invalid value, clitest reports the error(s) and does not execute the file. Multiple errors may be reported at once. @@ -632,6 +658,7 @@ All directives are validated at parse time. If any directive has an invalid valu | `@timeout` | Integer >= 0 (milliseconds) | `0` means no timeout (infinite wait). No duration suffixes. | | `@poll` | Integer > 0 (milliseconds) | Must be positive. No duration suffixes. | | `@env` | `KEY=VALUE` with non-empty key | Empty value after `=` is valid. Missing `=` is rejected. | +| `@var` | `KEY=VALUE` with non-empty key | Empty value after `=` is valid. Missing `=` is rejected. Duplicate key within one file is rejected. | Unknown directives are silently ignored. diff --git a/examples/09_var.clitest b/examples/09_var.clitest new file mode 100644 index 0000000..6b10103 --- /dev/null +++ b/examples/09_var.clitest @@ -0,0 +1,13 @@ +--- +@var SCENARIO=checkout/case-a +--- + +# All entries in this file see {{SCENARIO}} +echo "running {{SCENARIO}}" +EXIT 0 +running checkout/case-a + +# A sibling file for case-b would differ only in the frontmatter line above +echo "path: tests/{{SCENARIO}}/seller.json" +EXIT 0 +path: tests/checkout/case-a/seller.json From 3c4e0d29d9930ea77ea1a9c7a73df4da4f9200bc Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 11:53:57 +0200 Subject: [PATCH 07/10] test: add e2e coverage for @var resolution and precedence Co-Authored-By: Claude Haiku 4.5 --- test/_fixtures/syntax/var_cli_override.clitest | 7 +++++++ test/e2e/syntax/it_validates_directives.clitest | 4 ++++ test/e2e/var/it_captures_override_var.clitest | 13 +++++++++++++ test/e2e/var/it_sets_file_level_var.clitest | 7 +++++++ .../e2e/var/it_var_available_to_all_entries.clitest | 11 +++++++++++ 5 files changed, 42 insertions(+) create mode 100644 test/_fixtures/syntax/var_cli_override.clitest create mode 100644 test/e2e/var/it_captures_override_var.clitest create mode 100644 test/e2e/var/it_sets_file_level_var.clitest create mode 100644 test/e2e/var/it_var_available_to_all_entries.clitest diff --git a/test/_fixtures/syntax/var_cli_override.clitest b/test/_fixtures/syntax/var_cli_override.clitest new file mode 100644 index 0000000..7483652 --- /dev/null +++ b/test/_fixtures/syntax/var_cli_override.clitest @@ -0,0 +1,7 @@ +--- +@var NAME=file_value +--- + +# --var (passed on the command line invoking this fixture) beats @var +echo "{{NAME}}" +cli_value diff --git a/test/e2e/syntax/it_validates_directives.clitest b/test/e2e/syntax/it_validates_directives.clitest index 3751602..27cdbea 100644 --- a/test/e2e/syntax/it_validates_directives.clitest +++ b/test/e2e/syntax/it_validates_directives.clitest @@ -22,3 +22,7 @@ stdout contains "KEY=VALUE" # Validates that file-level @timeout is inherited by entries ./clitest test/_fixtures/syntax/file_timeout_inherit.clitest EXIT 0 + +# Validates that --var overrides file-level @var +./clitest --var NAME=cli_value test/_fixtures/syntax/var_cli_override.clitest +EXIT 0 diff --git a/test/e2e/var/it_captures_override_var.clitest b/test/e2e/var/it_captures_override_var.clitest new file mode 100644 index 0000000..a371ddd --- /dev/null +++ b/test/e2e/var/it_captures_override_var.clitest @@ -0,0 +1,13 @@ +--- +@var NAME=default_val +--- + +# Capture a value under the same key as the file-level var +echo "captured_val" +EXIT 0 +[captures] +NAME = stdout + +# Subsequent entries see the captured value, not the file-level default +echo "{{NAME}}" +captured_val diff --git a/test/e2e/var/it_sets_file_level_var.clitest b/test/e2e/var/it_sets_file_level_var.clitest new file mode 100644 index 0000000..86508e4 --- /dev/null +++ b/test/e2e/var/it_sets_file_level_var.clitest @@ -0,0 +1,7 @@ +--- +@var NAME=world +--- + +# File-level var resolves in the command +echo "hello {{NAME}}" +hello world diff --git a/test/e2e/var/it_var_available_to_all_entries.clitest b/test/e2e/var/it_var_available_to_all_entries.clitest new file mode 100644 index 0000000..7edb826 --- /dev/null +++ b/test/e2e/var/it_var_available_to_all_entries.clitest @@ -0,0 +1,11 @@ +--- +@var SCENARIO=case-a +--- + +# First entry sees the file-level var +echo "{{SCENARIO}}" +case-a + +# Second entry also sees it +echo "path/{{SCENARIO}}/file" +path/case-a/file From 83d0febf22ba99dfe1153f2a0af3c93364c4f620 Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 11:57:19 +0200 Subject: [PATCH 08/10] test: add e2e coverage for malformed and duplicate @var Co-Authored-By: Claude Haiku 4.5 --- test/_fixtures/syntax/bad_var.clitest | 6 ++++++ test/_fixtures/syntax/duplicate_var.clitest | 7 +++++++ test/e2e/syntax/it_validates_directives.clitest | 14 ++++++++++++++ 3 files changed, 27 insertions(+) create mode 100644 test/_fixtures/syntax/bad_var.clitest create mode 100644 test/_fixtures/syntax/duplicate_var.clitest diff --git a/test/_fixtures/syntax/bad_var.clitest b/test/_fixtures/syntax/bad_var.clitest new file mode 100644 index 0000000..b2d6e79 --- /dev/null +++ b/test/_fixtures/syntax/bad_var.clitest @@ -0,0 +1,6 @@ +--- +@var NOEQUALS +--- + +echo hello +EXIT 0 diff --git a/test/_fixtures/syntax/duplicate_var.clitest b/test/_fixtures/syntax/duplicate_var.clitest new file mode 100644 index 0000000..61c7742 --- /dev/null +++ b/test/_fixtures/syntax/duplicate_var.clitest @@ -0,0 +1,7 @@ +--- +@var X=first +@var X=second +--- + +echo hello +EXIT 0 diff --git a/test/e2e/syntax/it_validates_directives.clitest b/test/e2e/syntax/it_validates_directives.clitest index 27cdbea..9762c11 100644 --- a/test/e2e/syntax/it_validates_directives.clitest +++ b/test/e2e/syntax/it_validates_directives.clitest @@ -26,3 +26,17 @@ EXIT 0 # Validates that --var overrides file-level @var ./clitest --var NAME=cli_value test/_fixtures/syntax/var_cli_override.clitest EXIT 0 + +# Validates that malformed @var is rejected at parse time +./clitest test/_fixtures/syntax/bad_var.clitest +EXIT 1 +[asserts] +stdout contains "@var" +stdout contains "KEY=VALUE" + +# Validates that duplicate @var key is rejected at parse time +./clitest test/_fixtures/syntax/duplicate_var.clitest +EXIT 1 +[asserts] +stdout contains "@var" +stdout contains "duplicate" From 601f88b048db45cb9ec4fd197f1c3b2e81c0f7a5 Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 12:18:45 +0200 Subject: [PATCH 09/10] docs(spec): document @var's command-only substitution scope @var only expands {{KEY}} in an entry's command line, since it rides the same runtime path as [captures] substitution (vars.SubstituteCaptures, called only on entry.Command). It does not expand in expected body or [asserts] values, unlike --var, which is a raw whole-file-text pass run before parsing. Add a Scope bullet to SPEC.md's @var subsection stating this explicitly, plus a Variable substitution bullet parallel to @env's, noting that --var placeholders and $VAR/${VAR} env vars expand inside @var values themselves. Co-Authored-By: Claude Sonnet 5 --- SPEC.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/SPEC.md b/SPEC.md index f394542..04eb523 100644 --- a/SPEC.md +++ b/SPEC.md @@ -631,6 +631,10 @@ Sets a file-scoped default value for `{{KEY}}` template substitution. File-level **Values:** The value is everything after the first `=`. Values may contain `=`, spaces, and special characters. Empty values (`@var KEY=`) are valid. Malformed directives without `=` are rejected at parse time, as is a repeated key. +**Scope:** `@var` substitution expands `{{KEY}}` in the entry's command line only — it does NOT expand in the entry's expected body or `[asserts]` values. `--var`, by contrast, expands everywhere (command, body, and asserts), since it runs as a raw text substitution over the whole file before parsing. + +**Variable substitution:** `--var` placeholders (`{{name}}`) in `@var` values are expanded (since substitution runs before parsing). + **Use case:** sibling `.clitest` files that differ by exactly one value (a scenario directory, a tool version) can express that value once, instead of repeating it on every line that needs it: ``` From ded13a286a4e4c098c7a0c822060b4e5cf4db639 Mon Sep 17 00:00:00 2001 From: "Ronny.Herrgesell" Date: Thu, 17 Sep 2026 14:23:20 +0200 Subject: [PATCH 10/10] docs: bring design spec's Resolution/Changes sections in sync with implemented SeedCaptures approach MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The original correction (MergeCaptures -> SeedCaptures rewrite) was made before this branch's worktree was created and never made it into the cherry-picked commit — the worktree carried the stale MergeCaptures text while the plan and actual implementation already used SeedCaptures. --- ...6-09-17-file-level-var-directive-design.md | 42 ++++++++++++------- 1 file changed, 28 insertions(+), 14 deletions(-) diff --git a/docs/superpowers/specs/2026-09-17-file-level-var-directive-design.md b/docs/superpowers/specs/2026-09-17-file-level-var-directive-design.md index 9029eef..4f21399 100644 --- a/docs/superpowers/specs/2026-09-17-file-level-var-directive-design.md +++ b/docs/superpowers/specs/2026-09-17-file-level-var-directive-design.md @@ -50,13 +50,20 @@ literal in the parsed `entry.Command` string. touches `entry.Command`, not asserts/body — an existing limitation, unchanged here. -`@var` slots in at that same runtime step: the map passed to -`SubstituteCaptures` becomes `merge(fileVars, captures)`, with -`captures` overwriting `fileVars` on key collision. Because `--var` -already consumed matching placeholders at load time (before this step -ever runs), and `captures` overwrite `@var` in the merge, the net -effect is exactly `--var > captures > @var` with no change to the -load-time substitution pass and no per-entry re-architecture. +`@var` slots into that same runtime step, but even more directly than +originally planned: the `captures` map is a single instance, created +once per file run (`cmd/clitest/run.go:47`, `captures := map[string]string{}`) +and threaded by reference through every executor function that reads +or writes it — all 5 `SubstituteCaptures` call sites use this same +instance. Seeding that map with `f.Directives.Var` at creation (a +copy, so the original `FileDirectives.Var` is never mutated) means +every one of those call sites picks up `@var` values automatically, +with zero changes inside the executor package. Real captures +(`captures[c.Name] = val`, written as entries execute) then overwrite +the seeded value for the same key on the same map — ordinary map +write semantics, no merge function needed. Net effect is exactly +`--var > captures > @var`, with no change to the load-time +substitution pass and no per-entry re-architecture. ## Changes @@ -74,12 +81,18 @@ load-time substitution pass and no per-entry re-architecture. - `interpretEntryDirectives`: no change. **`internal/vars/vars.go`** -- `MergeCaptures(fileVars, captures map[string]string) map[string]string` - — returns a new map, `captures` entries win over `fileVars` on - collision. Used at all 5 existing `SubstituteCaptures` call sites - (`cmd/clitest/run.go` ×2, `internal/executor/executor.go` ×2, - `internal/executor/background.go` ×1) in place of the bare - `captures` map. +- `SeedCaptures(fileVars map[string]string) map[string]string` — + returns a fresh copy of `fileVars` (empty map if `fileVars` is nil), + never the original map. Used once, in place of the literal + `map[string]string{}` at `cmd/clitest/run.go:47`, to initialize the + `captures` map that's threaded through the rest of the run. + +**`cmd/clitest/run.go` / `cmd/clitest/root.go`** +- `runEntries`, `runEntriesVerbose`, `runEntriesCompact` each gain a + `fileVars map[string]string` parameter, threaded from + `parsed.Directives.Var` in `root.go` (where `parsed` is the + `*types.File` returned by `loadAndParse`) down to the `captures :=` + initialization in `runEntries`. **`SPEC.md`** - Variables → priority list: insert `@var` between `[captures]` and @@ -93,7 +106,8 @@ load-time substitution pass and no per-entry re-architecture. - **Unit** — `internal/parser/directives_test.go`: parses `@var`, rejects malformed (no `=`), rejects duplicate key. - `internal/vars/vars_test.go`: `MergeCaptures` precedence. + `internal/vars/vars_test.go`: `SeedCaptures` (copies, doesn't alias + the input map; nil input yields empty map). - **e2e** — `test/e2e/var/` (pattern: `test/e2e/env/`): - `it_sets_file_level_var.clitest` - `it_var_available_to_all_entries.clitest`