[codex] Harden cross-package quality checks

.codex/skills/tat-change-triage/SKILL.md

@@ -0,0 +1,9 @@
+# tat-change-triage
+
+Use this skill before modifying `tat-cli/`, `vscode-extension/`, or `packages/tat-shared/`.
+
+Checklist:
+- Map touched files to affected surfaces: schema, shared contracts, file parsing, CLI flags, exit codes, output shape, binary resolution, extension output formatting.
+- If any affected surface crosses package boundaries, require `npm run test:contracts` before completion.
+- If `tat-cli/` changes, explicitly decide whether `vscode-extension/` is impacted and record that in the handoff.
+- Prefer `npm run verify` as the default repo-wide safety check.

.codex/skills/tat-contract-guard/SKILL.md

@@ -0,0 +1,15 @@
+# tat-contract-guard
+
+Use this skill whenever a change touches any of:
+- `packages/tat-shared/src/contracts.ts`
+- `tat-cli/src/types.ts`
+- `tat-cli/src/schema.ts`
+- `tat-cli/src/cli.ts`
+- `vscode-extension/src/tatRunner.ts`
+- `vscode-extension/src/resultFormatting.ts`
+
+Requirements:
+- Update or add contract tests under `tests/contracts/`.
+- Keep exit codes `0 / 1 / 2` stable unless the user explicitly requests a breaking change.
+- Keep CLI JSON output parseable by `RunResultSchema`.
+- If the contract changes intentionally, update extension docs and repo rules in the same change.

.codex/skills/tat-harness-engineer/SKILL.md

@@ -0,0 +1,10 @@
+# tat-harness-engineer
+
+Use this skill when adding or modifying cross-package fixtures or regression coverage.
+
+Responsibilities:
+- Own fixtures under `tests/contracts/fixtures/`.
+- Prefer real CLI execution against the built artifact over spawn-only mocks.
+- Cover JSON, `.tat.yml`, and `.tat.yaml` paths when contract behavior might drift.
+- Route real CLI stdout through extension-side parsing/formatting helpers where possible.
+- Keep the harness fast and deterministic; avoid slow external services when a local test server is enough.

.codex/skills/tat-release-check/SKILL.md

@@ -0,0 +1,10 @@
+# tat-release-check
+
+Use this skill before publish or version-bump changes.
+
+Checklist:
+- Run `npm run verify`.
+- Run `npm run smoke:packaged-cli` before npm publish.
+- Confirm `tat --version` comes from `tat-cli/package.json`.
+- Confirm `schema.json` and shared file-format helpers remain aligned.
+- Confirm workflow changes keep using `npm ci`, not `npm install`.

.github/workflows/ci.yml

@@ -7,31 +7,25 @@ on:
     branches: [main]
 
 jobs:
-  check:
-    runs-on: ubuntu-latest
+  verify:
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+    runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v4
 
       - uses: actions/setup-node@v4
         with:
           node-version: 20
 
-      - name: Install CLI deps
-        working-directory: tat-cli
+      - name: Install workspace dependencies
         run: npm ci
 
-      - name: Install extension deps
-        working-directory: vscode-extension
-        run: npm install
+      - name: Verify workspace
+        run: npm run verify
 
-      - name: Lint CLI
-        working-directory: tat-cli
-        run: npm run lint
-
-      - name: Test CLI
-        working-directory: tat-cli
-        run: npm test
-
-      - name: Lint extension (catches type drift)
-        working-directory: vscode-extension
-        run: npm run lint
+      - name: Smoke packaged CLI
+        if: matrix.os == 'ubuntu-latest'
+        run: npm run smoke:packaged-cli

.github/workflows/examples-ci.yml

@@ -17,8 +17,7 @@ jobs:
         with:
           node-version: 20
 
-      - name: Install CLI deps
-        working-directory: tat-cli
+      - name: Install workspace dependencies
         run: npm ci
 
       - name: Build CLI

.github/workflows/publish-npm.yml

@@ -36,21 +36,14 @@ jobs:
           node-version: 20
           registry-url: https://registry.npmjs.org
 
-      - name: Install dependencies
-        working-directory: tat-cli
+      - name: Install workspace dependencies
         run: npm ci
 
-      - name: Lint
-        working-directory: tat-cli
-        run: npm run lint
+      - name: Verify workspace
+        run: npm run verify
 
-      - name: Test
-        working-directory: tat-cli
-        run: npm test
-
-      - name: Build
-        working-directory: tat-cli
-        run: npm run build
+      - name: Smoke packaged CLI
+        run: npm run smoke:packaged-cli
 
       - name: Publish to npm
         working-directory: tat-cli
@@ -73,5 +66,6 @@ jobs:
           title: "chore: bump version after npm publish"
           body: Automated patch bump after publishing the npm package.
           add-paths: |
+            package-lock.json
             tat-cli/package.json
             tat-cli/package-lock.json

.github/workflows/publish-vscode.yml

@@ -35,17 +35,11 @@ jobs:
         with:
           node-version: 20
 
-      - name: Install dependencies
-        working-directory: vscode-extension
-        run: npm install
-
-      - name: Lint
-        working-directory: vscode-extension
-        run: npm run lint
+      - name: Install workspace dependencies
+        run: npm ci
 
-      - name: Build
-        working-directory: vscode-extension
-        run: npm run build
+      - name: Verify workspace
+        run: npm run verify
 
       - name: Publish to VS Code Marketplace
         working-directory: vscode-extension
@@ -68,5 +62,6 @@ jobs:
           title: "chore(vscode): bump version after marketplace publish"
           body: Automated patch bump after publishing to VS Code Marketplace.
           add-paths: |
+            package-lock.json
             vscode-extension/package.json
             vscode-extension/package-lock.json

.gitignore

@@ -1,6 +1,7 @@
 node_modules/
 dist/
 coverage/
+.vite/
 *.log
 .DS_Store
 Thumbs.db

AGENTS.md

@@ -5,14 +5,20 @@ This file provides guidance to AI coding agents when working with code in this r
 ## Commands
 
 ```bash
+npm install                           # install all workspace deps from the repo root
+npm run verify                        # root quality gate: lint + tests + builds + contract harness
+npm run test:contracts                # cross-package CLI ↔ extension contract harness
 npm --prefix tat-cli run build        # compile with tsup → tat-cli/dist/cli.js
 npm --prefix tat-cli run lint         # type-check only (tsc --noEmit)
 npm --prefix tat-cli test             # run all tests once (vitest run)
 npm --prefix tat-cli run test:watch   # watch mode
 npm --prefix tat-cli exec vitest run tests/asserter.test.ts  # run a single test file
+npm --prefix vscode-extension run build
+npm --prefix vscode-extension run lint
 ```
 
 `tat-cli/` is the package root for the CLI. Build output is ESM only (`tat-cli/dist/cli.js`) with a `#!/usr/bin/env node` shebang injected by tsup. Entry point is `tat-cli/src/cli.ts`.
+`packages/tat-shared/` is the shared internal workspace package for file-format helpers and runtime-validated result contracts consumed by both the CLI and the VS Code extension.
 
 ## Cross-Package Impact Rule
 
@@ -28,10 +34,15 @@ At minimum, inspect `vscode-extension/` when a CLI change touches:
 
 If the extension is affected, update its source, docs, and tests in the same change. Prefer running:
 
+```bash
+npm run verify
+```
+
+For targeted cross-package checks during development:
+
 ```bash
 npm --prefix tat-cli exec vitest run vscode-extension/tests --config tat-cli/vitest.config.ts
-npm --prefix vscode-extension run lint
-npm --prefix vscode-extension run build
+npm --prefix tat-cli exec vitest run tests/contracts --config tat-cli/vitest.config.ts
 ```
 
 If the extension is not affected, mention that check in the PR summary or final handoff.
@@ -41,15 +52,17 @@ If the extension is not affected, mention that check in the PR summary or final
 The execution flow for `tat run <file>` is:
 
 1. **`tat-cli/src/cli.ts`** — Commander CLI entry. Exports `runCommand` and `validateCommand` for testability. Uses ESM main guard (`process.argv[1] === fileURLToPath(import.meta.url)`) to prevent `program.parse()` from running when imported in tests. Exit codes: 0 = pass, 1 = test failures, 2 = configuration/file errors or no matching suites.
-2. **`tat-cli/src/fileFormat.ts`** — File format detection and parsing. Exports `TAT_EXTENSIONS` (`.tat.json`, `.tat.yml`, `.tat.yaml`), `isTatFile` (extension check), and `parseFileContent` (routes to `JSON.parse` or `yaml.parse` based on extension).
+2. **`packages/tat-shared/src/fileFormat.ts`** — Shared file format detection and parsing. Owns `TAT_EXTENSIONS`, `isTatFile`, `isYamlTatFile`, and `parseTatFileContent` so CLI and extension stay in sync.
 3. **`tat-cli/src/runner.ts`** — Core logic: `loadAndValidate` (reads file, delegates parsing to `parseFileContent`, then Zod-validates), `resolveEnv` (inline object or external JSON file), `runSetup` (spawns shell command with `stdin`/`stderr` inherited so interactive prompts work, captures stdout as JSON env), `filterSuites` (tag/name filtering), `warnUndefinedVars` (pre-run scan for undefined `{{variables}}`), `run` (loops suites/tests, handles `skip` and `bail`, merges captures into vars between tests).
 4. **`tat-cli/src/asserter.ts`** — Builds the response context object (`$status`, `$headers`, `$body`, `$duration`, plus spread body fields) and evaluates assertion strings via `@nanotiny/json-expression`'s `evaluate()`.
 5. **`tat-cli/src/capturer.ts`** — Extracts values from the response context using `@nanotiny/json-expression`'s `query()`; captured values become `{{variable}}` interpolation vars for subsequent tests.
 6. **`tat-cli/src/interpolate.ts`** — `{{variable}}` substitution in strings and deep within objects/arrays.
 7. **`tat-cli/src/http.ts`** — Thin `fetch` wrapper; auto-sets `Content-Type: application/json` for object bodies. Accepts optional `timeoutMs` and uses `AbortController` to enforce it. Throws `TatRequestError` on network failure or timeout.
 8. **`tat-cli/src/reporter.ts`** — Formats `RunResult` as console (colored, with live streaming callbacks), JSON, or JUnit XML. Handles `skipped` tests in all three formats.
 9. **`tat-cli/src/schema.ts`** — Zod schema for the test file format.
-10. **`tat-cli/src/types.ts`** — Shared TypeScript interfaces (`Suite`, `Test`, `TestResult`, `RunResult`, etc.).
+10. **`packages/tat-shared/src/contracts.ts`** — Shared runtime-validated result schemas and types (`AssertionResult`, `TestResult`, `SuiteResult`, `RunResult`).
+11. **`tat-cli/src/types.ts`** — CLI-specific input types (`Suite`, `Test`, `HttpMethod`) plus re-exports of shared result contract types.
+12. **`tests/contracts/`** — Cross-package harness that runs the built CLI against JSON/YML/YAML fixtures and validates the output through the extension-side parser/formatter path.
 
 ### Assertion and capture expression syntax
 
@@ -58,3 +71,13 @@ Both assertions (`assert`) and capture paths (`capture`) are powered by [`@nanot
 ### Key data flow
 
 Captures accumulate across all suites in a single run. The merged env for each test is `{ ...staticEnv, ...captures }`. The `onSuiteStart` / `onTestResult` callbacks on `run()` are used by the CLI for live console streaming.
+
+## Repo-Local AI Assets
+
+- Repo-local skills live under `.codex/skills/`.
+- Durable project memory lives under `Rules/`.
+- When making cross-package changes, consult:
+  - `.codex/skills/tat-change-triage/SKILL.md`
+  - `.codex/skills/tat-contract-guard/SKILL.md`
+  - `.codex/skills/tat-harness-engineer/SKILL.md`
+  - `.codex/skills/tat-release-check/SKILL.md`

README.md

@@ -80,6 +80,10 @@ code --install-extension nanotiny.tat-test-runner
 ## Common Commands
 
 ```bash
+npm install
+npm run verify
+npm run test:contracts
+
 npm --prefix tat-cli run build
 npm --prefix tat-cli run lint
 npm --prefix tat-cli test

Rules/project-tat-binary-resolution-and-extension-assumptions.md

@@ -0,0 +1,26 @@
+---
+tags:
+  - rule
+  - rule/project/tat
+created: 2026-04-24
+updated: 2026-04-24
+scope: project/tat
+status: active
+---
+
+# Binary resolution order and extension assumptions are stable
+
+## Rule
+
+`vscode-extension/src/tatRunner.ts` must preserve its binary resolution order unless an intentional compatibility change is made:
+
+1. `tat.cliPath`
+2. `node_modules/.bin/tat[.cmd]` in the workspace
+3. `tat-cli/node_modules/.bin/tat[.cmd]` or `tat-cli/dist/cli.js` in the repo workspace
+4. `where tat` / `which tat`
+5. `npx @nanotiny/tiny-api-test`
+
+## Enforcement
+
+- Windows behavior is part of the contract; keep `.cmd` handling covered by tests.
+- Any change to resolution or stdout parsing assumptions must run through `tests/contracts/` and the extension test suite.

Rules/project-tat-cli-exit-codes.md

@@ -0,0 +1,24 @@
+---
+tags:
+  - rule
+  - rule/project/tat
+created: 2026-04-24
+updated: 2026-04-24
+scope: project/tat
+status: active
+---
+
+# CLI exit codes are stable
+
+## Rule
+
+`tat` must preserve this exit code contract unless an explicit breaking change is approved:
+
+- `0` = all selected tests passed
+- `1` = test failures, but stdout still contains valid `RunResult` JSON when `--output json` is used
+- `2` = configuration, validation, file discovery, or filter-selection errors
+
+## Enforcement
+
+- Any change touching CLI command handling, validation, or stdout/stderr behavior must update `tests/contracts/` if the behavior changes intentionally.
+- The VS Code extension relies on the `0/1/2` contract to distinguish test failures from invocation errors.

Rules/project-tat-json-result-contract.md

@@ -0,0 +1,21 @@
+---
+tags:
+  - rule
+  - rule/project/tat
+created: 2026-04-24
+updated: 2026-04-24
+scope: project/tat
+status: active
+---
+
+# JSON result shape comes from the shared contract
+
+## Rule
+
+`AssertionResult`, `TestResult`, `SuiteResult`, and `RunResult` are owned by `packages/tat-shared/src/contracts.ts`.
+
+## Enforcement
+
+- The CLI must keep `--output json` compatible with `RunResultSchema`.
+- The extension must validate CLI JSON with `RunResultSchema` instead of assuming shape by convention.
+- Any intentional contract change must update `tests/contracts/`, extension formatting/parsing, and relevant docs in the same change.

Rules/project-tat-schema-compatibility.md

@@ -0,0 +1,26 @@
+---
+tags:
+  - rule
+  - rule/project/tat
+created: 2026-04-24
+updated: 2026-04-24
+scope: project/tat
+status: active
+---
+
+# Test file parsing and schema compatibility must stay aligned
+
+## Rule
+
+The supported file extensions and raw parsing behavior for `.tat.json`, `.tat.yml`, and `.tat.yaml` must remain aligned across:
+
+- `packages/tat-shared/src/fileFormat.ts`
+- `tat-cli/src/schema.ts`
+- `tat-cli/schema.json`
+- `vscode-extension/src/fileParser.ts`
+- `vscode-extension/src/promptVariables.ts`
+
+## Enforcement
+
+- Changes to parsing or schema behavior require CLI tests and contract harness updates.
+- The extension may add editor-only behavior, but not a divergent parse contract.