feat: strict mode — test each Trouve before it is published

site-docs/docs/cli/compile.md

@@ -3,7 +3,7 @@
 Resolve the DAG and write generated SQL to `_clairtifacts/`. No Snowflake connection is made.
 
 ```bash
-clair compile [--project PATH] [--env NAME] [--select PATTERN]... [--run-mode MODE]
+clair compile [--project PATH] [--env NAME] [--select PATTERN]... [--run-mode MODE] [--strict]
 ```
 
 ## Example
@@ -17,6 +17,9 @@ clair compile --project . --env prod
 
 # Compile only the orders schema
 clair compile --project . --select='refined.orders.*'
+
+# Show the strict-mode plan: staging build, test checkpoint, promotion
+clair compile --project . --strict
 ```
 
 ## What it does
@@ -53,9 +56,11 @@ _clairtifacts/
 | `--env` | optional | Environment name. Required if you want routing applied to generated SQL. |
 | `--select` | all | Glob pattern to filter Trouves. Repeat to union patterns. |
 | `--run-mode` | `full_refresh` | `full_refresh` or `incremental`. Affects which SQL variant is generated. |
+| `--strict` | `false` | Emit the [strict-mode](../guides/strict-mode.md) plan instead of a direct write to the target. |
 
 ## See also
 
 - [DAG](../concepts/dag.md)
 - [Selectors](../guides/selectors.md)
+- [Strict Mode](../guides/strict-mode.md)
 - [clair clean](clean.md)

site-docs/docs/cli/run.md

@@ -3,7 +3,7 @@
 Execute Trouves against Snowflake in topological dependency order, then run data quality tests.
 
 ```bash
-clair run [--project PATH] [--env NAME] [--select PATTERN]... [--run-mode MODE] [--no-test] [--sample]
+clair run [--project PATH] [--env NAME] [--select PATTERN]... [--run-mode MODE] [--no-test] [--sample] [--strict]
 ```
 
 ## Example
@@ -20,6 +20,9 @@ clair run --project . --env prod --run-mode full_refresh
 
 # Skip tests
 clair run --project . --env dev --no-test
+
+# Only publish a Trouve once its tests pass
+clair run --project . --env prod --strict
 ```
 
 ## Execution order
@@ -32,6 +35,8 @@ SOURCE Trouves pass through (no SQL is executed against them).
 
 After each successful TABLE or VIEW, attached tests run automatically. If any test fails, the run exits with a non-zero status code. Use `--no-test` to skip tests.
 
+By default the target is written before its tests run, so a failing test means bad data is already in production. `--strict` builds each Trouve into a run-scoped staging object, tests it there, and promotes it into the real name only if every test passes — see [Strict Mode](../guides/strict-mode.md). It cannot be combined with `--no-test`.
+
 ## Flags
 
 | Flag | Default | Description |
@@ -42,14 +47,16 @@ After each successful TABLE or VIEW, attached tests run automatically. If any te
 | `--run-mode` | `full_refresh` | `full_refresh` or `incremental`. Overrides each Trouve's `run_config`. |
 | `--no-test` | `false` | Skip data quality tests |
 | `--sample` | `false` | Run tests against `SELECT TOP 1000 *` (skips `TestRowCount`) |
+| `--strict` | `false` | Build into a staging object, test, then promote into place. Incompatible with `--no-test`. |
 
 ## Exit codes
 
 - `0` — all Trouves succeeded and all tests passed
-- `1` — one or more Trouves failed, or one or more tests failed
+- `1` — one or more Trouves failed, or one or more tests failed (under `--strict`, a Trouve whose tests failed is itself reported as failed, its target is left unchanged, and the rejected candidate is retained for inspection)
 
 ## See also
 
 - [Selectors](../guides/selectors.md)
 - [Incrementality](../guides/incrementality.md)
 - [Data Quality Tests](../guides/data-quality-tests.md)
+- [Strict Mode](../guides/strict-mode.md)

site-docs/docs/guides/data-quality-tests.md

@@ -140,3 +140,11 @@ clair test --project . --env dev --sample
 | `TestUniqueColumns` | `columns: list[str]` (min 2) | No |
 
 See also: [Tests API reference](../reference/tests-api.md).
+
+## Testing before publishing
+
+Tests run after a Trouve has been materialized — that is the only point at which there is a table to query. By default that means a failing test tells you production is already wrong. [Strict Mode](strict-mode.md) inverts this: each Trouve is built into a staging object, tested there, and promoted into its real name only if every test passes.
+
+```bash
+clair run --project . --env prod --strict
+```

site-docs/docs/guides/index.md

@@ -5,6 +5,7 @@ How-to guides for common clair tasks.
 - **[Pandas-Native Transformations](pandas-native.md)** — write pipeline steps as Python functions using pandas
 - **[Incrementality](incrementality.md)** — APPEND and UPSERT strategies for large tables
 - **[Data Quality Tests](data-quality-tests.md)** — attach tests to Trouves
+- **[Strict Mode](strict-mode.md)** — publish a Trouve only after its tests pass
 - **[Selectors](selectors.md)** — run only a subset of your project
 - **[Routing Policies](routing.md)** — remap Snowflake targets per environment
 - **[Per-Database & Schema Config](per-database-schema-config.md)** — warehouse and role overrides per directory

site-docs/docs/guides/strict-mode.md

@@ -0,0 +1,93 @@
+# Strict Mode
+
+A table can only be tested once it has been materialized. In a normal run that means bad data lands in production first and the tests tell you about it afterwards — the table is already wrong, and anything reading it has already read the wrong numbers.
+
+Strict mode closes that window. Every Trouve is built into a run-scoped staging object, tested there, and only swapped into its real name once every test passes.
+
+```bash
+clair run --project . --env prod --strict
+```
+
+## What happens per Trouve
+
+1. **Build into staging.** The Trouve is materialized as `<table>__clair_<run_id>`, a sibling object in the same database and schema.
+2. **Test the staging object.** The Trouve's data quality tests run against the staging object, not the target.
+3. **Promote on pass.** For a `TABLE`, clair issues `CREATE OR REPLACE TABLE <target> CLONE <staging> COPY GRANTS` — a metadata-only operation whose cost does not scale with table size — then drops the staging copy. For a `VIEW`, the target is recreated with `CREATE OR REPLACE VIEW ... COPY GRANTS`.
+4. **Keep the candidate on fail.** The target is left exactly as it was, and the staging object is deliberately **not** dropped — see [When something fails](#when-something-fails). The Trouve is reported as a failure and everything downstream of it is skipped.
+
+Promotion happens immediately after each node's tests, before the next node starts. Dependents therefore always read their upstreams under the real names their SQL references — you never write `__clair_` anywhere yourself.
+
+## Grants
+
+Promotion uses `COPY GRANTS`, and it is not optional. Snowflake attaches privileges to the *object*, not to the name: an `ALTER TABLE ... SWAP WITH ...` carries a table's grants away under the staging name and leaves the production name holding only whatever the staging object was created with. Any privilege granted directly on a target would be silently revoked on every run.
+
+`COPY GRANTS` copies every privilege except `OWNERSHIP` from the object being replaced — or, when the target does not yet exist, from the clone source, which is why promotion needs no special case for a first run.
+
+`OWNERSHIP` is the exception: it lands on the role executing the run. If a production object is owned by some other role, strict mode changes its owner.
+
+## Incremental Trouves
+
+An incremental Trouve applies changes on top of state that already exists, so the staging object needs that state before the `INSERT` or `MERGE` can run. clair seeds it with a zero-copy clone:
+
+```sql
+-- strict: clone target into staging so incremental statements have a base
+CREATE OR REPLACE TABLE db.schema.orders__clair_<run_id> CLONE db.schema.orders
+
+INSERT INTO db.schema.orders__clair_<run_id>
+SELECT * FROM ( ... )
+
+-- tests run here
+
+CREATE OR REPLACE TABLE db.schema.orders CLONE db.schema.orders__clair_<run_id> COPY GRANTS
+DROP TABLE IF EXISTS db.schema.orders__clair_<run_id>
+```
+
+Snowflake clones are metadata-only, so seeding the staging table is constant-time no matter how large the target is. If the target does not exist yet, clair falls back to a full refresh for that Trouve and skips the seeding clone; promotion is unchanged.
+
+## Seeing the plan
+
+`clair compile --strict` writes the full plan to `_clairtifacts/`, including the clone, the staging build, a comment marking where tests run, and the promotion:
+
+```bash
+clair compile --project . --strict
+clair compile --project . --strict --run-mode incremental
+```
+
+## When something fails
+
+Nothing is thrown away. A staging object is dropped only after it has been successfully promoted.
+
+| What failed | Target | Staging object |
+|-------------|--------|----------------|
+| The build itself | Untouched | Retained, if it got far enough to be created |
+| A data quality test | Untouched | Retained — the rejected candidate |
+| The promotion | Untouched | Retained — it holds tested data and is the only record of the run |
+
+In all three cases the Trouve is reported as `FAILED`, its dependents are skipped, `clair run` exits `1`, and the error names the staging object so you can query it directly:
+
+```
+db.schema.orders ... FAILED (2.4s)
+      Error: strict mode: tests failed, db.schema.orders left unchanged
+             (rejected candidate retained at db.schema.orders__clair_<run_id>)
+```
+
+This is the fastest path to a diagnosis. The candidate is the exact data that failed the test, and rebuilding it otherwise means re-running everything upstream of it.
+
+The cost is that failed runs accumulate objects. Staging tables created by a full refresh are real copies; a retained incremental clone starts out sharing micro-partitions with its target but diverges — and therefore starts costing real storage — as the target changes. Drop them once you are done with them:
+
+```sql
+SHOW TABLES LIKE '%__clair_%' IN SCHEMA db.schema;
+```
+
+## Costs and caveats
+
+- **Storage.** Each Trouve briefly holds two copies. For a full refresh that is a real second copy for the duration of the node; for an incremental build the clone shares micro-partitions with the target and only diverges as rows change. Objects left behind by failures persist until you drop them.
+- **Tests are required.** `--strict` cannot be combined with `--no-test` — the whole point is to gate promotion on tests. A Trouve with no tests attached still goes through staging and is promoted; strict mode does not manufacture coverage you have not written.
+- **Name length.** The suffix adds 40 characters to the table component. Snowflake caps each identifier at 255 — the limit is per object name, not per fully-qualified path — so a Trouve whose table name is within 40 characters of the cap fails at the naming step with a clear error rather than mid-run.
+- **`--sample`.** Sampling applies to the staging object, so `--strict --sample` gates promotion on a `TOP 1000` check rather than the full table.
+
+## See also
+
+- [Data Quality Tests](data-quality-tests.md)
+- [Incrementality](incrementality.md)
+- [clair run](../cli/run.md)

site-docs/mkdocs.yml

@@ -78,6 +78,7 @@ nav:
     - Pandas-Native Transformations: guides/pandas-native.md
     - Incrementality: guides/incrementality.md
     - Data Quality Tests: guides/data-quality-tests.md
+    - Strict Mode: guides/strict-mode.md
     - Selectors: guides/selectors.md
     - Routing Policies: guides/routing.md
     - Per-Database & Schema Config: guides/per-database-schema-config.md

src/clair/cli/main.py

@@ -252,7 +252,13 @@ def _require(prompt_text: str, **kwargs) -> str:
     default="full_refresh",
     help="Run mode: full_refresh recreates all tables; incremental applies only new data.",
 )
-def compile_cmd(select: tuple[str, ...], exclude: tuple[str, ...], project: str, env: str | None, run_mode: str) -> None:
+@click.option(
+    "--strict",
+    is_flag=True,
+    default=False,
+    help="Show the strict-mode plan: build into a run-scoped staging object, test, then swap.",
+)
+def compile_cmd(select: tuple[str, ...], exclude: tuple[str, ...], project: str, env: str | None, run_mode: str, strict: bool) -> None:
     """Compile the project and show generated SQL (no Snowflake connection)."""
     project_root = Path(project).resolve()
     run_mode_enum = RunMode(run_mode)
@@ -301,7 +307,7 @@ def _on_node_compiled(node_info: CompiledNodeInfo) -> None:
             artifact_file = artifacts_dir / "/".join(parts[:-1]) / f"{parts[-1]}{extension}"
             logger.info("compile.node", trouve=node_info.name, dependencies=node_info.dependencies, artifact_file=str(artifact_file))
 
-        write_compile_output(dag, selected, project_root, on_node_compiled=_on_node_compiled, run_mode=run_mode_enum, run_id=run_id)
+        write_compile_output(dag, selected, project_root, on_node_compiled=_on_node_compiled, run_mode=run_mode_enum, run_id=run_id, strict=strict)
         logger.info("compile.complete", run_id=run_id, artifacts_dir=str(artifacts_dir))
 
     except ClairError as e:
@@ -429,12 +435,27 @@ def docs(project: str, port: int, host: str, no_browser: bool) -> None:
     default=False,
     help="Run post-run tests against a sample of each Trouve (skips row count tests).",
 )
-def run(select: tuple[str, ...], exclude: tuple[str, ...], project: str, env: str | None, run_mode: str, no_test: bool, sample: bool) -> None:
+@click.option(
+    "--strict",
+    is_flag=True,
+    default=False,
+    help=(
+        "Build each Trouve into a run-scoped staging object, test it, and only "
+        "promote it into its real name if every test passes. Promotion is a "
+        "constant-time clone that carries existing grants. A failing Trouve "
+        "leaves its target untouched and its rejected candidate in place."
+    ),
+)
+def run(select: tuple[str, ...], exclude: tuple[str, ...], project: str, env: str | None, run_mode: str, no_test: bool, sample: bool, strict: bool) -> None:
     """Run Trouves against Snowflake, then run data quality tests."""
     project_root = Path(project).resolve()
     run_mode_enum = RunMode(run_mode)
     run_id = uuid6.uuid7().hex
 
+    if strict and no_test:
+        logger.error("run.error", error="--strict cannot be combined with --no-test: strict mode promotes a Trouve only after its tests pass")
+        sys.exit(1)
+
     try:
         # Load environment
         env_name, environment = load_environment(env)
@@ -460,7 +481,7 @@ def run(select: tuple[str, ...], exclude: tuple[str, ...], project: str, env: st
             return
 
         recompile_for_selection(discovered, set(selected))
-        write_compile_output(dag, selected, project_root, run_mode=run_mode_enum, run_id=run_id)
+        write_compile_output(dag, selected, project_root, run_mode=run_mode_enum, run_id=run_id, strict=strict)
 
         # Warn if account_locator is missing (query URLs will be incomplete)
         if not environment.account_locator:
@@ -472,22 +493,27 @@ def run(select: tuple[str, ...], exclude: tuple[str, ...], project: str, env: st
 
         test_failures: list[str] = []
 
-        def on_node_success(node_name: str) -> bool:
-            node_test_results = run_tests(dag, [node_name], adapter, use_sample=sample)
+        def on_node_success(node_name: str, physical_name: str) -> bool:
+            node_test_results = run_tests(
+                dag, [node_name], adapter,
+                use_sample=sample,
+                physical_names={node_name: physical_name},
+            )
             passed = all(r.passed for r in node_test_results)
             if not passed:
                 test_failures.append(node_name)
             return passed
 
         try:
             total = len(selected)
-            logger.info("run.start", run_id=run_id, env=env_name, project=str(project_root), trouves=total, run_mode=run_mode)
+            logger.info("run.start", run_id=run_id, env=env_name, project=str(project_root), trouves=total, run_mode=run_mode, strict=strict)
 
             results = list(run_project(
                 dag, selected, adapter,
                 run_mode=run_mode_enum,
                 run_id=run_id,
                 after_node_success=on_node_success if not no_test else None,
+                strict=strict,
             ))
 
             counts = Counter(r.status for r in results)

src/clair/core/compiler.py

@@ -13,6 +13,12 @@
 from clair.core.dag import ClairDag
 from clair.core.discovery import ARTIFACTS_DIR_NAME
 from clair.core.runner import resolve_effective_mode
+from clair.core.strict import (
+    build_clone_statement,
+    build_drop_staging_statement,
+    build_promote_statement,
+    strict_staging_name,
+)
 from clair.exceptions import CompileError
 from clair.trouves.run_config import RunMode
 from clair.trouves.trouve import ExecutionType, Trouve, TrouveType
@@ -86,13 +92,54 @@ def render(self) -> str:
         return "\n".join(parts)
 
 
+def build_statements(
+    trouve: Trouve,
+    run_mode: RunMode,
+    run_id: str,
+    strict: bool = False,
+) -> list[str]:
+    """Build the SQL statements for one Trouve, as they would be executed.
+
+    Under strict mode the plan is shown end to end: the optional clone, the build
+    into the staging object, and the promotion that follows a passing test run.
+    The plan shown is the passing path -- a failing test run stops after the build
+    and leaves the staging object in place.
+    """
+    effective_mode = resolve_effective_mode(trouve, run_mode)
+
+    if not strict:
+        return trouve.build_sql(effective_mode, run_id=run_id)
+
+    target_name = trouve.full_name
+    staging_name = strict_staging_name(target_name, run_id)
+
+    statements: list[str] = []
+    if effective_mode == RunMode.INCREMENTAL:
+        statements.append(build_clone_statement(target_name, staging_name))
+    statements.extend(trouve.build_sql(effective_mode, run_id=run_id, target_name=staging_name))
+
+    assert trouve.compiled is not None
+    statements.append("-- strict: tests run against the staging object here")
+    statements.append(
+        build_promote_statement(
+            trouve.type,
+            staging_name=staging_name,
+            target_name=target_name,
+            resolved_sql=trouve.compiled.resolved_sql,
+        )
+    )
+    statements.append(build_drop_staging_statement(trouve.type, staging_name))
+    return statements
+
+
 def write_compile_output(
     dag: ClairDag,
     selected: list[str],
     project_root: Path,
     on_node_compiled: Callable[[CompiledNodeInfo], None] = lambda _: None,
     run_mode: RunMode = RunMode.FULL_REFRESH,
     run_id: str = "",
+    strict: bool = False,
 ) -> CompileOutput:
     """Write compiled SQL to _clairtifacts/<run_id>/ and return a structured output.
 
@@ -104,6 +151,8 @@ def write_compile_output(
             to disk, allowing callers to stream output.
         run_mode: The run mode to use when generating SQL statements.
         run_id: UUIDv7 hex string identifying this compile run.
+        strict: When True, emit the full strict-mode plan (staging build, test
+            checkpoint, promotion) rather than a direct write to the target.
 
     Returns:
         A CompileOutput with structured data and a .render() method.
@@ -173,8 +222,7 @@ def write_compile_output(
             artifact_path.parent.mkdir(parents=True, exist_ok=True)
             artifact_path.write_text(artifact_content)
         elif trouve.compiled.execution_type == ExecutionType.SNOWFLAKE:
-            effective_mode = resolve_effective_mode(trouve, run_mode)
-            statements = trouve.build_sql(effective_mode, run_id=run_id)
+            statements = build_statements(trouve, run_mode, run_id, strict=strict)
 
             node_info = CompiledNodeInfo(
                 name=name,