feat: split Trouve into TrouveAbc + Trouve + PandasTrouve

.claude/memory/project_docs_are_the_source_of_truth.md

@@ -12,11 +12,12 @@ metadata:
 `src/clair/auth/environments.py`, a path that does not exist. The documentation is the
 better source because users read it, so an error there gets a report.
 
-**But the documentation also rots.** The pandas guide, the landing page and the README
+**But the documentation also rots.** The pandas guide, the landing page and the README once
 documented a `PandasTrouve` class that was never built, while the API reference correctly
 documented the `df_fn` field that was. The pages came from a design spec, and nobody
 changed them when the implementation took a different shape. mkdocs does not execute the
-examples, so CI did not catch it.
+examples, so CI did not catch it. (The 2026-08-01 backend split built the real
+`PandasTrouve` and deleted `df_fn`, thus the two agree again — but the lesson holds.)
 
 **Therefore: the code is the final authority.** Read the documentation first for
 orientation, then confirm any API detail against `src/` or `example_projects/` before you
@@ -26,8 +27,8 @@ depend on it. When the two disagree, the code wins and the page is a bug.
 
 - To learn what a feature does, `grep` `site_docs/docs/` first. Then confirm the exact API
   against `src/` or a project in `example_projects/`.
-- Search for the field name, not only the class name. The pandas feature was invisible to a
-  search for `PandasTrouve`, because the real name is `df_fn`.
+- Search for the field name, not only the class name. The pandas feature was once invisible
+  to a search for `PandasTrouve`, because the name in the code was `df_fn`.
 - Map for orientation: `concepts/` (Trouve, DAG, project layout, environments),
   `guides/` (routing, incrementality, tests, selectors, pandas, per-database config),
   `cli/` (one page for each subcommand), `reference/` (API for Trouve, Column, RunConfig,

README.md

@@ -42,18 +42,15 @@ import pandas as pd
 from refined.products.catalog import trouve as catalog_trouve
 from refined.products.reviews import trouve as reviews_trouve
 
-from clair import Trouve
+from clair import PandasTrouve
 
 
-def summarize(
-    catalog: pd.DataFrame = catalog_trouve,  # type: ignore
-    reviews: pd.DataFrame = reviews_trouve,  # type: ignore
-) -> pd.DataFrame:
-    df = catalog.merge(reviews, on="product_id")
-    return df.groupby("name", as_index=False)["rating"].mean()
+def summarize(catalog: pd.DataFrame, reviews: pd.DataFrame) -> pd.DataFrame:
+    merged = catalog.merge(reviews, on="product_id")
+    return merged.groupby("name", as_index=False).agg(rating=("rating", "mean"))
 
 
-trouve = Trouve(df_fn=summarize)
+trouve = PandasTrouve(transform=summarize, inputs=[catalog_trouve, reviews_trouve])
 ```
 
 clair fetches the upstream tables from Snowflake, runs your function locally, then writes the result back. The DAG, lineage, `--select` filters, and data quality tests all work unchanged.
@@ -73,10 +70,10 @@ my_project/
         └── reviews.py   →   source.products.reviews
 ```
 
-A Trouve runs in one of two ways:
+Clair has one Trouve class for each backend. Both derive from `TrouveAbc`:
 
-- **`sql`** — compiles to a Snowflake `TABLE` or `VIEW`. Runs inside Snowflake.
-- **`df_fn`** — runs a Python function on the machine executing clair, then writes the result back to Snowflake.
+- **`Trouve`** — compiles `sql` to a Snowflake `TABLE` or `VIEW`. Runs inside Snowflake.
+- **`PandasTrouve`** — runs a Python function on the machine executing clair, then writes the result back to Snowflake.
 
 `Trouve` has three types:
 
@@ -388,31 +385,28 @@ Pass `--run-mode full_refresh` on the CLI to force a full rebuild of everything
 
 ## Pandas-native transformations
 
-When SQL isn't the right tool — complex reshaping, ML feature engineering, multi-step aggregations — give the Trouve a `df_fn` in place of `sql`. Your function receives upstream tables as DataFrames, runs locally on the machine executing clair, and the result is written back to Snowflake automatically.
+When SQL isn't the right tool — complex reshaping, ML feature engineering, multi-step aggregations — use a `PandasTrouve` in place of a `Trouve`. Your function receives upstream tables as DataFrames, runs locally on the machine executing clair, and clair writes the result back to Snowflake.
 
 ```python
 import pandas as pd
 from refined.products.catalog import trouve as catalog_trouve
 from refined.products.reviews import trouve as reviews_trouve
 
-from clair import Column, ColumnType, TestNotNull, Trouve
+from clair import Column, ColumnType, PandasTrouve, TestNotNull
 
 
-def top_rated(
-    catalog: pd.DataFrame = catalog_trouve,  # type: ignore
-    reviews: pd.DataFrame = reviews_trouve,  # type: ignore
-) -> pd.DataFrame:
-    df = catalog.merge(reviews, on="product_id")
+def top_rated(catalog: pd.DataFrame, reviews: pd.DataFrame) -> pd.DataFrame:
+    merged = catalog.merge(reviews, on="product_id")
     return (
-        df.groupby(["product_id", "name"], as_index=False)["rating"]
-        .mean()
-        .rename(columns={"rating": "avg_rating"})
+        merged.groupby(["product_id", "name"], as_index=False)
+        .agg(avg_rating=("rating", "mean"))
         .query("avg_rating >= 4")
     )
 
 
-trouve = Trouve(
-    df_fn=top_rated,
+trouve = PandasTrouve(
+    transform=top_rated,
+    inputs=[catalog_trouve, reviews_trouve],
     columns=[
         Column(name="product_id", type=ColumnType.STRING),
         Column(name="name",       type=ColumnType.STRING),
@@ -423,6 +417,8 @@ trouve = Trouve(
 )
 ```
 
+Clair binds `inputs` to the transform parameters by position. Because the transform takes plain DataFrames, you can call it directly in a unit test or in a notebook.
+
 > **Note:** pandas transformations run on the machine executing clair, not inside Snowflake. Keep this in mind for large tables.
 
 See the [Pandas-Native Transformations guide](https://rivage-sh.github.io/clair/guides/pandas-native/) for the full field reference, DAG integration details, and limitations.

example_projects/example_4/example_4_database/derived/daily_event_counts.py

@@ -1,22 +1,18 @@
 import pandas as pd
 from example_4_database.refined.events import trouve as example_4_database_refined_events
 
-from clair import Column, ColumnType, Trouve
+from clair import Column, ColumnType, PandasTrouve
 
 
-def daily_event_counts(
-    refined_events: pd.DataFrame = example_4_database_refined_events,  # type: ignore
-) -> pd.DataFrame:
-    return (
-        refined_events
-        .groupby(["event_date", "event_type"], as_index=False)
-        .size()
-        .rename(columns={"size": "event_count"})  # type: ignore
-    )
+def daily_event_counts(refined_events: pd.DataFrame) -> pd.DataFrame:
+    return refined_events.groupby(
+        ["event_date", "event_type"], as_index=False
+    ).agg(event_count=("event_type", "size"))
 
 
-trouve = Trouve(
-    df_fn=daily_event_counts,
+trouve = PandasTrouve(
+    transform=daily_event_counts,
+    inputs=[example_4_database_refined_events],
     docs="Daily count of each event type. This Trouve reads the refined events.",
     columns=[
         Column(name="event_date", type=ColumnType.DATE),

site_docs/docs/concepts/trouve.md

@@ -35,7 +35,6 @@ trouve = Trouve(
 | `tests` | `list[AnyTest]` | `[]` | Data quality tests. See [Tests](../guides/data-quality-tests.md). |
 | `docs` | `str` | `""` | Documentation string. `clair docs` shows it. |
 | `run_config` | `RunConfig` | full refresh | Materialization strategy. See [Incrementality](../guides/incrementality.md). |
-| `df_fn` | `Callable \| None` | `None` | Pandas execution mode (alternative to `sql`). TABLE-only, full-refresh-only. |
 
 ## Examples
 
@@ -109,46 +108,46 @@ trouve = Trouve(
 )
 ```
 
-## Pandas execution (`df_fn`)
+## Pandas execution (`PandasTrouve`)
 
-If SQL is not the correct tool, give the Trouve a `df_fn` in place of `sql`. You supply a Python function. clair reads the upstream tables from Snowflake as DataFrames. Then it calls your function on the machine that runs clair, and writes the result to Snowflake.
+If SQL is not the correct tool, use a `PandasTrouve` in place of a `Trouve`. You supply a Python function. clair reads the upstream tables from Snowflake as DataFrames. Then it calls your function on the machine that runs clair, and writes the result to Snowflake.
 
 ```python
 # derived/products/top_rated.py
 import pandas as pd
 from refined.products.catalog import trouve as catalog_trouve
 from refined.products.reviews import trouve as reviews_trouve
 
-from clair import Trouve
+from clair import PandasTrouve
 
 
-def top_rated(
-    catalog: pd.DataFrame = catalog_trouve,  # type: ignore
-    reviews: pd.DataFrame = reviews_trouve,  # type: ignore
-) -> pd.DataFrame:
-    df = catalog.merge(reviews, on="product_id")
+def top_rated(catalog: pd.DataFrame, reviews: pd.DataFrame) -> pd.DataFrame:
+    merged = catalog.merge(reviews, on="product_id")
     return (
-        df.groupby(["product_id", "name"], as_index=False)["rating"]
-        .mean()
+        merged.groupby(["product_id", "name"], as_index=False)
+        .agg(rating=("rating", "mean"))
         .query("rating >= 4")
     )
 
 
-trouve = Trouve(df_fn=top_rated)
+trouve = PandasTrouve(
+    transform=top_rated,
+    inputs=[catalog_trouve, reviews_trouve],
+)
 ```
 
-Each dependency is a parameter with the upstream Trouve as its default value.
+clair binds each item of `inputs` to a transform parameter by position. The transform takes plain DataFrames, thus you can call it directly in a test or in a notebook.
 
-The two run types have these differences:
+The two backends have these differences:
 
-| | `sql` | `df_fn` |
+| | `Trouve` | `PandasTrouve` |
 |---|---|---|
 | Runs | In Snowflake | On the clair machine |
 | Output type | TABLE or VIEW | TABLE only |
 | Incremental | Yes | Full refresh only |
-| Dependencies | f-string references | Parameter default values |
+| Dependencies | f-string references in `sql` | The `inputs` list |
 
-All the other behaviour is the same: the DAG, the `--select` flag, the data quality tests, and the `clair dag` output. You cannot use the two fields together. A Trouve with `sql` and `df_fn` causes an error.
+All the other behaviour is the same: the DAG, the `--select` flag, the data quality tests, and the `clair dag` output. The two classes have the same base class, `TrouveAbc`. That base class holds `columns`, `tests`, `docs`, and `run_config`.
 
 See the [Pandas-native guide](../guides/pandas-native.md) for a full example.
 

site_docs/docs/guides/pandas-native.md

@@ -1,8 +1,8 @@
 # Pandas-Native Transformations
 
-A `Trouve` with a `df_fn` lets you write a pipeline step as a plain Python function. clair reads your upstream tables from Snowflake as DataFrames. Then it calls your function on the machine that runs clair, and writes the result to Snowflake. The DAG, the lineage, the selectors, and the data quality tests all apply.
+A `PandasTrouve` lets you write a pipeline step as a plain Python function. clair reads your upstream tables from Snowflake as DataFrames. Then it calls your function on the machine that runs clair, and writes the result to Snowflake. The DAG, the lineage, the selectors, and the data quality tests all apply.
 
-## When to use `df_fn`
+## When to use `PandasTrouve`
 
 Use it if SQL is the incorrect tool for the task:
 
@@ -11,11 +11,11 @@ Use it if SQL is the incorrect tool for the task:
 - Aggregations of many steps that depend on Python state between the steps
 - Logic that you already have as pandas code
 
-For all other work, give the `Trouve` a `sql` string. The SQL runs in Snowflake, and the data does not move on the network.
+For all other work, use a `Trouve` with a `sql` string. The SQL runs in Snowflake, and the data does not move on the network.
 
 ## Installation
 
-`df_fn` needs no extra installation. pandas is a dependency of clair.
+`PandasTrouve` needs no extra installation. pandas is a dependency of clair.
 
 ## Basic example
 
@@ -25,39 +25,44 @@ import pandas as pd
 from refined.products.catalog import trouve as catalog_trouve
 from refined.products.reviews import trouve as reviews_trouve
 
-from clair import Trouve
+from clair import PandasTrouve
 
 
-def top_rated(
-    catalog: pd.DataFrame = catalog_trouve,  # type: ignore
-    reviews: pd.DataFrame = reviews_trouve,  # type: ignore
-) -> pd.DataFrame:
-    df = catalog.merge(reviews, on="product_id")
+def top_rated(catalog: pd.DataFrame, reviews: pd.DataFrame) -> pd.DataFrame:
+    merged = catalog.merge(reviews, on="product_id")
     return (
-        df.groupby(["product_id", "name"], as_index=False)["rating"]
-        .mean()
-        .rename(columns={"rating": "avg_rating"})
+        merged.groupby(["product_id", "name"], as_index=False)
+        .agg(avg_rating=("rating", "mean"))
         .query("avg_rating >= 4")
     )
 
 
-trouve = Trouve(df_fn=top_rated)
+trouve = PandasTrouve(
+    transform=top_rated,
+    inputs=[catalog_trouve, reviews_trouve],
+)
 ```
 
-You declare each dependency as a **parameter default value**: annotate the parameter as `pd.DataFrame` and give it the upstream Trouve object as its default. At run time clair replaces each default with the fetched DataFrame and calls your function. The parameter name is the name you use in the function body — it does not need to match the import.
+Clair binds `inputs` to the parameters of `transform` **by position**. The first Trouve in `inputs` becomes the first parameter, the second becomes the second parameter, and so on. The parameter names are yours to choose — they do not need to match the import names.
 
-!!! note
-    The `# type: ignore` comment is necessary. A type checker sees a `Trouve` object assigned to a `pd.DataFrame` parameter and reports a mismatch. clair substitutes the DataFrame before it calls the function, so the annotation is correct at run time.
+Because `transform` is an ordinary function that takes DataFrames, you can call it directly in a unit test or in a notebook:
+
+```python
+from derived.products.top_rated import top_rated
+
+result = top_rated(my_catalog_dataframe, my_reviews_dataframe)
+```
 
 ## With columns and tests
 
 `columns` and `tests` work the same as they do for SQL Trouves:
 
 ```python
-from clair import Column, ColumnType, TestNotNull, TestRowCount, Trouve
+from clair import Column, ColumnType, PandasTrouve, TestNotNull, TestRowCount
 
-trouve = Trouve(
-    df_fn=top_rated,
+trouve = PandasTrouve(
+    transform=top_rated,
+    inputs=[catalog_trouve, reviews_trouve],
     columns=[
         Column(name="product_id", type=ColumnType.STRING),
         Column(name="name",       type=ColumnType.STRING),
@@ -73,10 +78,10 @@ trouve = Trouve(
 
 ## How it runs
 
-`clair run` gives a `df_fn` Trouve these four steps:
+`clair run` gives a `PandasTrouve` these four steps:
 
-1. **Fetch** — for each parameter whose default is a Trouve, run `SELECT * FROM <full_name>` and load the result into a DataFrame. Column names become lowercase.
-2. **Transform** — call your function locally on the clair machine, with one keyword argument for each parameter.
+1. **Read** — for each Trouve in `inputs`, run `SELECT * FROM <full_name>` and load the result into a DataFrame. Column names become lowercase.
+2. **Transform** — call your function locally on the clair machine, with one DataFrame for each parameter, in the order of `inputs`.
 3. **Write** — write the DataFrame from your function to Snowflake. clair creates or replaces the table.
 4. **Test** — run the attached tests against the output table in Snowflake.
 
@@ -85,9 +90,16 @@ If your function returns a different type than `DataFrame`, the run fails with a
 !!! note
     clair reads the data into the memory of the machine that runs clair. For large upstream tables this is slow, and it uses much memory. Chunked reads are not available.
 
+## Validation
+
+Clair examines the `transform` signature when Python loads your file. Thus a mistake stops the run immediately, and it names the fault:
+
+- The count of `inputs` must equal the count of parameters. An error tells you both counts and lists the parameter names.
+- The transform must not use `*args` or `**kwargs`. Clair binds each input to a named parameter.
+
 ## DAG integration
 
-Dependencies come from the parameter defaults. No extra configuration is necessary. `clair dag` marks these nodes with a `[PANDAS]` tag, in place of the `[TABLE]` or `[VIEW]` tag:
+Dependencies come from `inputs`. No extra configuration is necessary. `clair dag` marks these nodes with a `[PANDAS]` tag, in place of the `[TABLE]` or `[VIEW]` tag:
 
 ```
 === Clair DAG: 3 models, 1 source ===
@@ -98,7 +110,7 @@ example_4_database.source.events  [SOURCE]
         └── example_4_database.derived.top_event_types  [TABLE]
 ```
 
-SQL Trouves can depend on the output of a `df_fn` Trouve, and a `df_fn` Trouve can depend on other `df_fn` Trouves. The tree above shows both: a `df_fn` node reads a SQL table, and a SQL table reads the `df_fn` output.
+A SQL `Trouve` can depend on the output of a `PandasTrouve`, and a `PandasTrouve` can depend on other `PandasTrouve` nodes. The tree above shows both: a pandas node reads a SQL table, and a SQL table reads the pandas output.
 
 ## Selectors
 
@@ -110,7 +122,7 @@ clair run --project=. --env=dev --select='derived.products.top_rated'
 
 ## Compile output
 
-`clair compile` writes a `.py` artifact for a `df_fn` Trouve, in place of the `.sql` file it writes for a SQL Trouve. The artifact holds a header, the imports of the source module, and the source of your function:
+`clair compile` writes a `.py` artifact for a `PandasTrouve`, in place of the `.sql` file it writes for a SQL Trouve. The artifact holds a header, the imports of the source module, and the source of your function. The header shows which upstream Trouve clair binds to each parameter:
 
 ```python
 # clair compiled: derived.products.top_rated
@@ -121,31 +133,28 @@ clair run --project=. --env=dev --select='derived.products.top_rated'
 
 import pandas as pd
 
-def top_rated(
-    catalog: pd.DataFrame = catalog_trouve,  # type: ignore
-    reviews: pd.DataFrame = reviews_trouve,  # type: ignore
-) -> pd.DataFrame:
+def top_rated(catalog: pd.DataFrame, reviews: pd.DataFrame) -> pd.DataFrame:
     ...
 ```
 
 ## Limitations
 
-- **Full-refresh only.** Incremental strategies are not available. A `df_fn` Trouve always replaces the table. A `RunConfig` with an incremental mode raises an error.
-- **TABLE output only.** Views are not available.
+- **Full-refresh only.** Incremental strategies are not available. A `PandasTrouve` always replaces the table. A `RunConfig` with an incremental mode raises an error.
+- **TABLE output only.** Views and sources are not available.
 - **A full table read.** clair reads all the upstream rows into memory. Chunked reads are not available.
-- **`sql` and `df_fn` are mutually exclusive.** A Trouve with both raises an error.
 
 ## Field reference
 
-These are the `Trouve` fields that apply to pandas execution. See the [Trouve API reference](../reference/trouve-api.md) for the full list.
+These are the `PandasTrouve` fields. See the [Trouve API reference](../reference/trouve-api.md) for the full list.
 
 | Field | Type | Default | Description |
 |-------|------|---------|-------------|
-| `df_fn` | `Callable \| None` | `None` | Python function that returns the output DataFrame. Its parameter defaults declare the upstream Trouves. Mutually exclusive with `sql`. |
+| `transform` | `Callable[..., pd.DataFrame]` | required | Python function that gives the output DataFrame. |
+| `inputs` | `list[TrouveAbc]` | `[]` | The upstream Trouves. clair binds each one to a transform parameter by position. |
 | `columns` | `list[Column]` | `[]` | Column definitions. Optional — clair uses them for the documentation. |
 | `tests` | `list[AnyTest]` | `[]` | Data quality tests. They run after clair writes the output. |
 | `docs` | `str` | `""` | Documentation string. `clair docs` shows it. |
 
 ## Complete example
 
-`example_projects/example_4/` in the repository is a runnable project that uses `df_fn`. See `example_4_database/derived/daily_event_counts.py`.
+`example_projects/example_4/` in the repository is a runnable project that uses `PandasTrouve`. See `example_4_database/derived/daily_event_counts.py`.