Skip to content

feat: split Trouve into TrouveAbc + Trouve + PandasTrouve - #18

Merged
OmerBaddour merged 3 commits into
mainfrom
worktree-pandas-trouve
Aug 2, 2026
Merged

feat: split Trouve into TrouveAbc + Trouve + PandasTrouve#18
OmerBaddour merged 3 commits into
mainfrom
worktree-pandas-trouve

Conversation

@OmerBaddour

Copy link
Copy Markdown
Member

What

Replaces the df_fn field with a PandasTrouve class, and splits Trouve into an abstract base plus one class per backend.

Before — the dependency rode on the parameter default, so the annotation was false and every parameter needed a # type: ignore:

def daily_event_counts(
    refined_events: pd.DataFrame = example_4_database_refined_events,  # type: ignore
) -> pd.DataFrame:
    ...

trouve = Trouve(df_fn=daily_event_counts)

After — the transform is an honest function; the dependencies live on the Trouve:

def daily_event_counts(refined_events: pd.DataFrame) -> pd.DataFrame:
    ...

trouve = PandasTrouve(
    transform=daily_event_counts,
    inputs=[example_4_database_refined_events],
)

Clair binds inputs to the transform parameters by position, so parameter names are free.

The class split

Class Backend Declares dependencies with
TrouveAbc abstract base — columns, tests, docs, run_config, compiled
Trouve Snowflake SQL f-string references in sql
PandasTrouve pandas the inputs list

Backend-specific validation moved off the shared base: _validate_sql now lives on Trouve only. This is what makes room for a DatabricksTrouve later.

Why this shape

  • The pd.DataFrame annotation is now true, so no # type: ignore — verified with ty.
  • The transform takes plain DataFrames, so it is directly callable in a unit test or notebook.
  • PandasTrouve validates transform arity at import time and rejects *args/**kwargs, with an error naming the parameters.
  • Because dag.get_trouve() returns TrouveAbc, ty now forces explicit narrowing before any .sql / .transform / .build_sql access. Several call sites gained isinstance asserts as a result.

Breaking change

df_fn is removed rather than deprecated — pre-1.0, and keeping it would defeat the split. All in-repo callers migrated.

Notes

  • The example's second # type: ignore came from pandas stubs (groupby().size() types as Series), not from clair. Rewritten as .agg(event_count=("event_type", "size")), which typechecks clean.
  • CompiledAttributes.resolved_df_fn renamed to resolved_transform.
  • build_dag / find_routing_collisions / recompile_for_selection take Sequence[TrouveAbc]list is invariant, so a list[Trouve] would not have been accepted.

Verification

  • uv run pytest tests/ — 529 passed
  • uvx ruff check . — clean
  • uvx ty check --exclude example_notebooks — clean
  • mkdocs build --strict — clean
  • clair dag and clair compile on example_projects/example_4 — DAG resolves with the [PANDAS] node in place, and the compiled artifact header maps refined_events -> example_4_database.refined.events

Docs updated: the pandas guide, the Trouve API reference, the Trouve concept page, the landing page, the reference index, and the README.

🤖 Generated with Claude Code

OmerBaddour and others added 3 commits August 1, 2026 19:26
Replace the df_fn field with a PandasTrouve class. A pandas transform is now
a plain function that takes DataFrames, and the upstream Trouves come from an
inputs list. Clair binds the inputs to the parameters by position.

Before, a transform declared each dependency as a parameter default value.
That made the pd.DataFrame annotation false, thus each parameter needed a
`# type: ignore`, and a user could not call the function directly.

The split introduces TrouveAbc, the abstract base that holds the attributes
of every backend. Trouve keeps the SQL backend and its validation. This makes
space for a Databricks backend later.

- PandasTrouve validates the arity of the transform, and rejects *args/**kwargs
- ClairDag, discovery, the compiler and the runner accept a TrouveAbc
- The example project and every documentation page use the new API

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Resolve conflicts in favour of the PandasTrouve API from this branch,
while it keeps the Simplified Technical English text and the routing
exports from main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Discovery gives a list[TrouveAbc] after the PandasTrouve split. The
function reads only .compiled and .type, which the base class holds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@OmerBaddour
OmerBaddour marked this pull request as ready for review August 2, 2026 23:44
@OmerBaddour
OmerBaddour merged commit 87a0d48 into main Aug 2, 2026
3 checks passed
@OmerBaddour
OmerBaddour deleted the worktree-pandas-trouve branch August 2, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant