Skip to content

About

A small SQLMesh project that runs locally on DuckDB and on dagctl.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

dagctl SQLMesh demo

A small SQLMesh project you can run on your laptop in a minute, and on dagctl in an afternoon. dagctl is a hosted service that runs SQLMesh projects in production: it keeps the project's state, runs the models on a schedule, holds every change for approval, and tells you when something breaks.

The project models a simple business: customers, their orders, and the two numbers a finance or sales team asks for, revenue by day and revenue by customer.

Run it locally

Needs Python 3.9 or newer. Everything runs against a local DuckDB file, so there are no credentials to set up.

pip install -r requirements.txt
sqlmesh plan --auto-apply
sqlmesh fetchdf "SELECT * FROM demo.daily_revenue ORDER BY order_date"

sqlmesh plan loads the two seed files, builds the views and tables, runs the audits, and shows you exactly what it did. Change a model and run sqlmesh plan again: SQLMesh shows what the change affects and rebuilds only that.

Run it on dagctl

With a dagctl organization and the dagctl CLI installed:

dagctl auth login --org your-org
dagctl use-project demo
dagctl config generate
sqlmesh plan

dagctl config generate writes a config.py that points SQLMesh at your project's hosted state, so a plan from your laptop and a scheduled run on dagctl see the same state. Point your project's warehouse gateway at Snowflake, BigQuery, Databricks, Redshift, or PostgreSQL in that config; DuckDB is for running locally.

What is in it

Path What it is
seeds/ Two CSV files: customers and orders.
models/seeds/ Loads the CSV files as SQLMesh seed models.
models/staging/ One view per source, cleaned and typed, with audits for uniqueness and nulls.
models/marts/customer_orders.sql One row per customer: completed orders, lifetime revenue, first and last order. Rebuilt in full.
models/marts/daily_revenue.sql Revenue by day. Incremental: a run only processes the days it is asked for.
audits/ A custom audit that fails if an order amount is not positive.
config.yaml The local DuckDB gateway.

License

Apache-2.0

About

A small SQLMesh project that runs locally on DuckDB and on dagctl.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors