Skip to content

Add WMATA Smart Data Hub open-source implementation#1

Open
chrisyamas wants to merge 1 commit into
mainfrom
wmata/april-2026-open-source
Open

Add WMATA Smart Data Hub open-source implementation#1
chrisyamas wants to merge 1 commit into
mainfrom
wmata/april-2026-open-source

Conversation

@chrisyamas
Copy link
Copy Markdown
Contributor

Publishing WMATA's Smart Data Hub code as the first implementer entry in this
repository, contributed under agencies/wmata/sdh-open-source/.

What's included

The WMATA team's open-source release of the Smart Data Hub, a transit data
lakehouse that ingests data from GTFS, AVL and APC, fare gate, and open-loop
fare payment sources, transforms it into TIDES-compliant tables, and serves
it for analytics and reporting. The release includes:

  • pipelines/ — Dagster pipelines for ingestion and orchestration
  • warehouse/ — dbt project with models, tests, macros, and seeds
  • tf/ — Terraform/OpenTofu infrastructure-as-code
  • docs/ — documentation and architecture diagrams
  • scripts/ — setup and deployment convenience scripts

If you want to see TIDES in action, warehouse/ is the place to look.
That's where the WMATA team's vendor-formatted source data (GTFS, AVL
and APC, fare gate, and open-loop fare payments) gets transformed into
TIDES tables, organized in dbt's layered structure: staging,
intermediate, mart, and metrics layers. Everything else in the
repository supports getting the data into the warehouse and running
the transformations reliably.

About this contribution

The contributed code is a redacted version of a production repository,
prepared by the WMATA team. Vendor names and sensitive details were
replaced with placeholders before the code was provided for publication.
Two additional placeholder substitutions were applied prior to this PR to
cover items surfaced in a pre-publication leakage review (a concrete Azure
Container Registry name in a debug-module example, and a user directory
segment in a pyproject.toml comment); both are noted inline in the files
where they appear.

Primary contact

Chum Chancharadeth, CChancharadeth@wmata.com, per the contributed
README.md.

Review notes

Reviewers should focus on subfolder structure and README clarity. The
technical content of the implementation itself is the WMATA team's work
to publish as they see fit; substantive review of that content sits with
WMATA, not with the TIDES maintainers of this repository.

@chrisyamas chrisyamas self-assigned this Apr 23, 2026
@chrisyamas chrisyamas added enhancement New feature or request labels Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants