This project simulates a real-world enterprise data migration and modernization strategy. It extracts transactional data from a simulated "On-Premise" environment (hosted on AWS EC2), performs heavy distributed processing using a Hadoop/Spark cluster, and ultimately serves the data via a Cloud-Native, serverless architecture to optimize costs .

A Data Warehousing project for retail sales using dimension modelling best practices with SCD type 2 on AWS Redshift. Utilizing AWS Lambda, Glue Workflows and Python Shell jobs to create and automate an ELT pipeline where batch data coming into S3 is loaded onto Redshift and necessary transformations are performed to meet requirements.

Build a data warehouse from scratch, including full load, daily incremental load, design schema, SCD Type 1 and 2.

Production-grade ELT pipeline classifying VIX volatility regimes. Features incremental models, SCD-2 snapshots, recursive CTEs, and Slim CI/CD. Stack: dbt, DuckDB/MotherDuck, Python.

End-to-end data lakehouse on Azure Databricks — Medallion Architecture, Star Schema, SCD Type 2, CI/CD, Data Quality Framework

End-to-End Retail Lakehouse on Microsoft Fabric | Medallion Architecture | PySpark | SCD Type-2 | DirectLake

Enterprise Data Warehouse with Star Schema, SCD Type 2, ETL pipelines, and multi-currency analytics (SQL Server + Python)

Greetings and welcome to my profile.

End-to-end Azure Databricks retail data engineering project using Medallion Architecture (Bronze, Silver, Gold). Implements Auto Loader, Unity Catalog, Delta Lake, SCD Type 1 & 2 dimensions, and Fact Orders for analytics-ready star schema modeling.

Working with SCD Type (Change Data Capture) and need a Data Vault model to test Azure Data Factory v2? - This Code with Help!

In this project we'll create real time healthcare patient data pipeline as data source and use arious services and tools like Azure Eventhubs, Azure Databricks, Delta lake and synapse analytics. also, implement medallion architecture, schema evolution and create facts and dimension tables and connect the cleaned and transformed data to PowerBI.

Complete Data Vault 2.0 (DV2) data warehouse implementation with Medallion Architecture on SQL Server. 67 objects, 60 SSIS packages, watermark-based incremental ETL.

End-to-end data engineering project using AWS S3, Snowflake, and dbt to implement Medallion Architecture with SCD Type 1 & Type 2 logic on Walmart sales data, followed by analytical visualizations using Seaborn and Plotly.

End-to-end data engineering project using AWS S3, Snowflake, and dbt to implement Medallion Architecture with SCD Type 1 & Type 2 logic on Walmart sales data, followed by analytical visualizations using Seaborn and Plotly.

E-commerce analytics pipeline using snowflake and dbt - staging, intermediate, marts, scd2 snapshots

Design and implement a full ELT data pipeline using Snowflake and S3, featuring star schema modelling, SCD Type 1 & 2 handling, and incremental load automation

Production-style Slowly Changing Dimension (SCD Type 2) pipeline built with Snowflake, dbt, and AWS S3. Demonstrates secure S3 ingestion, layered bronze/silver/gold modeling, dbt snapshots for historical tracking, and analytics-ready views identifying active vs historical records.

Production-grade data lakehouse on Databricks — PySpark · Delta Lake · Spark Streaming · MLflow · 19.4M real NYC taxi trips

End-to-end data pipeline: CSV → Bronze → Silver → Gold on AWS S3 with PySpark, Airflow, Great Expectations, SCD Type 2, and Streamlit dashboard

Enterprise-grade Microsoft Fabric Lakehouse project implementing Medallion architecture (Bronze, Silver, Gold) for financial transactio