Skip to content
View MsMansiDhruv's full-sized avatar

Block or report MsMansiDhruv

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
MsMansiDhruv/README.md
Mansi Dhruv — Lead Data Engineer & Solution Architect

Portfolio   Case Studies   LinkedIn   Email


Upstream Open Source & Engine Internals

Engine / Repository Pull Request Technical Scope Status
NVIDIA/cudf-spark #16863 Fix buffer and resource leaks on multithreaded file reader cancellation [re... Open (In Review) · 2026-11-19
NVIDIA/cudf-spark #16881 Fix multithreaded read when maxNumFilesParallel is below task file count [r... Open (In Review) · 2026-11-19
databricks/dbt-databricks #1767 feat: supplement delta.dataSkippingStatsColumns for snapshot tables (#1481) Open (In Review) · 2026-11-16
ibis-project/ibis #12214 fix(postgres): read query schema without temporary view DDL for read-only s... Open (In Review) · 2026-11-16
apache/iceberg-python #4265 Fix: assert ref snapshot id requirements during snapshot expiration (#4200) Open (In Review) · 2026-11-12
delta-io/delta-rs #5028 feat(python): support tuple filters in write, update, and delete predicates... Open (In Review) · 2026-11-08
databricks/dbt-databricks #1766 feat: allow passing query tags via environment variables (#1454) Open (In Review) · 2026-11-06
ibis-project/ibis #12211 fix(expr): type decimal mean as float64 and decimal product with combined s... Open (In Review) · 2026-11-03

Flagship Systems

Medallion Lakehouse on Databricks featuring Auto Loader, Delta CDF, SCD1/SCD2, Great Expectations quarantine gates, offline SQLGlot AST eval (~85ms), and AutoHeal-DE partition recovery.

Databricks · Delta Lake · sqlglot

In-memory vectorized columnar query execution engine exploring contiguous ColumnVector chunks, RecordBatch iteration, bitmask filters, and rule-based pushdown.

Columnar · Vectorized · Optimizer

PyPI PyPI Downloads

Open-source static AST optimizer, diagnostic engine, and built-in MCP server (dlt-doctor mcp) for Databricks DLT and PySpark Catalyst plans.

pip install dlt-doctor · MCP


Reference Lakehouse & AI Architecture

flowchart LR
    Ingest[Auto Loader / Kafka] --> Compute[Spark / RAPIDS GPU]
    Compute --> Gates{Quality & AST Gates}
    Gates -->|Valid| Storage[(Delta / Iceberg · Unity Catalog)]
    Gates -->|Quarantine| AutoHeal[AutoHeal-DE Partition Merge]
    Storage --> Serve[Databricks Genie / MCP Server]
Loading

Recent Architecture Notes & Case Studies


Certifications, Credentials & Honors

Anthropic MCP Advanced Topics   Anthropic Intro to MCP   Databricks Genie Code Foundations   Databricks Genie One & Agents   Databricks Spark Declarative Pipeline   Databricks Performance Optimization   NASA Open Science Essentials   AWS Certified

Value-able Award   GEM Award   GHCI Scholar   ACM-W Mention   MCA Merit Scholar   SQL for Data Analysis   Azure Databricks ETL   PySpark for Data Analysis   Big Data Analytics using Spark   Data Structures with Python   Machine Learning Using Python   Django 2.2   Docker Essentials   Node JS   Big Data Training   AI Website Creation


Python    AWS    Databricks    Apache Spark    Terraform    Docker    Apache Airflow    Apache Kafka    PostgreSQL    C++    NVIDIA

Complexity → operability  ·  portfolio-mansi-eight.vercel.app

Pinned Loading

  1. sentinel-lakehouse sentinel-lakehouse Public

    Production Medallion Lakehouse on Databricks with Auto Loader, Delta CDF, SCD1/SCD2, Great Expectations quarantine gates, and Unity Catalog.

    Python 1

  2. lifeline-os lifeline-os Public

    TypeScript

  3. dlt-doctor dlt-doctor Public

    Production optimization engine and quality accelerator for Databricks DLT, Auto Loader, and PySpark

    Python 1

  4. gquery-engine-clone gquery-engine-clone Public

    Vectorized columnar relational query engine laboratory exploring execution plans, predicate pushdown, and in-memory batch processing.

    Python 1