Skip to content

Issue 443: fix: support mixed snapshot and non-snapshot flows - #451

Merged
ravi-databricks merged 2 commits into
v0.1.1from
issue_443
Sep 17, 2026
Merged

ravi-databricks merged 2 commits into
v0.1.1from
issue_443

Conversation

@ravi-databricks

@ravi-databricks ravi-databricks commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

fix: support mixed snapshot and non-snapshot flows

Scope snapshot callbacks to snapshot specs so non-snapshot input views
are still registered when flows share a pipeline.

Add mixed snapshot and CloudFiles integration coverage.

- wrap append-flow readers with the configured custom transform
- support cloudFiles, Delta, Kafka, and Event Hubs append sources
- add unit coverage for transformed append-flow views
- validate processing timestamps in cloudfiles integration tests
Scope snapshot callbacks to snapshot specs so non-snapshot input views
are still registered when flows share a pipeline.

Add mixed snapshot and CloudFiles integration coverage.
@ravi-databricks ravi-databricks added this to the v0.1.1 milestone Sep 17, 2026
@ravi-databricks ravi-databricks self-assigned this Sep 17, 2026
@ravi-databricks ravi-databricks added the bug Something isn't working label Sep 17, 2026
@ravi-databricks ravi-databricks changed the title Issue 443 Issue 443: fix: support mixed snapshot and non-snapshot flows Sep 17, 2026
@codecov

codecov Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.30769% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 89.82%. Comparing base (350df0d) to head (420d3ed).
⚠️ Report is 1 commits behind head on v0.1.1.

Files with missing lines Patch % Lines
src/databricks/labs/sdp_meta/dataflow_pipeline.py 92.30% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           v0.1.1     #451      +/-   ##
==========================================
+ Coverage   89.80%   89.82%   +0.02%     
==========================================
  Files          18       18              
  Lines        4238     4247       +9     
  Branches      828      828              
==========================================
+ Hits         3806     3815       +9     
- Misses        242      243       +1     
+ Partials      190      189       -1     
Flag Coverage Δ
unittests 89.82% <92.30%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@brij-raghuwanshi-db brij-raghuwanshi-db left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@ravi-databricks
ravi-databricks merged commit 987c2d1 into v0.1.1 Sep 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants