Repository navigation
Filesystem: Expect no file after a refused ORC write - #426
Conversation
dlt-filesystem 0.22 encodes an ORC export in memory and writes the path only once PyArrow has accepted the schema, so a time column under Parquet staging now leaves no file at all instead of a zero-byte one. Assert the absence, drop the comment that explained the empty file, and require dlt-filesystem 0.22 so the test and the behaviour it pins ship together.
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one): 1
Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #426 +/- ##
==========================================
+ Coverage 60.11% 60.24% +0.13%
==========================================
Files 201 201
Lines 10311 10344 +33
==========================================
+ Hits 6198 6232 +34
+ Misses 4113 4112 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Summary
Fixes #424. dlt-filesystem 0.22 writes an ORC export only once PyArrow has accepted the schema (dlt-contrib/dlt-filesystem#26), so the refused write in
test_orc_cannot_take_a_time_column_under_parquet_stagingnow leaves no file at all where it used to leave a zero-byte one.staged.orcdoes not exist, and drops the comment that explained the empty file.>=0.22on the base dependency and the[filesystem]extra, so the test and the behaviour it pins ship together. The[iterable]and[vortex]extras resolve against the base floor.Test plan
mainwith dlt-filesystem 0.23.0, the test fails with theFileNotFoundErrorfrom Test casetest_orc_cannot_take_a_time_column_under_parquet_stagingstarted failing #424.assert not True), so the new assertion can fail.tests/main/filesystem/: 341 passed, 6 skipped (vortex extra not installed), Python 3.13.ruff format --check,ruff check,validate-pyprojectclean.Review
Two independent reviews from different model families read the commit: no findings.