Repository navigation
Commit 1e7edb4
committed
Use the full column name in the dynamic_partition_overwrite delete filter
Closes issue 4196
# Rationale for this change
`dynamic_partition_overwrite` built its delete filter from `find_field(source_id).name`, which is only the leaf name. For a partition sourced from the nested column `s.x`, the filter referenced `x`, so:
- with a top-level `x`, it deleted rows by the wrong column and kept the old rows of the replaced partition
- without one, binding the filter raised `ValueError`
The filter now uses `find_column_name`, the full column name, the same way `_determine_partitions` does since PR 2204.
## Are these changes tested?
Yes. `test_dynamic_partition_overwrite_by_nested_source_column` in `tests/table/test_init.py` runs the example from the issue against the memory and SQL catalogs:
- with this change, only partition `s.x = 5` is replaced: `[(5, 1), (9, 5)]`
- without it, the test fails with `[(1, 5), (9, 5)]`
## Are there any user-facing changes?
Yes, a bug fix: `dynamic_partition_overwrite` replaces the right partitions on tables partitioned by a nested column. Tables partitioned only by top-level columns behave as before.
## AI Disclosure
Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change. All submitted content was reviewed by a human before submission.
Signed-off-by: KK <37928252+dragonknightk@users.noreply.github.com>1 parent 09960d5 commit 1e7edb4
2 files changed
Lines changed: 31 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
643 | 643 | | |
644 | 644 | | |
645 | 645 | | |
646 | | - | |
647 | | - | |
648 | | - | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
649 | 653 | | |
650 | 654 | | |
651 | 655 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2489 | 2489 | | |
2490 | 2490 | | |
2491 | 2491 | | |
| 2492 | + | |
| 2493 | + | |
| 2494 | + | |
| 2495 | + | |
| 2496 | + | |
| 2497 | + | |
| 2498 | + | |
| 2499 | + | |
| 2500 | + | |
| 2501 | + | |
| 2502 | + | |
| 2503 | + | |
| 2504 | + | |
| 2505 | + | |
| 2506 | + | |
| 2507 | + | |
| 2508 | + | |
| 2509 | + | |
| 2510 | + | |
| 2511 | + | |
| 2512 | + | |
| 2513 | + | |
| 2514 | + | |
| 2515 | + | |
2492 | 2516 | | |
2493 | 2517 | | |
2494 | 2518 | | |
| |||
0 commit comments