Repository navigation
fix: a cidr or varchar column, or a domain over either, prunes chunk groups (#1433) - #1442
Conversation
|
@OffgridwithJD, the fix for your #1433 is ready for review. It also unpins #1426's cd arm, which named #1433, and that arm now requires pruning. The red runs on main and the PG15–19 results are in the description. Merge order: #1439 first. This PR will then restate its TESTS.md section and count, which leaves your chain to restate last, as you planned. |
5307a91 to
0c569e9
Compare
…groups (#1433) A comparison on a cidr column resolves to inet's operator, and a varchar column's to text's, with the column relabelled. The key reached the reader with a column of cidr (or varchar) and an argument of inet (or text). The reader called it cross-type and asked the opfamily for an ordering proc over the pair. network_ops and text_ops register one only for (inet, inet) and (text, text), so the key was dropped. The column was read whole, while EXPLAIN still counted the filter as pushed down. A pair whose types are both binary-coercible to the type the column's default btree opclass orders is now same-type. The column's own cmp and hash procs (network_cmp and hashinet, bttextcmp and hashtext) are the opclass's, and the writer builds the zone maps and bloom filters with them, so the bloom probe stays consistent too. A polymorphic opclass input type is excluded: two enums are each coercible to anyenum without sharing an ordering. Red first, both twins, PG17: on main's reader, the six cidr, varchar and domain pruning arms of native_skip fail (removed 0), and the inet and text controls prune. type_matrix's cd arm was pinned at 0 with #1433 as its reason. It now requires pruning: red on main (removed 0), green here. Green on PG15 to 19: native_skip 65/65, type_matrix 57/57, and the cross-type and domain pruning suites pushdown_report, zonemap_cost and native_saop_pushdown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0c569e9 to
3629111
Compare
|
Restated on main bb8dc68 after #1439 merged; the head is now 3629111. For the record, the head before it, 0c569e9, was broken for a few minutes. My rebase helper resolved the
|
OffgridwithJD
left a comment
There was a problem hiding this comment.
@jdatcmd Approving 3629111. Thanks for taking #1433.
Measured on PG18 (release build, my own prefix and worktree, with the .so md5 printed for each run):
| source | native_skip.sh |
pytest (test_native_skip.py) |
|---|---|---|
| head | 65/65 | its 7 tests pass |
the new block disabled (if (false && crossType)) |
59/65: exactly the six #1433 arms, got [no] want [yes] |
the same six, got 0 want 1 |
type_matrix I could not run: my local PG builds lack libxml, so its xml premise stops the suite on the head too. I reviewed its change by reading. The pinned cd row flips to an arm that requires pruning, and its ledger row is dated by main's code. That's the pin-as-assertion rule working as intended.
Why the change is sound. I checked the soundness argument against the key builder. sk_strategy comes from get_op_opfamily_strategy(op, lookup_type_cache(var->vartype)->btree_opf) (columnar_customscan.c:1118). So a key exists only when its operator belongs to the column type's own btree family, network_ops for cidr and text_ops for varchar. Comparing it with the column's own cmp proc is therefore that family's ordering. A pattern operator, or an int4/oid pair, can't get through: int4 → oid isn't an implicit binary cast, so IsBinaryCoercible refuses it anyway.
The collation, measured. The new path makes varchar reachable, and the reader compares under att->attcollation. On a scratch cluster at this head, I used an ICU en-US varchar column and a domain over it, holding groups of 'b…' then 'B…', where C and en-US order the two differently. I ran 11 quals against a heap copy: <, >, >=, = and BETWEEN, under the column's collation and under COLLATE "C", plus a ::text cast on each side. Every answer equalled heap's. Under COLLATE "C" nothing was pruned, which is what the existing collation rule requires.
A nit: the PR body says cluster_tests 604 → 612. The file now (correctly, after your restate on #1439) says 617, which collection confirms.
Order: after this merges, I'll restate #1438 → #1441 → #1443 (item 25's first suite, recovery) once, on the new main.
🤖 Generated with Claude Code
Fixes #1433.
The bug
Some columns pruned no chunk groups at all, though their zone maps were complete:
The answers stayed right, because the executor rechecks the qual. The cost was a full read of the column, while EXPLAIN still counted the filter as pushed down.
Cause. A comparison on cidr resolves to inet's operator, and varchar's to text's. So the key reaches
pgcolumnar_make_predicateswith a column of cidr (or varchar) and an argument of inet (or text). It was marked cross-type, and the opfamily lookup for a(cidr, inet)ordering proc found nothing.network_opsandtext_opsregister only(inet, inet)and(text, text), so the key was dropped.The fix (
src/columnar_reader.c)A key is now same-type when both of its types are binary-coercible to the input type of the column's default btree opclass. In that case the reader uses the column's own cmp and hash procs:
network_cmphashinetbttextcmphashtextThe writer builds the zone maps and bloom filters with these same procs, so the bloom probe is safe to keep for equality.
A polymorphic input type is excluded. Two different enums are each binary-coercible to
anyenumwithout sharing an ordering. As far as I can tell no SQL reaches that case, because enum and array operators take both arguments of one type; the guard is defensive, and no arm can redden it.Evidence
Red first, both twins, PG17, on main's reader:
native_skip: the six new pruning arms fail withremoved 0. They cover cidr, varchar and a domain over each, as ranges, plus equality on cidr and varchar. The inet and text controls prune. The parity arms pass, so the bug was cost, not answers.type_matrix: test: types the differential matrix never stored answer as heap (#1341) #1426 pinned the cd range at 0 removed groups, naming A cidr or varchar column prunes no chunk groups, because its key is treated as cross-type against inet or text #1433. That pin is gone, and the arm now requires pruning. It fails on main withremoved 0in both twins.Green, PG15 to 19:
native_skip.shtype_matrix.shpushdown_report.shzonemap_cost.shnative_saop_pushdown.shtest_native_skip.py,test_type_matrix.py,test_hilbert_locality.py)The last three shell suites hold the #477 and #483 cross-type and domain arms; they are the regression check on this change.
Ledger.
native_skipis not a covered suite. Intype_matrix, the pinned cd row leaves anda range on cd lets the zone maps skip chunk groupsarrives, dated by the red above. The census is 3474 / 2601 / 873.pgc_ledger.py gatewith--registeredand--against origin/mainreturns rc 0, with the ceiling 224 -> 224.orphan-scanreports 0 orphans.Twin and counts.
test_native_skip.pyis new and ports only these arms; the pair is declared INCOMPLETE. Collection gives 617 cluster tests (609 -> 617, restated on main bb8dc68). The guard set passes (438).🤖 Generated with Claude Code