Skip to content

fix: a cidr or varchar column, or a domain over either, prunes chunk groups (#1433) - #1442

Merged
jdatcmd merged 1 commit into
mainfrom
fix/1433-coercible-zone-maps
Oct 16, 2026
Merged

jdatcmd merged 1 commit into
mainfrom
fix/1433-coercible-zone-maps

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1433.

The bug

Some columns pruned no chunk groups at all, though their zone maps were complete:

  • a cidr column;
  • a varchar column;
  • a domain over either.

The answers stayed right, because the executor rechecks the qual. The cost was a full read of the column, while EXPLAIN still counted the filter as pushed down.

Cause. A comparison on cidr resolves to inet's operator, and varchar's to text's. So the key reaches pgcolumnar_make_predicates with a column of cidr (or varchar) and an argument of inet (or text). It was marked cross-type, and the opfamily lookup for a (cidr, inet) ordering proc found nothing. network_ops and text_ops register only (inet, inet) and (text, text), so the key was dropped.

The fix (src/columnar_reader.c)

A key is now same-type when both of its types are binary-coercible to the input type of the column's default btree opclass. In that case the reader uses the column's own cmp and hash procs:

column cmp hash
cidr network_cmp hashinet
varchar bttextcmp hashtext

The writer builds the zone maps and bloom filters with these same procs, so the bloom probe is safe to keep for equality.

A polymorphic input type is excluded. Two different enums are each binary-coercible to anyenum without sharing an ordering. As far as I can tell no SQL reaches that case, because enum and array operators take both arguments of one type; the guard is defensive, and no arm can redden it.

Evidence

Red first, both twins, PG17, on main's reader:

Green, PG15 to 19:

suite result on each major
native_skip.sh 65/65
type_matrix.sh 57/57
pushdown_report.sh 51/51
zonemap_cost.sh 19/19
native_saop_pushdown.sh 53/53
pytest (test_native_skip.py, test_type_matrix.py, test_hilbert_locality.py) 147 checks, 27 tests

The last three shell suites hold the #477 and #483 cross-type and domain arms; they are the regression check on this change.

Ledger. native_skip is not a covered suite. In type_matrix, the pinned cd row leaves and a range on cd lets the zone maps skip chunk groups arrives, dated by the red above. The census is 3474 / 2601 / 873. pgc_ledger.py gate with --registered and --against origin/main returns rc 0, with the ceiling 224 -> 224. orphan-scan reports 0 orphans.

Twin and counts. test_native_skip.py is new and ports only these arms; the pair is declared INCOMPLETE. Collection gives 617 cluster tests (609 -> 617, restated on main bb8dc68). The guard set passes (438).

🤖 Generated with Claude Code

@jdatcmd

jdatcmd commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, the fix for your #1433 is ready for review. It also unpins #1426's cd arm, which named #1433, and that arm now requires pruning. The red runs on main and the PG15–19 results are in the description. Merge order: #1439 first. This PR will then restate its TESTS.md section and count, which leaves your chain to restate last, as you planned.

…groups (#1433)

A comparison on a cidr column resolves to inet's operator, and a varchar
column's to text's, with the column relabelled. The key reached the reader with
a column of cidr (or varchar) and an argument of inet (or text). The reader
called it cross-type and asked the opfamily for an ordering proc over the pair.
network_ops and text_ops register one only for (inet, inet) and (text, text), so
the key was dropped. The column was read whole, while EXPLAIN still counted the
filter as pushed down.

A pair whose types are both binary-coercible to the type the column's default
btree opclass orders is now same-type. The column's own cmp and hash procs
(network_cmp and hashinet, bttextcmp and hashtext) are the opclass's, and the
writer builds the zone maps and bloom filters with them, so the bloom probe stays
consistent too. A polymorphic opclass input type is excluded: two enums are each
coercible to anyenum without sharing an ordering.

Red first, both twins, PG17: on main's reader, the six cidr, varchar and domain
pruning arms of native_skip fail (removed 0), and the inet and text controls
prune. type_matrix's cd arm was pinned at 0 with #1433 as its reason. It now
requires pruning: red on main (removed 0), green here. Green on PG15 to 19:
native_skip 65/65, type_matrix 57/57, and the cross-type and domain pruning
suites pushdown_report, zonemap_cost and native_saop_pushdown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd
jdatcmd force-pushed the fix/1433-coercible-zone-maps branch from 0c569e9 to 3629111 Compare October 16, 2026 00:14
@jdatcmd

jdatcmd commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator Author

Restated on main bb8dc68 after #1439 merged; the head is now 3629111. For the record, the head before it, 0c569e9, was broken for a few minutes. My rebase helper resolved the test_compare_to_bash.py conflict by sorting the union of both sides' lines. That is safe for a list of suite names, but it scrambled the two multi-line INCOMPLETE entries into a SyntaxError. I pushed before reading the guard run's error.

  • 3629111: both entries are rebuilt in order, and the file's diff against main is exactly the native_skip entry. The guards pass (438 tests, 1286 checks), and collection gives 617.
  • The helper now refuses any conflict that contains a line that is not a bare name. I tested it on a name list (unioned) and on a dict entry (refused, file untouched).
  • The code and tests are otherwise unchanged from 5307a91: the per-file patches are the same, apart from the TESTS.md section number (133) and the count (609 -> 617).

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Approving 3629111. Thanks for taking #1433.

Measured on PG18 (release build, my own prefix and worktree, with the .so md5 printed for each run):

source native_skip.sh pytest (test_native_skip.py)
head 65/65 its 7 tests pass
the new block disabled (if (false && crossType)) 59/65: exactly the six #1433 arms, got [no] want [yes] the same six, got 0 want 1

type_matrix I could not run: my local PG builds lack libxml, so its xml premise stops the suite on the head too. I reviewed its change by reading. The pinned cd row flips to an arm that requires pruning, and its ledger row is dated by main's code. That's the pin-as-assertion rule working as intended.

Why the change is sound. I checked the soundness argument against the key builder. sk_strategy comes from get_op_opfamily_strategy(op, lookup_type_cache(var->vartype)->btree_opf) (columnar_customscan.c:1118). So a key exists only when its operator belongs to the column type's own btree family, network_ops for cidr and text_ops for varchar. Comparing it with the column's own cmp proc is therefore that family's ordering. A pattern operator, or an int4/oid pair, can't get through: int4 → oid isn't an implicit binary cast, so IsBinaryCoercible refuses it anyway.

The collation, measured. The new path makes varchar reachable, and the reader compares under att->attcollation. On a scratch cluster at this head, I used an ICU en-US varchar column and a domain over it, holding groups of 'b…' then 'B…', where C and en-US order the two differently. I ran 11 quals against a heap copy: <, >, >=, = and BETWEEN, under the column's collation and under COLLATE "C", plus a ::text cast on each side. Every answer equalled heap's. Under COLLATE "C" nothing was pruned, which is what the existing collation rule requires.

A nit: the PR body says cluster_tests 604 → 612. The file now (correctly, after your restate on #1439) says 617, which collection confirms.

Order: after this merges, I'll restate #1438 → #1441 → #1443 (item 25's first suite, recovery) once, on the new main.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd merged commit 45eaa19 into main Oct 16, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A cidr or varchar column prunes no chunk groups, because its key is treated as cross-type against inet or text

2 participants