test: native_saop_pushdown gets its pytest twin, which reddens by name wherever the shell does (#1341)

src/columnar_vector.c

@@ -4196,9 +4196,11 @@ pgcolumnar_batch_gates_ok(const PgColumnarAggSpec *specs, int naggs,
 	 * PgColumnarQualsExactlyKeyed demands one EXACT key per clause, so those keys
 	 * never serve as the fold's WHERE even if pgcolumnar_clause_to_predicate were
 	 * later taught to accept a SAOP. The byval gather guard below is a second,
-	 * independent reason LIKE and other varlena filters fall back. Both
-	 * native_saop_pushdown.sh's vector-agg arms and ungrouped_vector_agg.sh's
-	 * #715 arms go red if this regresses.
+	 * independent reason LIKE and other varlena filters fall back.
+	 * ungrouped_vector_agg.sh's #715 arms go red if this regresses. For a SAOP
+	 * it is one of three gates: allConvertible above and the sk_flags test on
+	 * the keys below refuse it too, so native_saop_pushdown.sh's vector-agg
+	 * arms go red only with all three removed (measured, #1341).
 	 */
 	if (!PgColumnarQualsExactlyKeyed(quals, scanrelid, tupdesc))
 		return false;

test/check_ledger_budget.txt

@@ -15,7 +15,7 @@
 #
 #   Adding a check to a suite that is already covered does not move it, which is
 #   what makes it safe to bound.
-suites_not_covered 202
+suites_not_covered 201
 #
 # checks_never_observed_red -- A CENSUS. NOT a ceiling, and it must not become
 #   one.
@@ -3256,4 +3256,25 @@ suites_not_covered 202
 #     never         2889 + 34 = 2923
 #     dated         1118 +  7 = 1125
 #     2923 + 1125 == 4048
-checks_never_observed_red 2923
+#
+# #1341, native_saop_pushdown seeded, with its twin (OffgridwithJD): 2923 -> 2941,
+# and suites_not_covered 202 -> 201. The suite had no rows. It enters with all 53
+# checks from five-major runs (53/53 on each of 15-19). 35 are dated by twelve
+# mutations of the set key: the #704 fix reverted, the hull always taken, the
+# 128-element bound one short and one long, a NULL element bailing out, the single
+# distinct value emitted as a range, the reader's bloom and zone-map set loops, the
+# freeze of a mixed list and of a correlated PARAM_EXEC (which crashes the backend),
+# a cross-type key dropped, and the reader's reject expression cut to SK_ISNULL.
+# Eighteen arrive never: four premises; the NOT IN arm and the three batch-fold
+# arms, each behind more than one gate (useOr and the equality strategy; the fold's
+# allConvertible, exactness and sk_flags gates), so no single-gate mutation reddens
+# them; the parser-collapsed one-element list, which never reaches the set key;
+# seven exact counts no set-key mutation moves; SK_ISNULL, which cannot leave the
+# reject expression without reddening its premise; and the SK_SEARCHARRAY arm. The
+# merge also re-sorted five rewrite_storage_oid rows #1475 renamed in place.
+#
+#     ledger rows   4048 + 53 = 4101
+#     never         2923 + 18 = 2941
+#     dated         1125 + 35 = 1160
+#     2941 + 1160 == 4101
+checks_never_observed_red 2941

test/native_saop_pushdown.sh

@@ -162,8 +162,10 @@ check "the bloom-only multi-value set returns the exact empty answer" \
 
 # `IN (25)` parses to plain `= 25` (the parser collapses a one-element list),
 # so it exercises the OpExpr path; the `= ANY('{25,25}')` arm below is the
-# SAOP shape and is the removal proof for the single-distinct-value
-# equality-key refinement (as two range keys it removes 0).
+# SAOP shape. As two range keys [25,25] it removes 0, so it reddens when the
+# single-distinct-value case is emitted as a range. It does NOT prove the
+# equality-key refinement is needed: without it the list takes the set path,
+# whose bloom loop probes 25 too and removes all 20 (measured).
 check "a one-element IN-list (parser-collapsed to =) bloom-prunes (20 removed)" \
 	"$(groups_removed 'SELECT count(*) FROM t WHERE ov IN (25);')" "20"
 

test/pytest/TESTS.md

@@ -209,6 +209,7 @@ behaviour, the source of that number is named.
 - [161. test_native_parquet_schema.py: parquet_schema round trip](#161-test_native_parquet_schemapy-parquet_schema-round-trip)
 - [162. test_native_vecskip.py: per-vector skipping by the zone maps](#162-test_native_vecskippy-per-vector-skipping-by-the-zone-maps)
 - [163. test_native_index_projection.py: CREATE INDEX projects only what it needs](#163-test_native_index_projectionpy-create-index-projects-only-what-it-needs)
+- [164. test_native_saop_pushdown.py: IN-lists and = ANY(array) prune row groups](#164-test_native_saop_pushdownpy-in-lists-and--anyarray-prune-row-groups)
 
 ## 1. How to read a test in here
 
@@ -7641,3 +7642,20 @@ The pytest twin of `test/native_index_projection.sh` (ported for #1341), every c
 | `test_each_build_projects_only_what_it_needs` | every serial and parallel build projects only its key, expression and predicate columns |
 | `test_the_parallel_built_index_agrees_and_holds_each_row_once` | the parallel-built index agrees with a sequential scan and holds each row once |
 | `test_amcheck_finds_every_index_sound` | `bt_index_check` finds all eleven indexes sound |
+
+## 164. test_native_saop_pushdown.py: IN-lists and = ANY(array) prune row groups
+
+The pytest twin of `test/native_saop_pushdown.sh` (ported for #1341), every check by the shell's name. The scan-key builder keeps an IN-list or `= ANY(array)` of up to 128 non-NULL elements as one set predicate (#704), and a group survives when any element can lie in its zone map and bloom filter. Larger sets keep the bounded [min,max] hull. The fixture is 40000 rows in 20 monotonic groups, plus an overlap column whose every group has the range [10,88] and only even values, so only the bloom filter can prune an odd probe. NOT IN builds no keys. An empty or all-NULL array neither crashes nor mis-counts. A generic plan's array parameter is frozen at Begin, and a correlated one is not. The batch fold never takes the conservative set keys as its exact filter.
+
+### Every test
+
+| test | what it holds |
+| --- | --- |
+| `test_a_set_prunes_by_element_and_keeps_the_exact_answer` | an IN-list pushes one usable set filter that removes 19 groups, a scattered set prunes by element rather than by its hull, and both sides of the 128-element bound |
+| `test_nulls_types_and_collations_prune_exactly` | NULL elements, a by-reference text set, a cross-type array, a C-collated list and a domain-element array each prune and count exactly |
+| `test_only_the_bloom_filter_can_prune_the_overlap_column` | with 25 inside every group's range, the bloom filter alone removes all 20 groups for a set, a single value and a single distinct value |
+| `test_the_conservative_arms_build_nothing_wrong` | a range-spanning list prunes between its elements, NOT IN builds no range, and empty and all-NULL arrays count zero |
+| `test_a_parameter_array_is_frozen_at_begin` | a generic plan's array parameter and a mixed literal-and-parameter list prune |
+| `test_the_batch_fold_never_takes_the_set_keys_as_exact` | the batch fold refuses the IN-list but accepts the equivalent range, and the vectorized count stays exact |
+| `test_a_correlated_array_is_not_frozen_and_the_backend_lives` | a correlated LATERAL array is not frozen, and the session's backend is still alive; a crash is a named red |
+| `test_the_reader_refuses_every_special_flag_but_the_array_one` | the reader's reject expression names every special scan-key flag, and SK_SEARCHARRAY reaches its own arm |

test/pytest/expected_tests.txt

@@ -789,4 +789,6 @@ guard_tests 438
 # one test, by collection over the cluster files on this tree.
 # cluster_tests 698 -> 702 for #1341's test_native_index_projection.py, the twin of native_index_projection.sh:
 # four tests, by collection over the cluster files on this tree.
-cluster_tests 702
+# cluster_tests 702 -> 710 for #1341's test_native_saop_pushdown.py, the twin of native_saop_pushdown.sh:
+# eight tests, by collection over the cluster files on this tree.
+cluster_tests 710

test/pytest/test_compare_to_bash.py

@@ -143,6 +143,7 @@
             "native_reclaim_frag",
             "native_recluster",
             "native_repack",
+            "native_saop_pushdown",
             "native_vacuum_race",
             "native_vecskip",
             "native_writer",
@@ -411,7 +412,7 @@ def test_the_loop_reader_invents_nothing_in_this_corpus(expect):
     expect.text(", ".join(files),
                 "build_refusal, differential, join_runtime_filter, "
                 "native_index_projection, native_reclaim, native_repack, "
-                "native_vacuum_race, native_vecskip, sorted_pathkeys",
+                "native_saop_pushdown, native_vacuum_race, native_vecskip, sorted_pathkeys",
                 "and it is these files, so another appearing is a diff a reviewer "
                 "sees rather than a number that moved")