test: count group-list rows, not planning buffers, in the #1203 arm

[jdatcmd]

Closes #1203. @OffgridwithJD, this needs your review. I will not merge it without your approval on the head commit.

What changes

The arm a query with no ORDER BY does not read the group list to decide and its control now count the rows planning reads from pgcolumnar.row_group. Before, they counted every shared buffer planning touched. Both twins change, and the check names do not.

• The instrument is pg_stat_get_xact_tuples_returned on row_group plus row_group_pkey, in the planning backend's own transaction, around a plain EXPLAIN.
• The arm becomes an equality. The control requires at least one extra row per group.
• One premise is replaced. "the planning buffer count is a measurement" becomes "planning reads the group list at all, so a zero below is not an instrument that cannot see it".

Why this instrument

• It counts what the arm's name claims. The decision reads the row-group list through row_group_pkey. A per-relation breakdown of the transaction's reads shows it plainly. For the ORDER BY plan, row_group_pkey doubles from GUC off to GUC on: 2000 to 4000 in a breakdown that adds rows returned and rows fetched, or 1000 to 2000 counting returned only, which is what the instrument counts. Everything else moves by a few pg_class/pg_attribute index lookups at most.
• No other session can move it. The _xact_ counters are this transaction's own, so other backends and autovacuum cannot add to them. A plain EXPLAIN plans without executing.
• Heap and index are summed, so the count survives a change of access path.

Evidence, PG17, 1000 groups

Reading	GUC on	GUC off
no ORDER BY	1000	1000	12 readings, identical
ORDER BY	2000	1000	12 readings, identical
no ORDER BY, the same 12 with three sessions running DDL alongside	1000	1000	identical
no ORDER BY, has_useful_pathkeys early return removed	2000	1000	the arm fails

Removal proof, both twins, with the guard removed:

shell    112 passed + 1 failed   FAIL a query with no ORDER BY does not read the group list to decide:
                                      got [no (on=2000 off=1000 over 1000 groups)] want [yes]
pytest   AssertionError: ... got 'differs by 1000 (on=2000 off=1000 over 1000 groups)' want 'the same'

Green: shell 113 passed + 0 failed; the pytest module 48 passed. compare_to_bash.py: 110 literal matches, 0 missing.

What this does not establish

The cause of the CI noise is still unknown. I tried to reproduce it with dead row_group tuples (1600), a VACUUM of the catalog between readings, and three sessions running DDL during the readings. None of these moved the old buffer count either: 120/120 and 43/22 throughout. So this PR replaces the instrument with one that cannot see the noise source, whatever it is. It does not show what that source was.

sorted_pathkeys has no ledger rows, so the ledger is untouched.

🤖 Generated with Claude Code

[OffgridwithJD]

Approving f010157d (CI 15/15 green).

Reproduced in my own worktree at this head, PG 17:

  control                    sorted_pathkeys   113 passed + 0 failed
  early return disabled      sorted_pathkeys   112 passed + 1 failed
    a query with no ORDER BY does not read the group list to decide
        got [no (on=2000 off=1000 over 1000 groups)] want [yes]
  pytest twin                test_sorted_pathkeys.py   48 passed

The mutation is if (!has_useful_pathkeys(root, rel)) changed to if (false && ...) at columnar_customscan.c:1535, and it reddens that one arm and nothing else.

Stability, since stability is the issue: three more shell runs back to back gave identical readings each time:

  no ORDER BY   on=1000 off=1000 over 1000 groups
  ORDER BY      on=2000 off=1000 (needs at least 1000 more)

So the arm and its control are separated by exactly one row per group, against a counter no other backend can move. That is a deterministic instrument where the old one was a statistical one.

Agreed on what it does not establish. The old buffer count's noise source is still unidentified, and the description says so rather than claiming a cause. I'd keep #1203's diagnosis question alive somewhere rather than let this close it entirely, since the same noise could affect another buffer-counting arm. Not blocking.

[jdatcmd]

@OffgridwithJD, rebased onto 951b3ec after #1284 landed, because both PRs added a CHANGELOG entry at the same place. The new head is f825f97. Git resolved it without a conflict marker.

The patch did not change. Per-file md5 of the added and removed lines, against each base:

file	f010157 vs c1e253f	f825f97 vs 951b3ec
CHANGELOG.md	ccc54f3fee46	ccc54f3fee46
test/sorted_pathkeys.sh	f3338600f82b	f3338600f82b
test/pytest/test_sorted_pathkeys.py	93b6582294eb	93b6582294eb

Your approval is on f010157, so it needs re-stating on f825f97 before I merge. I will merge once CI is green there.

On keeping #1203's question alive: agreed. I will open a separate issue for the unidentified buffer noise when this merges, so the question survives #1203 closing.

🤖 Generated with Claude Code

[OffgridwithJD]

Re-approving at f825f975 (CI 15/15 green). Its parent is 951b3ec, so the rebase did happen.

Recomputed independently, counting the added and removed lines per file, with both hashes:

                                     f010157 vs c1e253f     f825f97 vs 951b3ec
test/sorted_pathkeys.sh              n=70  935f5710c34c     n=70  935f5710c34c
test/pytest/test_sorted_pathkeys.py  n=132 e55b397f3351     n=132 e55b397f3351
CHANGELOG.md                         n=10  281060088b92     n=9   ccc54f3fee46

The two test files are identical across the rebase, and none of those hashes is the empty-input one. CHANGELOG.md is not byte-identical, though your table shows the same hash on both sides. The only difference is one added empty line (+ with nothing after it). In the rebase, the separator between entries moved into context next to #1284's entry, so the text of the entry is unchanged. That's harmless. It does mean your hash method drops blank lines, so it would also miss a change that adds or removes only blank lines. Worth knowing for the next one; it doesn't block this.

Code under test is unchanged, so my reproduction at f010157 carries: control 113/113, early return disabled 112 + 1 on the named arm, and three stable repeats.

[jdatcmd]

@OffgridwithJD, you were right about the blank lines, thank you. My filter ^[-+][^-+] drops a line that is only + or -, so it could not see the separator move. Fixed method: git diff --unified=0, keep every ^[-+] line except the +++/--- headers, then count and md5.

It needed one more rebase: #1289 landed and also touched CHANGELOG.md. The new head is 37e4d80, on ede9a3c. With the corrected method:

file	before (vs 951b3ec)	after (vs ede9a3c)
CHANGELOG.md	n=10 98a1b416d016	n=10 98a1b416d016
test/sorted_pathkeys.sh	n=70 935f5710c34c	n=70 935f5710c34c
test/pytest/test_sorted_pathkeys.py	n=132 e55b397f3351	n=132 e55b397f3351

The two test-file hashes match yours exactly, and none is the empty-input value (d41d8cd98f00). This time the CHANGELOG text and its blank lines are both unchanged. It needs your approval on 37e4d80; I will merge once CI is green there.

🤖 Generated with Claude Code

[OffgridwithJD]

Approving 37e4d806: CI 15/15 green, parent ede9a3c = main. With your corrected method, the per-file patch is identical to the head I reproduced: n=10/70/132, CHANGELOG 98a1b416d016, sorted_pathkeys.sh 935f5710c34c, the pytest twin e55b397f3351.

[jdatcmd]

Merging now: OffgridwithJD approved 37e4d80, all 15 checks green on that head. Opening the follow-up issue for #1203's unidentified buffer noise as promised.