Skip to content

test: analyze() refuses a named column the table does not have, before writing any statistics; analyze_function seeds the ledger (#1341) - #1373

Merged
jdatcmd merged 30 commits into
commandprompt:mainfrom
OffgridwithJD:test/1341-analyze-unknown-column
Oct 15, 2026
Merged

jdatcmd merged 30 commits into
commandprompt:mainfrom
OffgridwithJD:test/1341-analyze-unknown-column

Conversation

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Part of #1341: the medium sql-objects item "pgcolumnar.analyze(rel, columns) refuses a named column that does not exist". The function's own comment says silently collecting nothing would be "the failure mode that looks exactly like success". No arm reached the check.

Arms (both twins; a new pytest function, test_a_named_column_that_does_not_exist_is_refused)

On a fresh table:

  • The refusal: analyze('af_unk', ARRAY['a','nosuch']) raises pgcolumnar.analyze(): column "nosuch" does not exist in af_unk, checked by SQLSTATE and message.
  • Before any work: pg_stats holds no row for the table, not even for a, so the refusal came first.
  • Control: the same call naming only a writes its statistics, so "none" isn't a table that never gets any.

pgcolumnar.analyze() needs PostgreSQL 18. On 15–17 the shell suite's single named version SKIP covers these too. The pytest function declines unnamed, exactly as its three siblings do, which my first push didn't: it failed on 15–17 before I gated it.

Mutation

The unknown-column check removed. PG18, main ff09c2c against this branch:

main this branch
analyze_function.sh / pytest 68/68, 5 passed the call succeeded ("got [ok]"), and wrote statistics for a ("got [1] want [0]")

Evidence

  • Clean: analyze_function 71/71 on 18–19, the named SKIP on 15–17. The pytest file passed on all five.
  • Ledger: analyze_function had no rows, so it is seeded with all 72 checks. 2353 → 2425 rows, the census 1798 → 1868, and suites_not_covered 246 → 245. The two arms are dated.
  • Gate: rc=0.
  • Counts: TESTS.md row, and cluster_tests 533 → 534. Other PRs of mine take 534 too, so whichever lands later restates.
  • Database-free set: 429 passed.

🤖 Generated with Claude Code

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving c61baf4 on content. Reproduced on PG18.

  • Green: shell 71/71; pytest 6 passed.
  • Red: I replaced the IF unknown IS NOT NULL THEN RAISE … block (pgcolumnar--1.0-alpha5.sql ~1560) with NULL;. The shell twin fails both arms:
    FAIL analyze() refuses a named column the table does not have, and names it: … got [] want [P0001 pgcolumnar.analyze(): column "nosuch" does not exist in af_unk]
    FAIL and writes no statistics, not even for the real column named beside it: got [1] want [0]
    pytest stops at the first: got 'ok' want 'P0001 …'. The control stays green.
  • Ledger: 72 rows for 71 checks plus the 15–17 SKIP name. No duplicate keys.

Notes, not blocking:

  • The shell arm reports got [] on success, where pytest says 'ok'. Echoing ok would match it.
  • A fresh 5000-row table is an autoanalyze candidate. An autoanalyze between the INSERT and the pg_stats count would redden "writes no statistics". The window is small; setting autovacuum_enabled = off on af_unk would close it. Unverified.

Before merge: a rebase. Restate the ceiling to 244 and the census on main's 1821, and settle the cluster_tests collision with #1372.

@OffgridwithJD

OffgridwithJD commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator Author

Rebased onto 57f6e0e (#1401), on top of #1403. The new head is 454c14a. Only the count files changed, so this needs your re-approval.

  • Same patch: outside the count files, the changed lines match the head you approved, except for TESTS.md section renumbers where main's new sections took the old numbers.
  • The ledger is restated on main's census and agrees with its tsv: 2858 rows, 2165 never, 693 dated, read back from the file. Each changed paragraph's arithmetic starts from main's last numbers.
  • cluster_tests 567, by collection over the cluster files on this tree. guard_tests 433 is unchanged.
  • Runs on the rebased tree at 217f23b, PG17 and PG18: PG17 analyze_function: 0 passed + 0 failed + 0 unrunnable + 1 skipped (analyze() needs PostgreSQL 18, so PG17 declines it by name); PG17 harness_selftest: 1245 passed + 0 failed + 0 unrunnable + 0 skipped; PG17 pytest: 6 passed in 1.83s; PG18 analyze_function: 71 passed + 0 failed + 0 unrunnable + 0 skipped; PG18 harness_selftest: 1245 passed + 0 failed + 0 unrunnable + 0 skipped; PG18 pytest: 6 passed in 9.55s. harness_selftest 1245/1245 and docs_style passed on both. fix: a read inside an aborted inner block no longer loses the enclosing statement's rows (#1360) #1403 has since changed the write path's subtransaction flush, which every suite here exercises, so I'm re-running them all on 758039f and will post the result. CI runs PG17 and PG18 on this head.

🤖 Generated with Claude Code

@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Re-run finished on the tree after #1403 (758039f or later). Everything is green, so the head above stands:

  • PG17 analyze_function: 0 passed + 0 failed + 0 unrunnable + 1 skipped (PG18-only analyze() arms, declined by name)
  • PG17 docs_style: passed
  • PG17 harness_selftest: 1245 passed + 0 failed + 0 unrunnable + 0 skipped
  • PG17 pytest: 6 passed in 2.33s (PG18-only analyze() arms, declined by name)
  • PG18 analyze_function: 71 passed + 0 failed + 0 unrunnable + 0 skipped
  • PG18 docs_style: passed
  • PG18 harness_selftest: 1245 passed + 0 failed + 0 unrunnable + 0 skipped
  • PG18 pytest: 6 passed in 8.21s

🤖 Generated with Claude Code

@OffgridwithJD
OffgridwithJD force-pushed the test/1341-analyze-unknown-column branch from 454c14a to 2db496a Compare October 15, 2026 17:06
@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Rebased onto 7da848d (#1432, #1409). The new head is 2db496a; only the count files changed, so please re-approve this head.

🤖 Generated with Claude Code

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at 2db496a. Compared with c61baf4, the head I approved, file by file: the changed lines are the same, apart from the count files (check_ledger_budget.txt, expected_tests.txt) and, in #1378/#1379/#1380/#1383, the TESTS.md section number, now 115.

🤖 Generated with Claude Code

@OffgridwithJD
OffgridwithJD force-pushed the test/1341-analyze-unknown-column branch from 2db496a to 9d8861f Compare October 15, 2026 17:53
@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Rebased onto 1f7b5da (#1411, #1412, #1417, #1418). The new head is 9d8861f. Only the count files changed, plus TESTS.md section numbers where main took 115, so please re-approve this head.

🤖 Generated with Claude Code

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at 9d8861f, the rebase onto 1f7b5da. File by file, the changed lines are the same as the head I approved before, apart from the count files and, in #1378/#1379/#1380/#1383, the TESTS.md section number.

🤖 Generated with Claude Code

@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

The PG18 re-run on this head (9d8861f, on 1f7b5da with #1412's index-build change) is all green:

  • analyze_function: 71 passed + 0 failed + 0 unrunnable + 0 skipped
  • docs_style: passed
  • harness_selftest: 1245 passed + 0 failed + 0 unrunnable + 0 skipped
  • pytest: 6 passed in 5.04s

🤖 Generated with Claude Code

@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Rebased onto 37ea557 (#1414, #1425). The new head is 4a2e517. Only the count files changed, so please re-approve this head.

🤖 Generated with Claude Code

@OffgridwithJD
OffgridwithJD force-pushed the test/1341-analyze-unknown-column branch from 9d8861f to 4a2e517 Compare October 15, 2026 18:28
@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Rebased onto f2bd5f1 (#1427). The new head is 8e59b7a. Only the count files changed, so please re-approve this head.

🤖 Generated with Claude Code

@OffgridwithJD
OffgridwithJD force-pushed the test/1341-analyze-unknown-column branch from 4a2e517 to 8e59b7a Compare October 15, 2026 18:38

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at 8e59b7a, the rebase onto f2bd5f1. File by file, the changed lines are the same as the head I approved before, apart from the count files and, in #1378/#1379/#1380/#1383, the TESTS.md section number.

🤖 Generated with Claude Code

@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Rebased onto d5c0a5e (#1391). The new head is d93ea54, so please re-approve this head.

🤖 Generated with Claude Code

@OffgridwithJD
OffgridwithJD force-pushed the test/1341-analyze-unknown-column branch from 8e59b7a to d93ea54 Compare October 15, 2026 19:26
@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

@jdatcmd CI's pytest (cluster tests, PG 17) was red on d93ea54 for a defect of mine: the new twin test, test_a_named_column_that_does_not_exist_is_refused, declines on PG15-17 as the other analyze() twins do, but I hadn't listed it in expected_unrunnable.txt. The job passed all 578 tests and exited 67 on the unexpected decline. My local PG17 runs showed the same 67 earlier, and I misread it as the expected decline.

Fixed in 59b4950: the registry gets the test for 15, 16 and 17. On PG17, test_analyze_function.py exits 67 without those rows and 0 with them, and the guard set passes 435. Please re-approve this head.

🤖 Generated with Claude Code

@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Rebased onto 2d1a77f (#1413–#1424). The new head is 257784b, so please re-approve this head.

🤖 Generated with Claude Code

@OffgridwithJD
OffgridwithJD force-pushed the test/1341-analyze-unknown-column branch from 59b4950 to 257784b Compare October 15, 2026 20:58

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at 257784b, the rebase onto 2d1a77f. The one addition beyond bookkeeping is three expected_unrunnable.txt lines declaring that test_a_named_column_that_does_not_exist_is_refused declines on PG15, 16 and 17. That's right: pgcolumnar.analyze() needs PostgreSQL 18.

🤖 Generated with Claude Code

jdatcmd and others added 5 commits October 15, 2026 21:15
…ws are reworded

The mutation column is a ';'-separated set, written back sorted. A --mutation
whose text held "; " was stored as two mutations, one a fragment starting with
a space, and every later merge re-sorted it. commandprompt#1427's recovery_minimal_wal text
did this to 8 rows, and commandprompt#1423, commandprompt#1424 and commandprompt#1426 each carried them as a change of
their own. Four older harness_selftest rows were split the same way.

- merge refuses a --mutation holding ';' (rc 2, ledger untouched).
- the reader refuses a ';'-piece with whitespace at an end, by line: the writer
  joins with a bare ';', so such a piece is a fragment, never a mutation.
- the 12 rows are reworded without ';', from their original text in history.
  Only field 6 changes; the census stays 3007 / 2282 / 725.

Red first: the two refusal arms got 0 want 2 on the unchanged tool, and the
committed-ledger arm was red at line 1783 with the refusal in and the rows not
yet reworded. guard_tests 435 -> 438 by collection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ment-mark mutation (commandprompt#1341 item 75)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
…wn privileges, in both twins (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
…d the refused tier writes no file, in both twins (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
jdatcmd and others added 16 commits October 15, 2026 21:16
…e mutations of the RLS guard (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
…x, and the twin fails it by name rather than raising (commandprompt#1383 review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
…name (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… arm runs with the configured list empty, in both twins (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n comparing two empty strings (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d the shell names 'empty' itself (commandprompt#1379 review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
…l's name (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…0) and leaves nothing declared, in both twins (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts connections (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Restated for the chain you asked for. The new head is 5879648, so please re-approve this head.

🤖 Generated with Claude Code

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at 5879648, its place in the chain on 17420be. This PR's own commits, measured from the link below it, change the same lines as the head I approved before, apart from the count files and the TESTS.md section number.

🤖 Generated with Claude Code

jdatcmd and others added 5 commits October 15, 2026 22:00
…emise: prefix (commandprompt#1341)

Review of commandprompt#1375 by jdatcmd: CI's suites (PG 17) ledger gate found the row seeded as
'premise: the drop really removed the projection' while projections.sh:410 and
test_projections.py:795 name the check without the prefix, so the gate saw one
check not in the ledger and one orphan row. The row is renamed and keeps its
other fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e writing any statistics, in both twins (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…blings do (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… dated (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ys so (commandprompt#1341)

test_a_named_column_that_does_not_exist_is_refused calls pgcolumnar.analyze(),
which needs PostgreSQL 18, so it declines on 15, 16 and 17 like the other
analyze_function twins. It was missing from expected_unrunnable.txt, so CI's
PG17 cluster job collected and passed 578 and still exited 67 on the
unexpected decline. Measured on PG17: test_analyze_function.py exits 67
without these rows and 0 with them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@OffgridwithJD
OffgridwithJD force-pushed the test/1341-analyze-unknown-column branch from 5879648 to e56de60 Compare October 15, 2026 22:02
@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Restacked onto #1375's fix (875429d, its projections row renamed). The new head is e56de60, so please re-approve this head. This PR's own commits are unchanged, and nothing in its diff moved.

🤖 Generated with Claude Code

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at e56de60. This PR's own commits are the same as the head I approved, rebased onto the fixed #1375.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd merged commit c5536f1 into commandprompt:main Oct 15, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants