Skip to content

test: import_arrow's dictionary refusal by SQLSTATE and message, and a compressed-body arm (#1341) - #1365

Merged
jdatcmd merged 2 commits into
commandprompt:mainfrom
OffgridwithJD:test/1341-arrow-unsupported
Oct 14, 2026
Merged

jdatcmd merged 2 commits into
commandprompt:mainfrom
OffgridwithJD:test/1341-arrow-unsupported

Conversation

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Part of #1341: the medium error-path item "import_arrow refuses dictionary-encoded files (0A000) and compressed record-batch bodies (0A000)".

  • Dictionary: the existing arm used expect_error, which any error satisfies.
  • Compressed bodies: no arm existed.

Arms

  • reject dictionary-encoded file keeps its name. It now requires 0A000 columnar.import_arrow does not support dictionary-encoded Arrow files, the refusal's SQLSTATE and message.
  • reject a compressed record-batch body is new. pyarrow writes a zstd or lz4 IPC stream, and the arm requires the 0A000 refusal and that no row landed. A premise checks that the codec was available, and both names join the suite's named-skip list for hosts without pyarrow.

PG17, main 1933577 against this branch:

mutation main this branch
the DictionaryBatch refusal removed 72/72: the import then fails later with "XX001 … too few buffers for the schema", and expect_error takes that "got [XX001 … too few buffers] want [0A000 … dictionary-encoded …]"
the compressed-body refusal removed 72/72: no arm "got [XX001 … value buffer too small …|0] want [0A000 … compressed Arrow bodies|0]"

The audit's reading holds: with its own check gone, the dictionary arm still passed on main, because a different error answered it.

Shell only: arrow_import has no pytest twin; that's its own #1341 item.

Evidence and ledger

🤖 Generated with Claude Code

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the content at a9531ed. On PG17 I reproduced the dictionary claim with the DictionaryBatch refusal disabled:

  • main: arrow_import passes 72/72. The old expect_error arm was satisfied by a different error, XX001 … too few buffers for the schema, exactly as the audit said.
  • this head: "reject dictionary-encoded file: got [XX001 …] want [0A000 …]".

Agreed on sequencing: land it after #1363, so arrow_import is seeded once, then date these two arms.

@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Rebased onto f601281: now 792d17a. Since you approved, #1363 seeded arrow_import, so this PR's checks need rows. One commit added for that; arrow_import.sh itself rebased cleanly and is unchanged.

  • The merged suite was rerun: 77/77 on each of 15–19.
  • Mutants (PG17): without the dictionary-batch refusal, the branch reddens reject dictionary-encoded file (got [XX001 ... too few buffers]). Without the compressed-body refusal, it reddens reject a compressed record-batch body (got [XX001 ... value buffer too small]). Main's suite survives both (75/75).
  • Ledger: 2 new rows (the compressed arm dated, its pyarrow premise never), and the dictionary arm is dated for the first time. 2460 + 2 = 2462; never 1893 + 1 − 1 = 1893; dated 569. The ceiling stays at 244 because the suite is already covered. The gate is clean against origin/main.

Next in queue after this is #1369, already restated at 527bb71.

🤖 Generated with Claude Code

jdatcmd and others added 2 commits October 14, 2026 17:27
…a compressed-body arm (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the ledger, both dated (commandprompt#1341)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
@OffgridwithJD
OffgridwithJD force-pushed the test/1341-arrow-unsupported branch from 792d17a to f614df9 Compare October 14, 2026 17:27
@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

Restated on 1269f79 (#1388 merged): now f614df9. Only the budget conflicted. Rows 2464 + 2 = 2466; never 1895; dated 571; ceiling 244. The gate is clean against origin/main, and the test files are unchanged from the 77/77 five-major runs. Its CI will hit main's advisory_lock_class race until #1390 lands, so #1390 should merge first.

🤖 Generated with Claude Code

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked f614df9 against my approval of a9531ed:

  • The test commit is unchanged: git range-diff shows d22a475 identical to a9531ed (=).
  • f614df9 touches only the ledger and budget: two new rows (the compressed-body arm dated, and the pyarrow premise never), and main's reject dictionary-encoded file moves from never to dated by the dictionary-batch mutation. No other main row changes. The dates rest on your mutation runs, which I didn't repeat.
  • Counts: 2466 rows / 1895 never; ceiling 244.
  • Base: 1269f79. Main has since moved only by #1390, which touches advisory_lock_class.sh alone, so the gate inputs are unchanged.

CI is 15/15 green. Approving and merging.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants