Skip to content

test: the docs tests read what a reader sees, and each arm can fail (#1339) - #1658

Merged
jdatcmd merged 1 commit into
mainfrom
test/1339-docs-tests
Oct 22, 2026
Merged

jdatcmd merged 1 commit into
mainfrom
test/1339-docs-tests

Conversation

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

@jdatcmd This fixes my eleven open #1339 entries in the four docs tests (batch B4 of my a–m triage). Each was shown green under its defect, then fixed and shown red by name. No cluster is needed; all four files are in NO_CLUSTER.

entry defect, planted on main main this branch
test_docs_join_clustering.py:38 [mislabelled] the star-schema heading renamed premise: the star-schema join heading is present red: a docs change reddens a premise (#1236) the file and heading checks lose premise:; the same plant reds the star-schema join heading is present
test_docs_join_clustering.py:41 [weak] "cluster" in one sentence, "join key" in another; or the phrase only in an HTML comment 2 passed, both plants and it names the join key as the clustering column: got 0, wanted at least 1, both plants
test_docs_join_clustering.py:52 [weak] best-practices clusters "on the timestamp", "join key" still mentioned elsewhere passed best-practices names clustering on the join key: got 0
test_docs_stripe_floor.py:90 [vacuous] FSST removed from the advice section only passed (the compression section names FSST) and names what lowering past it costs: 'fsst' is absent
test_docs_cross_database_query.py:19 [weak] the forbidden query in a ```sh psql -c block; under ```postgresql; spelled PG_DATABASE/PG_EXTENSION passed, all three got 1 want 0, all three
test_docs_cover_the_corpus.py:55 [weak] test_whole_row's own TESTS.md entry removed (its name is a prefix of test_whole_row_var_on_off's) passed got 'test_whole_row' want 'none'
:208 [weak] the real arm's duplicate rule mutated to count(n) > 2 the fixture stayed green (it had its own copy) one helper, _duplicated_names; the fixture reds
:278 [vacuous] the sweep's glob narrowed to test_d*.py each file counted once passed (dict keys against themselves) compared with an independent directory listing; names every missed file
:320 [vacuous] the no-totals arm stops reading the document the fixture stayed green (it compared two pinned tuples) it drives the arm's predicate, totals_line_refused: got 'none' want 'refused: 9 tests in 4 files'
:407 [weak] README's "28 refused" made "128 refused"; a second, contradicting "30 refused" added passed, both got '128' want '28'; got '28, 30' want '28'
:1172 [weak] the real empty-section reader counts blank lines as a body the control stayed green (it had its own copy) one reader, _empty_sections; control … got '' want '999. planted.py'
  • Clean: 46/46 across the four files, and the guard set passes.
  • Unchanged: no test function is renamed, so TESTS.md and guard_tests don't move. None of the changed check names has a ledger or lossy_arms.tsv row.
  • Speed: undocumented() now tokenizes the document once (whole words, plus x.py names), so the corpus file stays around 3 s.
  • Follow-up, not in this PR: test/docs_style.sh's arm of the same name still reads only ```sql blocks, case-sensitively. test: docs_style gains its pytest twin, and a sentence no reader sees no longer passes (#1341) #1653 rewrites that file, so I'll bring the shell arm level there rather than conflict with it.

The plants and harness are in ~/pgc-tools/i1339d/ on my side.

🤖 Generated with Claude Code

…1339)

Fixes the eleven #1339 entries in the four docs tests. Each was shown green
under its defect first, then red by name after the fix.

test_docs_join_clustering.py
- :38 [mislabelled] The file and heading checks lose "premise:". Renaming the
  heading reddened "premise: the star-schema join heading is present", so a docs
  change reddens them (#1236).
- :41 [weak] "and it names the join key as the clustering column" now needs a
  sentence that says to cluster ON the join key, read with HTML comments
  removed. "Cluster" in one sentence and "join key" in another passed, and so did
  the phrase inside an HTML comment.
- :52 [weak] "best-practices names clustering on the join key": the same rule.
  With the page clustering on the timestamp, "join key" elsewhere passed.

test_docs_stripe_floor.py
- :90 [vacuous] "and names what lowering past it costs" reads FSST in the
  section that gives the advice and states the floor. Page-wide, it passed with
  FSST removed from that section, because the compression section names it
  throughout.

test_docs_cross_database_query.py
- :19 [weak] It reads every SQL fence tag and shell blocks, matching
  case-insensitively. The forbidden query passed in a ```sh psql -c block, under
  ```postgresql, and spelled PG_DATABASE/PG_EXTENSION.

test_docs_cover_the_corpus.py
- :55 [weak] A name counts only as a whole word. 22 test names are prefixes of
  another's, so with `test_whole_row`'s own entry removed, its
  `test_whole_row_var_on_off` neighbour kept the arm green.
- :208 [weak] One duplicate rule, _duplicated_names, is used by the real arm and
  by the fixture. The fixture carried a copy, and stayed green with the real
  arm's rule broken (count > 2).
- :278 [vacuous] "each file counted once" compares the sweep with an
  independent listing of the directory. It compared a dict with its own keys.
  With the glob narrowed to test_d*.py it now names every file it missed.
- :320 [vacuous] The fixture drives the no-totals arm's predicate,
  totals_line_refused, and checks that it refuses a stated total. It compared two
  tuples both already pinned, and stayed green with the real arm reading
  nothing.
- :407 [weak] Every "N refused" README.md states must be the counted number.
  "128 refused" and a second, contradicting "30 refused" both passed.
- :1172 [weak] One empty-section reader, _empty_sections, serves the real arm
  and its control. The control carried a copy, and stayed green with the real
  reader counting blank lines as a body.

No test function is renamed, so TESTS.md and guard_tests do not move. Check
names change only in the two relabelled join-clustering premises and the :320
arm, and none has a ledger or lossy_arms row. test/docs_style.sh's arm of the
same name keeps reading only ```sql blocks. #1653 rewrites that file, so its
parity is a follow-up there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at 9c90c4d. I reran your :407 plant (README's "28 refused" made "128 refused" in test/pytest/README.md): main's test_docs_cover_the_corpus.py 40 passed; this head 1 failed, "README.md quotes the number of modes actually named as refused: got '128' want '28'". Clean, the four files: 46 passed in 2.93s. CI 15/15. Agreed that the shell arm's ```sql-only reading belongs with #1653.

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging at 9c90c4d with CI green; approved above.

@jdatcmd
jdatcmd merged commit f0b4380 into main Oct 22, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants