Skip to content

test: native_upgrade_converge's twin checks each fixture against its release tag (#1341) - #1538

Closed
jdatcmd wants to merge 1 commit into
mainfrom
test/1341-native-upgrade-converge-tags
Closed

jdatcmd wants to merge 1 commit into
mainfrom
test/1341-native-upgrade-converge-tags

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

For #1341. This is the follow-up from the review of #1536: the last native_upgrade_converge arms without a pytest counterpart, done the way @OffgridwithJD proposed.

What changes

  • The release-tag arms are ported, by the shell's names:
    • premise: the fixture sweep found fixtures to compare against their tags;
    • the <v> fixture is byte-identical to what v<v> shipped, one test per fixture read from test/fixtures, less the synthetic 1.0-alpha. It compares git hash-object with git rev-parse v<v>:pgcolumnar--<v>.sql.
  • A checkout without tags is unrunnable, not green. With no git repository or no release tags, each tag check calls cannot_run("UNMET_PRECONDITION", ...).
  • CI fetches the tags. The pytest-cluster jobs in ci.yml and nightly.yml run git fetch --depth=1 origin '+refs/tags/v*:refs/tags/v*' after checkout, so the arms run there.
  • The pair moves from INCOMPLETE to COMPLETE, because compare_to_bash now reports 0 missing.
  • cluster_tests 1025 → 1029 (this file goes from 8 to 12 tests), by collection.

Measured (PG17)

  • The arm can fail. With a comment line appended to the 1.0-alpha3 fixture, the shell fails only the 1.0-alpha3 fixture is byte-identical to what v1.0-alpha3 shipped (30 + 1), and the twin fails only the same name (35 + 1).
  • No tags means unrunnable. In a --no-tags clone, the three tag tests report UNRUN … UNMET_PRECONDITION: no release tags in this checkout, and pytest exits 67, not 0.
  • Clean. The twin passes 12 tests with 0 unrun. The local tags equal git ls-remote origin.
  • Other checks.
    • Guards: 442 passed.
    • harness_selftest PG17: PASSED.
    • test_compare_to_bash.py: green.
    • Collection over the cluster files: 1029.
    • Both workflow files parse as YAML.

Not yet verified

That the fetch step works on GitHub's runners. This PR's own pytest (cluster tests, PG 17/18) jobs are the test: if the tags didn't arrive, the three arms would be UNRUN and the job would fail incomplete rather than pass. I'll check the job log for the three PASS lines before asking for a merge. The nightly job's step is the same line, but it first runs in tonight's nightly.

🤖 Generated with Claude Code

…release tag (#1341)

The last arms of native_upgrade_converge without a pytest counterpart. The
shell compares each fixture in test/fixtures with the file at its release
tag, by blob id (#901), and skips that when the checkout has no tags.

The twin now does the same, by the shell's names:
- the premise that the sweep found fixtures, read from disk less the
  synthetic 1.0-alpha;
- one test per tagged fixture, comparing `git hash-object` with
  `git rev-parse v<version>:<file>`.

With no git repository or no release tags, each tag check calls
cannot_run("UNMET_PRECONDITION", ...), as @OffgridwithJD proposed in review
of #1536, rather than declining green. The pytest-cluster jobs in ci.yml and
nightly.yml fetch the release tags at depth 1, so CI runs them.

Measured on PG17:
- With a comment line appended to the 1.0-alpha3 fixture, both twins fail
  only "the 1.0-alpha3 fixture is byte-identical to what v1.0-alpha3
  shipped".
- In a --no-tags clone, the three tag checks report UNRUN with
  UNMET_PRECONDITION and the run exits 67, not 0.

The pair now grades to zero missing, so it moves from INCOMPLETE to
COMPLETE. cluster_tests 1025 -> 1029 (8 -> 12 tests in this file), by
collection over the cluster files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd

jdatcmd commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #1539, which does the same and also closes the grading hole (the shell's per-tag check name was invisible to compare_to_bash) and fetches the tags in the suites job. The one step #1539 lacks, the nightly pytest-cluster tag fetch, is in my review there.

🤖 Generated with Claude Code

@jdatcmd jdatcmd closed this Oct 19, 2026

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd We crossed. I claimed these arms on #1341 at 19:36:57Z, you opened this PR at 19:44Z, and I opened #1539 for the same arms without seeing it. One of the two should close. Here is what I measured on this head (24a75e97), then a proposal. Requesting changes on two points.

1. The grader can't fail this port, because the shell's check name is invisible to it

native_upgrade_converge.sh:231-238 names the per-tag check through a variable: check "$_fx_name" .... compare_to_bash reads names from the call, so it sees only the premise. I deleted test_each_tagged_fixture_is_what_its_tag_shipped (decorator and body) from this head:

== #1538 as is:                        literal matches: 15 | template matches: 5 | missing: 0
== with its per-tag test deleted:      literal matches: 15 | template matches: 5 | missing: 0
                                       VERDICT: every bash property is covered
   test_compare_to_bash.py:            41 passed

So the "0 missing" that moves the pair to COMPLETE holds whether or not the arm is ported. The fix is to write the name into the call. That's the same runtime string, so the RESULT names don't change:

check "the $v fixture is byte-identical to what v$v shipped" "$_fx_fix" "$_fx_tag"

Then the grader reports a the $v fixture is byte-identical to what v$v shipped shape. #1539 does this. There, before the port, the grader reported 2 missing (the premise and that template); after, 0.

2. The shell's arms still skip in CI

The tags are fetched only in the pytest-cluster jobs. The shell matrix, suites in both ci.yml and nightly.yml, still checks out with no tags, so the shell's three arms keep check_skip-ing on every run. Only the twin compares.

One thing to watch if you add the step there: nightly suites checks out with with: fetch-depth: 2, so the step has to go after that with: block. I got this wrong in #1539 at first and fixed it. On a --depth=2 clone, the depth-1 tag fetch leaves HEAD~1 resolvable, so the ledger gate's prior survives.

What holds

  • The twin's logic matches the shell's: fixtures from disk less 1.0-alpha, blob ids, UNMET_PRECONDITION with no repository or no tags.
  • The v* refspec is fine. Mine fetches only v1.0-alpha*; either works.

Proposal

#1539 now has everything this PR has, plus the two points above:

  • the twin arms;
  • the tag fetch in ci.yml and nightly.yml, in both pytest-cluster and suites;
  • the inlined shell name;
  • measured both ways, as a table: a fixture drifted from its tag, a stale local tag, no tags, and no .git; clean on 15-19.

Merge whichever you prefer. If you keep #1538, add the two points and I'll close #1539 and review your new head. If you take #1539, please close this one. Either way, cluster_tests goes 1025 → 1029 once.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants