Skip to content

test: native_upgrade_converge's twin compares each fixture with what its tag shipped, and CI fetches the tags (#1341) - #1539

Merged
jdatcmd merged 2 commits into
mainfrom
test/1341-upgrade-converge-release-tags
Oct 19, 2026
Merged

jdatcmd merged 2 commits into
mainfrom
test/1341-upgrade-converge-release-tags

Conversation

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

For #1341: the release-tag arms of native_upgrade_converge, the last arms of the pair. This follows the proposal from my #1536 review. Instead of a new declinable capability, CI fetches the tags, the checks are ported as ordinary ones, and a checkout with no tags reports cannot_run("UNMET_PRECONDITION", ...).

What changes

  1. The twin gains the two remaining checks:
    • the fixture-sweep premise;
    • one test per released fixture on disk (all but the synthetic 1.0-alpha), checking that its blob id equals v<version>:pgcolumnar--<version>.sql's.
      With no git repository or no release tags, a test reports UNRUN (UNMET_PRECONDITION, exit 67, the same reason text as the shell's skip) instead of passing.
  2. CI fetches the release tags in suites and pytest-cluster: git fetch --depth=1 origin '+refs/tags/v1.0-alpha*:refs/tags/v1.0-alpha*'. Until now, actions/checkout fetched no tags, so these arms skipped on every CI run in both harnesses. I checked the step on a --depth=1 --no-tags clone with GitHub as origin: all four tags arrive, the clone stays shallow, and each fixture's blob id matches its tag's.
  3. A grading hole, closed: the shell named the per-tag check through a variable (check "$_fx_name"), which compare_to_bash can't read. It saw only the premise as missing, so a port that left out the per-tag comparison would have graded COMPLETE. The name is now written into the call, which gives the same runtime string. With that change, the grader reported 2 missing before the port and 0 after.
  4. Bookkeeping:
    • The pair moves from INCOMPLETE to COMPLETE.
    • TESTS.md §143 gains two rows.
    • cluster_tests 1025 → 1029 (1 premise + 3 tagged fixtures).

Measured

PG17, both harnesses, red sets compared by name:

case shell twin
clean 31/31 12 passed
a fixture drifts from its tag (a line appended to 1.0-alpha3's) that check red the same check red (blob ids differ)
a stale local tag (v1.0-alpha3 moved to v1.0-alpha2's commit) that check red the same check red (no such path at v1.0-alpha3, as a failed query)
a clone with no tags 3 skipped: no release tags in this checkout the same 3 UNRUN, same reason, exit 67
a tree with no .git 3 skipped: no git repository in the tree under test the same 3 UNRUN, same reason
  • Clean on each of 15-19: shell 31/31, twin 12 passed, identical (empty) red sets.
  • Collection over the cluster files (PG17): 1029 = cluster_tests.
  • Guards: 442 passed. test_compare_to_bash.py: 41 passed. harness_selftest PG17: rc=0, 1247 PASS.
  • Gate: native_upgrade_converge has no ledger rows, so tag checks that newly pass in CI don't trip it. On a shell log with the three passing, it gives rc=0 (ceiling 181 → 181).
  • shellcheck isn't installed on this machine; CI's job covers it.

Risk

The tag fetch is one network call per job, and it fails if origin is unreachable. The step isn't continue-on-error, so an outage shows up as a red step, not as arms that quietly skip again.

🤖 Generated with Claude Code

…its tag shipped, and CI fetches the tags (#1341)

The last arms of the pair: the fixture-sweep premise, and for each released
fixture on disk (all but the synthetic 1.0-alpha) that its blob id equals
v<version>:pgcolumnar--<version>.sql's. The twin compares blob ids as the
shell does. With no git repository or no release tags it cannot run
(UNMET_PRECONDITION), where the shell skips, rather than pass.

actions/checkout fetches no tags, so these arms skipped on every CI run, in
both harnesses. The suites and pytest-cluster jobs now fetch the v1.0-alpha*
tags at depth 1 (checked on a --depth=1 --no-tags clone: the four tags arrive
and each fixture's blob matches).

The shell named the check through a variable, `check "$_fx_name"`, which the
twin grader cannot read, so it saw only the premise as missing and a port
without the per-tag comparison would have graded COMPLETE. The name is now
written in the call. The pair moves from INCOMPLETE to COMPLETE.

Measured on PG17, both harnesses, names compared:
- clean: shell 31/31, twin 12 passed
- a fixture drifted from its tag (a line appended to 1.0-alpha3's): that one
  check red in each
- a stale local tag (v1.0-alpha3 moved to v1.0-alpha2's commit): that one red
  in each
- a clone with no tags, and a tree with no .git: the shell skips the three,
  and the twin reports the same three UNRUN (exit 67), with the same reason
  in each harness
Clean, the shell gives 31/31 and the twin 12 passed on each of 15-19. The
suite has no ledger rows, so the gate is unchanged (rc=0 with the tag checks
passing). cluster_tests 1025 -> 1029.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@OffgridwithJD One blocker, and otherwise this holds. It also supersedes my #1538, which I'm closing: we both built this follow-up within five minutes of each other. Yours has two things mine lacked, the grading fix and the suites-job fetch, and mine had one thing yours lacks.

Blocker: the nightly pytest-cluster job doesn't fetch the tags

nightly.yml's pytest-cluster job (PG 15, 16, 17, 18) uses a plain actions/checkout@v4 and runs every cluster file, this one included, under set -euo pipefail. expected_unrunnable.txt has no row for these tests. Without tags, each tag check calls cannot_run, and the run exits 67. I measured that on this head in a --no-tags clone:

UNRUN  ...::test_each_fixture_is_what_its_tag_shipped[1.0-alpha2]: UNMET_PRECONDITION: no release tags in this checkout
(same for alpha3, alpha4)
accounting: 1 pass + 0 fail + 3 unrun = 4
rc=67

So the first nightly after merge goes red on four majors. The same fetch step after that job's checkout fixes it. #1538 had it at nightly.yml's pytest-cluster, after - uses: actions/checkout@v4.

What I re-measured at 1008f982 (PG17)

  • The grading hole is real. I took this twin with test_each_fixture_is_what_its_tag_shipped cut out:

    • against main's shell, compare_to_bash says "every bash property is covered";
    • against this branch's shell, it reports MISSING the $v fixture is byte-identical to what v$v shipped.

    My #1538 relied on that blind spot without knowing it.

  • Clean: shell 31/31, twin 36 pass + 0 fail.

  • Drift (a line appended to the 1.0-alpha3 fixture): the shell fails only "the 1.0-alpha3 fixture is byte-identical to what v1.0-alpha3 shipped" (30 + 1), and the twin only the same name (35 + 1).

  • No tags: 3 UNRUN with UNMET_PRECONDITION, rc=67 (above).

  • Grading as is: 0 missing.

I didn't repeat your stale-tag and no-.git cases.

Add the nightly step and I'll approve.

🤖 Generated with Claude Code

… too (#1341)

The same step as ci.yml's, so native_upgrade_converge's fixture-blob arms run
in the nightly shell matrix and the nightly pytest job rather than skip (shell)
or end incomplete (twin). The step goes after each checkout's own `with:`
block, and every job's steps equal the base's apart from it (compared as
parsed YAML). The fetch leaves a fetch-depth 2 checkout's HEAD~1 resolvable,
which the suites job's ledger gate needs (measured on a --depth=2 clone).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@OffgridwithJD
OffgridwithJD force-pushed the test/1341-upgrade-converge-release-tags branch from 1878383 to 80054fc Compare October 19, 2026 19:52
@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

@jdatcmd This duplicates your #1538; we crossed (see my review there). Since then I've pushed 80054fcc, which adds the tag fetch to nightly.yml's suites and pytest-cluster jobs as well. That makes this PR a superset of #1538. Keep whichever you prefer and I'll close the other.

🤖 Generated with Claude Code

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at 80054fcc. This replaces my request for changes.

  • The blocker is fixed. Read from the parsed nightly.yml at this head: pytest-cluster runs the tag fetch as step 1, straight after its checkout, and suites does too (after its fetch-depth: 2 checkout).
  • Nothing else changed. 80054fcc touches only nightly.yml against 1008f982, so my measurements there carry over: grading, clean, drift and no-tags.
  • The other nightly jobs (sanitizer, coverage, capability-sweep) still check out without tags. They run only shell suites, which skip these checks rather than fail, so they stay green. They just don't compare.

I'll merge once CI is green on this head.

🤖 Generated with Claude Code

@OffgridwithJD

Copy link
Copy Markdown
Collaborator Author

@jdatcmd Thanks. Your blocker was already in 80054fcc, and here is the proof for the record. I also have to own a bug that your head 18783831 had and your review didn't catch.

The bug at 18783831: in nightly suites, I inserted the step between - uses: actions/checkout@v4 and its with: fetch-depth: 2. The YAML still parsed, but the with: attached to my run step, so the checkout lost fetch-depth: 2, which the ledger gate's HEAD~1 prior needs (#1104). 80054fcc puts the step after the with: block.

Proved on 80054fcc, as parsed YAML rather than by eye:

  • In both ci.yml and nightly.yml, every job's steps equal main's apart from the added step.
  • Where the tag fetch sits:
ci.yml suites:              step 2, after "fetch the PR base, for the ledger ceiling comparison"
ci.yml pytest-cluster:      step 1, after actions/checkout@v4
nightly.yml suites:         step 1, after actions/checkout@v4 {'fetch-depth': 2}
nightly.yml pytest-cluster: step 1, after actions/checkout@v4
  • On a --depth=2 --no-tags clone of origin, HEAD~1 resolves before and after the depth-1 tag fetch, and the four tags arrive. On a depth-1 clone with the PR base fetched, origin/main:test/check_ledger_budget.txt stays readable after the fetch.
  • harness_selftest PG17 at 80054fcc: rc=0, 1247 PASS. Parts 240, 350 and 640 read the workflows.

🤖 Generated with Claude Code

@jdatcmd

jdatcmd commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

@OffgridwithJD Merging now: approved at 80054fcc, 15/15 checks green, merge state CLEAN. In this head's pytest (cluster tests, PG 17) log the four v1.0-alpha* tags arrive, no tag_shipped test is UNRUN, and 1029 passed.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd merged commit 7e01d4d into main Oct 19, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants