Repository navigation
test: native_upgrade_converge's twin compares each fixture with what its tag shipped, and CI fetches the tags (#1341) - #1539
Conversation
…its tag shipped, and CI fetches the tags (#1341) The last arms of the pair: the fixture-sweep premise, and for each released fixture on disk (all but the synthetic 1.0-alpha) that its blob id equals v<version>:pgcolumnar--<version>.sql's. The twin compares blob ids as the shell does. With no git repository or no release tags it cannot run (UNMET_PRECONDITION), where the shell skips, rather than pass. actions/checkout fetches no tags, so these arms skipped on every CI run, in both harnesses. The suites and pytest-cluster jobs now fetch the v1.0-alpha* tags at depth 1 (checked on a --depth=1 --no-tags clone: the four tags arrive and each fixture's blob matches). The shell named the check through a variable, `check "$_fx_name"`, which the twin grader cannot read, so it saw only the premise as missing and a port without the per-tag comparison would have graded COMPLETE. The name is now written in the call. The pair moves from INCOMPLETE to COMPLETE. Measured on PG17, both harnesses, names compared: - clean: shell 31/31, twin 12 passed - a fixture drifted from its tag (a line appended to 1.0-alpha3's): that one check red in each - a stale local tag (v1.0-alpha3 moved to v1.0-alpha2's commit): that one red in each - a clone with no tags, and a tree with no .git: the shell skips the three, and the twin reports the same three UNRUN (exit 67), with the same reason in each harness Clean, the shell gives 31/31 and the twin 12 passed on each of 15-19. The suite has no ledger rows, so the gate is unchanged (rc=0 with the tag checks passing). cluster_tests 1025 -> 1029. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jdatcmd
left a comment
There was a problem hiding this comment.
@OffgridwithJD One blocker, and otherwise this holds. It also supersedes my #1538, which I'm closing: we both built this follow-up within five minutes of each other. Yours has two things mine lacked, the grading fix and the suites-job fetch, and mine had one thing yours lacks.
Blocker: the nightly pytest-cluster job doesn't fetch the tags
nightly.yml's pytest-cluster job (PG 15, 16, 17, 18) uses a plain actions/checkout@v4 and runs every cluster file, this one included, under set -euo pipefail. expected_unrunnable.txt has no row for these tests. Without tags, each tag check calls cannot_run, and the run exits 67. I measured that on this head in a --no-tags clone:
UNRUN ...::test_each_fixture_is_what_its_tag_shipped[1.0-alpha2]: UNMET_PRECONDITION: no release tags in this checkout
(same for alpha3, alpha4)
accounting: 1 pass + 0 fail + 3 unrun = 4
rc=67
So the first nightly after merge goes red on four majors. The same fetch step after that job's checkout fixes it. #1538 had it at nightly.yml's pytest-cluster, after - uses: actions/checkout@v4.
What I re-measured at 1008f982 (PG17)
-
The grading hole is real. I took this twin with
test_each_fixture_is_what_its_tag_shippedcut out:- against main's shell,
compare_to_bashsays "every bash property is covered"; - against this branch's shell, it reports
MISSING the $v fixture is byte-identical to what v$v shipped.
My #1538 relied on that blind spot without knowing it.
- against main's shell,
-
Clean: shell 31/31, twin 36 pass + 0 fail.
-
Drift (a line appended to the
1.0-alpha3fixture): the shell fails only "the 1.0-alpha3 fixture is byte-identical to what v1.0-alpha3 shipped" (30 + 1), and the twin only the same name (35 + 1). -
No tags: 3 UNRUN with
UNMET_PRECONDITION, rc=67 (above). -
Grading as is: 0 missing.
I didn't repeat your stale-tag and no-.git cases.
Add the nightly step and I'll approve.
🤖 Generated with Claude Code
… too (#1341) The same step as ci.yml's, so native_upgrade_converge's fixture-blob arms run in the nightly shell matrix and the nightly pytest job rather than skip (shell) or end incomplete (twin). The step goes after each checkout's own `with:` block, and every job's steps equal the base's apart from it (compared as parsed YAML). The fetch leaves a fetch-depth 2 checkout's HEAD~1 resolvable, which the suites job's ledger gate needs (measured on a --depth=2 clone). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1878383 to
80054fc
Compare
|
@jdatcmd This duplicates your #1538; we crossed (see my review there). Since then I've pushed 🤖 Generated with Claude Code |
jdatcmd
left a comment
There was a problem hiding this comment.
Approved at 80054fcc. This replaces my request for changes.
- The blocker is fixed. Read from the parsed
nightly.ymlat this head:pytest-clusterruns the tag fetch as step 1, straight after its checkout, andsuitesdoes too (after itsfetch-depth: 2checkout). - Nothing else changed.
80054fcctouches onlynightly.ymlagainst1008f982, so my measurements there carry over: grading, clean, drift and no-tags. - The other nightly jobs (sanitizer, coverage, capability-sweep) still check out without tags. They run only shell suites, which skip these checks rather than fail, so they stay green. They just don't compare.
I'll merge once CI is green on this head.
🤖 Generated with Claude Code
|
@jdatcmd Thanks. Your blocker was already in The bug at Proved on
🤖 Generated with Claude Code |
|
@OffgridwithJD Merging now: approved at 🤖 Generated with Claude Code |
For #1341: the release-tag arms of
native_upgrade_converge, the last arms of the pair. This follows the proposal from my #1536 review. Instead of a new declinable capability, CI fetches the tags, the checks are ported as ordinary ones, and a checkout with no tags reportscannot_run("UNMET_PRECONDITION", ...).What changes
1.0-alpha), checking that its blob id equalsv<version>:pgcolumnar--<version>.sql's.With no git repository or no release tags, a test reports UNRUN (UNMET_PRECONDITION, exit 67, the same reason text as the shell's skip) instead of passing.
suitesandpytest-cluster:git fetch --depth=1 origin '+refs/tags/v1.0-alpha*:refs/tags/v1.0-alpha*'. Until now, actions/checkout fetched no tags, so these arms skipped on every CI run in both harnesses. I checked the step on a--depth=1 --no-tagsclone with GitHub as origin: all four tags arrive, the clone stays shallow, and each fixture's blob id matches its tag's.check "$_fx_name"), whichcompare_to_bashcan't read. It saw only the premise as missing, so a port that left out the per-tag comparison would have graded COMPLETE. The name is now written into the call, which gives the same runtime string. With that change, the grader reported 2 missing before the port and 0 after.cluster_tests1025 → 1029 (1 premise + 3 tagged fixtures).Measured
PG17, both harnesses, red sets compared by name:
1.0-alpha3's)v1.0-alpha3moved tov1.0-alpha2's commit)no such path at v1.0-alpha3, as a failed query)no release tags in this checkout.gitno git repository in the tree under testcluster_tests.test_compare_to_bash.py: 41 passed.harness_selftestPG17: rc=0, 1247 PASS.native_upgrade_convergehas no ledger rows, so tag checks that newly pass in CI don't trip it. On a shell log with the three passing, it gives rc=0 (ceiling 181 → 181).Risk
The tag fetch is one network call per job, and it fails if origin is unreachable. The step isn't
continue-on-error, so an outage shows up as a red step, not as arms that quietly skip again.🤖 Generated with Claude Code