Lint Catalog fails a boxel PR whose merged Merges-after catalog PR isn't in the pin

.claude/skills/catalog-pairing/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: catalog-pairing
-description: How a boxel pull request and a boxel-catalog pull request that depend on each other are paired — a `Merges before:` / `Merges after:` line in each description naming the other, declared from both sides — and which merges first. Covers when a change needs a pair (a boxel change the catalog's cards must follow, such as renaming or removing a host tool, command, base export or type the catalog imports; or a catalog definition boxel's tests need first), the declaration syntax and the rules a declaration must meet, merge order in each direction and in the three-PR flow, what boxel's Lint Catalog does with the pairing (it lints the paired catalog pull request's head and refuses to let catalog main be left broken behind a pull request that isn't ready), and how to read its failures. Use before opening or editing any pull request in cardstack/boxel-catalog; before opening a boxel pull request that changes something boxel-catalog's cards import or that pins or tests a catalog change; whenever Lint Catalog fails; and whenever a check says `pairing:`, "doesn't name … back", "waiting on cardstack/boxel-catalog#… to merge", "is not approved yet", "pair the two", or "Merges before" / "Merges after".
+description: How a boxel pull request and a boxel-catalog pull request that depend on each other are paired — a `Merges before:` / `Merges after:` line in each description naming the other, declared from both sides — and which merges first. Covers when a change needs a pair (a boxel change the catalog's cards must follow, such as renaming or removing a host tool, command, base export or type the catalog imports; or a catalog definition boxel's tests need first), the declaration syntax and the rules a declaration must meet, merge order in each direction and in the three-PR flow, what boxel's Lint Catalog does with the pairing (it lints the paired catalog pull request's head and refuses to let catalog main be left broken behind a pull request that isn't ready), and how to read its failures. Use before opening or editing any pull request in cardstack/boxel-catalog; before opening a boxel pull request that changes something boxel-catalog's cards import or that pins or tests a catalog change; whenever Lint Catalog fails; and whenever a check says `pairing:`, "doesn't name … back", "waiting on cardstack/boxel-catalog#… to merge", "is not approved yet", "pair the two", "Merges before" / "Merges after", "which predates it", or "can't contain it yet".
 ---
 
 # Pairing boxel and boxel-catalog pull requests
@@ -53,6 +53,7 @@ Edit a description with `gh pr edit <n> --repo <owner/repo> --body-file <file>`
 **boxel's Lint Catalog** (`.github/workflows/lint-catalog.yaml`, `packages/catalog/scripts/pairing.ts` and `lint-sweep.ts`):
 
 1. It reads this pull request's declaration and checks it from both sides. A problem fails the job immediately, naming the line to add or fix and the pull request it belongs on.
+   The catalog revision this pull request pins in `packages/catalog/test-subset.json` must contain its `Merges after` pair's merge commit (for a pair stacked in its own repository, the merge commit its stack reached `main` through), since production deploys the catalog at the revision boxel pins. While that pair is open, the job's last step fails with `… hasn't merged, so … can't contain it yet`, after the lint steps have judged the pair's head. Nothing re-runs the job when the pair merges, so the push that re-pins with `pnpm --dir packages/catalog catalog:test-subset --bump` is what runs it again. A pin that predates the merge commit fails with `cardstack/boxel-catalog#N merged as <sha>, but … pins <rev>, which predates it`.
 2. It lints the paired catalog pull request's head against the change, in place of catalog `main`. That is the `Merges after` pair if it's unmerged, since it lands first, otherwise the `Merges before` pair. Errors there are for the catalog pull request to fix.
 3. It also lints catalog `main` against the change, against the base branch too, so only errors this change adds count. If the change adds errors to catalog `main`:
    - **`Merges after` pair unmerged**: fails with `waiting on cardstack/boxel-catalog#N to merge`. Re-run after it merges. This comes first, because that pull request has to land before this one.
@@ -67,5 +68,5 @@ Edit a description with `gh pr edit <n> --repo <owner/repo> --body-file <file>`
 ## Merging
 
 - **Boxel first:** merge the boxel pull request only once its catalog pull request is approved and neither one is stacked, then merge the catalog one right after. Until the catalog one merges, catalog `main` fails against boxel `main`, and boxel-catalog CI is red for everyone.
-- **Catalog first:** merge the catalog pull request, re-pin the boxel pull request onto catalog `main` (`pnpm --dir packages/catalog catalog:test-subset --bump`), re-run its checks, then merge it.
+- **Catalog first:** merge the catalog pull request, re-pin the boxel pull request onto catalog `main` (`pnpm --dir packages/catalog catalog:test-subset --bump`), re-run its checks, then merge it. Lint Catalog enforces the re-pin: it fails the boxel pull request that declares `Merges after:` the catalog pull request until that has merged and the pin contains its merge commit, since production deploys the catalog at the revision boxel pins.
 - **Each environment gets the catalog on its own schedule.** Staging syncs catalog `main` on every catalog merge. Production gets only what boxel-catalog's "Deploy to production" deploys: the revision boxel pins, before and after Manual Deploy [boxel] to production releases, or a catalog `main` revision someone deploys by hand. That deploy refuses while a catalog pull request in it says `Merges after:` a boxel pull request production doesn't run. So a catalog change that needs boxel code declares its pair even when Lint Catalog wouldn't require one. The `catalog-deploy` skill covers the deploy and its refusals.

.claude/skills/catalog-test-subset/SKILL.md

@@ -81,7 +81,7 @@ Never commit a pin that exists only in a local catalog checkout (the check fails
 
 **For merge.** Every subset file at the pin must match catalog `main` byte for byte. `--check-pin` enforces this on every boxel PR that changes the manifest.
 
-- `waiting on cardstack/boxel-catalog#<N> to merge`: the pin is a commit in an open catalog PR. Merge that PR first, then re-run the check. A pin from a catalog PR passes once the PR merges, whether as a merge commit, a squash or a rebase, so the pin doesn't have to move.
+- `waiting on cardstack/boxel-catalog#<N> to merge`: the pin is a commit in an open catalog PR. Merge that PR first, then re-run the check. A pin from a catalog PR passes this check once the PR merges, whether as a merge commit, a squash or a rebase. A boxel PR that declares `Merges after:` that catalog PR still has to re-pin with `--bump` (below).
 - "main has changed … since": catalog `main` has a newer version of a subset file. Re-pin with `--bump`, run the sync, and re-run the consumers. If the message names a boxel PR that already pins a matching revision, merging boxel `main` after it lands works too.
 
 ## Changing a subset definition, or adding one
@@ -97,7 +97,7 @@ The order matters, because the deployed catalog is boxel-catalog `main`, not the
 
 3. **Push the boxel branch, with its manifest entry, before you push the boxel-catalog branch or open its PR.** boxel-catalog's `Boxel Test Subset` workflow decides which files to test from the manifest on the same-named boxel branch, looked up at the moment it runs. If that branch isn't on GitHub yet, the workflow falls back to boxel `main`. That manifest doesn't list a file you are adding, so the check passes without running a single test. A changed file fails loudly against boxel `main` instead of passing, but the tests it ran still weren't yours. If the catalog workflow ran before the boxel branch existed, re-run it once the branch is pushed. Then open the boxel-catalog PR, set the manifest's `revision` to its head sha (a commit on a pushed branch fetches fine), and push the boxel branch again. Re-pin whenever you push to the catalog PR. Pin before you open or merge the boxel PR.
 4. **Validate the catalog PR against boxel.** boxel-catalog's `Boxel Test Subset` workflow runs the manifest's `tests` consumers whenever a PR touches a subset path. It runs them against the boxel branch of the same name when one exists, otherwise against boxel `main`, so matching branch names are how a catalog change and the boxel change that depends on it are tested together before either merges. A manual run takes any `boxel_ref`. The catalog workflow only decides whether to run and which boxel ref to use. The tests run in boxel's reusable `.github/workflows/catalog-test-subset-consumers.yml`, which the catalog calls at boxel `main`, so a CI setup change belongs in boxel, not in the catalog. The workflow's steps, and those of the `test-web-assets.yaml` it calls, come from boxel `main`. What the steps run comes from the chosen ref: its mise tasks, composite actions, manifest and tests, and the source the host dist is built from. So a paired boxel branch's edit to one of those steps reaches catalog runs only once it merges; the branch's own boxel PR run is what tests it. And if the catalog run fails on a task or argument the paired branch doesn't have, the branch is behind `main`: merge `main` into it. While the pin is a catalog PR commit, boxel's `Catalog Test Subset` check fails with `waiting on cardstack/boxel-catalog#<N> to merge`. That is expected until the catalog merges.
-5. **Merge the catalog PR first.** Then re-run the boxel PR's `Catalog Test Subset` check. It passes once every subset file at the pin matches catalog `main`, whether the catalog PR merged as a merge commit, a squash or a rebase, so the pin doesn't have to move. If it fails with "main has changed … since" (the catalog PR got commits after you pinned, or another catalog PR changed a subset file), re-pin with `pnpm --dir packages/catalog catalog:test-subset --bump`, run the sync, and re-run the consumers. Then merge the boxel PR.
+5. **Merge the catalog PR first.** Then re-pin the boxel PR onto catalog `main` with `pnpm --dir packages/catalog catalog:test-subset --bump`, run the sync, re-run the consumers, and push. Production deploys the catalog at the revision the deployed boxel pins, so the pin has to contain the catalog PR's merge commit, not only its head: Lint Catalog fails a boxel PR that declares `Merges after:` a catalog PR until that has merged and the pin contains its merge commit. If catalog `main` moves again before the boxel PR merges and `Catalog Test Subset` fails with "main has changed … since", bump again. Then merge the boxel PR.
 
 A catalog change to a subset file reaches deployments as soon as the catalog merges, but it reaches boxel's tests only when the pin is bumped. So it must stay compatible with boxel `main`. Bump the pin in the same boxel PR as any platform or test change that depends on the new definition. A catalog change that merges with no boxel PR paired with it still needs a follow-up boxel PR that only runs `--bump`, so boxel keeps testing what deployments serve. Until that lands, the next boxel PR that touches the manifest fails `--check-pin` and has to bump.
 

.github/workflows/lint-catalog.yaml

@@ -51,6 +51,9 @@ jobs:
       # The catalog change a pull request depends on, or that follows it, is
       # the one its description pairs it with (scripts/pairing.ts). Lint that
       # pull request's head in place of main, and catalog main as well below.
+      # Production deploys the catalog at the revision boxel pins, so a catalog
+      # pull request this change merges after must be in the pin once it has
+      # merged.
       - name: Read the catalog pairing
         id: pairing
         if: github.event_name == 'pull_request'
@@ -65,6 +68,7 @@ jobs:
             --number="$PR_NUMBER" \
             --base="$PR_BASE" \
             --counterpart=cardstack/boxel-catalog \
+            --pin=packages/catalog/test-subset.json \
             --out="$RUNNER_TEMP/catalog-pairing.json"
           # An open pair this change merges after is what it builds on, and
           # lands first; otherwise an open pair it merges before is the
@@ -193,3 +197,12 @@ jobs:
             args+=(--baseline="$RUNNER_TEMP/catalog-main-lint-base.json")
           fi
           node "$RUNNER_TEMP/lint-sweep.ts" "${args[@]}"
+
+      # A catalog pull request this change merges after has to be in the pin
+      # before this change merges, and nothing re-runs this job when that pull
+      # request merges. So while it is open this fails, after the lint above
+      # has judged its head, and the push that re-pins once it merges is the
+      # run that compares the pin with its merge commit.
+      - name: Wait for the re-pin
+        if: ${{ !cancelled() && steps.pairing.outcome == 'success' }}
+        run: node packages/catalog/scripts/pairing.ts --pin-verdict="$RUNNER_TEMP/catalog-pairing.json"