ci: rosetta-audit without pins -- baseline-classified, refuses to no-op (#2682)

.github/workflows/golden-crucible.yml

@@ -57,7 +57,7 @@ jobs:
           # no credentials -- the only thing a fork run actually lacks is the ref *string*,
           # and that string is already committed in-repo as PINNED_TAG. So fall back to it
           # and give fork contributors real signal. Same escape-hatch shape as
-          # rosetta-audit.yml's `vars.KEYWORD_ROSETTA_REF || 'main'`, but pinned exactly
+          # rosetta-audit.yml's unpinned checkout of keyword-rosetta main, but pinned exactly
           # rather than floating on main. See tests/_crucible_pin.py's docstring for why the
           # pin lives in two places, and remember to bump BOTH when moving it.
           ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}"

.github/workflows/rosetta-audit.yml

@@ -1,26 +1,34 @@
 name: rosetta-audit
 
-# Cross-language consistency gate (#2557, Phase 5 of the control-corpus plan):
-# checks out keyword-rosetta at the pinned KEYWORD_ROSETTA_REF and runs its
-# verifier across ALL 46 language folders against THIS PR's engine build, plus
-# the corpus's baseline-gated n/a review audit. An engine change that shifts
-# corpus-observed counts, or nulls a rule without a validated ledger entry,
-# fails HERE -- in the PR that caused it -- instead of days later in the
-# corpus repo (how the #2552 cobol drift was found).
+# Cross-language consistency audit (#2557, redesigned in #2682): runs the keyword-rosetta
+# control corpus's verifier across ALL language folders against THIS PR's engine build and
+# reports which languages the build moves. Modelled on tri-comparison-audit.yml: this job
+# only ever MEASURES, never writes, and is baseline-gated -- a language that already fails
+# against engine main (the corpus has not caught up with a merged engine change yet) is a
+# notice, not a failure, so an unrelated PR stays green while the corpus is behind.
 #
-# Intentional drift is re-baselined via the cross-repo protocol in
-# docs/self_scan/BUMPING_THE_ROSETTA_PIN.md (the corpus repo's ENGINE_REF
-# escape hatch + a bump of KEYWORD_ROSETTA_REF in the engine PR).
+# There is no pin. The corpus is checked out at its main; its own CI verifies against THIS
+# repo's main. Both are advisory (neither repo requires a status check to merge), so an
+# intentional, corpus-visible engine change simply merges first and the corpus re-blesses
+# against engine main afterwards -- see docs/self_scan/ROSETTA_AUDIT.md. Add the
+# `rosetta:rebless-owed` label to such a PR and its regressions become warnings.
+#
+# Every step that can silently do nothing is asserted: rosetta_audit.py exits 2 unless it
+# actually verified every language folder, and a verifier crash is "broken", never "fail".
 
 on:
   pull_request:
     branches: [main, v6-dev]
     paths:
       - "gitgalaxy/**"
       - "pyproject.toml"
+      - "tests/tools/rosetta_audit.py"
       - ".github/workflows/rosetta-audit.yml"
   workflow_dispatch:
 
+permissions:
+  contents: read
+
 jobs:
   rosetta-audit:
     runs-on: ubuntu-latest
@@ -31,11 +39,20 @@ jobs:
           path: gitgalaxy
           persist-credentials: false
 
-      - name: Checkout keyword-rosetta (pinned)
+      # The baseline build: the branch this PR targets. Failing languages are re-run against
+      # it so the audit can tell "this PR moved the corpus" from "the corpus is behind main".
+      - name: Checkout GitGalaxy baseline (PR base branch)
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.base.ref || github.ref_name }}
+          path: gitgalaxy-baseline
+          persist-credentials: false
+
+      - name: Checkout keyword-rosetta (main)
         uses: actions/checkout@v4
         with:
           repository: squid-protocol/keyword-rosetta
-          ref: ${{ vars.KEYWORD_ROSETTA_REF || 'main' }}
+          ref: main
           path: keyword-rosetta
           persist-credentials: false
 
@@ -46,33 +63,40 @@ jobs:
           cache: "pip"
 
       # Full-precision deps: without all of them galaxyscope silently drops to
-      # Zero-Dependency Mode and observed counts drift from what the corpus
-      # manifests were locked against.
-      - name: Install GitGalaxy (full precision)
+      # Zero-Dependency Mode and observed counts drift from what the corpus manifests
+      # were locked against. The baseline gets its own venv (a snapshot install, not
+      # editable) so the two builds cannot alias each other.
+      - name: Install GitGalaxy PR build (full precision)
         run: |
           python -m pip install --upgrade pip
           pip install PyYAML networkx tiktoken numpy pandas xgboost
           pip install -e ./gitgalaxy
 
-      - name: Run all language gates
-        working-directory: keyword-rosetta
+      - name: Install GitGalaxy baseline build (full precision, separate venv)
+        run: |
+          python -m venv baseline-venv
+          baseline-venv/bin/pip install --upgrade pip
+          baseline-venv/bin/pip install PyYAML networkx tiktoken numpy pandas xgboost
+          baseline-venv/bin/pip install ./gitgalaxy-baseline
+
+      - name: Run all language gates, classify against the baseline
         env:
           GITGALAXY_PATH: ${{ github.workspace }}/gitgalaxy
         run: |
-          rc=0
-          for d in data/*/; do
-            L=$(basename "$d")
-            python tools/verify_language.py "$L" || rc=1
-          done
-          exit $rc
+          allow=""
+          if [ "${{ contains(github.event.pull_request.labels.*.name, 'rosetta:rebless-owed') }}" = "true" ]; then
+            allow="--allow-regressions"
+            echo "::notice::rosetta:rebless-owed label present -- regressions reported as warnings"
+          fi
+          python gitgalaxy/tests/tools/rosetta_audit.py \
+            --corpus keyword-rosetta \
+            --baseline-bin "${{ github.workspace }}/baseline-venv/bin/galaxyscope" \
+            $allow
 
+      # Baseline-gated: fails only on unreviewed rule-absent (n/a) cells beyond the corpus's
+      # docs/na_baseline.json -- an engine PR cannot quietly null a rule nobody has ledgered.
       - name: n/a review audit (baseline-gated)
         working-directory: keyword-rosetta
         env:
           GITGALAXY_PATH: ${{ github.workspace }}/gitgalaxy
-        run: |
-          if [ -f tools/na_check.py ]; then
-            python tools/na_check.py --ci
-          else
-            echo "::notice::pinned keyword-rosetta ref predates tools/na_check.py -- bump KEYWORD_ROSETTA_REF to activate the n/a audit"
-          fi
+        run: python tools/na_check.py --ci

.github/workflows/tree-sitter-accuracy-audit.yml

@@ -59,7 +59,7 @@ jobs:
           # no credentials -- the only thing a fork run actually lacks is the ref *string*,
           # and that string is already committed in-repo as PINNED_TAG. So fall back to it
           # and give fork contributors real signal. Same escape-hatch shape as
-          # rosetta-audit.yml's `vars.KEYWORD_ROSETTA_REF || 'main'`, but pinned exactly
+          # rosetta-audit.yml's unpinned checkout of keyword-rosetta main, but pinned exactly
           # rather than floating on main. See tests/_crucible_pin.py's docstring for why the
           # pin lives in two places, and remember to bump BOTH when moving it.
           ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}"

.github/workflows/tree-sitter-accuracy-history.yml

@@ -82,7 +82,7 @@ jobs:
           # no credentials -- the only thing a fork run actually lacks is the ref *string*,
           # and that string is already committed in-repo as PINNED_TAG. So fall back to it
           # and give fork contributors real signal. Same escape-hatch shape as
-          # rosetta-audit.yml's `vars.KEYWORD_ROSETTA_REF || 'main'`, but pinned exactly
+          # rosetta-audit.yml's unpinned checkout of keyword-rosetta main, but pinned exactly
           # rather than floating on main. See tests/_crucible_pin.py's docstring for why the
           # pin lives in two places, and remember to bump BOTH when moving it.
           ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}"

.github/workflows/tri-comparison-audit.yml

@@ -84,7 +84,7 @@ jobs:
           # no credentials -- the only thing a fork run actually lacks is the ref *string*,
           # and that string is already committed in-repo as PINNED_TAG. So fall back to it
           # and give fork contributors real signal. Same escape-hatch shape as
-          # rosetta-audit.yml's `vars.KEYWORD_ROSETTA_REF || 'main'`, but pinned exactly
+          # rosetta-audit.yml's unpinned checkout of keyword-rosetta main, but pinned exactly
           # rather than floating on main. See tests/_crucible_pin.py's docstring for why the
           # pin lives in two places, and remember to bump BOTH when moving it.
           ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}"

.github/workflows/tri-comparison-history.yml

@@ -93,7 +93,7 @@ jobs:
           # no credentials -- the only thing a fork run actually lacks is the ref *string*,
           # and that string is already committed in-repo as PINNED_TAG. So fall back to it
           # and give fork contributors real signal. Same escape-hatch shape as
-          # rosetta-audit.yml's `vars.KEYWORD_ROSETTA_REF || 'main'`, but pinned exactly
+          # rosetta-audit.yml's unpinned checkout of keyword-rosetta main, but pinned exactly
           # rather than floating on main. See tests/_crucible_pin.py's docstring for why the
           # pin lives in two places, and remember to bump BOTH when moving it.
           ref="${{ vars.LANGUAGE_CRUCIBLE_REF }}"

CLAUDE.md

@@ -289,20 +289,21 @@ corpus-wide whenever any node's mass changes, so a single-language fix ripples c
 across every language in the corpus. Those are attributable as a class; filter them out and
 scope the remainder, which is what actually needs a per-language explanation.
 
-**The two accuracy audits exit 0 while doing nothing if `galaxyscope` is off `PATH`.**
-`tests/tools/tree_sitter_accuracy_audit.py` and `tests/tools/tri_comparison_chart.py` report
-`galaxyscope not found on PATH` per language, skip every one, and still print
-`all OK` with exit code 0 -- a false green that looks exactly like a pass. They need the **main
-`.venv`**, which has both `galaxyscope` and `tree_sitter_language_pack`; neither crucible venv
-does (`full_precision` has `galaxyscope` but not the tree-sitter pack). Always run them as:
+**The corpus-backed audit runners need the main `.venv`, and they refuse to pass on nothing.**
+`tests/tools/tree_sitter_accuracy_audit.py`, `tests/tools/tri_comparison_chart.py` and
+`tests/tools/rosetta_audit.py` all need `galaxyscope` on `PATH` (the main `.venv` has it plus
+`tree_sitter_language_pack`; neither crucible venv has both). Since #2682 a run that checked zero
+languages, or a baselined language whose scan could not run, is a hard failure — previously
+`tri_comparison_chart.py --ci` skipped every language and printed `all OK` with exit 0 when
+`galaxyscope` was missing. Run them as:
 
 ```sh
 PATH="$PWD/.venv/bin:$PATH" .venv/bin/python tests/tools/tree_sitter_accuracy_audit.py --ci --all
 PATH="$PWD/.venv/bin:$PATH" .venv/bin/python tests/tools/tri_comparison_chart.py --all --ci
+PATH="$PWD/.venv/bin:$PATH" .venv/bin/python tests/tools/rosetta_audit.py      # needs ../keyword-rosetta
 ```
 
-and confirm the summary line names a plausible language count (30 and 3 respectively as of
-2026-09-02) -- "0 languages checked, all OK" is a failure wearing a pass.
+and still glance at the summary line's language count (30, 3 and 46 respectively as of 2026-09-03).
 
 ## Logging cases where GitGalaxy beats tree-sitter/AST ground truth
 

CONTRIBUTING.md

@@ -80,17 +80,17 @@ When you modify GitGalaxy's core engine, several CI workflows will rigorously te
 
 4. **Cross-Language Consistency (`rosetta-audit`, the [keyword-rosetta](https://github.com/squid-protocol/keyword-rosetta) corpus)**
    This runs the control corpus's verifier across all 46 language folders against your engine build,
-   so a change that shifts corpus-observed counts fails **in the PR that caused it**. Unlike the three
-   baselines above, the expected values live in *another repository* — you cannot re-bless them here.
-   - Run one language locally: `GITGALAXY_PATH=<path-to-this-repo> python tools/verify_language.py <lang>` from a keyword-rosetta checkout.
-   - If the drift is an unintentional regression: fix the engine change. **Do not touch the pin.**
-   - If the drift is an intentional, corpus-visible improvement: it needs a companion re-baseline PR
-     in keyword-rosetta *before* this one can merge. Follow
-     [`docs/self_scan/BUMPING_THE_ROSETTA_PIN.md`](docs/self_scan/BUMPING_THE_ROSETTA_PIN.md) — in short,
-     set that repo's committed `ENGINE_REF` file to `pull/<your PR number>/head` so its gates run green
-     against your unmerged branch, then a maintainer restores it and bumps `KEYWORD_ROSETTA_REF` here.
-   - Bumping the pin needs repo admin, so **ask a maintainer** rather than trying to do it yourself.
-     Bumping a pin is never a way to make a red check go away.
+   then re-runs anything that failed against a build of the branch you target, so it can tell
+   "this PR moves the corpus" from "the corpus is behind main" (the latter is a notice, not a
+   failure). Unlike the three baselines above, the expected values live in *another repository* —
+   you cannot re-bless them here, and you do not have to before merging.
+   - Run it locally: `python tests/tools/rosetta_audit.py` (needs `../keyword-rosetta` and `galaxyscope` on PATH).
+   - If the drift is an unintentional regression: fix the engine change.
+   - If the drift is an intentional, corpus-visible improvement: add the `rosetta:rebless-owed` label
+     (the check goes green with the languages still listed), merge, then open the re-bless PR in
+     keyword-rosetta against engine main. Full detail in
+     [`docs/self_scan/ROSETTA_AUDIT.md`](docs/self_scan/ROSETTA_AUDIT.md). There is no pin on
+     either side and nothing to reset afterwards.
 
 If your PR touches any baseline fixtures, **explain why in the PR description** (e.g. "improved the Rust parser, now correctly detects async trait bounds"). A CI check flags any PR that modifies these files so it's never invisible in a large diff.
 
@@ -123,8 +123,8 @@ If you are moving the pin, bump it in **both** places — `tests/_crucible_pin.p
 `LANGUAGE_CRUCIBLE_REF` repository variable. Nothing enforces that they match; see that file's
 docstring for why the pin is deliberately duplicated.
 
-`rosetta-audit` uses the same escape hatch against a different corpus (it falls back to
-keyword-rosetta's `main`), and every other check — `full-suite`, the `smoke-test` matrix,
+`rosetta-audit` needs no escape hatch: it always checks out keyword-rosetta's `main` (that repo is
+public), so it runs for real on a fork PR too. Every other check — `full-suite`, the `smoke-test` matrix,
 `ruff-audit`, `mypy-audit`, `ast-accuracy-audit` — runs normally on a fork PR too.
 
 ---

docs/ecosystem.md

@@ -16,7 +16,7 @@ Canonical local layout on the dev machine — all siblings under `/srv/storage_1
 |---|---|---|
 | **gitgalaxy** | `gitgalaxy/v6` | **The engine** (this repo). AST-free, LLM-free static analysis: bounded-regex structural signatures → knowledge graph → risk scoring / SBOM / 3D map. Everything else in this table exists to feed, verify, or showcase it. |
 | **language-crucible** | `all_language_repo` | Zero-execution structural-parser **benchmark corpus** (`data/<language>/<repo-folder>/`, per-category `SOURCES.md`, machine-readable `PROVENANCE.json`). gitgalaxy CI pins it to a release tag (`LANGUAGE_CRUCIBLE_REF` GH Actions var + `tests/_crucible_pin.py`) and diffs golden masters against it. Releases per its `RELEASING.md`. |
-| **keyword-rosetta** | `keyword-rosetta` | **Control corpus**: one identical 12-probe program shell in all 46 signature-bearing languages, exact planted keyword counts — measures whether the engine treats identical intent identically across languages (cross-language bias). Gates via `tools/verify_language.py`; deviations live in `deviation_ledger.json` per its `docs/GATING.md`. Its CI checks out gitgalaxy **main** (see choreography below). |
+| **keyword-rosetta** | `keyword-rosetta` | **Control corpus**: one identical 12-probe program shell in all 46 signature-bearing languages, exact planted keyword counts — measures whether the engine treats identical intent identically across languages (cross-language bias). Gates via `tools/verify_language.py`; deviations live in `deviation_ledger.json` per its `docs/GATING.md`. Its CI checks out gitgalaxy **main**, and gitgalaxy's `rosetta-audit` checks out its **main** — no pins in either direction (#2682). Its `bias-history.yml` regenerates the bias chart after every corpus push and daily. |
 | **gitgalaxy-raw-output** | `gitgalaxy-raw-output` | Real, unedited **scan outputs** on independently-chosen production repos (`v<engine-version>/<repo>/<repo>_galaxy_llm.md` + gzipped audit/SBOM) plus speed charts. Evidence source for README claims and `docs/language_status/` §8 sections. |
 | **squid-telemetry** | `squid-telemetry` | **Distribution/adoption analytics** (the engine itself is air-gapped and phones nothing home; this pipeline scrapes public GitHub/GitLab/PyPI fetch metrics daily via Actions and commits regenerated chart PNGs). |
 | **gitgalaxy-population-analyses** | `gitgalaxy-population-analyses` | Offline **statistical analyses** over scan populations (risk-distribution ridgeplots, archetype clustering, threat-prediction distribution studies). Reads raw inputs from gitgalaxy-raw-output; never on any CI path. |
@@ -80,7 +80,7 @@ rule, sync all three.
 | Workflow | Repos touched (in merge order) | Documented in |
 |---|---|---|
 | **Crucible corpus growth → release → pin bump** | language-crucible (data PRs, tag per `RELEASING.md`) → gitgalaxy (`docs/self_scan/BUMPING_THE_CRUCIBLE_PIN.md`: regen golden masters + tri-comparison + tree-sitter artifacts, bump `LANGUAGE_CRUCIBLE_REF` + `PINNED_TAG`) | crucible `RELEASING.md`; gitgalaxy `BUMPING_THE_CRUCIBLE_PIN.md` |
-| **Rosetta sweep** (work one language's bias issue) | gitgalaxy engine PR first (its `rosetta-audit` check fails at the source — expected) → keyword-rosetta corpus re-baseline PR with `ENGINE_REF=pull/<N>/head` (green immediately, no draft limbo) → restore `ENGINE_REF=main`, merge corpus → engine PR bumps `KEYWORD_ROSETTA_REF` var, merges green. Capstone lands back in gitgalaxy `docs/language_status/<lang>.md` §10. | keyword-rosetta `rosetta-language-sweep` skill; gitgalaxy `docs/self_scan/BUMPING_THE_ROSETTA_PIN.md` |
+| **Rosetta sweep** (work one language's bias issue) | gitgalaxy engine PR first (its `rosetta-audit` lists the languages it moves; add the `rosetta:rebless-owed` label if that is intended) → merge → keyword-rosetta corpus re-bless PR against engine main (green by construction; manifests + ledger) → merge; its `bias-history.yml` regenerates the chart. Capstone lands back in gitgalaxy `docs/language_status/<lang>.md` §10. | keyword-rosetta `rosetta-language-sweep` skill; gitgalaxy `docs/self_scan/ROSETTA_AUDIT.md` |
 | **Adding a language to the engine** | gitgalaxy (`standards/how_to_add_a_language.md`, includes authoring the rosetta control folder) → keyword-rosetta (`SPEC.md` shell + manifest) → optionally language-crucible (`expand-language-coverage`) | those three docs |
 | **Tri-comparison / accuracy verification** | gitgalaxy only (ledger, chart, `manual_verification.json`), but reads the pinned crucible corpus | gitgalaxy `docs/self_scan/tri_comparison_README.md` |
 | **README / evidence claims** | gitgalaxy README cites gitgalaxy-raw-output artifacts and the keyword-rosetta chart (embedded from that repo's raw main URL — it self-updates when rosetta main moves) | gitgalaxy `readme-maintenance` skill |