From a6fd15c7fd6902e0fe78adcc25cf90a392041874 Mon Sep 17 00:00:00 2001 From: Nick Date: Thu, 13 Aug 2026 19:47:47 +0300 Subject: [PATCH 1/3] ci(journeys): run the browser probes, so their green stops decaying MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `OCU_BROWSER_E2E` was set nowhere — no workflow, no stand script. So the twenty- two frame-egress probes only ever ran when I ran them by hand, and "proven by running probes" was decaying into "proven the day it merged". Twelve of them need a live stand (a pane, a portal, a render frame) and stay opt-in. Four do not: the sink control, the unpoliced-channel control, and both halves of the script-execution probe reproduce the isolation primitives standalone. Measured — they pass on a bare runner with only chromium. The gate is SET in this job on purpose. `_require_browser` fails rather than skips when the gate is set and chromium is missing, so a broken install cannot read as a pass. The vacuity guard is an equality, not a floor, and it is the same failure this file already guards for the journeys: a suite where everything skipped renders green. Four is the measured count; a floor would hide a probe silently dropping out of the selection, which is the regression worth catching. Mutation-checked against a real junit report: with the gate unset, 3 of 4 skip and the guard reds. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/journeys-collect.yml | 55 ++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/.github/workflows/journeys-collect.yml b/.github/workflows/journeys-collect.yml index 5502a6b9..fdbc7244 100644 --- a/.github/workflows/journeys-collect.yml +++ b/.github/workflows/journeys-collect.yml @@ -95,3 +95,58 @@ jobs: f"subset collapsed. Failing the gate.") sys.exit(1) PY + + # --------------------------------------------------------------------------- + # The N group's browser-only probes. + # + # Twelve of the twenty-two need a live stand (a pane, a portal, a render frame) + # and stay opt-in. Four do not: the sink control, the unpoliced-channel + # control, and both halves of the script-execution probe reproduce the + # isolation primitives standalone. Without this job they only ever ran by + # hand, and "proven by running probes" decays to "proven the day it merged". + # + # OCU_BROWSER_E2E is SET here on purpose: with the gate set and no chromium, + # `_require_browser` FAILS rather than skipping, so a broken install cannot + # read as a pass. + # --------------------------------------------------------------------------- + frame-egress-probes: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0 + with: + python-version: "3.12" + - name: Install playwright + chromium + run: | + python -m pip install --upgrade pip playwright pytest + python -m playwright install --with-deps chromium + - name: Run the standalone probes + env: + OCU_BROWSER_E2E: "1" + # -k selects the four that need no stand. The stand-bound twelve are not + # deselected silently: they SKIP with their own reason, visible in -rs. + run: | + pytest -rs -o xfail_strict=true --strict-markers \ + --junit-xml=n-group-report.xml \ + -k "n0 or n2 or n4" \ + deploy/tests/journeys/test_n_frame_egress.py + - name: Guard against a vacuous browser-probe pass + # Same failure this file already guards for the journeys: a suite where + # everything skipped renders green. Four is the measured count, and it + # is an equality rather than a floor -- a probe silently dropping out of + # the selection is the regression, and a floor would hide it. + run: | + python3 - <<'PY' + import sys, xml.etree.ElementTree as ET + root = ET.parse("n-group-report.xml").getroot() + suites = root.findall("testsuite") or [root] + total = sum(int(s.get("tests", 0)) for s in suites) + skipped = sum(int(s.get("skips", s.get("skipped", 0))) for s in suites) + executed = total - skipped + print(f"browser probes: total={total} skipped={skipped} executed={executed}") + if executed != 4: + print(f"EXPECTED 4 executed browser probes, got {executed}. Either a " + f"probe stopped running or one was added without updating this " + f"count. Both need a look.") + sys.exit(1) + PY From 48dddee35e42a6d527a18b4e142d7429c0fb2cc1 Mon Sep 17 00:00:00 2001 From: Nick Date: Thu, 13 Aug 2026 19:59:11 +0300 Subject: [PATCH 2/3] ci(journeys): the comment misstated the gate's own verification properties MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fable's ruling, and he was right on both counts. The comment claimed the stand-bound probes "are not deselected silently: they SKIP with their own reason, visible in -rs". Measured: `-k` DESELECTS them, deselected tests never reach `-rs` and never enter the junit, and `-rs` reports zero SKIPPED lines here. It also said twelve; the arithmetic is 22 - 4 = eighteen test ids. A comment that misstates a gate's verification properties is exactly the decay this job exists to stop, so it says the true thing now: the eighteen are deselected, their absence is detectable only through the equality guard, and that guard is the load-bearing half of the job. Also states plainly that this covers 4 of 22 and is not "the N group covered". Two follow-ons he named, both cheap enough to do here rather than queue: A weekly schedule plus workflow_dispatch. Chromium installs at run time, so the probes can rot from upstream drift with no repo change — and that rot would surface as a red on some unrelated deploy PR. Per-PR alone was still "proven the day it merged", only with a longer day. Pinned playwright and pytest. A floating toolchain changes what the probe MEANS between runs. Both pins are the versions the probes were actually measured on (1.62.0 / 9.1.1) — I first wrote 1.58.0 and 8.3.4 from memory, which would have pinned a contract I never ran. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/journeys-collect.yml | 35 ++++++++++++++++++++------ 1 file changed, 27 insertions(+), 8 deletions(-) diff --git a/.github/workflows/journeys-collect.yml b/.github/workflows/journeys-collect.yml index fdbc7244..23f0534d 100644 --- a/.github/workflows/journeys-collect.yml +++ b/.github/workflows/journeys-collect.yml @@ -16,6 +16,14 @@ on: paths: - "deploy/**" - ".github/workflows/journeys-collect.yml" + # The browser probes install chromium at run time, so they can rot from + # upstream drift with no repo change at all — and that rot would then surface + # as a red on some unrelated future deploy PR. A weekly run finds it on its + # own schedule instead. Per-PR alone was still "proven the day it merged", + # only with a longer day. + schedule: + - cron: "17 6 * * 1" + workflow_dispatch: permissions: contents: read @@ -99,11 +107,15 @@ jobs: # --------------------------------------------------------------------------- # The N group's browser-only probes. # - # Twelve of the twenty-two need a live stand (a pane, a portal, a render frame) - # and stay opt-in. Four do not: the sink control, the unpoliced-channel - # control, and both halves of the script-execution probe reproduce the - # isolation primitives standalone. Without this job they only ever ran by - # hand, and "proven by running probes" decays to "proven the day it merged". + # Four of the twenty-two test ids need no stand: the sink control, the + # unpoliced-channel control, and both halves of the script-execution probe + # reproduce the isolation primitives standalone. Without this job they only + # ever ran by hand, and "proven by running probes" decays to "proven the day + # it merged". + # + # This job covers 4 of 22 — the controls and the script-execution primitive. + # It is NOT "the N group covered": the other eighteen need a live pane, portal + # and render frame, and they are the tier-2 gate against a real stand. # # OCU_BROWSER_E2E is SET here on purpose: with the gate set and no chromium, # `_require_browser` FAILS rather than skipping, so a broken install cannot @@ -117,14 +129,21 @@ jobs: with: python-version: "3.12" - name: Install playwright + chromium + # Pinned, like the sibling job's requirements file. A floating + # playwright changes what the probe MEANS between runs: the same green + # would be measuring a different browser contract each week. run: | - python -m pip install --upgrade pip playwright pytest + python -m pip install --upgrade pip + python -m pip install "playwright==1.62.0" "pytest==9.1.1" python -m playwright install --with-deps chromium - name: Run the standalone probes env: OCU_BROWSER_E2E: "1" - # -k selects the four that need no stand. The stand-bound twelve are not - # deselected silently: they SKIP with their own reason, visible in -rs. + # -k selects the four that need no stand and DESELECTS the other + # eighteen. Deselected is not skipped: they never run, never reach -rs, + # and never enter the junit — measured, `-rs` reports zero SKIPPED lines + # here. Their absence is detectable only through the equality guard + # below, which is why that guard is the load-bearing half of this job. run: | pytest -rs -o xfail_strict=true --strict-markers \ --junit-xml=n-group-report.xml \ From a93e5c5d9c24a84375893c579ac7e56f9f598517 Mon Sep 17 00:00:00 2001 From: Nick Date: Thu, 13 Aug 2026 20:01:27 +0300 Subject: [PATCH 3/3] ci(journeys): install the suite's own requirements before the probes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The job died at collection with ModuleNotFoundError: no module named 'yaml' — the journeys conftest imports it, and I had installed only playwright and pytest. My local venv already had it, which is the whole reason the CI run is the one that counts. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/journeys-collect.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/workflows/journeys-collect.yml b/.github/workflows/journeys-collect.yml index 23f0534d..355f0867 100644 --- a/.github/workflows/journeys-collect.yml +++ b/.github/workflows/journeys-collect.yml @@ -134,6 +134,11 @@ jobs: # would be measuring a different browser contract each week. run: | python -m pip install --upgrade pip + # The journeys conftest imports yaml, so the suite's own requirements + # come first — measured: without them the job dies at collection with + # ModuleNotFoundError before a single probe runs. My local venv had it + # already, which is exactly why the CI run is the one that counts. + python -m pip install -r deploy/tests/journeys/requirements.txt python -m pip install "playwright==1.62.0" "pytest==9.1.1" python -m playwright install --with-deps chromium - name: Run the standalone probes