Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions .github/workflows/journeys-collect.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,14 @@ on:
paths:
- "deploy/**"
- ".github/workflows/journeys-collect.yml"
# The browser probes install chromium at run time, so they can rot from
# upstream drift with no repo change at all — and that rot would then surface
# as a red on some unrelated future deploy PR. A weekly run finds it on its
# own schedule instead. Per-PR alone was still "proven the day it merged",
# only with a longer day.
schedule:
- cron: "17 6 * * 1"
workflow_dispatch:

permissions:
contents: read
Expand Down Expand Up @@ -95,3 +103,74 @@ jobs:
f"subset collapsed. Failing the gate.")
sys.exit(1)
PY

# ---------------------------------------------------------------------------
# The N group's browser-only probes.
#
# Four of the twenty-two test ids need no stand: the sink control, the
# unpoliced-channel control, and both halves of the script-execution probe
# reproduce the isolation primitives standalone. Without this job they only
# ever ran by hand, and "proven by running probes" decays to "proven the day
# it merged".
#
# This job covers 4 of 22 — the controls and the script-execution primitive.
# It is NOT "the N group covered": the other eighteen need a live pane, portal
# and render frame, and they are the tier-2 gate against a real stand.
#
# OCU_BROWSER_E2E is SET here on purpose: with the gate set and no chromium,
# `_require_browser` FAILS rather than skipping, so a broken install cannot
# read as a pass.
# ---------------------------------------------------------------------------
frame-egress-probes:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: "3.12"
- name: Install playwright + chromium
# Pinned, like the sibling job's requirements file. A floating
# playwright changes what the probe MEANS between runs: the same green
# would be measuring a different browser contract each week.
run: |
python -m pip install --upgrade pip
# The journeys conftest imports yaml, so the suite's own requirements
# come first — measured: without them the job dies at collection with
# ModuleNotFoundError before a single probe runs. My local venv had it
# already, which is exactly why the CI run is the one that counts.
python -m pip install -r deploy/tests/journeys/requirements.txt
python -m pip install "playwright==1.62.0" "pytest==9.1.1"
python -m playwright install --with-deps chromium
- name: Run the standalone probes
env:
OCU_BROWSER_E2E: "1"
# -k selects the four that need no stand and DESELECTS the other
# eighteen. Deselected is not skipped: they never run, never reach -rs,
# and never enter the junit — measured, `-rs` reports zero SKIPPED lines
# here. Their absence is detectable only through the equality guard
# below, which is why that guard is the load-bearing half of this job.
run: |
pytest -rs -o xfail_strict=true --strict-markers \
--junit-xml=n-group-report.xml \
-k "n0 or n2 or n4" \
deploy/tests/journeys/test_n_frame_egress.py
- name: Guard against a vacuous browser-probe pass
# Same failure this file already guards for the journeys: a suite where
# everything skipped renders green. Four is the measured count, and it
# is an equality rather than a floor -- a probe silently dropping out of
# the selection is the regression, and a floor would hide it.
run: |
python3 - <<'PY'
import sys, xml.etree.ElementTree as ET
root = ET.parse("n-group-report.xml").getroot()
suites = root.findall("testsuite") or [root]
total = sum(int(s.get("tests", 0)) for s in suites)
skipped = sum(int(s.get("skips", s.get("skipped", 0))) for s in suites)
executed = total - skipped
print(f"browser probes: total={total} skipped={skipped} executed={executed}")
if executed != 4:
print(f"EXPECTED 4 executed browser probes, got {executed}. Either a "
f"probe stopped running or one was added without updating this "
f"count. Both need a look.")
sys.exit(1)
PY
Loading