Skip to content

ci: keep Code Quality green on push (org-level readiness gates) and fail a stale uv.lock early - #498

Merged
eaitbrahim merged 2 commits into
mainfrom
ci-quality-and-lockcheck
Aug 22, 2026
Merged

ci: keep Code Quality green on push (org-level readiness gates) and fail a stale uv.lock early#498
eaitbrahim merged 2 commits into
mainfrom
ci-quality-and-lockcheck

Conversation

@eaitbrahim

Copy link
Copy Markdown
Contributor

Closes #402, Closes #424

Two CI failures that pointed at the wrong thing, fixed at the point where the mistake is made.

#402 — Code Quality push-runs fail on every main commit

Root cause (from the live run logs)

The issue title says "secrets/org config only in PR context"; the investigation found something more specific. This workflow has no pull_request trigger at all (deliberately, per its own header — fork PRs receive no secrets), so what passes on PRs is the SonarCloud GitHub App, a separate integration. The workflow's own failures started on 2026-08-18 — the day SONAR_TOKEN and SNYK_TOKEN were created (gh secret list confirms both, created 09:45/09:51 that day). The preflight's single configured flag flipped to true, both scan jobs ran for the first time, and both failed on org-level prerequisites that no repository secret can carry:

  • SonarQube scanERROR You must define the following mandatory properties for 'CodeGateSoftware_keel': sonar.organization (run 32553995252). sonar.organization is mandatory on SonarQube Cloud and is deliberately commented out in sonar-project.properties — a wrong value fails with a misleading "project not found", so it was left for the org owner to fill in. The token alone was never enough.
  • Snyk test / Snyk monitor422 Unprocessable Entity, SNYK-OS-PYTHON-0013 "Missing required packages". The token authenticates; the Snyk organization on the snyk.io side rejects the scan — org-level setup the repository cannot perform or even detect from YAML (the snyk monitor step has carried a --org=<keel-snyk-org-id> "SET BEFORE FIRST USE" placeholder since the workflow was written).

Net effect: a permanently red main, which is exactly the outcome this workflow's preflight was designed to prevent ("a red X nobody reads is not honest").

The fix — option (b): gate each scan on its real readiness, so it lights up the moment the org config appears

Main-branch analysis is genuinely wanted here (the push: [main] trigger was added deliberately in #272 once the repo went public), so restricting the jobs to pull_request (option a) would delete intended behaviour — and there is no PR-context behaviour in this workflow to preserve. Instead the existing preflight pattern was deepened, per scanner:

  • preflight now emits sonar_ready (token present and an active sonar.organization= line in sonar-project.properties) and snyk_ready (token present and the new SNYK_ORG repository variable, which the scan steps pass as --org — turning the old placeholder into real wiring).
  • Each scan job is gated on its own output, so one scanner's gap no longer keeps the other dark — and each lights up the moment its own missing piece is configured. No flag day.
  • The documented asymmetry is preserved and pinned by test: workflow_dispatch still fails loudly when a scan is not ready (silently ignoring a direct request is worse); push/schedule skip with a run summary naming exactly what is missing and where to set it.
  • The skipped-step notice names both the property (sonar.organization, in sonar-project.properties) and the variable (SNYK_ORG, Settings → Secrets and variables → Actions → Variables). A missing sonar-project.properties in preflight fails loudly rather than skipping the Sonar scan forever.

What an org owner still needs to do to light the scans up (the repository cannot): set sonar.organization in sonar-project.properties to the real SonarCloud key, and create/wire the Snyk organization then set SNYK_ORG. The moment either happens, its scan runs on the next push — no further workflow change.

#424 — a stale uv.lock fails the release three steps late as a dirty-tree error

Root cause

The 0.10.0 release: #422 bumped the version in all seven pyproject.toml files and missed the tracked uv.lock. uv sync silently re-locked the checkout ("Sync dependencies"), the stamp step then recorded a dirty tree, and the failure surfaced five steps later as ::error::artifact reports a dirty tree — sending the reader to keel/version.py and the stamp step instead of the lockfile, after Lint, Type-check, Test and Build had all run and passed.

The fix

New step "The lockfile must already be current" in release.yml, placed immediately after the interpreter is in place and immediately before "Sync dependencies" — the first step that can mutate the tree (and still well before "Test"). It runs uv lock --check under set -euo pipefail (flag verified against uv 0.11.31: "Check if the lockfile is up-to-date", no write), and on failure prints:

::error::uv.lock is stale for this pyproject -- run 'uv lock' locally and commit the result with the version bump. (A stale lock is otherwise discovered five steps later as 'artifact reports a dirty tree'.)

docs/RELEASING.md step 1 now says the bump is pyproject.toml and uv lock — the doc gap the issue named. The workflow still never writes to main (the issue's explicit "not proposed").

Test delta

  • tests/test_security_scans.py: the guard test now pins per-scanner gates (sonar_ready / snyk_ready declared and used by the token-referencing jobs) instead of the single shared configured string; new tests pin that preflight checks the org-level config (sonar.organization, SNYK_ORG), that the snyk job passes --org, and that the dispatch-loud / push-skip asymmetry survives.
  • tests/test_desktop_packaging.py: new test pins that the uv lock --check step exists, runs with set -euo pipefail, precedes both "Sync dependencies" and "Test" by step order, and that its failure text names the cause ("stale") and the remedy (uv lock) with an ::error annotation.
  • All assertions parse the workflows with the strict no-duplicate-keys loader (the PR fix(release): repair release.yml on main — glued comment made the workflow unparseable #484 discipline).

Gates

  • uv run pytest -q — 4245 passed, 3 skipped
  • uv run ruff check tests — clean
  • uv run mypy — clean (332 files)
  • Both workflows strict-parsed for duplicate keys; every run: script passes bash -n; the preflight script was dry-run locally under five configuration scenarios (missing-everything, today's real state, sonar-only-ready, fully-configured × push/dispatch)

The tokens were created on 2026-08-18, the preflight's single
'configured' flag flipped true, both scans ran for the first time --
and both failed on ORG-level prerequisites no repository secret can
carry: SonarQube Cloud's mandatory sonar.organization (still commented
out in sonar-project.properties) and the Snyk organization (the server
answered every scan with a 422). Every push to main went red for
reasons the repository could not fix -- exactly the permanently-red
main this preflight exists to prevent.

Readiness is now measured PER SCANNER and beyond token presence:
preflight outputs sonar_ready (token + active sonar.organization in
sonar-project.properties) and snyk_ready (token + the SNYK_ORG
repository variable, passed as --org), each job gated on its own
output, so each scan lights up the moment its missing piece appears.
Dispatch still fails loudly; push/schedule skip with a run summary
naming what to set and where. The snyk monitor --org placeholder
becomes that variable wiring.
The 0.10.0 release failed five steps after the mistake: #422 bumped
seven pyproject.toml files and not the tracked uv.lock; 'uv sync'
silently re-locked the checkout, the stamp recorded a dirty tree, and
the reader was sent chasing 'artifact reports a dirty tree' through
keel/version.py and the stamp step instead of the lockfile -- after
lint, types, tests and a full build had all run and passed.

'The lockfile must already be current' now runs right after the
interpreter is in place and immediately before 'uv sync' (the first
step that can mutate the tree): uv lock --check re-resolves against
the manifests, exits non-zero WITHOUT writing, and the failure message
names the remedy -- run uv lock locally and commit the result with the
version bump. RELEASING.md step 1 now says the bump is pyproject.toml
AND uv lock.
@eaitbrahim eaitbrahim self-assigned this Aug 22, 2026
@eaitbrahim
eaitbrahim merged commit 53d5bb3 into main Aug 22, 2026
5 checks passed
@eaitbrahim
eaitbrahim deleted the ci-quality-and-lockcheck branch August 22, 2026 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant