Skip to content

ci: add CodeQL and ZAP baseline scanning - #525

Merged
JimWallace merged 5 commits into
mainfrom
claude/quirky-ride-2ef1fa
May 17, 2026
Merged

ci: add CodeQL and ZAP baseline scanning#525
JimWallace merged 5 commits into
mainfrom
claude/quirky-ride-2ef1fa

Conversation

@JimWallace

@JimWallace JimWallace commented May 16, 2026

Copy link
Copy Markdown
Owner

Summary

Adds two new CI security workflows. Trivy is already wired into docker-build.yml so it is intentionally not duplicated.

What was already present in .github/workflows/

  • docker-build.yml — builds the image, runs aquasecurity/trivy-action@v0.35.0 against the locally-built image on every PR + push to main (CRITICAL only, exit-code: 1, ignore-unfixed: true, table output, no SARIF upload).
  • swift-tests.yml, test-coverage.yml, jupyterlite.yml, release.yml — unrelated to security scanning.

What Dependabot is monitoring (no changes here)

Soft-launch posture

  • CodeQL JS/TS runs on push to main, PRs targeting main, and weekly (Mon 06:00 UTC) on ubuntu-latest. Skips Public/jupyterlite/** (generated). SARIF lands in the Security tab. No fail-on-severity yet. First-run baseline: zero alerts.
  • ZAP runs weekly (Mon 07:00 UTC) and on workflow_dispatch. No PR trigger — too slow and noisy to gate every PR. Brings up docker compose with AUTH_MODE=local + ENABLE_NON_SSO_AUTH_MODES=true, polls /health (60s timeout), runs zaproxy/action-baseline@v0.12.0 with -a, fail_action: false. Opens an issue on findings. Server logs uploaded as an artifact.
  • Existing Trivy step in docker-build.yml unchanged.

Known gap: no CodeQL Swift

Three configurations were attempted during this PR; all failed:

Attempt Result
ubuntu-latest matrix entry ❌ CodeQL refuses — "Swift analysis is only supported on macOS runner images"
macos-latest + swift build ❌ Fails compiling swift-configuration: Data.bytes only exists in swift-foundation, not Apple's Darwin Foundation
macos-latest + build-mode: none "Swift does not support the none build mode"

The Darwin-vs-swift-foundation API gap is structural and will keep biting us as transitive deps adopt swift-foundation-only APIs, even if we patch around the current swift-configuration issue. Shipping without Swift SAST for now. Revisit if/when CodeQL ships a Linux Swift extractor or Apple closes the Foundation gap.

Actions are pinned to @v4 for CodeQL action (Node 24)

github/codeql-action@v4 clears the node20 deprecation warning. v4 ships the same CodeQL bundle as v3 — the only difference is Node 24 runtime.

Verification done locally

  • YAML parses cleanly (Ruby yaml lib — actionlint/PyYAML not installed locally).
  • Both files have top-level permissions: blocks.
  • Neither file uses pull_request_target.
  • CodeQL analyze@v4 step has no if: guard.

Follow-up checklist

  • Triage the first weekly ZAP run; add .zap/rules.tsv ignores for CI-only false positives (Cookie Without Secure Flag will fire because CI runs HTTP).
  • Decide whether to add a separate container-scan.yml that complements docker-build.yml's Trivy step with HIGH-severity coverage, SARIF upload, and a weekly cadence.
  • After a few weeks of clean ZAP runs, decide whether to enable a PR trigger (likely off — slow) and/or switch from issue-creation to SARIF upload.
  • Revisit CodeQL Swift if CodeQL ships a Linux Swift extractor or Apple Foundation gains swift-foundation parity.
  • Branch protection rules are intentionally out of scope; revisit after baseline triage.

Test plan

  • CodeQL JS/TS workflow appears in the Actions tab and completes (verified — 0 alerts).
  • ZAP does not run on this PR (verified — schedule + dispatch only).
  • After merge, manually trigger ZAP via workflow_dispatch once to verify the docker-compose-up / health-wait / scan / teardown loop end-to-end.

🤖 Generated with Claude Code

Soft-launch posture: both scanners report findings but do not fail
the build. Trivy is already wired into docker-build.yml and is left
untouched.

- codeql.yml: Swift + JS/TS matrix on push/PR/weekly. Swift runs in
  the swift:6.3-jammy container (matches swift-tests.yml). JS/TS run
  excludes Public/jupyterlite/** (generated output). SARIF uploads
  to the Security tab via the analyze action.

- zap-baseline.yml: weekly + workflow_dispatch only (no PR trigger).
  Brings up docker compose with AUTH_MODE=local +
  ENABLE_NON_SSO_AUTH_MODES=true, waits on /health, runs
  zaproxy/action-baseline@v0.12.0 with -a (alpha passive rules),
  fail_action: false. Opens issues on findings. Server logs uploaded
  as an artifact for debugging.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-advanced-security

Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

JimWallace and others added 4 commits May 16, 2026 16:03
CodeQL Swift analysis is macOS-only as of CodeQL 2.25 — the unified
ubuntu-latest matrix in the previous commit fails fast with "Swift
analysis is only supported on macOS runner images." Split into two
files so the JS/TS half can keep running cheaply on Linux while
Swift gets the macOS runner it requires.

- codeql-js.yml: ubuntu-latest, every push/PR/weekly. Same posture
  as before.
- codeql-swift.yml: macos-latest, paths-filtered to Sources/**,
  Tests/**, Package.swift, Package.resolved, and the workflow itself.
  Weekly cron still runs full scan. Uses swift-actions/setup-swift@v2
  with swift-version 6.3 to match swift-tools-version in Package.swift.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
swift-actions/setup-swift@v2 doesn't have Swift 6.3 in its version
catalog — first run on the macOS CodeQL job failed with "Version
'6.3' is not available". SwiftyLab/setup-swift is the actively
maintained fork; v1.14.0 (Apr 2026) supports the current Swift line.
Package.swift pins swift-tools-version:6.3 so we can't fall back to
an older toolchain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Swift build on macOS fails because a transitive SPM dependency
(swift-configuration) calls Data.bytes, an API that exists in
swift-foundation but not in Apple's Darwin Foundation. Rather than
yak-shave the toolchain, switch to CodeQL's buildless Swift extractor
(build-mode: none). Trade-off documented in the workflow comment:
buildless is experimental and may produce less complete results than
tracing extraction, but it works without a build and runs in a few
minutes instead of 17+.

Also bump github/codeql-action from v3 to v4 in both workflows to
clear the Node 20 deprecation warning. v4 ships the same CodeQL
bundle as v3; the difference is Node 24 runtime.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tried three configurations on the original PR thread:

  1. ubuntu-latest matrix entry      → CodeQL refuses ("Swift analysis
                                       is only supported on macOS")
  2. macos-latest + swift build      → fails compiling swift-configuration:
                                       Data.bytes only exists in
                                       swift-foundation, not Apple's
                                       Darwin Foundation
  3. macos-latest + build-mode: none → "Swift does not support the none
                                       build mode"

The Darwin-vs-swift-foundation gap is structural and will keep biting
us as transitive deps adopt swift-foundation-only APIs, even if we
patch around the current swift-configuration issue. Ship without
Swift SAST for now; revisit if/when CodeQL ships a Linux Swift
extractor or Apple closes the Foundation gap.

JS/TS CodeQL on Linux continues to run on every PR; ZAP baseline
runs weekly. Trivy in docker-build.yml is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@JimWallace
JimWallace merged commit f8b8986 into main May 17, 2026
10 checks passed
@JimWallace
JimWallace deleted the claude/quirky-ride-2ef1fa branch May 17, 2026 00:13
JimWallace added a commit that referenced this pull request May 17, 2026
The runner service had a bare `environment:` key with no value, which
older docker compose versions tolerated as a no-op but newer versions
reject with:

  validating docker-compose.yml: services.runner.environment must be a mapping

This surfaced when the new ZAP baseline workflow (added in #525) tried
to run `docker compose up` in CI. The runner doesn't actually need any
env vars passed via compose — RUNNER_WORKER_ID and RUNNER_MAX_JOBS are
referenced in the `command:` block and substituted from the shell at
parse time. Remove the empty key entirely.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants