Skip to content

Use shared workflows for change detection, prek, and gate job #2104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
KRRT7 merged 2 commits into from
Apr 23, 2026
Merged

Use shared workflows for change detection, prek, and gate job #2104

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c4bc18e
ci: use shared workflows for change detection, prek, and gate job
KRRT7 Apr 23, 2026
c613fda
Merge main, resolve prek conflict (keep shared workflow)
KRRT7 Apr 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 27 additions & 119 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,87 +83,26 @@ jobs:
# ---------------------------------------------------------------------------
# Change detection — decides which downstream jobs actually run.
# On push/workflow_dispatch every flag is true so all jobs execute.
# On pull_request we diff against the merge base (same approach as astral-sh/ruff).
# On pull_request we diff against the merge base.
# ---------------------------------------------------------------------------
determine-changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
unit_tests: ${{ github.event_name != 'pull_request' || steps.check.outputs.unit_tests == 'true' }}
type_check: ${{ github.event_name != 'pull_request' || steps.check.outputs.type_check == 'true' }}
e2e: ${{ github.event_name != 'pull_request' || steps.check.outputs.e2e == 'true' }}
e2e_js: ${{ github.event_name != 'pull_request' || steps.check.outputs.e2e_js == 'true' }}
e2e_java: ${{ github.event_name != 'pull_request' || steps.check.outputs.e2e_java == 'true' }}
steps:
- uses: actions/checkout@v6
if: github.event_name == 'pull_request'
with:
fetch-depth: 0

- name: Determine merge base
if: github.event_name == 'pull_request'
id: merge_base
run: |
sha=$(git merge-base HEAD "origin/${{ github.event.pull_request.base.ref }}")
echo "sha=${sha}" >> "$GITHUB_OUTPUT"

- name: Check changed paths
if: github.event_name == 'pull_request'
id: check
run: |
check_paths() {
local name="$1"; shift
if ! git diff --quiet "$MERGE_BASE...HEAD" -- "$@" 2>/dev/null; then
echo "${name}=true" >> "$GITHUB_OUTPUT"
else
echo "${name}=false" >> "$GITHUB_OUTPUT"
fi
}

# Unit tests: code + test infra + packages + build config
check_paths unit_tests \
'codeflash/' 'codeflash-benchmark/' \
'tests/' 'packages/' 'pyproject.toml' 'uv.lock'

# Type checking: code + build config + mypy config
check_paths type_check \
'codeflash/' 'pyproject.toml' 'uv.lock' 'mypy_allowlist.txt'

# E2E tests: Python pipeline + tests + build config (excludes java/ and javascript/)
check_paths e2e \
'codeflash/*.py' \
'codeflash/api/' 'codeflash/benchmarking/' 'codeflash/cli_cmds/' \
'codeflash/code_utils/' 'codeflash/discovery/' 'codeflash/github/' \
'codeflash/languages/python/' 'codeflash/languages/*.py' \
'codeflash/lsp/' 'codeflash/models/' 'codeflash/optimization/' \
'codeflash/picklepatch/' 'codeflash/result/' 'codeflash/setup/' \
'codeflash/telemetry/' 'codeflash/tracing/' 'codeflash/verification/' \
'tests/' 'pyproject.toml' 'uv.lock'

# JS E2E tests: JS language support + shared pipeline + packages + test fixtures
check_paths e2e_js \
'codeflash/languages/javascript/' 'codeflash/languages/base.py' \
'codeflash/languages/registry.py' 'codeflash/optimization/' \
'codeflash/verification/' 'packages/' \
'code_to_optimize/js/' 'tests/scripts/end_to_end_test_js*'

# Java E2E tests: Java language support + shared pipeline + runtime
check_paths e2e_java \
'codeflash/languages/java/' 'codeflash/languages/base.py' \
'codeflash/languages/registry.py' 'codeflash/optimization/' \
'codeflash/verification/' 'codeflash-java-runtime/' \
'code_to_optimize/java/' 'tests/scripts/end_to_end_test_java*' \
'tests/test_languages/fixtures/java_tracer_e2e/'
env:
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
uses: codeflash-ai/github-workflows/.github/workflows/determine-changes.yml@main
with:
path-filters: |
{
"unit_tests": ["codeflash/", "codeflash-benchmark/", "tests/", "packages/", "pyproject.toml", "uv.lock"],
"type_check": ["codeflash/", "pyproject.toml", "uv.lock", "mypy_allowlist.txt"],
"e2e": ["codeflash/*.py", "codeflash/api/", "codeflash/benchmarking/", "codeflash/cli_cmds/", "codeflash/code_utils/", "codeflash/discovery/", "codeflash/github/", "codeflash/languages/python/", "codeflash/languages/*.py", "codeflash/lsp/", "codeflash/models/", "codeflash/optimization/", "codeflash/picklepatch/", "codeflash/result/", "codeflash/setup/", "codeflash/telemetry/", "codeflash/tracing/", "codeflash/verification/", "tests/", "pyproject.toml", "uv.lock"],
"e2e_js": ["codeflash/languages/javascript/", "codeflash/languages/base.py", "codeflash/languages/registry.py", "codeflash/optimization/", "codeflash/verification/", "packages/", "code_to_optimize/js/", "tests/scripts/end_to_end_test_js*"],
"e2e_java": ["codeflash/languages/java/", "codeflash/languages/base.py", "codeflash/languages/registry.py", "codeflash/optimization/", "codeflash/verification/", "codeflash-java-runtime/", "code_to_optimize/java/", "tests/scripts/end_to_end_test_java*", "tests/test_languages/fixtures/java_tracer_e2e/"]
}

# ---------------------------------------------------------------------------
# Unit tests — 6 Linux + 1 Windows matrix
# ---------------------------------------------------------------------------
unit-tests:
needs: determine-changes
if: needs.determine-changes.outputs.unit_tests == 'true'
if: fromJSON(needs.determine-changes.outputs.flags).unit_tests == 'true'
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -214,7 +153,7 @@ jobs:
# ---------------------------------------------------------------------------
coverage:
needs: determine-changes
if: needs.determine-changes.outputs.unit_tests == 'true'
if: fromJSON(needs.determine-changes.outputs.flags).unit_tests == 'true'
runs-on: ubuntu-latest
env:
PYTHONIOENCODING: utf-8
Expand Down Expand Up @@ -252,7 +191,7 @@ jobs:
# ---------------------------------------------------------------------------
type-check:
needs: determine-changes
if: needs.determine-changes.outputs.type_check == 'true'
if: fromJSON(needs.determine-changes.outputs.flags).type_check == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
Expand All @@ -278,39 +217,15 @@ jobs:
needs: determine-changes
if: >-
github.event_name == 'pull_request'
&& (needs.determine-changes.outputs.e2e == 'true'
|| needs.determine-changes.outputs.e2e_js == 'true')
runs-on: ubuntu-latest
&& (fromJSON(needs.determine-changes.outputs.flags).e2e == 'true'
|| fromJSON(needs.determine-changes.outputs.flags).e2e_js == 'true')
uses: codeflash-ai/github-workflows/.github/workflows/prek-lint.yml@main
permissions:
contents: write
steps:
- uses: actions/checkout@v6
with:
ref: ${{ github.head_ref }}
fetch-depth: 0
- uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true

- name: Auto-fix formatting
run: |
uv run ruff check --fix . || true
uv run ruff format .
# uv-dynamic-versioning rewrites version.py on every `uv run` — discard those changes
git checkout HEAD -- codeflash/version.py codeflash-benchmark/codeflash_benchmark/version.py 2>/dev/null || true

- name: Commit and push fixes
run: |
git diff --quiet && exit 0
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -u
git commit -m "style: auto-format with ruff"
git push

- uses: j178/prek-action@v2
with:
extra-args: '--from-ref origin/${{ github.base_ref }} --to-ref HEAD'
with:
auto-fix: true
checkout-ref: ${{ github.head_ref }}
restore-paths: "codeflash/version.py codeflash-benchmark/codeflash_benchmark/version.py"

# ---------------------------------------------------------------------------
# E2E tests — only on pull_request and workflow_dispatch (not push to main)
Expand All @@ -320,7 +235,7 @@ jobs:
e2e-python:
needs: determine-changes
if: >-
needs.determine-changes.outputs.e2e == 'true'
fromJSON(needs.determine-changes.outputs.flags).e2e == 'true'
&& github.event_name != 'push'
&& github.actor != 'dependabot[bot]'
strategy:
Expand Down Expand Up @@ -424,7 +339,7 @@ jobs:
e2e-js:
needs: determine-changes
if: >-
needs.determine-changes.outputs.e2e_js == 'true'
fromJSON(needs.determine-changes.outputs.flags).e2e_js == 'true'
&& github.event_name != 'push'
&& github.actor != 'dependabot[bot]'
strategy:
Expand Down Expand Up @@ -508,7 +423,7 @@ jobs:
e2e-java:
needs: determine-changes
if: >-
needs.determine-changes.outputs.e2e_java == 'true'
fromJSON(needs.determine-changes.outputs.flags).e2e_java == 'true'
&& github.event_name != 'push'
&& github.actor != 'dependabot[bot]'
strategy:
Expand Down Expand Up @@ -617,13 +532,6 @@ jobs:
- e2e-java
runs-on: ubuntu-latest
steps:
- name: Verify all required jobs passed
run: |
failing=$(echo "$NEEDS_JSON" | jq -r 'to_entries[] | select(.value.result != "success" and .value.result != "skipped") | "\(.key): \(.value.result)"')
if [ -n "$failing" ]; then
echo "Required jobs failed or were cancelled:"
echo "$failing"
exit 1
fi
env:
NEEDS_JSON: ${{ toJSON(needs) }}
- uses: codeflash-ai/github-workflows/.github/actions/required-checks-gate@main
with:
needs-json: ${{ toJSON(needs) }}