diff --git a/.github/RELEASE.md b/.github/RELEASE.md index b499f9f..0aaaa9f 100644 --- a/.github/RELEASE.md +++ b/.github/RELEASE.md @@ -1,110 +1,134 @@ # Release Process -## Automated Release Workflow +## Release Workflow ### 1. Create PR with Changes Create a PR targeting the `main` branch with your changes. -### 2. Add Version Bump Label -Add ONE OR MORE of these labels to indicate version change: -- `bump:major` - Breaking changes (2025.8.11 → 2026.0.0) -- `bump:minor` - New features (2025.8.11 → 2025.9.0) -- `bump:patch` - Bug fixes (2025.8.11 → 2025.8.12) -- `bump:stable` - Remove pre-release suffix -- `bump:alpha`, `bump:beta`, `bump:rc`, `bump:post`, `bump:dev` - Pre-release +**Tip**: Write a clear PR description - it will become your GitHub Release notes! -The version-bump workflow will automatically: -- Update `pyproject.toml` and `uv.lock` -- Commit changes to your PR -- Enable auto-merge (PR will merge when all checks pass) +### 2. Bump Version Manually +On your local branch, bump the version using `uv`: + +```bash +# Choose one or more bump types: +uv version --bump patch # Bug fixes: 2025.8.11 → 2025.8.12 +uv version --bump minor # New features: 2025.8.11 → 2025.9.0 +uv version --bump major # Breaking changes: 2025.8.11 → 2026.0.0 + +# Pre-release versions: +uv version --bump rc # Release candidate: 2025.9.0 → 2025.9.0rc1 +uv version --bump beta # Beta: 2025.9.0 → 2025.9.0b1 +uv version --bump alpha # Alpha: 2025.9.0 → 2025.9.0a1 +uv version --bump stable # Remove suffix: 2025.9.0rc1 → 2025.9.0 + +# Multiple bumps (applied sequentially): +uv version --bump minor --bump rc # 2025.8.11 → 2025.9.0rc1 +``` + +Commit and push the version change: +```bash +git add pyproject.toml uv.lock +git commit -m "Bump version to $(uv version --short)" +git push +``` ### 3. Add Release Type Label -Add ONE of these labels: +Add ONE of these labels to your PR: - `release` - Publish to PyPI (production) - `test-release` - Publish to TestPyPI (testing) -### 4. Wait for Checks -Required checks must pass: -- ✅ Tests (`test.yml`) -- ✅ Version bump (`version-bump.yml`, if bump labels present) - -### 5. Wait for Auto-Merge -The version-bump workflow enables auto-merge, so the PR will automatically merge once all required checks pass. +### 4. Wait for Tests +Wait for the test workflow to pass. -You can also manually merge if needed via the GitHub UI. +### 5. Merge PR +Once tests pass, merge the PR via GitHub UI. ### 6. Automatic Publishing -After merge: -- Package is built and tested -- Published to PyPI or TestPyPI -- Git tag created (e.g., `v2025.9.0`) -- GitHub Release created with: +After merge, the release workflow automatically: +- Builds package +- Runs tests +- Publishes to PyPI or TestPyPI (based on label) +- Creates git tag (e.g., `v2025.9.0`) +- Creates GitHub Release with: - PR title and body as release notes - Installation instructions (`pipx install quarto-batch-convert`) - Links to package, PR, and documentation -**Tip**: Write a clear PR description - it will become your release notes! +--- -## Example: Patch Release to PyPI +## Examples -1. Create PR: "Fix bug in file collection" -2. Add labels: `bump:patch`, `release` -3. Version bump workflow: `2025.8.11` → `2025.8.12` -4. Merge PR -5. Release workflow publishes to PyPI and creates tag `v2025.8.12` +### Example 1: Patch Release to PyPI -## Example: Test Release +```bash +# 1. Make your bug fix changes +# 2. Bump version +uv version --bump patch +git add pyproject.toml uv.lock +git commit -m "Bump version to $(uv version --short)" +git push -1. Create PR: "Test new feature" -2. Add labels: `bump:minor`, `bump:beta`, `test-release` -3. Version bump workflow: `2025.8.11` → `2025.9.0b1` -4. Merge PR -5. Release workflow publishes to TestPyPI (not production PyPI) +# 3. Add 'release' label to PR in GitHub UI +# 4. Merge PR +# 5. Release workflow publishes to PyPI +``` -## Multiple Version Bumps +### Example 2: Test Release with RC -You can apply multiple bump labels simultaneously. They will be applied sequentially in this order: major, minor, patch, stable, alpha, beta, rc, post, dev. +```bash +# 1. Make your feature changes +# 2. Bump version +uv version --bump minor --bump rc +git add pyproject.toml uv.lock +git commit -m "Bump version to $(uv version --short)" +git push -Example: -- Labels: `bump:minor`, `bump:rc` -- Result: `2025.8.11` → `2025.9.0rc1` +# 3. Add 'test-release' label to PR in GitHub UI +# 4. Merge PR +# 5. Release workflow publishes to TestPyPI (not production) +``` + +--- ## Workflow Details -### Version Bump Workflow (`version-bump.yml`) -- **Trigger**: When bump labels are added to a PR targeting main -- **Actions**: - - Applies version bumps using `uv version --bump ` - - Commits updated `pyproject.toml` and `uv.lock` to PR branch - - Posts comment with version change - - Acts as required status check +### Test Workflow (`test.yml`) +- **Trigger**: On pull requests to main +- **Actions**: Runs pytest test suite +- **Concurrency**: Cancels previous test runs when new commits pushed ### Release Workflow (`release.yml`) -- **Trigger**: When PR with release label is merged to main +- **Trigger**: When PR with `release` or `test-release` label is merged to main - **Actions**: - - Builds package - - Runs tests + - Extracts PR number from merge commit + - Builds package and runs tests - Publishes to PyPI or TestPyPI based on label - Creates git tag matching version - - Creates GitHub Release - - Test releases marked as pre-release + - Creates GitHub Release with PR content as notes + - Marks test releases as pre-release -## Troubleshooting +--- -### Version bump workflow doesn't run -- Ensure at least one `bump:*` label is applied -- Check that PR targets the `main` branch -- Verify workflow file is present in `.github/workflows/version-bump.yml` +## Troubleshooting ### Release workflow doesn't run -- Ensure PR was merged (not closed) +- Ensure PR was **merged** (not closed) - Ensure either `release` or `test-release` label is applied -- Check workflow logs for errors +- Check workflow logs in Actions tab for errors ### PyPI publish fails - Verify PyPI Trusted Publishing is configured correctly - Check that GitHub environments (`pypi`, `testpypi`) are set up - Ensure version doesn't already exist on PyPI +- Check that package builds successfully (run `uv build` locally) + +### Wrong version published +- Check that you committed the version bump to the PR +- Run `uv version --short` to verify current version +- Ensure `pyproject.toml` and `uv.lock` are both committed + +--- ## Configuration Requirements @@ -120,7 +144,28 @@ Configure at https://pypi.org/manage/account/publishing/: - Environment: `pypi` (and separately for `testpypi`) ### Branch Protection -Main branch should have: +Main branch protection recommended settings: - Require pull request before merging -- Require status checks: `test`, `bump-version` +- Require status check: `test` - Require branches to be up to date + +--- + +## Notes + +### Why Manual Version Bumping? + +The automated version-bump workflow was archived due to GitHub Actions limitations: +- Commits from `GITHUB_TOKEN` don't trigger other workflows +- This prevented tests from running after automated version bumps +- Caused auto-merge to block indefinitely waiting for test status + +See `.github/workflows/archive/version-bump.yml` for details and potential future solutions using GitHub Apps. + +### Version Bump Labels (Deprecated) + +The following labels exist but are **not used** in the current manual workflow: +- `bump:major`, `bump:minor`, `bump:patch` +- `bump:stable`, `bump:alpha`, `bump:beta`, `bump:rc`, `bump:post`, `bump:dev` + +These may be used in the future if automated version bumping is re-implemented with a GitHub App. diff --git a/.github/workflows/version-bump.yml b/.github/workflows/archive/version-bump.yml similarity index 66% rename from .github/workflows/version-bump.yml rename to .github/workflows/archive/version-bump.yml index 9cf6b39..b10494d 100644 --- a/.github/workflows/version-bump.yml +++ b/.github/workflows/archive/version-bump.yml @@ -1,3 +1,50 @@ +# ARCHIVED - Version Bump Workflow +# +# This workflow was archived because of fundamental GitHub Actions limitations: +# +# Problem 1: GITHUB_TOKEN commits don't trigger other workflows +# - Commits made with secrets.GITHUB_TOKEN deliberately don't trigger workflows +# (GitHub's safeguard against infinite loops) +# - After version bump commits, test workflow never runs +# - Branch protection waits for 'test' check that never comes +# - Auto-merge blocked indefinitely +# +# Problem 2: workflow_run doesn't report status to PR +# - Tried using workflow_run to trigger tests after version bump +# - workflow_run executes in main branch context, not PR context +# - Status checks created are for wrong commit SHA +# - Branch protection on PR can't see these checks +# +# Problem 3: Multiple workflow runs with labels +# - Adding multiple labels (e.g., bump:rc + test-release) triggers separate runs +# - Even with concurrency control, cancellation happens after work starts +# - Results in duplicate version bump commits before cancellation +# +# Attempted Solutions That Failed: +# - [skip ci] in commit message (doesn't work, workflow never triggers anyway) +# - workflow_run trigger (runs in wrong context, status not reported to PR) +# - Concurrency control (helps but doesn't prevent all duplicates) +# - Manual status check via API (hacky, doesn't actually re-test) +# +# Working Solutions (not implemented here): +# Option A: Use GitHub App or PAT instead of GITHUB_TOKEN +# - Commits from app/PAT DO trigger workflows +# - Requires creating GitHub App or storing PAT as secret +# - Industry standard solution (used by Renovate, Dependabot, etc.) +# +# Option B: Manual version bumping (CHOSEN) +# - Run `uv version --bump ` manually +# - Commit to PR branch +# - Simple, reliable, no workflow complexity +# - Trade-off: one manual step for simplicity +# +# This workflow is preserved for future reference if we decide to implement +# Option A (GitHub App/PAT) for full automation. +# +# Last used: 2025-10-21 +# Archived by: Claude Code +# Related: _todo/proposal/fix-workflow-status-reporting.md + name: Version Bump on: diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index ad0c882..c06192b 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -3,40 +3,20 @@ name: Run Tests on: pull_request: branches: [ main ] - workflow_run: - workflows: ["Version Bump"] - types: [completed] + +concurrency: + group: test-${{ github.event.pull_request.number }} + cancel-in-progress: true permissions: contents: read jobs: test: - # Only run on PRs or successful version bump completions - if: | - github.event_name == 'pull_request' || - (github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success') runs-on: ubuntu-latest steps: - - name: Get PR details - if: github.event_name == 'workflow_run' - id: pr - env: - GH_TOKEN: ${{ github.token }} - run: | - # Get PR number from the workflow run - PR_NUMBER=$(gh api repos/${{ github.repository }}/pulls \ - --jq ".[] | select(.head.sha == \"${{ github.event.workflow_run.head_sha }}\") | .number" \ - | head -1) - - echo "number=$PR_NUMBER" >> $GITHUB_OUTPUT - echo "PR number: $PR_NUMBER" - - - name: Checkout PR branch - uses: actions/checkout@v4 - with: - ref: ${{ github.event_name == 'workflow_run' && github.event.workflow_run.head_sha || github.event.pull_request.head.sha }} + - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 diff --git a/_todo/proposal/fix-workflow-status-reporting.md b/_todo/proposal/fix-workflow-status-reporting.md new file mode 100644 index 0000000..a7be18b --- /dev/null +++ b/_todo/proposal/fix-workflow-status-reporting.md @@ -0,0 +1,225 @@ +# Fix Workflow Status Reporting and Auto-Merge + +## Problem Analysis + +### Current Issues + +1. **workflow_run doesn't report status to PR** + - Test workflow triggered via `workflow_run` runs in **main branch context** + - Status check not associated with PR commit + - Branch protection can't see the test result + - Auto-merge blocked indefinitely + +2. **Version bump triggers multiple times** + - Concurrency control working but cancellations happen after work starts + - First run (25s) commits version before being cancelled + - Second run commits again (different version) + - Wasteful and confusing + +3. **Root cause: GITHUB_TOKEN limitations** + - Commits made with `secrets.GITHUB_TOKEN` deliberately **don't trigger workflows** + - GitHub's safeguard against infinite workflow loops + - We're fighting against this design + +### Why workflow_run Doesn't Work + +From GitHub docs and common issues: +- `workflow_run` executes in the context of the **default branch** (main) +- It does NOT run "on behalf of" the PR +- Status checks created are for the wrong commit SHA +- Branch protection rules on PR don't see these checks + +## Best Practice Solutions + +### Option A: Use GitHub App or PAT (Recommended) + +**How it works:** +1. Create GitHub App or use Personal Access Token (PAT) +2. Store token as repository secret (e.g., `BOT_TOKEN`) +3. Use this token for version bump commits instead of `GITHUB_TOKEN` +4. Commits from this token **DO trigger workflows normally** +5. Test runs automatically, reports to PR, auto-merge works + +**Pros:** +- ✅ Clean, standard solution used by major projects +- ✅ No workflow_run hacks +- ✅ Status checks report correctly +- ✅ No manual API calls needed + +**Cons:** +- ⚠️ Requires creating GitHub App or PAT +- ⚠️ Additional secret to manage +- ⚠️ Security consideration (token has write access) + +**Implementation:** +```yaml +# version-bump.yml +- uses: actions/checkout@v4 + with: + token: ${{ secrets.BOT_TOKEN }} # Instead of GITHUB_TOKEN + +# Rest of workflow unchanged - commits will trigger test.yml normally +``` + +**Examples in the wild:** +- Renovate Bot (uses GitHub App) +- Dependabot (uses GitHub App) +- Many semantic-release setups (use PAT) + +### Option B: Manual Status Check via API + +**How it works:** +1. Version bump workflow commits changes +2. Version bump workflow manually creates passing "test" status via API +3. References the test run that already passed before version bump +4. Auto-merge sees the status and proceeds + +**Pros:** +- ✅ No additional tokens needed +- ✅ Uses existing test results + +**Cons:** +- ⚠️ Hacky - we're asserting tests pass without re-running them +- ⚠️ Tests don't actually run on bumped version +- ⚠️ Complex API calls in workflow + +**Implementation:** +```yaml +# version-bump.yml - after commit +- name: Create test status check + env: + GH_TOKEN: ${{ github.token }} + run: | + gh api repos/${{ github.repository }}/statuses/${{ github.event.pull_request.head.sha }} \ + -f state=success \ + -f context=test \ + -f description="Tests passed before version bump" +``` + +### Option C: Remove Auto-Merge (Simplest) + +**How it works:** +1. Version bump commits changes +2. User reviews the version bump +3. User manually merges (or clicks merge button) +4. Release workflow triggers + +**Pros:** +- ✅ Extremely simple +- ✅ No workflow complexity +- ✅ Human verification of version +- ✅ No token management + +**Cons:** +- ⚠️ Not fully automated +- ⚠️ Requires manual action + +**Implementation:** +- Remove auto-merge enable step from version-bump.yml +- User merges when ready + +### Option D: workflow_dispatch Chain + +**How it works:** +1. Version bump commits changes +2. Version bump triggers test via `workflow_dispatch` +3. Test workflow runs in PR context (because we pass the ref) +4. Status reported correctly + +**Pros:** +- ✅ No tokens needed +- ✅ Tests run on bumped version +- ✅ Status reports correctly + +**Cons:** +- ⚠️ More complex workflow coordination +- ⚠️ Need to pass PR context manually + +**Implementation:** +```yaml +# version-bump.yml - after commit +- name: Trigger test workflow + env: + GH_TOKEN: ${{ github.token }} + run: | + gh workflow run test.yml \ + -f pr_number=${{ github.event.pull_request.number }} \ + -f sha=${{ github.event.pull_request.head.sha }} + +# test.yml - add workflow_dispatch trigger +on: + pull_request: + workflow_dispatch: + inputs: + pr_number: + required: true + sha: + required: true +``` + +## Recommendation + +**Option A (GitHub App/PAT)** is the industry standard and cleanest solution, BUT requires setup. + +**Option C (Remove Auto-Merge)** is simplest if you don't mind one manual step. + +**For this project, I recommend:** +1. **Short-term: Option C** - Remove auto-merge, keep workflows simple +2. **Long-term: Option A** - If you want full automation, set up GitHub App + +## Proposed Implementation (Option C - Simple) + +### Changes: +1. **Remove auto-merge step** from version-bump.yml +2. **Remove workflow_run trigger** from test.yml (back to pull_request only) +3. **Update documentation** - user merges after reviewing version bump + +### Updated Flow: +1. Create PR with changes +2. Add `bump:*` labels +3. Version bump workflow commits new version +4. **User reviews version bump** +5. **User merges PR** (via UI or gh CLI) +6. Release workflow publishes package + +### Pros: +- ✅ Simple, no complex workflow orchestration +- ✅ Human verification of version before release +- ✅ No token management +- ✅ No status check issues +- ✅ Tests run normally on PR + +### Cons: +- One manual step (clicking merge button) + +## Alternative Recommendation (Option A - Fully Automated) + +If you want full automation: + +### Setup GitHub App (one-time): +1. Create GitHub App with repo write permissions +2. Install on repository +3. Store private key or create installation token +4. Add as repository secret: `BOT_TOKEN` + +### Changes: +1. version-bump.yml: Use `BOT_TOKEN` instead of `GITHUB_TOKEN` +2. test.yml: Back to simple `pull_request` trigger +3. Keep auto-merge step + +### Flow: +1. Add labels → version bump commits → test runs → auto-merge → release + +Fully automated, no manual steps. + +## Questions + +1. Do you want full automation (Option A - requires GitHub App setup)? +2. Or accept one manual step for simplicity (Option C - remove auto-merge)? +3. Or try the workflow_dispatch approach (Option D - no tokens, moderately complex)? + + \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index 1d92c50..7ed44b0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "quarto-batch-convert" -version = "2025.9.1rc1" +version = "2025.9.1rc5" description = "Converts multiple Jupyter notebooks to Quarto documents at once" readme = "README.md" license = "MIT" diff --git a/uv.lock b/uv.lock index 7b580bc..581f024 100644 --- a/uv.lock +++ b/uv.lock @@ -81,7 +81,7 @@ wheels = [ [[package]] name = "quarto-batch-convert" -version = "2025.9.1rc1" +version = "2025.9.1rc5" source = { editable = "." } dependencies = [ { name = "click" },