Skip to content

Add more nuanced unmet requirements for external PRs#17

Merged
tidy-dev merged 8 commits into
mainfrom
tidy-dev/smart-pr-triage
Apr 29, 2026
Merged

Add more nuanced unmet requirements for external PRs#17
tidy-dev merged 8 commits into
mainfrom
tidy-dev/smart-pr-triage

Conversation

@tidy-dev
Copy link
Copy Markdown
Contributor

@tidy-dev tidy-dev commented Apr 24, 2026

Summary

Adds opt-in extended PR screening to the triage-pr-requirements.yml reusable workflow. When enable_pr_screening: true is set, four new behaviors activate for external PRs:

Behavior What happens Default thresholds
Zero-change close PRs with 0 file changes are instantly closed
Resubmission detection Same-author PRs matching recently closed unmet-requirements PRs (file overlap, 30-day window) are closed ≥2 overlapping files
Small-fix fast-track Small, well-described PRs skip unmet-requirements and go straight to review ≤30 LOC, ≤3 files, ≥100 char body
Large PR accelerated close Large unsolicited PRs with no team reviews get a shorter close deadline >200 LOC, >5 files, 3 days

All existing behavior is unchanged when enable_pr_screening is not set (default: false).

New Inputs

Input Type Default Description
enable_pr_screening boolean false Master toggle for all screening behaviors
fast_track_max_loc number 30 Max lines changed for fast-track
fast_track_max_files number 3 Max files changed for fast-track
fast_track_min_body number 100 Min body length (normalized) for fast-track
fast_track_label string ready-for-review Label applied to fast-tracked PRs
large_pr_min_loc number 200 Min lines to trigger large PR path
large_pr_min_files number 5 Min files to trigger large PR path
large_pr_days_until_close number 3 Days before closing large PRs

Pipeline Order

spam (zero-change) → resubmission → requirements check → fast-track → large PR → standard unmet-requirements

First match wins — subsequent steps are skipped.

Comment Improvements

  • All unmet-requirements comments now show the specific reason upfront with full requirements in a collapsed <details> block
  • Large PR comments include LOC/file stats and direct guidance to open an issue first
  • Fast-tracked PRs get an encouraging message with a note about linking issues

Scheduled Close Enhancements

The close-unmet-requirements scheduled job now checks PR size at close time:

  • Standard PRs: closed after days_until_close (default 7 days)
  • Large PRs (when screening enabled): closed after large_pr_days_until_close (default 3 days)

This ensures the accelerated timeline promised in the comment is actually enforced.

Security & Robustness

  • Review bypass protection: Large PR detection filters reviews by authorAssociation (MEMBER/OWNER/COLLABORATOR only) — external drive-by reviews can't defeat the accelerated close
  • Self-hosted runner safety: Resubmission detection uses $RUNNER_TEMP instead of /tmp to avoid cross-job interference
  • Missing label guard: Fast-track label application uses || true so a missing label doesn't abort the job
  • Configurable fast-track label: New fast_track_label input (default: ready-for-review) so consumers can customize

Testing

Tested all 4 paths in tidy-dev/playground with a workflow referencing this branch:

Test PR Scenario Result
#12 Zero-change (empty commit) ✅ Instantly closed
#13 Large unsolicited (248 LOC, 8 files) unmet-requirements + 3-day message
#14 Small fix (1 LOC, 1 file, good body) ready-for-review label applied
#15 Standard unmet (medium, no help-wanted) unmet-requirements + 7-day message

Files Changed

  • .github/workflows/triage-pr-requirements.yml — All new logic + scheduler enhancement
  • example-usage/triage-prs.yml — Commented example showing opt-in usage
  • README.md — Updated workflow description, labels table, and configurable inputs

tidy-dev and others added 8 commits April 24, 2026 15:01
Add three new opt-in behaviors to triage-pr-requirements.yml, all gated
behind enable_smart_triage: true (existing behavior unchanged by default):

- Instant-close spam PRs: zero changed files, empty body (< 50 chars
  after stripping HTML comments), or all files in configurable irrelevant
  paths
- Detect same-author resubmissions: finds closed PRs from the same
  author with unmet-requirements in the last 30 days with overlapping
  files, closes immediately with reference to the original
- Fast-track small fixes: routes PRs with ≤ 30 LOC, ≤ 3 files, and a
  well-written description (≥ 100 chars) to needs-triage instead of
  auto-closing

Based on analysis of 91 external PRs across desktop/desktop and cli/cli
that received the unmet-requirements label. The heuristics would have
correctly triaged ~80% of those PRs with high confidence.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A short body alone is not spam — contributors may update the body after
opening. The existing requirements check already handles short bodies
with a 7-day grace period. Spam detection now only triggers on
unambiguous signals: zero changed files or all files in irrelevant paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add accelerated close timer (default 3 days) for large PRs (>200 LOC,
>5 files) with no help-wanted issue and no team reviews. Data showed
100% close rate on these across 16 PRs in the analysis.

Rework unmet-requirements comments to lead with the specific reason the
PR was flagged and collapse the full requirements list into a <details>
block for clarity.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The irrelevant-paths check only caught 2 of 91 PRs and both were
already covered by the zero-files check or standard requirements flow.
Not worth the per-repo configuration burden.

Simplify spam detection to a single unambiguous signal: zero changed
files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the 'smart_triage' branding (which could imply AI) with names
that describe what each input controls:

  enable_smart_triage       → enable_pr_screening
  smart_triage_max_loc      → fast_track_max_loc
  smart_triage_max_files    → fast_track_max_files
  smart_triage_min_body     → fast_track_min_body
  smart_triage_large_pr_loc → large_pr_min_loc
  smart_triage_large_pr_files → large_pr_min_files
  smart_triage_large_pr_days  → large_pr_days_until_close

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Small, well-described PRs that pass the fast-track heuristic should go
directly to review rather than sitting in a triage queue.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Update workflow description to mention opt-in screening
- Add ready-for-review label to pr-requirements in labels table
- Add enable_pr_screening to key configurable inputs table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Scheduler now checks PR size at close time, applying
  large_pr_days_until_close for large PRs instead of standard deadline
- Filter reviews by authorAssociation (MEMBER/OWNER/COLLABORATOR)
  so external reviews can't bypass large-PR detection
- Use $RUNNER_TEMP instead of /tmp for resubmission temp files
  (safe on self-hosted runners)
- Add fast_track_label input (default: ready-for-review) so consumers
  can customize the label, with || true guard for missing labels

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tidy-dev tidy-dev changed the title Add smart triage for external PRs Add smarter unmet requirements for external PRs Apr 28, 2026
@tidy-dev tidy-dev marked this pull request as ready for review April 28, 2026 17:11
Copilot AI review requested due to automatic review settings April 28, 2026 17:11
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in “extended PR screening” mode to the reusable triage-pr-requirements.yml workflow to better handle external PRs (auto-close empty PRs, detect resubmissions, fast-track small fixes, and accelerate closure of large unsolicited PRs), while keeping existing behavior unchanged unless enabled.

Changes:

  • Introduces new workflow_call inputs and screening steps (zero-change close, resubmission detection, small-fix fast-track, large PR path) in triage-pr-requirements.yml.
  • Updates unmet-requirements/large-PR comment formatting to include a reason-first message with full requirements in a <details> block.
  • Documents the opt-in feature in README and adds a commented example configuration.
Show a summary per file
File Description
.github/workflows/triage-pr-requirements.yml Adds opt-in screening logic, new inputs, updated commenting, and scheduled close adjustments.
example-usage/triage-prs.yml Adds commented example showing how to enable screening via with: inputs.
README.md Documents the new screening capability and updates labels/inputs tables accordingly.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

.github/workflows/triage-pr-requirements.yml:393

  • This comment body is indented inside the quoted string, which will render as a Markdown code block and likely break the intended formatting (including the new <details> block). Use a heredoc/--body-file (without leading indentation) so the reasons and <details> collapse render properly.
          gh pr comment "$NUMBER" --body "Thanks for your pull request! Unfortunately, it doesn't meet the requirements for review:

          $REASONS

          **Please update your PR to address the above.** This PR will be automatically closed in **$DAYS days** if these requirements are not met.

          <details>
          <summary>Full contribution requirements</summary>

          1. Include a detailed description of what this PR does
          2. Link to an issue with the \`${{ inputs.help_wanted_label || 'help wanted' }}\` label (use \`Fixes #123\` or \`Closes #123\`)
          </details>"
  • Files reviewed: 3/3 changed files
  • Comments generated: 2

Comment thread .github/workflows/triage-pr-requirements.yml
Comment thread .github/workflows/triage-pr-requirements.yml
@tidy-dev tidy-dev enabled auto-merge April 28, 2026 17:20
@tidy-dev tidy-dev disabled auto-merge April 28, 2026 17:20
@tidy-dev tidy-dev changed the title Add smarter unmet requirements for external PRs Add more nuanced unmet requirements for external PRs Apr 28, 2026
@tidy-dev tidy-dev merged commit 48d5222 into main Apr 29, 2026
6 checks passed
tidy-dev added a commit to cli/cli that referenced this pull request Apr 29, 2026
Opts in to the new PR screening features in the shared triage workflow:
- Instantly closes PRs with zero file changes
- Detects same-author resubmissions of recently closed PRs
- Fast-tracks small, well-described fixes to ready-for-review
- Accelerates closure of large unsolicited PRs (3 days vs 7)

Depends on desktop/gh-cli-and-desktop-shared-workflows#17

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
pull Bot pushed a commit to h308427/desktop that referenced this pull request May 4, 2026
Opts in to the new PR screening features in the shared triage workflow:
- Instantly closes PRs with zero file changes
- Detects same-author resubmissions of recently closed PRs
- Fast-tracks small, well-described fixes to ready-for-review
- Accelerates closure of large unsolicited PRs (3 days vs 7)

Depends on desktop/gh-cli-and-desktop-shared-workflows#17

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants