[prompt-analysis] Copilot PR Prompt Analysis - 2026-09-02

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-09-02

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 839 (83.9%) | Closed: 149 (14.9%) | Open: 12

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Bug Fix	900	754	83.8%
Feature Addition	84	74	88.1%
Refactoring	0	0	0%
Documentation	1	1	100.0%
Testing	6	4	66.7%
Other	9	6	66.7%

Note: Categorization uses simple keyword matching against title+body (first matching category wins, in order bug_fix → feature → refactor → docs → tests), so most prompts land in bug_fix since Copilot PR titles/bodies commonly include words like "fix"/"issue".

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 215.8 words
• Most common keywords: github, workflow, com, sous, chef, https, run, actions

Example successful prompts:

1. PR [blog] Agent of the Day – 2026-09-02 #57966: [blog] Agent of the Day – 2026-09-02 → Merged
2. PR Add model inventory validation to compile #57958: Add model inventory validation to compile → Merged
3. PR Fix temporary ID substitution in safe-output comments #57955: Fix temporary ID substitution in safe-output comments → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 248.3 words
• Most common keywords: github, workflow, agent, copilot, issue, com, run, https

Example unsuccessful prompts:

1. PR Suppress runner-guard alert for Playwright loopback healthcheck #57859: Suppress runner-guard alert for Playwright loopback healthcheck → Closed
2. PR Add Claude Code LSP support #57258: Add Claude Code LSP support → Closed
3. PR Document current OpenAI Codex LSP support #57257: Document current OpenAI Codex LSP support → Closed

Key Insights

• Length matters modestly: Closed PR prompts averaged 248.3 words vs 215.8 for merged ones — longer, more sprawling prompts trend toward closure, though the gap is not huge, suggesting length alone isn't decisive.
• CVE/security-report language appears disproportionately in closed PRs: keywords like cve, report, engine rank highly among closed-PR terms but not merged ones, hinting that security-scan-triggered or exploratory security PRs are harder to land cleanly (likely due to reviewer scrutiny or false positives).
• Feature-labeled prompts have the highest success rate (88.1%) versus bug-fix prompts (83.8%), possibly because feature requests tend to be scoped with clearer acceptance criteria while bug-fix prompts vary widely in specificity.

Recommendations

Based on today's analysis:

1. DO: Keep prompts focused and moderately concise — the closed-PR average is noticeably longer, suggesting scope creep or ambiguity reduces success.
2. DO: For security/CVE-related fixes, provide precise reproduction details and reference exact files/lines, since these prompts show a lower merge rate overall.
3. AVOID: Vague "improve"/"report" style prompts without a concrete actionable ask — they cluster with closed PRs.

Historical Trends

Date	PRs	Success Rate	Top Category
2026-08-31	994	84.3%	Bug Fix
2026-09-01	1000	84.1%	Bug Fix
2026-09-02	1000	83.9%	Bug Fix

Trend: Merge rate has stayed relatively stable in the 82-85% range over the last several observations, with bug-fix prompts consistently the dominant category by volume.

---

Generated by Copilot PR Prompt Analysis (Run: 33680771971)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · 26.1 AIC · ⌖ 5.95 AIC · ⊞ 11K · ◷

• expires on Sep 3, 2026, 12:48 PM UTC-08:00