CS student project - need help testing my code reviewer on real PRs

[yashwant86]

Hey, I'm a CS student in Germany and I built MergeMonkey, a code reviewer for pull requests. My university gave me a grant to run a pilot program so I can get real feedback and improve it.

Getting companies to try it as a pilot has been rough - they want proof before they commit. So I'm reaching out to open-source projects to build that trust and get real feedback. It's completely free, I cover the API costs myself.

Saw Appwrite has 256 open PRs so figured it could be useful here. Let me know if you're interested, would really help me out.

mergemonkey.dev

[hardikkaurani]

Interesting project, especially because PR review tooling is becoming one of the highest leverage areas for AI-assisted development.

Reviewing pull requests is fundamentally different from generating code — it requires understanding:

• repository conventions
• architectural intent
• test expectations
• security implications
• codebase history
• maintainers’ style preferences

So getting feedback from real OSS repositories is probably the right approach if you want the system to improve meaningfully.

The important part will likely be signal quality rather than review quantity. A lot of automated review tools struggle because they:

• generate noisy comments
• repeat obvious lint feedback
• miss architectural context
• suggest stylistic churn
• create reviewer fatigue

If MergeMonkey can consistently surface:

• real bug risks
• missing edge cases
• security concerns
• test gaps
• concurrency/state issues
• API contract inconsistencies

without overwhelming maintainers, that’s where it becomes genuinely valuable.

One thing that could help adoption in OSS projects is granular control, for example:

• review only changed files
• skip documentation-only PRs
• configurable review strictness
• label-based triggering
• confidence scoring
• “suggestion vs warning” separation
• ignore generated/vendor files
• repository-specific review rules

It would also be interesting to know:

• how you handle monorepos
• whether the system uses repository indexing/context windows
• if comments are deterministic
• how hallucinated review feedback is mitigated
• whether maintainers can train/customize project-specific review behavior

For open-source maintainers, trust and noise control are usually the biggest factors for adoption.

Also respect for funding the API costs yourself while validating the idea. Getting real-world feedback loops from active repositories is probably much more valuable than synthetic benchmark evaluations alone.

Website:
https://mergemonkey.dev/