GitHub Discovery is a daily GitHub repository discovery pipeline that surfaces projects while they are still accelerating, for developers who want early signal instead of yesterday's popularity.
Website · Quick Start · Features · Development
English | 简体中文
GitHub Trending shows you what's popular today.
GitHub Discovery shows you what's about to be popular — repos with unusual growth patterns, community picks from Hacker News, and early-stage projects gaining traction.
Every day it collects signals from 6 data sources, runs them through a smart scoring system (100 points), and delivers curated results via email and web.
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ GitHub │ │ GitHub │ │ Hacker │
│ Trending │ │ Search │ │ News │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘
│ │ │
┌──────┴──────┐ ┌──────┴──────┐ ┌──────┴──────┐
│ Rising │ │ AI/ML │ │ HF Daily │
│ Detection │ │ Keywords │ │ Papers │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘
│ │ │
└────────────────┼────────────────┘
▼
┌───────────────────────┐
│ Smart Scorer │
│ (100 points) │
│ ───────────────── │
│ acceleration : 40 │
│ quality : 30 │
│ anti-spam : 30 │
└───────────┬───────────┘
▼
┌───────────────────────┐
│ Cross-day Dedup │
│ (7-day window) │
└───────────┬───────────┘
▼
┌────────────────┴────────────────┐
▼ ▼
┌─────────────┐ ┌─────────────┐
│ 📧 Email │ │ 🌐 GitHub │
│ Digest │ │ Pages │
└─────────────┘ └─────────────┘
| Source | Signal | What it catches |
|---|---|---|
| GitHub Trending | Popularity | Daily trending repositories |
| GitHub Search | New & rising | Repos created in the last 7 days with fast star growth |
| Hacker News | Community picks | GitHub repos from Show HN posts |
| Rising Detection | Early signal | Repos created in the last 3 days carrying unusually high fork counts — forks as an early usage signal |
| AI/ML Keyword Sweep | AI focus | GitHub Search over an AI/ML keyword list, 5 keywords per day on a rotating schedule (inspired by OSSInsight trending/ai, implemented against the GitHub Search API) |
| HF Daily Papers | Research signal | GitHub repos linked from trending Hugging Face papers — paper upvotes lead GitHub stars by days |
| Dimension | Points | What it measures |
|---|---|---|
| Acceleration | 40 | Real day-over-day star growth (from daily snapshots) + acceleration vs lifetime average |
| Quality | 30 | Age, language, license, content completeness |
| Anti-spam | 30 | Starts at 30 and deducts: star/fork ratio > 50, age < 3 days with 5000+ stars, marketing buzzwords, gaming-trainer or random-username patterns |
| Code Quality | +20 | README, CI config, commit frequency — scaled into the quality dimension rather than added on top |
| Suspicious Stars | -20 / -15 | Age ≤ 1 day with 1000+ stars (-20); age ≤ 2 days with 2000+ stars (-15); 1000+ stars/day with an empty description (-15) |
| Batch Fraud | up to -40 | Same owner with ≥ 3 repos in one batch (-15), ≥ 2 repos under 7 days old with 200+ stars (-15), template-similar descriptions (-10) |
- Star fraud detection: age ≤ 1 day with 1000+ stars, age ≤ 2 days with 2000+ stars, or 1000+ stars/day with an empty description → flagged
- Batch fraud detection: same owner with ≥ 3 repos in one batch, or ≥ 2 repos under 7 days old already past 200 stars → flagged
- Content quality: No description or no README → penalty
- Cross-day dedup: 7-day window, no duplicate recommendations
- Daily curated repos delivered to your inbox
- Dark mode support (Apple Mail / iOS)
- Powered by Resend API
- Prefer a feed reader over email? Subscribe to the Atom feed:
https://alloevil.github.io/github-discovery/feed.xml - Last ~14 days of picks, regenerated with every daily run
- Modern, professional web interface
- Filter by date and language
- Real-time scoring display
Nothing to install: the pipeline uses the Python standard library only, there is no requirements.txt, and CI runs Python 3.11.
git clone https://github.com/alloevil/github-discovery.git
cd github-discovery
python scripts/main.py # one full run: collect → score → dedup → write output/ + data/Sending the email digest needs a RESEND_API_KEY. Collection, scoring, the published site and the Atom feed need no key beyond GitHub's own GITHUB_TOKEN. To run it as a daily automation rather than locally, follow Quick Start below.
- You want to see a library, tool or model in the days before it hits 10k stars, not after.
- You want the discovery decision to be inspectable: every recommendation carries its score, and every run's input data stays committed under
data/. - You want zero infrastructure — GitHub Actions plus one API key for email. No server, no database.
- You care about filtering manufactured popularity: star-authenticity checks, cross-repo batch-fraud detection and description/README quality checks all subtract from the score.
- You want editorial judgement. This is a scoring function; it will sometimes rank a repo highly on acceleration alone.
- You want per-topic or per-user subscriptions. The six sources are fixed in
scripts/sources.pyand the thresholds inscripts/config.py; the site filters by date and language only after the fact. - You want proof that a high score predicts long-term success. The committed backtest covers a 7-day window over 20 repos at one point in time — enough to sanity-check the score, not enough to establish predictive power.
- You need every rising repo. Only repos that surface in one of the six sources can be scored, cross-day dedup suppresses a repo for 7 days after it is recommended, and the expensive per-repo quality and star-authenticity checks run only on the top
DEEP_CHECK_TOP_K(20) candidates after coarse ranking. - You want the AI/ML sweep to be exhaustive on a given day: it uses 5 keywords per day out of a rotating list, so one day covers only part of that space.
Click the Fork button in the top right corner.
Go to Settings → Secrets and variables → Actions and add:
| Secret | Required | Description |
|---|---|---|
RESEND_API_KEY |
✅ | Resend API Key for sending emails |
GITHUB_TOKEN |
❌ | GitHub Personal Access Token (optional, uses GITHUB_TOKEN by default) |
FIRECRAWL_API_KEY |
❌ | Firecrawl key. Makes GitHub Trending parsing robust (scrapes the page instead of regex-matching raw HTML). Without it, Trending falls back to direct HTML scraping. |
Go to Actions and click I understand my workflows, go ahead and enable them.
Go to Actions → Daily Discovery → Run workflow to trigger a test run.
- GitHub Pages: Visit
https://<your-username>.github.io/github-discovery/ - Email: Subscribers receive daily digests
github-discovery/
├── scripts/
│ ├── sources.py # 6 data source collectors
│ ├── scorer.py # Scoring algorithm
│ ├── quality.py # Code quality detection
│ ├── anti_spam.py # Anti-spam scoring dimension
│ ├── dedup.py # Cross-day deduplication (7-day window)
│ ├── fraud_detection.py # Batch fraud detection
│ ├── snapshots.py # Daily star snapshots (real growth)
│ ├── verify_scoring.py # Scoring verification / backtesting
│ ├── generate_site.py # GitHub Pages site + Atom feed
│ ├── subscribe_handler.gs # Google Apps Script subscribe endpoint
│ ├── main.py # Entry point
│ └── config.py # Configuration
├── tests/ # Unit tests (pytest)
├── docs/ # GitHub Pages (index.html, feed.xml)
├── .github/workflows/ # Daily automation
└── subscribers.txt # Email subscriber list
git clone https://github.com/alloevil/github-discovery.git
cd github-discovery
python scripts/main.pypip install pytest
python -m pytest tests/ -v- Add a new
fetch_xxx()function inscripts/sources.py - Call it in
fetch_all() - Add tests in
tests/test_sources.py - Submit a PR
Scoring logic is in scripts/scorer.py. The dimension limits are constants in scripts/config.py:
ACCELERATION_MAX = 40
QUALITY_MAX = 30
ANTISPAM_MAX = 30
QUALITY_BONUS_MAX = 20 # deep code-quality bonus, scaled into the quality dimension
DEEP_CHECK_TOP_K = 20 # how many coarse-ranked candidates get the expensive checksRun backtesting to verify whether high-scored repos actually took off. The
backtest reads the committed daily JSON reports (data/discovery-*.json),
so it works on a fresh clone with no local state:
python scripts/verify_scoring.py --days 30How are the 100 points allocated? Acceleration is worth up to 40 and measures real day-over-day star growth from the project's own committed snapshots plus acceleration against the repo's lifetime average. Quality is worth up to 30 (age, language, license, content completeness). Anti-spam starts at 30 and deducts for a star/fork ratio above 50, an age under 3 days with 5000+ stars, marketing buzzwords, and gaming-trainer or random-username patterns. A deep code-quality check adds a bonus of up to 20 that is scaled into the quality dimension, star-authenticity checks subtract 15 or 20, and batch fraud subtracts up to 40.
Do I need any paid service? Only for email. RESEND_API_KEY sends the digest through Resend; FIRECRAWL_API_KEY is optional and only makes GitHub Trending parsing more robust than regex-matching raw HTML. Everything else runs on the GitHub Actions free tier with the default GITHUB_TOKEN.
Will the same repo be recommended every day while it is hot? No. Cross-day dedup blocks any repo recommended in the previous 7 days, using the history committed at data/recommend_history.json; records older than 30 days are cleaned up. The history has to be a committed file because every CI run starts from a fresh checkout with no local state.
Can I trust the numbers on the published page? The page is a build artifact: scripts/generate_site.py renders docs/index.html from docs/template.html plus the committed reports, and the workflow publishes docs/ to the gh-pages branch. The authoritative copies are the committed JSON under data/ and the Markdown digests under output/ — which is also what verify_scoring.py reads, so any claim on the page can be recomputed from a fresh clone.
How do I add a data source? Add a fetch_xxx() function in scripts/sources.py, call it from fetch_all(), and add tests in tests/test_sources.py. Sources return the same repo dict shape, so scoring, dedup and rendering need no changes.
Contributions are welcome! Please follow these steps:
- Fork this repo
- Create a feature branch:
git checkout -b feature/your-feature - Commit your changes:
git commit -m 'feat: add your feature' - Push the branch:
git push origin feature/your-feature - Submit a Pull Request
- 📡 Add new data sources
- 🎯 Optimize scoring algorithm
- 🐛 Fix bugs
- 📖 Improve documentation
- ✅ Add tests
This project is licensed under the MIT License.
- GitHub API
- Hacker News API
- OSSInsight — AI/ML repository trends and analytics
- Resend
- Firecrawl — robust web scraping for GitHub Trending
⭐ If you find this useful, please give it a star!