ai-crawler-detection

Here are 2 public repositories matching this topic...

stefanoseggio / ai-crawler-content-signal-permission-monitor

Apify Actor that delta-monitors robots.txt AI-crawler permissions (GPTBot, ClaudeBot, etc.), Cloudflare Content-Signal headers, and llms.txt for changes, billed Pay-Per-Event.

typescript data-engineering web-scraping robots-txt regtech change-detection apify seo-monitoring apify-actor gptbot llms-txt pay-per-event claudebot content-signal ai-crawler-detection delta-monitoring

Updated Oct 16, 2026

Octanium91 / ua-parser

Star

High-performance, cross-platform User-Agent parser: one Go core for Go, Java, Node.js, Python, plus a REST service and WASM/browser. Client Hints (Sec-CH-UA) priority, bot & AI-crawler detection, and a regex + correction database that hot-updates at runtime — no redeploy.

Updated Aug 31, 2026
Go

Add this topic to your repo

To associate your repository with the ai-crawler-detection topic, visit your repo's landing page and select "manage topics."

Learn more