Launching today
Cockroach Crawler
Built to be the best AI crawler for governed agents.
1 follower
Built to be the best AI crawler for governed agents.
1 follower
Give AI agents the web without handing them unrestricted network access. Cockroach Crawler is an open-source Node.js toolkit for static and rendered crawling, adaptive traversal, screenshots, PDFs, structured extraction, public GitHub and YouTube reads, MCP, Docker, and Cloudflare Workers. It returns Markdown, JSON, and evidence records through explicit origins, budgets, and runtime controls.
