Hey Product Hunt - I built Cockroach Crawler after hitting the same problem again and again: an AI agent could reason about the web, but giving it a normal browser or unrestricted network tool meant giving it way too much authority. The safer tools I tried solved that by limiting authority, but they also removed so many useful capabilities that the agent could not finish real work. I wanted the point where both sides meet - broad crawling, browser, extraction, and agent features, while the host still owns the origins, budgets, hooks, credentials, and evidence trail. So I built one Node.js package that can crawl static pages, render JavaScript apps, follow adaptive crawl paths, capture screenshots and PDFs, extract Markdown or structured data, read public GitHub and YouTube sources, and connect through JavaScript, CLI, MCP, Docker, or a Worker. It is open source, works on Node 22, 24, and 26, and the full test and benchmark evidence is public. I would genuinely love people here to break it. If you test it, tell me the OS, Node version, public URL, and the smallest reproducible failure. That feedback is worth more than a generic "looks cool" comment. - Ajnas

Cockroach Crawler

Built to be the best AI crawler for governed agents.

Built to be the best AI crawler for governed agents.