Crawl any website for free. Discover every page, get titles, meta descriptions, HTTP status codes, response times, indexability, and more — all in a clean report.
Replies
Best
Maker
📌
Hey Product Hunt! 👋
I built webcrawler.buzz to solve a problem I kept running into — needing a quick, no-fuss way to crawl a website and see every page it has.
Whether you're doing an SEO audit, checking for broken links, mapping out a competitor's site structure, or just want a full inventory of pages on your domain — webcrawler.buzz handles it in seconds.
How it works:
- Enter any URL and hit crawl
- Watch real-time progress as it discovers pages using BFS (breadth-first search)
- Browse paginated results with page titles, URLs, and status codes
- Export everything to CSV with one click
- Get email notifications when your crawl is done
- No sign-up required. No usage limits. Completely free.
I'd love to hear your feedback — what features would make this more useful for your workflow? Site comparison? Broken link detection? Scheduled crawls?
Thanks for checking it out! 🚀
Replies