Supacrawler
Turn any website into any structured LLM data you want
Start new thread
Antoine Ross

Supacrawler - Headless browser tool for scraping anything on the internet

by•
Supacrawler is a simple API for developers to reliably crawl sites, scrape content, and capture screenshots. Built and priced for individual builders and researchers. Available today in Public Beta.

Add a comment

Replies

Best
Antoine Ross
Maker
📌
Hi ProductHunt! I originally built Supacrawler for myself. I needed a reliable way to scrape web data for a project, and when I looked at the existing tools, I noticed a pattern: they were all priced for enterprise. That pricing model doesn't work if you're a student, a researcher, or a small company. So, I built Supacrawler to fix this. It's a powerful scraping engine designed for builders, not just big businesses. It does all the hard work without the enterprise price tag. Supacrawler was built with developers in mind first and now has the capabilities of: scraping pages, automatically crawling multiple pages such as docs, and taking screenshots with multiple configurations! My goal is simple: if you have an idea that needs data, you should have access to the tools to get it. I hope you find it useful. Let me know what you think!
Mu Joe
GPT-4o

This is realy cool! Ngl, finding an affordable but powerful scraping tool has been a pain, especially when you're not an enterprise. The focus on builders and not just big businesses is a game-changer. How robust is the automatic multi-page crawling for very deeply nested sites?

Antoine Ross
Maker

Hi all! Happy to share that I'm releasing the second part of this release:

Here are the SDKs!

https://github.com/Supacrawler/supacrawler-py

https://github.com/Supacrawler/supacrawler-js

I've included examples on how to use the SDK for every api route we have and will add more guides for more use cases soon! Stay tuned for that in the docs and the blog as well! 

See how to install here: https://docs.supacrawler.com/api/install

FYI users who sign up get free 500 tokens, and if you subscribe in the following month you will also get an additional free 5000 tokens...on us! 👀