Supacrawler - Simple API for extracting structured content from the Web
by•
Supacrawler is now open-source!
We also released a new feature: Parse.
What if you could ask ChatGPT to list all of the careers on openai.com/careers and receive everything back in a CSV? In JSON? Or any other website?
That's exactly what it does. Try it!
Replies
Thanks everyone for the feedback so far. A quick update on this release:
This release is focused on making Supacrawler fully open source. Over the next month, we’ll be wrapping up beta. After that, Supacrawler will move into general availability.
The biggest change is the new Parse endpoint. It lets you extract structured content from any website and return it in JSON, CSV, or Markdown. Instead of just pulling raw HTML, Parse handles the heavy lifting of turning a page into clean, usable data. For example, you can crawl an entire job board and get the listings back as a structured CSV with one request. Docs are here if you want to see how it works: https://docs.supacrawler.com/api...
For now, if you subscribe to the Starter plan during beta, you’ll get 2x the tokens in your first month. This won’t carry over once we launch publicly, so it’s a good time to try it out if you’ve been on the fence.
Would love to hear what you’d build with Parse, and what features you think we should prioritize before GA. Thanks for taking a look!