Spider

Turn websites into organized data without code.

#4 Product of the DayApril 11, 2019
Featured Embed
Spider is a smart point & click web scraping tool. With it, you can turn websites into organized data without code. Simply start clicking and download the JSON/CSV when ready.
Reviews
  • al
    alPhoto creds @emileseguin
    Pros: 

    Tremendous product - comparable performance to many of the paid automatic scrapers out there on the market. Clean and intuitive UI/UX.

    Cons: 

    Onboarding docs need work. For a free product there's really not much more you can ask for.

    https://www.webscraper.io/ and https://getdata.io/ are two of the bigger competitors in the space. Will be very interesting to see how these guys will handle pricing/marketing/features that contrast with the current paid scraping companies in the field. My personal recommendation would be to productize the underlying technology to make it more accessible for the average user. This could be either be marketing towards one specific customer profile (fashion, entrepreneurship, etc.) and providing demonstrable value in that regard (tracking price dips/inflated black friday prices) OR providing more powerful hooks for enterprise clients (zapier, google sheets, etc.). I've used most of the big paid and free scraping tools on the internet for the companies that I've worked with and have high hopes for Amie! (If you guys don't know her, she has one of the best tech blogs on the internet.)

    al has used this product for one day.
  • Pros: 

    +ZERO experience needed, easy to use. +Intelligent column identification & selection. +Nice UI.

    Cons: 

    -Tool appears as overlay, hiding bottom page contents (including next page button !!😞 ) -Some false positives -json button didn't work

    Feature request: *Detachable window/new tab. *Auto forward pages & collect column data. *Rearrange columns both vertically & horizontally. *Should be able to delete specific items from a column. Overall great product. My best wishes.

    Shivam Singhania has used this product for one day.
Discussion
Hunter
Makers
You need to become a Contributor to join the discussion.
Amie Chen
Amie ChenMaker@amiechen01 · Designer & Developer. Love making stuff
Hi hunters! I made Spider, an easy-to-use web scraping tool that turns websites into organized data. It requires 0 configuration or programming experience, simply just start clicking! I was trying to gather a specific set of data that was scattered among blog posts and websites. Most of the scraping tools out there focus on recurring scraping of multi-page mega sites like Yelp. Although they do provide analytic data around the scraped results, it was not what I needed. I need something that allows me to easily scrap multiple webpages as I browse (like websites I only visit once), 0 friction (no upfront configuration needed), and persists the data (so I can keep growing the list as I scrape), and then generates a CSV. Also cheap. Hence the building of Spider, and it really did solve my problem! If you constantly find yourself scraping data, I hope this tool helps you too! I'm working on some new features so please sign up here(https://spider.amie-chen.com/) and let me know your use case. I'd love to make it useful for everyone.
Lola Tesla
Lola Tesla@lolatesla · Renaissance Woman
Yess!! I’ve been waiting for something like this!
Amie Chen
Amie ChenMaker@amiechen01 · Designer & Developer. Love making stuff
@lolatesla Hope it's helpful for you! DO let me know if you've any feedback :D
Hoang Nguyen
Hoang Nguyen@hoang_nguyen
How can I get image URL?
Amie Chen
Amie ChenMaker@amiechen01 · Designer & Developer. Love making stuff
@hoang_nguyen Thanks for trying it out! I didn't implement image URL atm, but will in the next version
Jacob Bøtter
Jacob Bøtter@boetter · Founder, author, etc.
Just tried this out and was skeptical at first, but it really works as promised. Keep it up, and keep this part of it free! Maybe make a live-updating hosted JSON feed that you'd have to pay for? ;-)
Amie Chen
Amie ChenMaker@amiechen01 · Designer & Developer. Love making stuff
@boetter Thanks Jacob! We have the same ideas - I absolutely want to keep this part free because I think it should. Still working on the next phase -- which may be a hosted thing-- depends on the feedbacks from this launch.
Benoit Chabert
Benoit Chabert@benoit2 · Lead Product Designer
Great Product Amie, super impressed! I'll give more feedback once I get to use it on a client's project.
Amie Chen
Amie ChenMaker@amiechen01 · Designer & Developer. Love making stuff
@benoit2 Thank you, please do!