Spider Free - Turn websites into organized data without code.

Hi hunters! I made Spider, an easy-to-use web scraping tool that turns websites into organized data. It requires 0 configuration or programming experience, simply just start clicking! I was trying to gather a specific set of data that was scattered among blog posts and websites. Most of the scraping tools out there focus on recurring scraping of multi-page mega sites like Yelp. Although they do provide analytic data around the scraped results, it was not what I needed. I need something that allows me to easily scrap multiple webpages as I browse (like websites I only visit once), 0 friction (no upfront configuration needed), and persists the data (so I can keep growing the list as I scrape), and then generates a CSV. Also cheap. Hence the building of Spider, and it really did solve my problem! If you constantly find yourself scraping data, I hope this tool helps you too! I'm working on some new features so please sign up here(https://spider.amie-chen.com/) and let me know your use case. I'd love to make it useful for everyone.

How can I get image URL?

Yess!! I’ve been waiting for something like this!

Feature request: *Detachable window/new tab. *Auto forward pages & collect column data. *Rearrange columns both vertically & horizontally. *Should be able to delete specific items from a column. Overall great product. My best wishes.

Pros:

+ZERO experience needed, easy to use. +Intelligent column identification & selection. +Nice UI.

Cons:

-Tool appears as overlay, hiding bottom page contents (including next page button !!😞 ) -Some false positives -json button didn't work

Just tried this out and was skeptical at first, but it really works as promised. Keep it up, and keep this part of it free! Maybe make a live-updating hosted JSON feed that you'd have to pay for? ;-)

https://www.webscraper.io/ and https://getdata.io/ are two of the bigger competitors in the space. Will be very interesting to see how these guys will handle pricing/marketing/features that contrast with the current paid scraping companies in the field. My personal recommendation would be to productize the underlying technology to make it more accessible for the average user. This could be either be marketing towards one specific customer profile (fashion, entrepreneurship, etc.) and providing demonstrable value in that regard (tracking price dips/inflated black friday prices) OR providing more powerful hooks for enterprise clients (zapier, google sheets, etc.). I've used most of the big paid and free scraping tools on the internet for the companies that I've worked with and have high hopes for Amie! (If you guys don't know her, she has one of the best tech blogs on the internet.)

Pros:

Tremendous product - comparable performance to many of the paid automatic scrapers out there on the market. Clean and intuitive UI/UX.

Cons:

Onboarding docs need work. For a free product there's really not much more you can ask for.

Quite a clever idea, Amie! I just sent you a DM on twitter about a good use case.

nice tool! It's possible to scrap image files? png or jpg

Great Product Amie, super impressed! I'll give more feedback once I get to use it on a client's project.

Brilliant!

Been looking for this for so long! Thanks for making it!

Hey @amiechen01, great tool that comes in so handy :) I've flipped you a message on Twitter with something I've noticed 👍️ Keep the great work up! -- Peter

Oooooh this is neat!

This looks awesome, great work!

Sent you an email regarding some typos I found

Very useful! Will you make a firefox extension?

Hey Amie! really useful little tool. We use scrapers in SEO a lot, but there were mainly not that easy to operate. Yours comes really handy for small analysis projects. Kudos!

@amiechen01 Love the product Amie. Can't wait to use it every time ;')

Hello. I recently bought the product, but I am having problems installing

This looks great Are you able to scrape from pages that you have logged into? Or just public pages? Thanks!