Portia

Scrape websites visually

Featured

Discussion

most upvoted
Kumar Thangudu — 2X Saas Founder + Angel Investor
Sam Doshi — Co Founder @ Relayo.com
@datarade Thanks for posting - educational !
Kumar Thangudu — 2X Saas Founder + Angel Investor
Deandre Durr☀️ — Growth Hacker
@datarade sounds like you should make a collection of Scrapers.
Enoch Tang — Cofounder of Zlement.com
@datarade Wow this is an amazing list! Thanks!
Stuti — Fake Nomad. Mobiefit. India
@datarade This is really helpful. Thanks for sharing!
Nate Ritter — CEO, Perfect Space
@datarade kimonolabs.com was bought and is now defunct
Micah Carroll — Designer at Ring Video Doorbell
@datarade You're the man 😀
Joe Minock — AthleteProgress.com / Rails & JS Dev
@datarade Really appreciate this list! Especially bummed that Kimono is gone, though.
Sahil Chaturvedi — Co-founder, Ader (eSports sponsorships)
@datarade Jeez that's a lot! Are they all similar, or just different use cases?
GIANNI Ð'ALΞRTA — Director of Marketing @ ActivePBX.com
@datarade love this!
Kumar Thangudu — 2X Saas Founder + Angel Investor
@giannidalerta follow me and check out my blog. My plan is to continue these type of comments. ;)
Nick Kwan — Head of Growth, Pakible (YC W15)
@datarade I've seen a lot of your epic scraper comments on PH, so looks like you are the scraper king haha.

What are your go-to scrapers nowadays for social networks like Linkedin + Twitter? Looking for something web based / usable on Mac. Data-miner.io is good, but a bit complex for us non-technicals. Many sites like Portia, Import, etc don't work on these sites.
Kumar Thangudu — 2X Saas Founder + Angel Investor
@nwkwan
Scraping king? No.
Have I used and/or looked at more enterprise and consumer apps than anyone on the planet? potentially.

I would reachout to jakub@apifier and talk shop with him about this. He's your best bet. Knows the most about this space.
Nick Kwan — Head of Growth, Pakible (YC W15)
@jakubbalada thoughts? Checking out @apifier now.. thx @datarade
Jakub Balada — Co-founder, Apifier
@nwkwan @apifier @datarade You can use Apifier for scraping Linkedin, but you would need a lot of proxies for that site. Let me know if you need some help with crawler configuration...
GuessBox — Official GuessBox ProductHunt Profile
Gabriel Puliatti — Founder, Emptor
I've worked for Scrapinghub for the past two years, happy to answer any questions about Portia, its big brother Scrapy, or any part of our platform... or even web scraping in general!

Us and our users are currently crawling 3.5 pages billion per month, or around 80,000 pages per minute. So we know a little bit about scraping. :)
Jake Miller — hyperLincoln
@gpuliatti I'm curious to hear your perspective to) Pala tired acquisition and sudden shutdo not of kimono labs and how they handled that?
Gabriel Puliatti — Founder, Emptor
@jpmillions We're open source guys (and gals)… so we're definitely saddened to see users of a closed platform treated this way.

OTOH, we've seen a lot of customers coming to try Portia from Kimono. :) We're actively working on ways to help people port their Kimono crawlers, so keen on hearing anyone in this boat! Email me directly (gabriel@scrapinghub.com) or sign up to the mailing list at the bottom of this post (https://goo.gl/CGxsFl).

Both Portia and Scrapy are fully open source, and any crawlers (created or running) in our platform are fully exportable and interoperable with open technologies.

While we are focused on the long-term and so doubt our platform will be shut down any time soon, if that ever happened, all of our users would be able to export their crawlers and use them on their own infrastructure. We've done this ourselves for some of our Professional Services clients who want us to build scrapers but also run things on their own infrastructure.
Evan Lodge — Co-Founder, HigherMe
@gpuliatti I tried scraping a list of 27,000 urls... the browser crashed. Is there any easier way of adding URLs to the scrape?
Commenting is limited to those invited by others in the community.
Login to continue.