WrapAPI

Build an API on top of any website

#3 Product of the DayApril 05, 2016

Reviews

Discussion

Makers
You need to become a Contributor to join the discussion - Find out how.
Prathamesh Juvatkar
Prathamesh Juvatkar@pjuvatkar · CTO at NanoNets.com
In the product we are building, we need to use some scrapers to pull a particular parameter from website. Usually we write a simple script in python using Beautiful Soup or some other library to pull data. But biggest challenge has been that any updates in DOM of that website will make the script unusable. How do you handle this issue?
Peter Xu
Peter XuMaker@peterxu
@pjuvatkar It's still a manual process, but we try to help a little bit! Each API endpoint you create is versioned, and we let you send requests to /your/api/element/latest tag and just publish an updated version of the endpoint so that you don't have to change any of the calling code.
Prathamesh Juvatkar
Prathamesh Juvatkar@pjuvatkar · CTO at NanoNets.com
@peterxu I am very excited about the crowdsourcing angle to such problems. Someone in community can submit a fix and everybody in community benefits
Sarah Cornwell
Sarah Cornwell@sccornwell · @appbackr, @forum280, PH Contributor
Can this replace the Kimono extension? It shut down in Feb.
maxkolysh
maxkolysh@maxkolysh · Founder, Zinc.io
@sccornwell yes I'll reach out to him :)
Peter Xu
Peter XuMaker@peterxu
@sccornwell Yes, it can (mostly) replace the Kimono extension, though it has some more features and a potentially slightly steeper learning curve, as it creates general-purpose APIs (including those with POST requests backing them), nested outputs (vs. collections), and is shared amongst the community.
Sarah Cornwell
Sarah Cornwell@sccornwell · @appbackr, @forum280, PH Contributor
@peterxu Fantastic news! Thrilled to hear that!
Dylan Buckley
Dylan Buckley@dylankbuckley · Founder @ HackerMetrics
Does anyone know the Maker Peter Xu? Would be cool to ask some questions!
Peter Xu
Peter XuMaker@peterxu
@dylankbuckley Hey! I'm now on ProductHunt; either message here or email me at peter@wrapapi.com, and I'd be happy to help
Bruno Skvorc
Bruno Skvorc@bitfalls · Bitfalls.com, Coinvendor.io, Diffbot.com
I work for Diffbot, a competing (but not really) company, and we use AI to auto-classify content and deliver it as JSON via our APIs. (If you're interested, here's a bunch of tutorials: http://sitepoint.com/tag/diffbot) - I'm mainly curious about how you handle changes in the DOM, and different content being "API-fied", i.e. products vs. text vs. images, etc. Do you have a machine learning side of things, or is it mainly manual?
robertprince
robertprince@robertprince · VP Engineering, ActiveRain
Clever, but if I found you were circumventing my API using this, I'd blackhole you.
Cody
Cody@rdbrdd · Founder of Redbearded
@robertprince so hostile...
robertprince
robertprince@robertprince · VP Engineering, ActiveRain
@rdbrdd No, just serious about my business and savvy to API 'usage'. You?