Turn any website into an API

#2 Product of the WeekAugust 08, 2019
Dashblock turns websites into APIs thanks to machine learning.
Go to a page, right-click on the data you want and save your custom API. That's it, you can now query this API with similar URLs to automatically receive the structured data!
Would you recommend this product?
31 Reviews4.4/5
Dashblock created a powerful algorithm and an intuitive interface that makes web scraping really easy! Check it out
Upvote (14)Share
@gustaf Thanks for the support! =)
Very intuitive, Would make a very powerful tool for web scraping with a bit more intelligence. It was just a quick try on a random airbnb property page, I think with more time spent on tagging all different part, the result could have been nice. Very curious about the progress of this project, I will keep in touch, in a meanwhile, Dashblock gets my vote.
@timothee_grosjean Thanks for your feedback Timothée !
Looks very good. Could I use this for notion or any sites that require a login?
@jwelch Thanks James ! We don't support authentication yet but it will come really soon and you will be able to go to any page, even behind a login, and query your API the same way =)
@maxime_corbani NOW we're talking!!! :-)
@jwelch @maxime_corbani Was going to ask the same thing. Is there a time frame for this kind of implementation?
@jwelch @maxime_corbani in that case we will need to provide you with our authentication passwords ? that seems fishy (honest question)
@jwelch @maxime_corbani @francoolaami We are trying to figure out what is the best way to implement this feature but in any case if you want to scrape behind a login, we'll need to authenticate on your behalf on our servers.
This is really good. Is it possible to get an elements attribute rather than the content? Eg getting a company or review rating from Yelp - on page this is represented by images but the rating in text is in the container elements title or alt attribute. Also it would be good to filter what's matched with a regular expression or some presets like first word/number/etc . Eg if an element contains a price "Only $12.99" but I want the API to return {price:"12.99", currency:"$"} .
@bob_gamble Hi Bob, thanks for your feedback ! Unfortunately, It is not possible at the moment, but it will be added soon. You will be able to extract any attributes from elements and also format the data (date, currency, number...)
Works intuitively, but does not understands well repeating objects. For example, if I'm scraping PH main page (list of products) I'm getting a JSON which has title, image, description, score, etc, and each one of those is an array. What I would really like to get is an array of objects, and each one would have title, image, description, score, etc. Still, well done for an early product. Got my vote.
@danbars Hi Dan, thanks for your feedback and your upvote ! To work properly you have to make sure all the elements you want to extract are colored and give few examples for the same attribute until our model understand the pattern (usually 2 or 3 samples are enough). If you do that, you will have an array of JSON !