Website Metadata Scraper API - Extract email, phone number and more from just a URL.

@aazar_ali_shad thanks for the kind words!

@tabansi thanks joshua :)

@jezzacrisp thanks for checking us out! to be honest, there are other companies who are specialized to provide that kind of info (eg. clearbit). the idea for this api was to provide a simple + cheap solution for a very specific problem. hope this makes sense

hey @edecaria! there are two factors at play here: 1. we are going through quite a big queue at the moment, so it's taking longer than usual. i can see it's run through for you now though :) 2. we go through up to 30 pages per URL to find the components we need, and only save the data we find at the end of this. i will make an update now so that the data is saved as it's found, which will show results a lot quicker. thanks for giving this a try!

@philip_kallberg Thanks for the reply. And yes I see my previous pending calls have gone through. So what do you thinking a typical turnaround time will be for a pair of POST + GET calls (successful)? Is it first-come, first-serve through the queue or do paying customers get bumped to front of the line? This API seems pretty solid. I'm currently using Diffbot to scrape social links, but tbh it doesn't have a very high success rate. For example, it doesn't pick up the sephora social handles, whereas your API does. Same for nike.com, which I just tried. I also see that my nike call is still crawl_status:pending even though I see the social handles on my GET request. Impressively quick work if you already made the change you noted above -- thanks!

@edecaria yeah our queue prioritises paying customers, the higher the plan the higher the priority. glad to hear you like it! and yes i did update that for you, i should have thought of that before haha :) also note that some big sites will have captchas and other blocking mechanisms which isn't a problem but add to the load time.

@philip_kallberg Cool. That makes sense. Mind explaining why the GET requests count toward the limit? Seems that the real unit of value is the POST. Especially if I start making multiple GET calls to try to pick up the response as soon as it's done, seems like I could burn through even 1,000 monthly calls pretty quickly. I'd suggest a webhook/callback when the crawl finishes or better yet when a specified field is populated. That's a side issue, though. Really just trying to figure out how to ensure a reliably fast response. This API seems like a perfect fit for a product of mine, but if it's 5-10 seconds sometimes and 1+ minutes other times, I can't really depend on it.

@edecaria yeah i agree the pricing isn't optimal, happy to discuss. why don't you try running through 20 urls and seeing the response time? if it's social info you want i'm sure that'll be pretty fast. most sites have those in footer so it'll be picked up on the first page. please join me on the website chat and we can discuss your use case and see how this could fit in.

@subhanjan_sarkar which URL are you using? some details would help :)

Hey @rachel_cook3, thanks for checking us out! I'm afraid we don't have that option for the time being, we can offer you a 30 day money back guarantee though if that helps.