SkydiveAI coworkers that actually get work done
Promoted
Mindcase is the infrastructure layer for extracting web data in a structured, usable format. Built for developers and AI teams that need reliable web data without managing scraping infrastructure. Access APIs across popular sources, or get anything across the web built as a custom API for your specific use case.
Vadoo AI
Congratulations on the launch!
answering Saurabh's question directly - we're a B2B SaaS and the source I'd actually pay for is G2/Capterra review pages for our competitors: review text, star rating, and reviewer company size band. Maybe 500-1000 rows a month, nothing crazy. Every competitive intel tool I've tried treats review sites as an afterthought behind the "big" platforms. Also curious about the pricing question Raunak asked above - if I fetch the same post twice in a day, am I paying twice or is there any caching on the backend?
Mindcase
@galdayan Appreciate you answering the question properly Gal. Review text, rating and reviewer size band across G2 and Capterra at around 1,000 rows a month is a well shaped request and we will come back to you in this thread with a straight yes or no on both sites rather than a maybe.
On pricing, you pay per collection, so calling the same page twice bills twice. Whether that matters depends entirely on the source. For reddit or twitter, as I said to Raunak above, the second call is usually the point, since the upvotes and comment counts have moved and you are collecting a second observation rather than the same row. Reviews sit at the other end of that. They accumulate rather than change, so a scheduled pull plus a dedupe on review id at your end is cheaper and gives you the same picture.
Curious to see how this handles websites that constantly change layouts or add new restrictions. If the reliability holds up, this could become a useful foundation for many data-driven products.
Mindcase
@ryankeller Good thing to be sceptical about Ryan.
The structural advantage is not that we are cleverer about layout changes, it is that when a site changes it breaks for everyone using that agent at once, so we hear about it immediately and fix it once for all of them.
Maintaining it in house means you find out alone, usually late, and fix it alone every time. We are not going to claim we never break. We are claiming you are not the one who has to notice or repair it.
Buffup.AI
The custom API angle is interesting. A lot of scraping tools work well for known sources, but the anything across the web problem is where teams usually get stuck.
Mindcase
@sansa_grey Thank you, and worth saying plainly: anything across the web is the ambition, not a guarantee.
Some sources are genuinely a bad idea and we would rather tell you that than take the work and disappoint you two months in. What we can commit to is a straight answer either way, quickly.
If you have a source that has stalled a project before, name it and we will tell you honestly where it sits.
Netlify
Normalizing web extraction behind a single API call shape is a total game-changer for AI teams. Super cool build, congrats!
Mindcase
@thisiskp_ Thank you KP, so glad you like it.
That was the main thing we wanted to get right. Good to hear that comes across from outside.
The maintenance tax is the part people underestimate, and the failure mode that cost us most was not scrapers breaking loudly. It was scrapers succeeding quietly. A selector drifts, the extractor returns an empty string, and the pipeline records a clean zero instead of an error. We spent weeks counting those as real results before we noticed.
So the question I would put to any extraction layer: when a page changes shape and the parser finds nothing, does the API return success with an empty payload, or does it tell me it could not read the page? Those are very different products downstream.
Curious how you handle that, and whether the custom APIs come with any drift detection when a source silently changes.
Mindcase
@whateverneveranywhere This is the right question and the clean zero is exactly the thing we set out not to do. When the parser cannot read a page you get an error you can see, not a success with nothing in it. A false zero is worse than a failed run, because a failed run gets retried and a false zero gets believed, which is what happened to you for those weeks.
Custom agents are treated the same as the catalogue ones, same monitoring, they are not a second class thing we hand over and forget. If you ever get a silent empty back from us, that is a bug and we would want to hear about it directly.
A failed run gets retried and a false zero gets believed is a better sentence than the one I wrote, and it is exactly the shape of it. The retry path has a guardian. The believed path has nobody.
The rule that eventually saved us was refusing to let the pipeline score itself on its own output. If our own read of the page is the only evidence, it does not count as evidence. That was expensive to retrofit and I would have paid real money for it to have been the default.
Good luck with the launch, and thanks for answering the actual question.
SpeedLegal
This is actually pretty useful. Keeping scrapers working is such a pain, so having it all behind one API makes a lot of sense. Nice launch 👏
Mindcase
@rajat_kapoor05 Putting it behind an API is partly a technical decision and mostly a way of making sure that knowledge is not sitting with one person.